The managed files hook's content read looped the file's model mappings and asked each deployment for the file. A file LiteLLM stored itself maps every model to its storage url, so the read sent that internal id to the upstream server, failed, and the batch rate limiter failed open: a key's TPM limit did not apply to a LiteLLM-executed batch. The hook now returns the stored bytes from the file's storage backend before it consults any deployment
An explicit target_storage=litellm_db upload was accepted for any model, so an OpenAI model's litellm_db://<uuid> id was sent to OpenAI as input_file_id and a model-less upload left a content row nothing can read; it now answers 400 on target_storage. An explicit target_storage skips the files api probe and the purpose and single-target gates, which only decide whether LiteLLM keeps the file itself, so an azure_storage user_data upload for a vLLM model reaches the storage path again as it did before this branch. cancel_batch authorizes the model of every LiteLLM-managed batch id before it branches, the way retrieve_batch already does, so the LiteLLM-executed branch gets the check its provider sibling had. Restores the test_afile_delete_passes_trusted_model_credentials_to_router definition line an earlier commit dropped
A batch upload naming a model on a LiteLLM-executed provider now checks
that the key may call that model before the upstream server is probed for
a Files API, matching the order batch create already uses. Only targets on
an executed provider are checked here, so provider-model uploads keep
their existing behavior.
File content reads and writes move out of the storage backend into
ManagedFileContentRepository, so the backend no longer queries Prisma
directly.
* fix(otel v2): summarize embedding vectors as Langfuse observation output
The v2 LLM span built its output only from response choices, so /v1/embeddings
rendered a Langfuse generation with input, usage and cost but a blank output.
Embedding calls now carry an EmbeddingOutput(count, dimensions) summary that the
Langfuse mapper serializes as the observation output, and they are exported with
the embedding observation type instead of generation. Chat and Responses output
mapping is unchanged.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel v2): keep embedding observations typed as generation in Langfuse
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The config file winning over the database was silent. An admin who had set
a value through the UI and later pinned the same key in the file saw their
stored value quietly stop applying, with nothing said at boot and nothing
said when a later write was refused.
Startup now warns once per key whose stored value differs from the file's,
naming the key and what to do about it. The refusal raised on a write to a
config-owned key carries the same sentence, so the log and the 400 read
identically, and both call out that a stored value exists and will never be
applied. The /config/update refusal gained the same detail.
Keys the file does not declare are untouched: the database still owns them,
and a stored value equal to the file's is not worth a warning.
The v2 LLM span built its output only from response choices, so /v1/embeddings
rendered a Langfuse generation with input, usage and cost but a blank output.
Embedding calls now carry an EmbeddingOutput(count, dimensions) summary that the
Langfuse mapper serializes as the observation output, and they are exported with
the embedding observation type instead of generation. Chat and Responses output
mapping is unchanged.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Add an aws_textract OCR provider on the Rust route, with no Python path. The
detect-document-text model returns plain lines and analyze-document renders
layout and tables as markdown. Both use Textract's synchronous API, so a
multi-page PDF or TIFF is rejected with an error that names the single-page
limit. A call with no region fails instead of falling back to Bedrock's default
SigV4 covers the request body, and host hooks can rewrite that body before it
is sent. litellm-http now has OutboundRequest, which serializes the body once,
shows those bytes to a RequestSigner and is the only thing a route can send.
Chat, audio transcription and OCR build it after their hooks ran, so a callback
that redacts the body still produces a valid Bedrock or Textract signature
ChatCompletionsAuth and AudioTranscriptionAuth are replaced by
litellm_auth::RequestAuth, and one helper in core turns it into a signed or
unsigned request. Audio transcription now signs only the AWS header set and
rejects a forwarded header that SigV4 computes, the same as chat
The OCR catalog routes aws_textract as Rust required, and the dispatch context
reads the provider from the model prefix so a provider scoped rule can match
The OCR route called back into Python's get_secret_str for every env
fallback. With no secret manager configured that is os.environ behind a GIL
hop, and with one configured it blocked a tokio worker on vault I/O and also
sent the Azure and GCP identity variables, which Python reads with os.getenv,
to the vault. The other Rust routes already read the process environment.
Read the process environment here too. When litellm would read secrets from
a secret manager, decline the Rust route so the Python route serves the call
with the vault-backed keys
The azure row of test_get_model_info_falls_back_from_dated_snapshot_to_undated_entry used gpt-5.6-luna-2026-07-09, which main's cost map carries as an exact azure key, so the lookup returned the dated key and the required misc test job failed on main. All three dated snapshot tests now use a 2099-01-01 snapshot date, so they keep exercising the strip path whatever real snapshots the map gains
DeepSeek charges half the listed rate outside 01:00-04:00 and 06:00-10:00 UTC
Monday to Friday, so every deepseek-flash, deepseek-v4-flash,
deepseek-v4-flash-vision-exp, and deepseek-v4-pro entry now carries an
off_peak_pricing block with those windows and the halved input, output, and
cache-hit rates. The generated cost map schema picks up the block, and the
regression tests pin the peak and off-peak cost of one call at fixed moments.
The S3 Vectors ingestion embedded every chunk with the request's
embedding.model or the default, never the embedding_model the store was
registered with, while search on the same store embeds with the
registered model. A registered store uploaded to by id alone therefore
embedded with the wrong model and AWS rejected the vectors on the
dimension mismatch. The store's embedding model now wins for S3 Vectors
ingestion through a helper next to the one search already uses
