The strip in text_tokens_without_nested_reasoning is capped at the reasoning
share, the reported text, and the over-sum past completion_tokens. Dropping the
caps to a bare over-sum passed every existing test, so this pins each cap with
a parametrized helper test plus one billing test where text over-reports past
the reasoning share and only the nested share may be netted out
The scoped xargs list missed a ruff-tests.toml rule change and skipped ruff on
deletions, so the block now runs test-linting.yml's exact command over tests/.
ruff-tests.toml, test-quality-budget.json, and scripts/check_test_quality.py
trigger the block too, and it sits after the background launches so the
dashboard and gen:api jobs overlap it.
* fix(batches): register ownership for every batch create path
Since the team isolation change, the managed files hook decided whether a
response came from a create by looking for the managed input file id on it,
which only the unified input path sets. Batches created from a model-encoded
input file id, a model param, or a raw provider id with ?provider= never got
an ownership row, so they vanished from GET /v1/batches for the key that
created them.
The create endpoint now stamps a create marker on the response before the
hooks run, and the hook keys ownership registration and the batch-created
metric on that marker instead of on the input id format.
* test(batches): assert ownership registration through the managed files hook
The endpoint tests asserted the private create marker, which is wiring, not
behaviour. They now run the create through the real managed files hook and
assert the ownership row is written for the creating key on every create
path, with the unified path driven by a genuine encoded input file id
instead of patched decoders.
* fix(mcp): let config.yaml MCP servers pin server_id
A config-defined MCP server's id is a hash of server_name|url|transport|
auth_type|alias, recomputed on every config load, so editing any of those
fields mints a new id. Every key and team granted the old id via
object_permission.mcp_servers keeps pointing at an id that no longer exists,
and the server disappears from tools/list for them with nothing logged.
load_servers_from_config now uses an explicit server_id from the server's
config entry when present and falls back to the existing hash otherwise, so
grants survive url/name/alias edits. Rejected at config load: a blank or
non-string server_id, two entries claiming the same id, a pinned id already
held by a database-backed server, and a pinned id that is another entry's
server_name or alias (expand_permission_list matches ids before names, so
that one would capture the other server's grants). Because the database
registry loads after the config on startup, a database row that lands on a
pinned config id is reported as a warning from the database reload instead,
where it is decidable; the warning is latched on the shadowed set so the
config-reload timer does not reprint it every interval.
Deployments that do not set server_id keep the exact id they have today.
* fix(mcp): close two more pinned-id capture paths
A pinned server_id equal to an alias supplied through litellm_settings
mcp_aliases was accepted, because the collision index only held the entry's
own alias field. expand_permission_list matches ids before names, so grants
written for the aliased server resolved to the pinning one. mcp_aliases keys
whose target is a config server are now reserved the same way.
A pinned server_id equal to a database-backed server's name, server_name or
alias had the same effect against the database side, and could not be
rejected at config load because the database registry is not loaded yet. The
database reload now warns about it, latched like the existing shadow warning.
* fix(mcp): reserve only the aliases the loader actually assigns
Reserving every mcp_aliases key targeting a config server was too broad in
two ways: the mapping is ignored when the entry sets its own alias, and only
the first mapping for a server is ever applied. Both cases made a pinned
server_id that could never have collided abort proxy startup. Reserve only
the name load_servers_from_config will really assign.
The database capture warning also fired for a database server whose own id is
the config server_id. There the database row wins the id outright through
get_registry precedence, so the shadow warning above it is the accurate one
and the capture message contradicted it. Skip those rows.
Also mark the two litellm-internal patches in the reload test helper, which
the test-quality gate counts; the database reload has no other seam.
* fix(mcp): match the loader's alias check exactly, is None not falsiness
load_servers_from_config consults mcp_aliases only when the entry has no
alias key at all, so an entry setting alias: "" gets no mapped alias. The
collision index used falsiness and reserved the mapped name anyway, which
failed startup on a pinned server_id that could never have collided with it.
* fix(mcp): skip one identifier, not the whole database row
A database row can shadow one config server_id by id and capture another by
name at the same time. Skipping the entire row when its id shadowed a config
entry dropped the second warning, leaving the operator with half a diagnosis.
Skip only the identifier equal to the row's own id.
* fix(mcp): reject conflicting self-pinned server ids
* fix(mcp): validate config server names before building the identifier index
The collision check reads every entry's body up front, so a malformed
entry under an invalid name surfaced as an AttributeError instead of the
name validation error the loader gave before this change.
CI's required lint job runs ruff with ruff-tests.toml over tests/ and the
test-quality budget gate, but scripts/pre_commit_lint.sh only triggered make
lint on litellm/ files, so a tests-only commit passed make check with a no-op
note and then failed CI (a duplicate test name, ruff F811, did exactly that).
When tests/ Python files are in scope and no litellm/ files are, run ruff
with ruff-tests.toml over the changed test files and make lint-test-quality,
with the matching partial-staging warning, summary line, and no-op condition.
validate_url raises HostResolutionError, a SSRFError subclass, for the two
DNS outcomes (lookup failed, no addresses). The image fetch helper maps
that to a "host could not be resolved" message and keeps the
user_url_allowed_hosts hint for the policy verdicts it can actually fix.
The image edit handler now awaits BaseImageEditConfig.async_transform_image_edit_request, and
Black Forest Labs overrides it so URL images and masks download through async_safe_get instead of
the blocking safe_get on the event loop. Rejected image fetches raise a fixed policy message with
the user_url_allowed_hosts hint rather than echoing the resolver's verdict (resolved IP, DNS
failure) back to the caller. The test fixture also fails any request-path call of the sync
convert_url_to_base64 so a regression cannot pass unnoticed.
The handshake error path already runs client-facing error strings through
_redact_string; the relay's _close_client did not, so a secret echoed in an
upstream close reason could reach the client verbatim. Mirror the handshake
path and scrub the close message and reason before relaying them.
Health checks probed every OCR deployment with a PDF, which Cohere Parse
rejects, so /health, background health checks, and the UI Test Connection
button marked Cohere Parse deployments unhealthy. BaseOCRConfig gains a
get_health_check_document hook (PDF by default) that CohereParseConfig
overrides with a 1x1 PNG data URI. cohere also gains ocr in the provider
endpoint matrix
The realtime usage writer passed the provider's output_token_details through as sent, so spend logs and callbacks kept a text_tokens that still contained reasoning_tokens while every other completion_tokens_details producer stores the partitioned share. The writer now applies the same rule the cost calculator uses, moved to litellm/types/utils.py so both read one definition, and the calculator keeps it for usage objects that arrive nested from elsewhere
* fix(responses): decode JSON-string tool schemas before sending to the provider
A caller that hands a tool schema over already JSON-encoded reached the
Responses API with a string `parameters`, and the provider rejected the
request with a 400 naming the routed model instead of the offending tool.
Decode it at the one place every Responses request converges, and refuse
anything that is neither an object nor a string encoding one.
Collapses the duplicated input/tool sanitization block shared by the
request and compact-request builders into a single owner, so the decode
cannot be wired into one path and not the other.
* test(responses): pin null tool schemas as accepted, and type the parametrized cases
The Responses API serves `parameters: null` and an omitted schema alike, so
neither may raise. Pin both against a future tightening, annotate the
parametrized inputs, and trim the docstrings back to what the code does not
already say.
The refusal predicate also required the session log to be empty, but that
log is not limited to upstream frames. With gemini_live_defer_setup the
handler stores a synthetic session.created before the relay starts, and
the transcription usage flush appends a usage event before the check
runs, so an upstream policy close with no received frames was still
logged as a $0 success. Key the check off the received-frames flag only
The relay's failure dispatch runs the async handler and then the legacy sync
failure_handler for the proxy's callable callbacks. The realtime logging object
carried no async marker, so failure_handler treated the session as a sync SDK
call and fired every CustomLogger's sync failure hook on top of the async one:
Langfuse recorded two ERROR observations per refused session, and OpenTelemetry,
MLflow, Braintrust, Literal AI, DeepEval and New Relic implement the same sync
hook. Plant the _arealtime marker in litellm_params the way aanthropic_messages
and agenerate_content already do, so both dispatchers classify the session async.
The SSRF check in async_safe_get resolved DNS on the event loop and a blocked
address was retried three times; validate_url now runs in a thread and an
SSRFError fails the fetch on the first attempt in both fetchers. The shared
HTTP handler signed the request and ran pre_call logging on the loop after an
async transform; both now run in a thread. Vertex AI Gemini still fetched
http:// images and https images without an inferrable mime type with the sync
converter inside its async body builder; the walker takes a should_inline
predicate and Vertex AI inlines exactly those URLs, leaving https images with a
known mime type and Files API refs to Google. When one download fails the
other in-flight downloads for that request are now cancelled instead of
finishing in the background
A shadow eval job could only be scoped by identity, so "this user's traffic on model X
across every key they own" was not expressible and a models field on the start body was
silently dropped. The job now carries a models list that every target is narrowed to,
matched on the requested model group with model_group_alias resolved on both sides. An
unresolvable name is a 400 at start. Empty means every model, which is what every existing
row reads as. The dashboard start form gains an "Only on models" picker and the job
headline shows the scope.
* feat(otel): stamp litellm.request.route on the LLM call span
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(otel): drop redundant comment on REQUEST_ROUTE
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(otel): Final-annotate route test locals, drop field comment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): read litellm.request.route off the server span
The LLM call span took the auth-normalized literal path from logging
metadata, which disagrees with the SERVER span wherever FastAPI matched a
template: on /engines/{model:path}/chat/completions the LLM span spelled the
model name while http.route carried the template, so the two spans grouped
into different buckets and the PR's premise did not hold.
Read the value off the span that already holds it. The request's root SERVER
span is anchored per request for parenting, and its attributes stay readable
after it ends, so request_root_http_route() answers from the async close
callback with the same http.route the SERVER span exports: the route template
on a normal route, the literal path where the passthrough hook rewrote it, and
the mount point on an MCP call. Nothing has to re-derive any of that, so the
two spans cannot drift apart.
The route the proxy recorded at auth stays as the backstop for a deployment
whose FastAPI instrumentation never mounted, where there is no server span to
disagree with. Off the proxy the attribute is omitted rather than empty.
---------
Co-authored-by: shivam <shivam@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Yucheng He <yucheng@berri.ai>
Suppression state moves out of request metadata into a request-scoped ContextVar.
refresh_proxy_server_request_body_snapshot copies metadata into
proxy_server_request.body, which deployments persist to spend logs, so the marker
naming each suppressed guardrail was readable by the caller whose request produced
it. Recovering it was enough to replay {token}:{name} for any CustomGuardrail and
switch off a PII or content-filter guardrail, since the check never verified the
named guardrail was a compression one. Nothing is read from metadata now, so there
is no marker to forge and the per-process token is no longer needed.
Routing-side compression reads the live messages instead of a pre-guardrail copy.
arm_pre_call runs before the pre-call hook, so its snapshot held the prompt as it
was before any masking guardrail rewrote it, and messages_for_routing handed that
to a compression guardrail which POSTs it to an external service. Masked content
left the proxy anyway. The cost is one combination: when the model hop compressed
and the hops differ, routing now classifies on the compressed text, since no
uncompressed copy survives that a masking guardrail has already seen.
policy_for_model no longer falls back to a marker scoped to tags the request does
not carry, which applied an 'eu' policy to a 'us' request on config order alone.
Each fix carries a regression test; all three fail when the fix is reverted.
