* fix(vertex_ai): stop advertising OpenAI platform-only params on Gemma and Llama routes
The Anthropic /v1/messages bridge derives prompt_cache_key from Claude Code's
session id whenever the provider config advertises it, and every Vertex
OpenAI-compatible route (gemma/, openai/<endpoint>, meta/) inherited the full
OpenAI list, so the Model Garden vLLM container rejected each turn with a
pydantic extra_forbidden 400. Vertex's Llama and Gemma configs now filter one
shared list of platform-only params (prompt_cache_key, prompt_cache_retention,
safety_identifier, service_tier, store, web_search_options, modalities,
prediction, audio, max_retries) out of their supported params, so the bridge
no longer derives the key and drop_params drops an explicit one.
* fix(vertex_ai): scope the platform-param filter to self-deployed Model Garden endpoints
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(fal_ai): price nano-banana-2 and nano-banana-pro image generations by resolution
* fix(fal_ai): bill passthrough submits per requested image and register the resolution price keys
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(vertex_ai): surface the Gemma container's own error inside a 200 :predict response
* refactor(vertex_ai): move the gemma container error parser next to its adapter
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(proxy): let team admins update member key budgets when enabled
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): drop casts flagged by LIT006 in member key budgets change
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): send budget-only key updates when a team admin edits a member key
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): block spend echo in team admin member key updates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): only send dirty budget fields in team admin member key updates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): replace class method monkeypatch with module symbol patch
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(lint): LIT013 caps comprehensions at one for and one if clause
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(lint): rewrap the type discipline gate rule list
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(lint): honor comprehension-ok on any line a comprehension spans
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(lint): scope comprehension-ok to the innermost comprehension spanning it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(lint): break equal-span suppression ties toward the inner comprehension
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(lint): let single-line and only violating comprehensions own comprehension-ok markers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(lint): type tmp_path in LIT014 tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(vertex_ai): return chunk content, extractive text, and structData from search_api vector store hits
* fix(vertex_ai): report a chunk hit's relevanceScore as the search result score
* test(vertex_ai): type the search response helper and parametrized case
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(e2e): pin end-user and tag attribution from Codex-style headers on /v1/responses
Codex CLI has no body field for the end user, so its config.toml
http_headers attach x-litellm-customer-id or x-litellm-end-user-id plus
x-litellm-tags to every /v1/responses call. The proxy already honors
those headers on the Responses route, but nothing in the e2e stack
pinned it. The new case sends that exact wire shape with each standard
customer header and fails unless the spend row carries the end user,
the tags, the aresponses call type, and a nonzero cost.
* test(e2e): assert the customer's /customer/info total matches the header-attributed Responses row
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(spend): attribute CLI session spend to the per-user cli-session alias instead of the hashed session token
A CLI session token is a fresh random secret on every login, so since v1.99 each
login's spend rows carried a different sha256 hash as api_key and the usage APIs
could resolve neither key_alias nor user_email for them. Spend rows and logging
callbacks now attribute a session request to its stable alias,
cli-session-<user_id>, and the usage endpoints derive that alias and owner from
the key itself instead of scanning for a matching digest
* fix(spend): resolve the CLI session team from the user's first team in usage metadata
A cli-session key carries no team of its own in the DB, so the usage
breakdown showed team_id None for it and the export grouped it as
Unassigned. The login attaches the user's first team to the session, so
the recovery mirrors that rule for cli-session keys only.
* fix(spend): claim the session team only for a single-team user
The CLI login attaches a team on its own only when the user has exactly
one; a user in several teams picks one per login, so usage metadata for
the alias would otherwise name a team the login may not have used.
* test(pass_through): mark the mocked auth object as a plain key
The logged key follows the alias only for a session token; a bare
MagicMock reads as one, so the test names the field it relies on.
* fix(spend): attribute CLI session pass-through, queue, and managed batch spend to the cli-session alias
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend): only treat the exact cli-session-<created_by> value as a batch key alias
A managed object row written by an older build can still carry the raw per-login
session token, which shares the cli-session- prefix. Matching on the prefix alone
would have surfaced that token as a trusted alias and persisted it verbatim in the
batch cost spend log, so the alias check now requires the exact per-user value and
every other prefixed value keeps going through redaction
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend): log proxy executed batch rows under the cli-session alias instead of the session token
_row_metadata set user_api_key from the raw bearer token while user_api_key_hash carried the alias, so the spend log redaction rejected the alias as untrusted and hashed the random session token instead
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend): attribute semantic search embedding spend to the cli-session alias
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend): scope /key/spend/report for a CLI session to the cli-session alias
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend): use the cli-session alias for websearch spend, prometheus failure labels and the parallel limiter
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(spend): drop explanatory docstrings on get_logged_api_key and attach_user_details
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend): only recover cli-session usage keys whose suffix is a known user
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(ocr): remove the Python OCR execution path and require the Rust route
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fmt
* refactor(ocr): tidy the native OCR passthrough binding
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(ocr): ruff format the azure passthrough transformation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(ocr): resolve passthrough OCR costing in one Rust call
Replace passthrough_url/passthrough_transform with passthrough_response,
which matches the relayed endpoint against each Azure config's path
segments instead of building a fake request to call get_complete_url.
The binding drops the unused headers, status and api_base arguments.
Catch the ValueError/RuntimeError the binding raises so a relayed body
that is not OCR-shaped falls back to the passthrough object instead of
failing logging, and cover the relay against the real binding.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(ocr): drop the unused LlmProviders import from health check helpers
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* ci: drop the ocr_testing job now that tests/ocr_tests is gone
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(ocr): restore the live OCR matrix and the ocr_testing job
The public litellm.ocr / aocr / Router interface is unchanged by the Rust
migration, so the live provider matrix still applies. Drops the stale VCR skip
list for the deleted test_rust_bridge.py.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(ocr): import Final in the health check helper tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* done
* fix(rust_bridge): run Python secret reads under the caller's contextvars
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(rust_bridge): run every blocking Python call under the caller's contextvars
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): enforce tpm_limit and rpm_limit set on tag objects
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): keep tag rate limit helpers within type discipline budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): drop descriptive docstrings from tag rate limit helpers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover tag object rpm and tpm limits
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): type the fake tag batch helper parameters
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): name the over-limit tag in 429 errors
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover tag rpm limit shared across teams, orgs and users
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(proxy): format the tag descriptor match in the v3 limiter
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): drop Final annotation inside loop for pyright
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: kerry <kerry@berri.ai>
* fix(bedrock): map Anthropic batch row params the way real time does
* fix(bedrock): let a batch row's allowed_openai_params reach the mapper
* test(bedrock): assert the batch thinking value matches the real-time mapping
* fix(bedrock): keep json_mode out of Anthropic batch rows and pin route-prefixed deployments
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(router): match provider-prefixed fallback keys for bare model groups
* fix(router): infer the fallback key's provider the way routing does for bare model groups
A bare model group served by a wildcard deployment (claude-sonnet-4-6 routed to anthropic/*) now finds a fallback keyed <provider>/<group>. The provider is inferred through one shared helper, inferred_provider, which the pattern router already used inline, so the fallback lookup and routing agree on the prefix. The lookup only infers a provider when some fallback key ends in /<group>, so alias-style groups never hit the resolver
* fix(router): resolve context window and content policy fallback keys through the shared lookup
---------
Co-authored-by: Jason Dougherty <jasondoc3@gmail.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(integration): reproduce encrypted_content_affinity 503 when origin has no boundary peer
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): keep encrypted content affinity turn one out of the response cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): degrade encrypted_content_affinity when the origin has no encryption-boundary peer
A routed-group candidate that is currently unavailable and shares its
(api_base, api_key) with no healthy deployment used to raise a proxy-level
503/429 from _unavailable_origin_error, even though healthy siblings in the
same model group could still serve the turn. Strip the encrypted reasoning
and dispatch to the healthy pool instead, matching the existing cross-group
behavior, and log a warning naming the origin model_id and routed group
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): bound the degraded-affinity log marker and assert the strip on the wire
Address review findings: restore num_retries alongside
optional_pre_call_checks in the integration test teardown, record scenario
request bodies on the scripted upstream so the tests can assert no encrypted
reasoning reaches the sibling, and truncate the client-supplied model_id in
the degraded-dispatch warning
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(tests): only record JSON bodies on the scripted upstream
Multipart uploads to scripted POST routes have no JSON body, so gate the
observation recording on the request content-type
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): make encrypted_content_affinity runtime-toggleable so /config/update can turn it off
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* ci: run the claude_code harness unit-test trees in the lint job
* ci: gate the harness step on Python files plus its installer and workflow
* ci: fire the harness step on dependency manifest changes too
* test: cover the harness gate's installer, manifest, and workflow triggers
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(spend): capture-rate check of LiteLLM spend against the OpenAI bill
* fix(spend): claim the alert lock after the check, NaN gauge on no rate, 180-day range cap, live settings, OpenAI adapter under llms
* fix(spend): chart the capture-rate gauge in the all-metrics dashboard and clear it when the check is removed
* fix(prometheus): record the capture-rate gauge when api_provider is an excluded label
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(mcp): cap an agent key's tools at what the invoking user and team may call
The invoking user's and team's x-litellm-user-id / x-litellm-team-id, echoed back by the
agent, already narrowed which MCP servers the agent key could reach, but not which tools on
those servers. An agent granted every tool on a server kept them all when acting for a user
who may only call a subset. The caller's team and user tool grants now intersect the agent's
tool list on each server, mirroring the servers axis, so the headers only ever narrow.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(mcp): pick the caller principal explicitly instead of getattr in the tool grant stub
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): return immutable tool sequences from the agent caller tool ceiling
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(e2e): carve harness tests out of the no-unit-tests hard rule
The Hard Rules bullet in tests/e2e/AGENTS.md banned unit tests of any
kind, the harness's own included, while the same file's claude_code/
and load/ entries, CONTRIBUTING.md, and the harness conftest all
describe markerless harness tests that run without a proxy. Reword the
rule to keep the product-feature and no-mocks bans, name the harness
trees as the one exception with the standard they are judged by, and
put the harness sentence back in the marker paragraph so the two docs
agree
* docs(e2e): name env vars set through monkeypatch as inputs, not patches
The rule banned monkeypatching anywhere under tests/e2e while its harness
carve-out named root-level test_*.py files that set env vars through
pytest's monkeypatch fixture. Say the ban is about patching code and that
an env var set that way is an input, so the examples and the rule agree
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(vertex_ai): translate /v1/responses batch rows through the Responses-to-Chat bridge
Vertex batch uploads treated every non-embeddings JSONL row as a chat
completions body, so a /v1/responses row lost its input and reached GCS
as a blank text part. Route detection now recognizes /v1/responses rows
and bridges them to chat through the same Responses-to-Chat bridge the
real-time path uses. That bridge call moves out of the Bedrock files
transformation into a shared helper both providers call, forwarding the
record's fields as sent, like real time, instead of validating them
against the SDK TypedDicts whose required keys clients omit.
* chore(batches): type the Vertex responses test helper and drop the quoted input cast
* fix(batches): translate developer messages to system on Vertex and Bedrock batch rows like real time
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* ci: fix the litellm-tests unit job with sysmon coverage, an env allowlist and coverage upload on failure
* test: replace key-dependent proxy, enterprise and mcp unit tests with synthetic values and integration and e2e coverage
* test: drop key reads at the legacy proxy, enterprise and mcp paths and wire the gemini pass-through split
* ci: move caching, proxy-extras, gateway and enterprise tests into tests/unit and run them from litellm-tests under their legacy flags
* ci: move caching, proxy-extras, gateway and enterprise tests into tests/unit and run them from litellm-tests under their legacy flags
* ci: move tests/proxy_unit_tests to tests/unit/proxy and run the proxy-db shards from litellm-tests
* ci: move provider-independent MCP tests into tests/unit and run mcp-integration from litellm-tests
* ci: fail the unit shard when circleci tests split errors
* test: drop restating comments from the gemini pass-through split
* build: point the local proxy unit targets at the nested tests/unit/proxy tree
* ci: exit the unit shard cleanly when circleci tests split assigns it no files
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* ci: fix the litellm-tests unit job with sysmon coverage, an env allowlist and coverage upload on failure
* test: replace key-dependent proxy, enterprise and mcp unit tests with synthetic values and integration and e2e coverage
* test: drop key reads at the legacy proxy, enterprise and mcp paths and wire the gemini pass-through split
* ci: move caching, proxy-extras, gateway and enterprise tests into tests/unit and run them from litellm-tests under their legacy flags
* ci: move caching, proxy-extras, gateway and enterprise tests into tests/unit and run them from litellm-tests under their legacy flags
* ci: move tests/proxy_unit_tests to tests/unit/proxy and run the proxy-db shards from litellm-tests
* ci: fail the unit shard when circleci tests split errors
* test: drop restating comments from the gemini pass-through split
* build: point the local proxy unit targets at the nested tests/unit/proxy tree
* ci: exit the unit shard cleanly when circleci tests split assigns it no files
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* feat(proxy): let callbacks filter the model listing routes per caller
* fix(proxy): offer every listed name to the listing callback, agent groups and deployment lookups included
* fix(proxy): hide aliases of a team model by its public name and offer /model/info lookups the listed name
* fix(proxy): map a malformed model listing filter return to the proxy error contract and document legacy team names
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* ci: fix the litellm-tests unit job with sysmon coverage, an env allowlist and coverage upload on failure
* test: replace key-dependent proxy, enterprise and mcp unit tests with synthetic values and integration and e2e coverage
* test: drop key reads at the legacy proxy, enterprise and mcp paths and wire the gemini pass-through split
* ci: move caching, proxy-extras, gateway and enterprise tests into tests/unit and run them from litellm-tests under their legacy flags
* ci: move caching, proxy-extras, gateway and enterprise tests into tests/unit and run them from litellm-tests under their legacy flags
* ci: fail the unit shard when circleci tests split errors
* test: drop restating comments from the gemini pass-through split
* ci: exit the unit shard cleanly when circleci tests split assigns it no files
---------
Co-authored-by: yuneng <yuneng@berri.ai>
npm keeps an unpublished version's timestamp in the packument's time map but drops it from versions, so the resolver could hand npm install a version it refuses with ETARGET. Only versions still present in versions are candidates now.
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(proxy_cli): add --validate_config dry-run flag
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(tests): format --validate_config CliRunner calls
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy_cli): run --validate_config before the ollama auto-start
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy_cli): restore file and add ollama validate_config regression test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
* ci: fix the litellm-tests unit job with sysmon coverage, an env allowlist and coverage upload on failure
* test: replace key-dependent proxy, enterprise and mcp unit tests with synthetic values and integration and e2e coverage
* test: drop key reads at the legacy proxy, enterprise and mcp paths and wire the gemini pass-through split
* ci: fail the unit shard when circleci tests split errors
* test: drop restating comments from the gemini pass-through split
* ci: exit the unit shard cleanly when circleci tests split assigns it no files
---------
Co-authored-by: yuneng <yuneng@berri.ai>
#42443 pins a floating LiteAdmin button to the bottom-right corner, where it covers the logs page's next-page control. Flip the per-user Hide LiteAdmin switch during global setup so every spec reusing the admin storage state loads with the button hidden.
* feat(compat-matrix): resolve and install the Claude Code CLI per run
* chore(compat-matrix): drop the installer header comment
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(vertex_ai): keep batch output_file_id null until Vertex reports outputInfo
Vertex only sets outputInfo.gcsOutputDirectory once a batch job has written
output. Falling back to outputConfig's outputUriPrefix named the per-model
directory shared by every batch of the deployment, an object that never
exists, so the proxy minted a managed file for it under the first key and
every other key's file calls on that id were 403s
* fix(vertex_ai): treat a null gcsOutputDirectory as no output file yet
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Prisma types a raw array parameter from the first batch a connection sees. After a flush in
which every member cost was a whole number (a free model), the connection's cached statement
expected int8[] and every later fractional batch on it failed with "improper binary format in
array element", so member spend silently stopped landing while team spend kept rising.
The rows now travel as one JSON document unpacked by jsonb_to_recordset with the column types
declared in SQL, so Postgres types the numbers and the batch shape no longer matters.
* feat(proxy): add uncapped server-side team usage export route
GET /team/daily/activity/export answers the same scoping as
/team/daily/activity/aggregated with one unbounded rollup query, so keys
past USAGE_TOP_API_KEYS_LIMIT are included. Supports daily,
daily_with_keys, daily_with_users and daily_with_models export types as
CSV (default) or JSON. The PTU flat-cost sentinel stays in the plain
daily rollup and is excluded from the keyed and per-model exports
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): export team usage server-side when the key list was truncated
When the aggregated spend response reports api_key truncation, EntityUsage
passes a serverExport into the export modal that downloads CSV or JSON
from GET /team/daily/activity/export instead of building the file from
the truncated on-screen data. apiClient gains a responseType option so
the download can arrive as a Blob, and truncation no longer blocks the
export button
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): cover team usage export types, sentinel handling and scope
Unit tests pin the uncapped key rollup past USAGE_TOP_API_KEYS_LIMIT,
PTU sentinel inclusion in the daily rollup and exclusion elsewhere, the
per-user fold, and the CSV column layout. Integration tests exercise the
route against a live proxy, including member scope denial
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): tidy team usage export route
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): use membership test for export type branch (PLR1714)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(ui): format exportBlockedReason test with prettier
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): satisfy type-discipline gate in team usage export
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): pass export rows as a sequence to the response model
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy-behavior): cover team usage export in the daily activity scope matrix
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): carry PTU flat cost and escape formulas in team usage export
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): keep the truncation export block on surfaces without a server export
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): drop redundant comments in team export call and modal test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): audit cells for team usage export
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): tighten team usage export audit cells
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): type the export params tuple and fold user keys in one pass
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(ui): bring entity usage export helpers under eslint budgets
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(ui): prettier-format UsagePageView after merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): route unmapped openai family model ids to converse
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(bedrock): rename the e2e openai family backend constant
global.openai.gpt-6-sol has a cost-map row now, so the constant no longer
names an unmapped model
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(logging): pass provider response headers to callbacks on every endpoint
Custom callbacks only received kwargs["response_headers"] for chat
completions. Responses, image generation and edit, speech, and
transcription calls either never recorded the provider's headers or
recorded them in one place and not the other.
Every handler now records the provider's httpx headers on the response's
hidden params as "headers" (raw) and "additional_headers" (processed,
with LiteLLM's own entries winning on a clash), and the logging object
derives model_call_details["response_headers"] from those hidden params
before cost calculation on the non-stream and both streaming success
paths, keeping a handler-set value authoritative. Binary speech responses
expose their hidden params to the standard logging payload, and the sync
OpenAI transcription request always fetches the raw response.
* test(images): point the legacy image and speech fakes at the raw response surface
Image generation now goes through the SDK's raw response so the provider headers can be read, and the speech binary response now carries hidden params. The unit fakes in the image generation, xinference, proxy provider, image edit, Vertex speech, and otel suites still pinned the old call surface and the old "no hidden params" assertion, so they read an uncalled mock or a fake response without headers.
* test(images): drop the rewritten mock comments and the generated edit PNGs
* test(images): move the llm-span test's image fake to the raw response surface
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(vertex): native batch JSONL passthrough with cost tracking
Add a per-request `passthrough=true` multipart field on `POST /v1/files`
(and the same kwarg on `litellm.create_file`) that uploads a native
Vertex AI batch JSONL to the deployment's GCS bucket unchanged, so rows
using `googleSearch` and other Gemini-only features run as written and
the output, `groundingMetadata` included, comes back untouched.
Passthrough is sticky through the GCS object path
(`litellm-vertex-files/passthrough/...`), so batch create and output
retrieval inherit it without new state. Native output rows are costed
from their `usageMetadata` with the deployment's model and model_info,
in the polling and retrieve paths and for the existing global
`disable_vertex_batch_output_transformation` flag, which billed $0
before.
The proxy requires the target to resolve to vertex_ai deployments only,
refuses `passthrough` with a non-batch purpose, a non-default
`target_storage`, or pre-call guardrails, and validates native rows on
`request` instead of the OpenAI batch keys.
* refactor(vertex): keep native batch row pricing inside the Vertex adapter
Moves native Vertex batch row detection, response parsing, and per-row
pricing from litellm/batches/batch_utils.py into
litellm/llms/vertex_ai/batches/transformation.py, so batch_utils only
aggregates the rows it gets back. Adds tests/test_litellm/files to the
misc unit shard so the new test directory is claimed by a shard.
* fix(files): say what a passthrough batch upload takes when a row is not native
The missing-key 400 listed bare key names, so an OpenAI-shaped row under
passthrough=true read "Each line must be a JSON object with keys request".
The batch line shape now carries its own hint, and the passthrough one says
a passthrough upload takes native Vertex batch rows with a request key
* fix(batches): bill native Vertex embedding batch rows on the native cost path
A native Vertex output row whose response holds an embedding was validated as a
generateContent response, so the documented tokenCount-only shape counted as a failed
row. Price embedding rows from their own usage (promptTokenCount, else tokenCount) with
the helper the transformed embeddings path already used, and drop the prompt-details
helper nothing calls anymore.
* fix(batches): keep modality batch rates on native Vertex embedding rows
An embedding row that carries usageMetadata was billed from promptTokenCount alone, so
its promptTokensDetails no longer reached the audio, image, and video batch rates the
way it did before the native cost path. Run every row with usageMetadata through the
Gemini usage parser and keep the flat tokenCount fallback for embedding rows without it.
* fix(batches): price native Vertex batch rows by modelVersion under a wildcard deployment
A `vertex_ai/*` deployment hands the batch cost path `*` as the deployment model, which
no cost map resolves, so every native (passthrough or flag-on) row was billed at $0. A
wildcard deployment model now defers to the row's own `modelVersion`, the way the
transformed path already prices by the row's `model`.
Also moves the native passthrough tests under tests/test_litellm, the tree codecov
reads, and covers the raw upload chunking, the embedding output translation, the
unpriceable-row path, and the flag-on dispatch.
* fix(batches): keep explicit deployment prices for native Vertex rows without a modelVersion
Under a wildcard deployment a native batch row that carries no modelVersion (an embedding
row, or a generateContent row Vertex returned without one) was billed at $0 even when the
deployment's model_info sets explicit batch prices, because the cost calculator was never
called. The row now falls back to the wildcard name, which the cost calculator prices from
the explicit model_info, and only a row with neither a modelVersion nor a deployment model
is billed at $0 with the warning
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(prometheus): add model_group label to deployment request and rate limit metrics
litellm_deployment_total_requests, litellm_deployment_success_responses,
litellm_deployment_failure_responses, litellm_deployment_tpm_limit and
litellm_deployment_rpm_limit had no way to identify which model_group a
pooled deployment belongs to, only requested_model, litellm_model_name and
model_id, none of which name the alias a model_name resolves through when
it fans out to more than one deployment.
model_group was already resolved onto enum_values for every request in
async_log_success_event, so this is a label-list addition for the metrics
built directly from that enum_values (the two request counters). The
failure counter builds its own UserAPIKeyLabelValues locally and had a
model_group variable already in scope that it never passed through, and
the tpm/rpm limit gauges are set from a helper that took no model_group
parameter at all even though its only caller already had it on
enum_values. Both now thread the value through.
* test(prometheus): expect model_group in deployment success/total request labels
test_set_llm_deployment_success_metrics asserts the exact label set passed
to litellm_deployment_success_responses.labels() and
litellm_deployment_total_requests.labels(), which now includes model_group
since it was added to those metrics' label list.
* fix(prometheus): bound model_group on deployment failure metrics
On a pre-routing reject (no deployment selected), model_group is
caller-supplied via litellm_params.metadata and was passed through
unbounded, letting an unrecognized value mint unlimited label series on
litellm_deployment_failure_responses / litellm_deployment_total_requests.
Bound it with the same _bounded_requested_model_label used for
requested_model on this path. When a deployment is actually selected,
model_group is router-resolved and passed through as-is.
Also documents the model_group parameter on
_set_deployment_tpm_rpm_limit_metrics and the bounding behavior on
set_llm_deployment_failure_metrics.
---------
Co-authored-by: ahamedshaik16 <24526479+ahamedshaik16@users.noreply.github.com>
* test(rust): encode anthropic response serialization shape as rstest cases
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(rust): shape Anthropic Messages requests natively
The Rust Messages route only relayed the body. It now runs the request shaping
the Python handler does for the direct Anthropic provider: history sanitizers
(empty blocks, tool ids, replayed web search results, provider_specific_fields,
encrypted reasoning, advisor blocks), reasoning_effort and adaptive/legacy
thinking translation against the model's capability flags, the sampling and
speed gates under drop_params, the metadata allowlist, additional_drop_params,
reasoning auto summary, OAuth and ANTHROPIC_AUTH_TOKEN credentials,
provider_specific_header merging and anthropic-beta injection. Capability flags
and LiteLLM settings reach Rust through route_host.shaping(). A request the
route rejects before the call now maps to BadRequestError instead of
APIConnectionError
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(rust): port Anthropic Messages shaping tests and pin comment contracts as cases
Every Python unit test that exercises the ported shaping for the direct
Anthropic provider now has a named rstest counterpart, and every comment that
stated a behavior contract is deleted in favor of a case that pins it. Measured
with cargo-mutants over the touched files, all viable mutants are caught
Porting the tests surfaced parity gaps, fixed here to match Python: every
casing of a forwarded anthropic-beta header is merged, replayed web search
results are rewritten from their own block (an empty result keeps its slot and
a server_tool_use with a non-string query stays), an empty output_config.effort
falls back to medium, speed and reasoning effort errors quote values the way
Python does, additional_drop_params apply after metadata validation and the
auto summary and never touch model or messages, and a non-string
metadata.user_id is rejected before the call
* fix(rust): resolve Messages credentials through the secret source and scope headers by resolved provider
The native Messages route read ANTHROPIC_API_KEY, ANTHROPIC_AUTH_TOKEN and the
base URL straight from the process environment, so a key or base held in a
configured secret manager was never found. Each provider config now declares
its secret names and the route resolves them through the same SecretSource the
OCR route uses, with the Python bridge passing in litellm's configured manager
provider_specific_header entries were scoped by the explicit
custom_llm_provider only, falling back to anthropic, so an azure_ai/ model
lost its azure_ai scoped headers. Scoping now happens in the route after the
provider is resolved from the model, as Python's handler does
The Azure config now adds the same anthropic-beta feature headers Python's
Azure route adds, and the metadata allowlist, reasoning auto summary and
history sanitizers move from the core route into the llms crate, mirroring
their home in Python's messages handler
* test(rust): escape the dot in the metadata.user_id match pattern
* refactor(rust-bridge): project Messages capabilities without mutable dicts
The capability flags and effort tiers were built as dict comprehensions,
which the type-discipline gate counts as mutable construction, and the
asdict call carried a mutable-ok suppression that suppressed nothing. The
flags are now passed one by one and the effort tiers are a frozen dataclass,
which asdict projects to the same map the native side reads
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* test(guardrails): run the cache-hit redis outage test on the shared owned_redis helper
test_redis_outage_keeps_serving_in_memory_hits (#42780) spawned `redis-server`
straight from PATH. The CircleCI integration machine has no redis-server
binary, so the test died with FileNotFoundError before reaching the proxy.
Use integration._support.redis_process.owned_redis, which runs the local
binary when there is one and otherwise the job's redis-cache container, and
take the outage with its stop()/start() pair the way test_redis_recovery
already does. The assertions are unchanged.
* test(guardrails): describe the owned_redis stop as an outage, not a kill
#42791 switched the MCP management endpoints from create_mcp_server to
create_mcp_server_if_identifier_free, which keeps the same arguments and
returns the created row on success. Two tests in
tests/store_model_in_db_tests/test_mcp_servers.py still patched the old name,
so mock.patch raised AttributeError before the tests ran and
proxy_store_model_in_db_tests has been red on main since.
test_malformed_bodies_missing_users_and_foreign_servers_are_rejected used
`body` as a comprehension variable and then called the module-level `body()`
helper a few lines later. CPython 3.12.2, which the CircleCI integration job
runs, compiles that later call as a local read, so the test raised
UnboundLocalError on every integration-mcp run since #42652. Newer 3.12
patch releases and 3.13 compile it as a global read, which is why it passes
locally.
Renaming the comprehension variable makes both reads unambiguous.
The new CircleCI tests pipeline (#42773) runs tests/unit under pytest-cov on
CPython 3.12.2, where coverage traces every line through sys.settrace. The two
tracemalloc peak comparisons in test_vertex_ai_files_streaming.py drive 8000-row
payloads through both pipelines and slow from ~10s to over 3 minutes under that
tracer, so both hit the 90s pytest-timeout on every run.
Mark them no_cover so pytest-cov pauses tracing for just these two. Their
assertions are unchanged and every other test in the file still reports coverage.
#42463 made Logout revoke the dashboard session key on the server. Both
logout specs ran on the shared ADMIN_STORAGE_PATH session that globalSetup
mints once, so clicking Logout revoked the key every later admin spec reuses.
The CircleCI run is serial, and from the auth/ folder on, every admin-session
spec failed with "Invalid proxy server token passed" (80 failures, up from 7)
while the internal-user, internal-viewer and team-admin specs kept passing.
Each logout spec now starts from an empty storage state and logs in through
the login page, so the session it revokes is its own. The login steps live in
a shared logInThroughLoginPage helper next to the other onboarding helpers.
* fix(ui): pass is_proxy_admin for proxy admins on the models page team drill-in
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): drop explanatory comments from the models page team drill-in tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): exclude view-only admins from is_proxy_admin on the models page team drill-in
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): add browser integration contract for the team guardrail kill switch on the models page
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
test_ollama_image returned the request's images list as the mocked `response`
field and read it back from message content. Since #42838 the completion
transform validates `response` as a string, so the list is dropped and the test
fails with "string index out of range". Ollama always sends a string there.
The mock now returns a real string reply and the test asserts on the images the
transform actually sent, which is what it was checking all along.
