* docs(adr): add architecture decision records, starting with provider usage extras transport
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(adrs): correct where provider usage extras survive and scope the live proof bar to runtime changes
* docs(adrs): say which chat transforms keep unknown usage keys and where a new field is passed
* docs(adrs): correct ADR 001's account of the xAI billing PR, date its code facts, and move the billing recipe to CONTRIBUTING.md
* docs(adrs): say the stream builder carries Anthropic's inference_geo by name and drops other extras
* docs(adrs): cover /v1/messages, the cost_per_token arm, and the Gemini tier side channel in the usage-field ADR and contribution guide
* docs(adrs): decide that usage extras reach callers and shared paths copy them, scope out provider-reported cost, and fix the proof steps
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(cost): a provider slug the price map lacks no longer bills the turn at zero
Anthropic names the dated build in `message_start`, so a streamed turn carries
`provider_response_model="claude-opus-5-20250930"` while the price map holds
`claude-opus-5`. `_select_model_name_for_cost_calc` prefers the reported name
over `response.model`, and `get_model_info` answers a model it has never heard
of with zero rates rather than raising -- so the turn is billed 0.0 with nothing
in the logs to explain it.
Same usage, only that field varying, on main:
provider_response_model absent -> 0.00079
provider_response_model=claude-opus-5 -> 0.00079
provider_response_model=claude-opus-5-20250930 -> 0.0
Missing is harmless; the damage is a name that is present and unpriced.
The reported name still wins whenever it resolves to a rate -- it is the most
specific truth about what served the turn. It is only passed over when charging
by it would produce a silent zero, and then the response's own model is used:
the name the deployment resolved to, and the one an unstreamed turn is already
priced by.
This is not specific to Anthropic or to streaming. Any provider reporting a more
specific slug than the map carries -- dated, regional, build-suffixed -- is
affected, and only streamed turns carry the field, which is why it reads as a
streaming bug from the outside.
Reported in #42161.
* address review: drop explanatory comments, stop pinning vendor slugs in tests
Three points from the review, two of which were right.
The helper docstrings explained behaviour the code already shows, which AGENTS.md
rules out. Removed.
The tests asserted that `claude-opus-5` is in the price map and
`claude-opus-5-20250930` is not. Both are facts we do not own: if Anthropic registers
the dated build, the test breaks without any litellm change. They now use a name that
is ours, and assert the invariant instead, that what the provider reports cannot change
the price of a turn the map cannot quote it by.
Picking that name needed care. An arbitrary string makes `get_model_info` raise, which
a caller upstream already handles, so the test passed with or without the fix and
defended nothing. A `claude-` prefixed name reaches the zero-rate fallback that this
bug rides on, so `claude-test-42161` reproduces it and stays ours.
The third point, that lines 801 and 828 exceed 120 characters, does not hold: they are
98 and 39.
* Pass completion_response_model as str or None to _priced_provider_response_model
basedpyright counted the bare variable as partially unknown (reportUnknownArgumentType +1 over the budget).
* test(cost): isolate runtime registration replay cases
---------
Co-authored-by: Eduardo Pessin <eduardopessin@users.noreply.github.com>
* refactor(anthropic): rename llms/anthropic/pass_through to llms/anthropic/messages
Move the unified /v1/messages code out of a directory called pass_through, since it is not the
passthrough feature. The inner messages sub-package becomes native, and the Vertex AI partner
model experimental_pass_through package becomes messages. The old packages stay as import shims
that warn once and alias every submodule in sys.modules, so old deep imports still resolve
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(anthropic): keep Final variables out of loops in the rename shim
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(anthropic): match the warning text in the rename shim test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(anthropic): resolve old pass_through imports lazily through a meta path finder
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(anthropic): mark the alias loader spec restore for the type discipline gate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(anthropic): format the renamed package shim
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(anthropic): keep the rebind-ok marker on the spec restore line
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(anthropic): carry the new package search path on alias specs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(anthropic): declare the shim test locals Final
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
gpt_5_transformation.py becomes gpt_reasoning_transformation.py (OpenAI and Azure),
the Bedrock anthropic_claude3_transformation.py files become anthropic_claude_transformation.py,
anthropic_claude2_transformation.py becomes anthropic_legacy_text_completions_transformation.py,
and vertex_ai_partner_models/llama3 becomes vertex_ai_partner_models/llama. Class and symbol
names are unchanged.
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: move key-gated tests/test_litellm SDK tests into tests/llm_translation and drop empty folders
* test: make token counter and health check unit tests run offline
* ci: point unit shards, rust path filter, Makefile and docs at tests/unit
* docs: fix stale test_litellm run paths in moved llm_translation tests
* fix: correct databricks e2e sys.path depth and contributing example path
---------
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
_PROXY_MaxBudgetLimiter re-checked spend:user:{id} against user_max_budget in
async_pre_call_hook without the zero-cost model exemption that
_user_max_budget_check applies in auth, so free models were rejected with
"Max budget limit reached." once a user was over budget. Auth already owns
this check, so the hook is deleted rather than taught the exemption again
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Several file references under proxy/management_helpers/ and other
paths no longer exist; the code moved to proxy/common_utils/,
proxy/db/db_transaction_queue/, litellm_enterprise/proxy/common_utils/,
proxy/hooks/litellm_skills/, and litellm_core_utils/.
