mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-29 06:33:28 +00:00
e5fe716d69
* fix(cost): a provider slug the price map lacks no longer bills the turn at zero
Anthropic names the dated build in `message_start`, so a streamed turn carries
`provider_response_model="claude-opus-5-20250930"` while the price map holds
`claude-opus-5`. `_select_model_name_for_cost_calc` prefers the reported name
over `response.model`, and `get_model_info` answers a model it has never heard
of with zero rates rather than raising -- so the turn is billed 0.0 with nothing
in the logs to explain it.
Same usage, only that field varying, on main:
provider_response_model absent -> 0.00079
provider_response_model=claude-opus-5 -> 0.00079
provider_response_model=claude-opus-5-20250930 -> 0.0
Missing is harmless; the damage is a name that is present and unpriced.
The reported name still wins whenever it resolves to a rate -- it is the most
specific truth about what served the turn. It is only passed over when charging
by it would produce a silent zero, and then the response's own model is used:
the name the deployment resolved to, and the one an unstreamed turn is already
priced by.
This is not specific to Anthropic or to streaming. Any provider reporting a more
specific slug than the map carries -- dated, regional, build-suffixed -- is
affected, and only streamed turns carry the field, which is why it reads as a
streaming bug from the outside.
Reported in #42161.
* address review: drop explanatory comments, stop pinning vendor slugs in tests
Three points from the review, two of which were right.
The helper docstrings explained behaviour the code already shows, which AGENTS.md
rules out. Removed.
The tests asserted that `claude-opus-5` is in the price map and
`claude-opus-5-20250930` is not. Both are facts we do not own: if Anthropic registers
the dated build, the test breaks without any litellm change. They now use a name that
is ours, and assert the invariant instead, that what the provider reports cannot change
the price of a turn the map cannot quote it by.
Picking that name needed care. An arbitrary string makes `get_model_info` raise, which
a caller upstream already handles, so the test passed with or without the fix and
defended nothing. A `claude-` prefixed name reaches the zero-rate fallback that this
bug rides on, so `claude-test-42161` reproduces it and stays ours.
The third point, that lines 801 and 828 exceed 120 characters, does not hold: they are
98 and 39.
* Pass completion_response_model as str or None to _priced_provider_response_model
basedpyright counted the bare variable as partially unknown (reportUnknownArgumentType +1 over the budget).
* test(cost): isolate runtime registration replay cases
---------
Co-authored-by: Eduardo Pessin <eduardopessin@users.noreply.github.com>
|
||
|---|---|---|
| .. | ||
| _support | ||
| audio_tests | ||
| base_sdk_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| e2e_harness | ||
| fixtures | ||
| harness_e2e | ||
| integration | ||
| rust-python-harness | ||
| test_litellm_rust | ||
| typing | ||
| unit | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _master_key.py | ||
| _openai_record_replay_proxy.py | ||
| _process_helpers.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| _wait_helpers.py | ||
| _ws_vcr.py | ||
| AGENTS.md | ||
| capturing_transport.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_logging.conf | ||
| test_rust_python_harness.py | ||
| VCR_TESTING.md | ||
| white_100x100.png | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/unit
This folder can only run mock tests.