mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-29 06:33:28 +00:00
test(cost): cover unpriced provider slugs and explicit zero rates (#42467)
* fix(cost): a provider slug the price map lacks no longer bills the turn at zero
Anthropic names the dated build in `message_start`, so a streamed turn carries
`provider_response_model="claude-opus-5-20250930"` while the price map holds
`claude-opus-5`. `_select_model_name_for_cost_calc` prefers the reported name
over `response.model`, and `get_model_info` answers a model it has never heard
of with zero rates rather than raising -- so the turn is billed 0.0 with nothing
in the logs to explain it.
Same usage, only that field varying, on main:
provider_response_model absent -> 0.00079
provider_response_model=claude-opus-5 -> 0.00079
provider_response_model=claude-opus-5-20250930 -> 0.0
Missing is harmless; the damage is a name that is present and unpriced.
The reported name still wins whenever it resolves to a rate -- it is the most
specific truth about what served the turn. It is only passed over when charging
by it would produce a silent zero, and then the response's own model is used:
the name the deployment resolved to, and the one an unstreamed turn is already
priced by.
This is not specific to Anthropic or to streaming. Any provider reporting a more
specific slug than the map carries -- dated, regional, build-suffixed -- is
affected, and only streamed turns carry the field, which is why it reads as a
streaming bug from the outside.
Reported in #42161.
* address review: drop explanatory comments, stop pinning vendor slugs in tests
Three points from the review, two of which were right.
The helper docstrings explained behaviour the code already shows, which AGENTS.md
rules out. Removed.
The tests asserted that `claude-opus-5` is in the price map and
`claude-opus-5-20250930` is not. Both are facts we do not own: if Anthropic registers
the dated build, the test breaks without any litellm change. They now use a name that
is ours, and assert the invariant instead, that what the provider reports cannot change
the price of a turn the map cannot quote it by.
Picking that name needed care. An arbitrary string makes `get_model_info` raise, which
a caller upstream already handles, so the test passed with or without the fix and
defended nothing. A `claude-` prefixed name reaches the zero-rate fallback that this
bug rides on, so `claude-test-42161` reproduces it and stays ours.
The third point, that lines 801 and 828 exceed 120 characters, does not hold: they are
98 and 39.
* Pass completion_response_model as str or None to _priced_provider_response_model
basedpyright counted the bare variable as partially unknown (reportUnknownArgumentType +1 over the budget).
* test(cost): isolate runtime registration replay cases
---------
Co-authored-by: Eduardo Pessin <eduardopessin@users.noreply.github.com>
This commit is contained in:
parent
7a23e497bd
commit
e5fe716d69
3 changed files with 71 additions and 0 deletions
|
|
@ -239,6 +239,8 @@ graph LR
|
|||
7. `DBSpendUpdateWriter.update_database()` queues spend increments to Redis
|
||||
8. Background job `update_spend` flushes queued spend to PostgreSQL every 60s
|
||||
|
||||
Cost lookup falls back to the response model when the provider-reported name cannot be priced. Explicitly registered zero rates remain zero, and custom deployment pricing or an explicit `base_model` retain precedence
|
||||
|
||||
### Data Access Layer (Models & Repositories)
|
||||
|
||||
Database entities and the operations on them live in two packages at the root of `litellm/` so both the gateway (`proxy/`) and the SDK can use them without importing proxy internals:
|
||||
|
|
|
|||
|
|
@ -94,6 +94,8 @@ Add your tests to the [`tests/unit/` directory](https://github.com/BerriAI/litel
|
|||
- **Only add mocked tests** - no real LLM API calls in this directory
|
||||
- For integration tests with real APIs, use the appropriate test directories
|
||||
|
||||
Tests that call `litellm.register_model` or change `litellm.model_cost` should use the `local_model_cost_map` fixture from `tests/unit/conftest.py`. It provides a fresh catalog and restores the previous map, so pricing and capability overrides cannot leak into later tests
|
||||
|
||||
### File Naming Convention
|
||||
|
||||
The `tests/unit/` directory follows the same structure as `litellm/`:
|
||||
|
|
|
|||
|
|
@ -6810,6 +6810,73 @@ def test_completion_cost_bills_base_when_gemini_serves_on_demand(
|
|||
assert cost == pytest.approx(100 * 0.001 + 50 * 0.002)
|
||||
|
||||
|
||||
def test_an_unpriced_provider_slug_does_not_bill_the_turn_at_zero(
|
||||
_local_model_cost_map: None, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
response_model: Final = "anthropic/claude-test-42161-response"
|
||||
reported_model: Final = "claude-test-42161-unmapped"
|
||||
monkeypatch.delitem(litellm.model_cost, reported_model, raising=False)
|
||||
monkeypatch.delitem(litellm.model_cost, f"anthropic/{reported_model}", raising=False)
|
||||
monkeypatch.setitem(
|
||||
litellm.model_cost,
|
||||
response_model,
|
||||
{
|
||||
"input_cost_per_token": 0.001,
|
||||
"output_cost_per_token": 0.002,
|
||||
"litellm_provider": "anthropic",
|
||||
"mode": "chat",
|
||||
},
|
||||
)
|
||||
response: Final = ModelResponse(
|
||||
model=response_model,
|
||||
choices=[],
|
||||
usage=Usage(prompt_tokens=38, completion_tokens=24, total_tokens=62),
|
||||
)
|
||||
response._hidden_params["provider_response_model"] = reported_model
|
||||
|
||||
cost: Final = completion_cost(completion_response=response, custom_llm_provider="anthropic")
|
||||
|
||||
assert cost == pytest.approx(38 * 0.001 + 24 * 0.002)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("reported_rate", (0.0, 0.01))
|
||||
def test_a_priced_provider_slug_still_decides_the_rate(
|
||||
_local_model_cost_map: None, monkeypatch: pytest.MonkeyPatch, reported_rate: float
|
||||
) -> None:
|
||||
response_model: Final = "anthropic/claude-test-42161-response"
|
||||
reported_model: Final = "claude-test-42161-reported"
|
||||
monkeypatch.setitem(
|
||||
litellm.model_cost,
|
||||
response_model,
|
||||
{
|
||||
"input_cost_per_token": 0.001,
|
||||
"output_cost_per_token": 0.002,
|
||||
"litellm_provider": "anthropic",
|
||||
"mode": "chat",
|
||||
},
|
||||
)
|
||||
monkeypatch.setitem(
|
||||
litellm.model_cost,
|
||||
f"anthropic/{reported_model}",
|
||||
{
|
||||
"input_cost_per_token": reported_rate,
|
||||
"output_cost_per_token": reported_rate * 2,
|
||||
"litellm_provider": "anthropic",
|
||||
"mode": "chat",
|
||||
},
|
||||
)
|
||||
response: Final = ModelResponse(
|
||||
model=response_model,
|
||||
choices=[],
|
||||
usage=Usage(prompt_tokens=38, completion_tokens=24, total_tokens=62),
|
||||
)
|
||||
response._hidden_params["provider_response_model"] = reported_model
|
||||
|
||||
cost: Final = completion_cost(completion_response=response, custom_llm_provider="anthropic")
|
||||
|
||||
assert cost == pytest.approx(38 * reported_rate + 24 * reported_rate * 2)
|
||||
|
||||
|
||||
def test_custom_pricing_as_completion_cost_param():
|
||||
from litellm import Choices, Message, ModelResponse
|
||||
from litellm.utils import Usage
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue