test(cost): cover unpriced provider slugs and explicit zero rates (#42467)

* fix(cost): a provider slug the price map lacks no longer bills the turn at zero

Anthropic names the dated build in `message_start`, so a streamed turn carries
`provider_response_model="claude-opus-5-20250930"` while the price map holds
`claude-opus-5`. `_select_model_name_for_cost_calc` prefers the reported name
over `response.model`, and `get_model_info` answers a model it has never heard
of with zero rates rather than raising -- so the turn is billed 0.0 with nothing
in the logs to explain it.

Same usage, only that field varying, on main:

    provider_response_model absent                  ->  0.00079
    provider_response_model=claude-opus-5           ->  0.00079
    provider_response_model=claude-opus-5-20250930  ->  0.0

Missing is harmless; the damage is a name that is present and unpriced.

The reported name still wins whenever it resolves to a rate -- it is the most
specific truth about what served the turn. It is only passed over when charging
by it would produce a silent zero, and then the response's own model is used:
the name the deployment resolved to, and the one an unstreamed turn is already
priced by.

This is not specific to Anthropic or to streaming. Any provider reporting a more
specific slug than the map carries -- dated, regional, build-suffixed -- is
affected, and only streamed turns carry the field, which is why it reads as a
streaming bug from the outside.

Reported in #42161.

* address review: drop explanatory comments, stop pinning vendor slugs in tests

Three points from the review, two of which were right.

The helper docstrings explained behaviour the code already shows, which AGENTS.md
rules out. Removed.

The tests asserted that `claude-opus-5` is in the price map and
`claude-opus-5-20250930` is not. Both are facts we do not own: if Anthropic registers
the dated build, the test breaks without any litellm change. They now use a name that
is ours, and assert the invariant instead, that what the provider reports cannot change
the price of a turn the map cannot quote it by.

Picking that name needed care. An arbitrary string makes `get_model_info` raise, which
a caller upstream already handles, so the test passed with or without the fix and
defended nothing. A `claude-` prefixed name reaches the zero-rate fallback that this
bug rides on, so `claude-test-42161` reproduces it and stays ours.

The third point, that lines 801 and 828 exceed 120 characters, does not hold: they are
98 and 39.

* Pass completion_response_model as str or None to _priced_provider_response_model

basedpyright counted the bare variable as partially unknown (reportUnknownArgumentType +1 over the budget).

* test(cost): isolate runtime registration replay cases

---------

Co-authored-by: Eduardo Pessin <eduardopessin@users.noreply.github.com>
This commit is contained in:
Eduardo Pessin 2026-10-28 01:00:40 +00:00 • committed by GitHub
parent 7a23e497bd
commit e5fe716d69
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 71 additions and 0 deletions

View file

@ -239,6 +239,8 @@ graph LR
7. `DBSpendUpdateWriter.update_database()` queues spend increments to Redis
8. Background job `update_spend` flushes queued spend to PostgreSQL every 60s
Cost lookup falls back to the response model when the provider-reported name cannot be priced. Explicitly registered zero rates remain zero, and custom deployment pricing or an explicit `base_model` retain precedence
### Data Access Layer (Models & Repositories)
Database entities and the operations on them live in two packages at the root of `litellm/` so both the gateway (`proxy/`) and the SDK can use them without importing proxy internals:

View file

@ -94,6 +94,8 @@ Add your tests to the [`tests/unit/` directory](https://github.com/BerriAI/litel
- **Only add mocked tests** - no real LLM API calls in this directory
- For integration tests with real APIs, use the appropriate test directories
Tests that call `litellm.register_model` or change `litellm.model_cost` should use the `local_model_cost_map` fixture from `tests/unit/conftest.py`. It provides a fresh catalog and restores the previous map, so pricing and capability overrides cannot leak into later tests
### File Naming Convention
The `tests/unit/` directory follows the same structure as `litellm/`:

View file

@ -6810,6 +6810,73 @@ def test_completion_cost_bills_base_when_gemini_serves_on_demand(
assert cost == pytest.approx(100 * 0.001 + 50 * 0.002)
def test_an_unpriced_provider_slug_does_not_bill_the_turn_at_zero(
_local_model_cost_map: None, monkeypatch: pytest.MonkeyPatch
) -> None:
response_model: Final = "anthropic/claude-test-42161-response"
reported_model: Final = "claude-test-42161-unmapped"
monkeypatch.delitem(litellm.model_cost, reported_model, raising=False)
monkeypatch.delitem(litellm.model_cost, f"anthropic/{reported_model}", raising=False)
monkeypatch.setitem(
litellm.model_cost,
response_model,
{
"input_cost_per_token": 0.001,
"output_cost_per_token": 0.002,
"litellm_provider": "anthropic",
"mode": "chat",
},
)
response: Final = ModelResponse(
model=response_model,
choices=[],
usage=Usage(prompt_tokens=38, completion_tokens=24, total_tokens=62),
)
response._hidden_params["provider_response_model"] = reported_model
cost: Final = completion_cost(completion_response=response, custom_llm_provider="anthropic")
assert cost == pytest.approx(38 * 0.001 + 24 * 0.002)
@pytest.mark.parametrize("reported_rate", (0.0, 0.01))
def test_a_priced_provider_slug_still_decides_the_rate(
_local_model_cost_map: None, monkeypatch: pytest.MonkeyPatch, reported_rate: float
) -> None:
response_model: Final = "anthropic/claude-test-42161-response"
reported_model: Final = "claude-test-42161-reported"
monkeypatch.setitem(
litellm.model_cost,
response_model,
{
"input_cost_per_token": 0.001,
"output_cost_per_token": 0.002,
"litellm_provider": "anthropic",
"mode": "chat",
},
)
monkeypatch.setitem(
litellm.model_cost,
f"anthropic/{reported_model}",
{
"input_cost_per_token": reported_rate,
"output_cost_per_token": reported_rate * 2,
"litellm_provider": "anthropic",
"mode": "chat",
},
)
response: Final = ModelResponse(
model=response_model,
choices=[],
usage=Usage(prompt_tokens=38, completion_tokens=24, total_tokens=62),
)
response._hidden_params["provider_response_model"] = reported_model
cost: Final = completion_cost(completion_response=response, custom_llm_provider="anthropic")
assert cost == pytest.approx(38 * reported_rate + 24 * reported_rate * 2)
def test_custom_pricing_as_completion_cost_param():
from litellm import Choices, Message, ModelResponse
from litellm.utils import Usage