mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-17 23:51:30 +00:00
9c48ad41ac
Two follow-ups from review on the upstream-reported usage contract. An unusable cost header fell through to the endpoint's flat cost_per_request instead of the zero the contract promises, so a target that contradicted itself got billed an estimate it had just disowned. A target that speaks this contract now owns the cost for the request whether or not the value it sent parsed. The reported total also cannot be split into prompt and completion, so reading one out of it under token_rate_limit_type input or output yielded zero and left the TPM window uncharged; pass-through traffic then ran past a limit it is meant to share with the general API. Usage that carries no split now charges its total under every limit type, while usage that does carry one is untouched. |
||
|---|---|---|
| .. | ||
| llm_provider_handlers | ||
| test_carry_guardrail_logging_info.py | ||
| test_llm_pass_through_endpoints.py | ||
| test_method_specific_routing.py | ||
| test_pass_through_endpoints.py | ||
| test_passthrough_auth_default.py | ||
| test_passthrough_endpoints_common_utils.py | ||
| test_passthrough_guardrail_block_otel_span.py | ||
| test_passthrough_guardrails.py | ||
| test_passthrough_guardrails_field_targeting.py | ||
| test_passthrough_post_call_guardrails.py | ||
| test_streaming_handler_interrupt.py | ||
| test_upstream_usage_headers.py | ||
| test_vertex_ai_batch_passthrough.py | ||
| test_vertex_passthrough_load_balancing.py | ||
| test_watsonx_proxy_route.py | ||