scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-11-09 08:18:12 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 67
litellm/tests/unit/llms
History
devin-ai-integration[bot] 444423a5ca
fix(cost): bill a deployment's per-second rates on native Anthropic /v1/messages (#42417) 
* fix(cost): honor per-second custom pricing on chat completions for every provider

* test(cost): assert a per-second priced deployment bills instead of staying at $0

The zero-cost diagnostic tests from #42345 used a per-second-only entry as their
misconfigured fixture, which this branch now bills. Switch that fixture to a
per-query-only entry, which is still selected as the deployment's own pricing and
still prices chat usage at $0, and add a per-second test asserting the call
duration is billed with no diagnostic

Also let a caller's explicit total_time outrank the logging window in
completion_cost, so the SDK precedence stays stamped response, caller, logging

* test(response_metadata): move the per-second pricing regression into the mapped tests/unit file

* fix(cost): bill a deployment's per-second rates on /v1/responses and /v1/messages

The proxy builds the request's logging state before the router picks a
deployment, and only the chat path re-read the deployment's
input_cost_per_second / output_cost_per_second afterwards. The shared
update step every other route runs (Logging.update_from_kwargs) now
seeds those custom pricing keys from the router-merged kwargs too, so
custom_pricing flips on and the cost calculator prices the call against
the router-registered deployment id on every surface.

* fix(cost): price a /v1/messages call off the response time the proxy reports

A /v1/messages result is a plain dict, so the proxy keeps its response time on
the logging object. The cost step now reads it there for a non-streamed call,
so the cost header and the spend log bill the same duration.

* fix(anthropic): bill a deployment's per-second rate only on the native /v1/messages route

Move the custom pricing copy from the shared logging refresh into the native
Messages handler, reusing the helper the Responses handler already had, so
generateContent and other routes keep their current billing

* fix(cost): pass only per-second rates to native /v1/messages cost tracking

Other deployment overrides keep billing as on main, matching chat's per-second gate

* test(anthropic): pin per-second-only pricing on native /v1/messages with a long-context-rate deployment

* fix(cost): bill native /v1/messages from logging setup, as main did, to the end the proxy reports

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-29 05:29:03 -07:00
..
a2a chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
aiml chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
aiohttp_openai fix(aiohttp_openai): forward service_tier in transform_response (#47263) 2026-10-28 13:14:44 -07:00
anthropic fix(cost): bill a deployment's per-second rates on native Anthropic /v1/messages (#42417) 2026-10-29 05:29:03 -07:00
apiserpent test(unit): add package markers to migrated unit test directories 2026-09-20 11:53:14 +00:00
aws_polly fix(aws_polly): read the engine from the bare model name (#47963) 2026-10-23 18:10:51 -07:00
azure fix(openai): warn when the GPT-5 reasoning effort gate drops a level under drop_params (#47761) 2026-10-29 04:46:29 -07:00
azure_ai fix(cohere): price native Cohere embed calls on Foundry relays and the Cohere passthrough (#48978) 2026-10-29 05:17:11 -07:00
base_llm fix(llms): move later developer messages for models declared system-first (#48897) 2026-10-29 04:12:13 -07:00
baseten fix(baseten): list current Model APIs instead of retired 2023 deployment IDs (#46945) 2026-10-19 16:40:13 +00:00
bedrock fix(openai): warn when the GPT-5 reasoning effort gate drops a level under drop_params (#47761) 2026-10-29 04:46:29 -07:00
bedrock_mantle fix(openai): warn when the GPT-5 reasoning effort gate drops a level under drop_params (#47761) 2026-10-29 04:46:29 -07:00
bigmodel feat(providers): add BigModel support and model pricing (#48563) 2026-10-29 00:39:11 -07:00
black_forest_labs refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
brave chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
bytez test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
cerebras test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
chat fix(router): read the first stream chunk before returning so a failure before any token runs through the retry and fallback budget (#46493) 2026-10-14 14:19:25 -07:00
chatgpt fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
clarifai refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
claude_code refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
cloudflare fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
codestral fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
codex refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
cohere fix(cohere): price native Cohere embed calls on Foundry relays and the Cohere passthrough (#48978) 2026-10-29 05:17:11 -07:00
cometapi chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
compactifai refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
crusoe fix(crusoe): update base URL to current endpoint (#33121) 2026-10-15 19:10:58 -07:00
custom_httpx fix(websearch)!: keep the provider prefix on the agentic follow-up model (#38885) 2026-10-29 04:47:18 -07:00
dashscope feat(dashscope): add separate explicit and implicit cache pricing (#45567) 2026-10-20 04:54:35 +00:00
databricks fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
dataforseo chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
datarobot test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
deepagents feat: add litellm.agent() to run claude code, codex, opencode and deep agents through the ai gateway (#43885) 2026-10-01 22:27:49 +00:00
deepgram refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
deepinfra refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
deepseek refactor(anthropic): rename llms/anthropic/pass_through to llms/anthropic/messages (#48510) 2026-10-27 10:08:25 -07:00
docker_model_runner test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
duckduckgo chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
e2b refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
edenai feat: generate provider_endpoints_support.json from the cost map (#48619) 2026-10-27 12:06:55 -07:00
elevenlabs fix(elevenlabs): map wav and pcm TTS formats to 24kHz output formats (#48455) 2026-10-26 13:42:01 -07:00
exa_ai fix(exa): return highlights, highlight scores, score and the response envelope from Exa search (#47423) 2026-10-19 15:45:56 -07:00
fal_ai chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
fastcrw fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
featherless_ai refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
firecrawl fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
fireworks_ai fix(llms): move later developer messages for models declared system-first (#48897) 2026-10-29 04:12:13 -07:00
friendliai fix(friendliai): drop litellm-internal message fields before sending (#45092) 2026-10-20 01:22:07 +00:00
gdc test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
gemini fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
gigachat fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
github_copilot fix(github_copilot): treat a malformed api-key.json like a missing one (#44723) 2026-10-29 03:46:44 -07:00
google_pse chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
gradient_ai fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
groq fix(groq): allow streamed reasoning replies in history (#47931) (#48268) 2026-10-25 00:52:01 +00:00
heroku fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
hosted_vllm fix(llms): move later developer messages for models declared system-first (#48897) 2026-10-29 04:12:13 -07:00
huggingface fix(token_counter)!: keep a loaded Hugging Face tokenizer when a slower download fails (#48916) 2026-10-29 03:50:58 -07:00
hyperbolic refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
inception refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
infinity chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
jina_ai fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
lambda_ai test: move the unit half of 126 mixed legacy files into tests/unit (#45090) 2026-10-07 14:07:43 -07:00
langflow fix(ci): stop stale CI reds, keep unit tests off the host env, retry CyberArk policy conflicts (#43294) 2026-09-26 09:25:13 -07:00
langgraph fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
laya feat: add Bespoke Nimble gateway and OSS classifier support (#44246) 2026-10-02 19:37:08 -07:00
lemonade chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
linkup fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
litellm_proxy fix(llms): move later developer messages for models declared system-first (#48897) 2026-10-29 04:12:13 -07:00
llamafile refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
lm_studio refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
manus test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
meta fix(realtime)!: keep turn_detection.type when a transcript guardrail disables auto-response (#44939) 2026-10-16 15:24:40 +00:00
meta_llama refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
microsoft_365_copilot refactor(anthropic): rename llms/anthropic/pass_through to llms/anthropic/messages (#48510) 2026-10-27 10:08:25 -07:00
milvus test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
minimax feat(minimax): add MiniMax-H3 and H3-Max video generation via /v1/videos (#43705) 2026-10-15 05:48:36 -07:00
mistral fix(mistral): replay assistant reasoning as a Mistral thinking chunk (#48778) 2026-10-28 11:41:33 -07:00
modelscope fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
mongodb test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
moonshot fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
morph feat(morph): support open models and cached input pricing (#48676) 2026-10-27 18:06:14 -07:00
nadir test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
nebius test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
neosantara refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
nimble test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
nlp_cloud chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
novita test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
nscale test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
nvidia_nim fix(openai): send chat completions through client.post so the SDK's per-message param transform stops holding the event loop (#46967) 2026-10-16 18:06:01 -07:00
nvidia_riva feat(nvidia_riva)!: add riva_offline mode for offline-only ASR models (#41021) 2026-10-20 23:03:56 +00:00
oci fix(oci): drop non-function tools under drop_params and gate reasoning_effort on declared levels (#40732) 2026-10-15 22:20:02 +00:00
ocr test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
ollama fix(ollama): forward embedding credentials, headers and timeout (#48707) 2026-10-28 09:32:47 -07:00
oobabooga fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
openai fix(openai): warn when the GPT-5 reasoning effort gate drops a level under drop_params (#47761) 2026-10-29 04:46:29 -07:00
openai_like feat(providers): add Kyma API as an OpenAI-compatible provider (#48756) 2026-10-29 05:19:21 -07:00
opencode fix(opencode): bill OpenCode Go requests from the reported list_cost (#48254) 2026-10-24 16:44:29 -07:00
openrouter chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
opensandbox chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
ovhcloud chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
parallel_ai test: add __init__.py to phase 12 unit test directories 2026-09-20 12:36:15 +00:00
parasail refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
pass_through fix(guardrails): exclude litellm_metadata from pass-through guardrail scan text (#48276) 2026-10-26 12:07:58 -07:00
perplexity fix(responses): drop the null created_by OpenAI rejects when output items are replayed (#48783) 2026-10-28 21:11:49 -07:00
pg_vector fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
pi feat(harness): add pi harness for litellm.agent() (#45467) 2026-10-20 09:01:40 +00:00
publicai test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
ragflow test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
recraft refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
reducto test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
replicate chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
runwayml refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
s3_vectors test: add __init__.py to intermediate phase 12 unit test directories 2026-09-20 12:37:21 +00:00
sagemaker fix(streaming)!: send the include_usage chunk with an empty choices array (#47109) 2026-10-21 02:26:05 +00:00
sail test(e2e): align completion, SAIL, and spend-log fixtures with supported contracts (#43902) 2026-09-30 14:21:22 -07:00
sambanova chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
sap fix(sap): correctly handle cache_control (#39122) 2026-10-06 08:54:06 -07:00
scaledown feat(scaledown): add native text provider support (#44168) 2026-10-10 14:01:17 -07:00
scaleway test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
searchapi chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
searxng chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
serper chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
snowflake fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
soniox feat(soniox): decode proxy form fields for context, translation and flags (#40605) 2026-10-16 06:37:34 +00:00
stability refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
strands_decider feat(strands_decider): call a Strands Decider on an AgentCore runtime ARN (#45678) 2026-10-09 22:51:19 -07:00
tavily fix(tavily): return raw_content, score, favicon and the response envelope from Tavily search (#48684) 2026-10-27 18:52:06 -07:00
tencent feat(tencent): add Seedream image generation and editing through TokenHub (#48735) 2026-10-28 15:53:00 -07:00
tinyfish test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
together_ai fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444) 2026-10-14 21:40:14 -07:00
tool_loop feat(harness): add Harness.TOOL_LOOP, a minimal in-process tool-calling loop (#44391) 2026-10-03 17:33:01 +00:00
topaz fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
triton fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
v0 test: move the unit half of 126 mixed legacy files into tests/unit (#45090) 2026-10-07 14:07:43 -07:00
valkey test: migrate legacy provider tests to tests/unit (wave 2, phase 13) 2026-09-20 14:18:49 +00:00
vercel_ai_gateway refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
vertex_ai fix(vertex_ai): build the cachedContents collection URL under a custom api_base (#44783) 2026-10-29 03:46:35 -07:00
vllm fix(vllm): write spend log rows for /vllm passthrough calls to router models (#45733) 2026-10-14 12:40:48 -07:00
volcengine fix(openai): send chat completions through client.post so the SDK's per-message param transform stops holding the event loop (#46967) 2026-10-16 18:06:01 -07:00
voyage fix(llms): sanitize trailing slashes in provider get_complete_url (#42887) 2026-10-15 02:59:46 -07:00
wandb test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
watsonx chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
xai feat(search): add native xAI X Search support (#38505) (#48677) 2026-10-28 00:28:51 +00:00
xinference test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
you_com test: migrate phase 14 wave 2 provider tests to tests/unit 2026-09-20 11:01:40 +00:00
zai test: add __init__.py to every tests/unit directory this migration touches 2026-09-20 11:52:09 +00:00
__init__.py test(unit): add package markers to migrated unit test directories 2026-09-20 11:53:14 +00:00
test_api_base_trailing_slash_routes.py fix(llms): join api_base and route with one shared helper (#47535) 2026-10-21 06:59:32 +00:00
test_cache_control_and_reasoning.py test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
test_custom_llm.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_file_content_block.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
test_file_search_responses.py fix(responses)!: count input tokens the way inference sends the request (#47363) 2026-10-20 15:13:59 +00:00
test_get_complete_url_trailing_slash.py refactor(anthropic): rename llms/anthropic/pass_through to llms/anthropic/messages (#48510) 2026-10-27 10:08:25 -07:00
test_lifecycle_fix.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
test_oss_decision.py feat(decisions): add Laya and Bespoke providers to the Add Model form (#45260) 2026-10-14 08:11:11 -07:00
test_polling_url_origin_match.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_predibase_transformation.py fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00