scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-13 03:59:12 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 66
litellm/tests/unit/llms
History
devin-ai-integration[bot] abd3af422d
fix(caching): never store or serve a chat completion with no choices (#44709) 
* fix(caching): never store or serve a chat completion with no choices

A provider response with empty choices was written to the response cache and served on every identical request until the TTL ended, with no provider call in between. The cache now skips storing such a response and treats an already stored one as a miss, so the next request goes back to the provider and its answer replaces the entry.

* fix(caching): skip responses with no output on the Responses API and Anthropic Messages too

* fix(caching): skip streams and stored entries that carry no output

A chat or text completion stream whose chunks carried no choice is closed
by the stream wrapper with one empty choice of its own, so the assembled
response passed the choices check and was cached. The assembled stream is
now judged on its content: a stream with no text, tool call, or other
output in any choice is never stored, on the async and sync writers alike.
The Responses API stream writer and the Anthropic Messages stream writer
apply the same no-output check before storing.

A stored entry with no output read through the worker memory tier is now
evicted from that tier on the miss, so the next read reaches Redis where
the refill lands; the text completion and messages writers only write to
Redis, and the memory copy otherwise kept missing until its own TTL.

* test(integration): response cache cells for answers without output

Deterministic cells for the response cache on every unified endpoint,
streamed and not, through the OpenAI and Anthropic SDKs and raw httpx,
plus the sync SDK paths, stale entries, malformed answers, per-request
TTLs, cache delete, and chaos (Redis stopped or paused mid burst, a
worker killed, in-memory cache mode). The scripted upstream counts only
POSTs as deployment calls, since the proxy's boot-time GET /v1/models
discovery of a config deployment is not one.

* test(caching): pin the stored entry timestamp in the worker-copy test

* test(integration): drop the restating comments from the chaos cells

* test(integration): close the breaker on the first call after the Redis restart

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-06 20:07:30 +00:00
..
a2a refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
aiml test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
aiohttp_openai refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
anthropic fix(caching): never store or serve a chat completion with no choices (#44709) 2026-10-06 20:07:30 +00:00
apiserpent test(unit): add package markers to migrated unit test directories 2026-09-20 11:53:14 +00:00
azure refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
azure_ai refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
base_llm refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
baseten test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
bedrock refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
bedrock_mantle fix(bedrock_mantle): route Claude chat completions to the native Messages endpoint (#43646) 2026-10-03 15:24:41 -07:00
black_forest_labs refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
bytez test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
cerebras test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
chat fix(params): validate stream_chunk_size once, before any provider call (#43222) 2026-09-26 23:01:20 +00:00
chatgpt fix(chatgpt,github_copilot): refuse device-code login inside an event loop or worker thread (#39585) 2026-10-06 03:20:07 +00:00
clarifai refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
claude_code refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
cloudflare test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
codex refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
cohere refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
cometapi refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
compactifai test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
crusoe test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
custom_httpx fix(lens): price agent traces by joining gen_ai.response.id to spend logs (#44738) 2026-10-06 19:55:26 +00:00
dashscope refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
databricks fix(streaming): keep the served service_tier on streamed chunks and spend rows (#42870) 2026-09-29 12:54:17 -07:00
datarobot test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
deepagents feat: add litellm.agent() to run claude code, codex, opencode and deep agents through the ai gateway (#43885) 2026-10-01 22:27:49 +00:00
deepgram refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
deepinfra test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
deepseek refactor(anthropic): rename experimental_pass_through to pass_through (#43329) 2026-09-26 13:00:50 -07:00
docker_model_runner test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
e2b refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
edenai test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
elevenlabs refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
exa_ai fix(exa): fall back to highlights/summary when text is missing (#42213) 2026-10-02 14:15:15 -07:00
fal_ai refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
fastcrw test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
featherless_ai test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
fireworks_ai refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
gdc test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
gemini refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
gigachat refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
github_copilot fix(chatgpt,github_copilot): refuse device-code login inside an event loop or worker thread (#39585) 2026-10-06 03:20:07 +00:00
gradient_ai test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
groq test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
heroku test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
hosted_vllm fix(hosted_vllm): keep reasoning_content on replayed assistant messages (#43599) 2026-09-30 13:12:28 -07:00
huggingface test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
inception test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
jina_ai refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
langflow fix(ci): stop stale CI reds, keep unit tests off the host env, retry CyberArk policy conflicts (#43294) 2026-09-26 09:25:13 -07:00
laya feat: add Bespoke Nimble gateway and OSS classifier support (#44246) 2026-10-02 19:37:08 -07:00
lemonade test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
litellm_proxy refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
llamafile test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
lm_studio test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
manus refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
meta test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
meta_llama test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
minimax refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
mistral test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
modelscope refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
mongodb test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
moonshot chore(cost-map): remove models past their deprecation date (#42435) 2026-09-22 21:19:26 +00:00
nadir test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
nebius test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
neosantara test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
nimble test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
novita test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
nscale test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
nvidia_nim refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
nvidia_riva test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
oci refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
ocr test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
ollama test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
oobabooga test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
openai refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
openai_like feat: add Reka as an OpenAI-compatible provider (#44278) 2026-10-05 17:47:23 -07:00
opencode feat: add litellm.agent() to run claude code, codex, opencode and deep agents through the ai gateway (#43885) 2026-10-01 22:27:49 +00:00
openrouter refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
ovhcloud refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
parallel_ai test: add __init__.py to phase 12 unit test directories 2026-09-20 12:36:15 +00:00
parasail test: add __init__.py to phase 12 unit test directories 2026-09-20 12:36:15 +00:00
pass_through test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
perplexity test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
pg_vector fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
publicai test: add __init__.py to phase 12 unit test directories 2026-09-20 12:36:15 +00:00
ragflow test: add __init__.py to intermediate phase 12 unit test directories 2026-09-20 12:37:21 +00:00
recraft refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
reducto test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
replicate refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
runwayml test: add __init__.py to phase 12 unit test directories 2026-09-20 12:36:15 +00:00
s3_vectors test: add __init__.py to intermediate phase 12 unit test directories 2026-09-20 12:37:21 +00:00
sagemaker refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
sail test(e2e): align completion, SAIL, and spend-log fixtures with supported contracts (#43902) 2026-09-30 14:21:22 -07:00
sambanova test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
sap fix(sap): correctly handle cache_control (#39122) 2026-10-06 08:54:06 -07:00
scaleway feat(scaleway): add rerank support (#44160) 2026-10-02 21:20:05 -07:00
snowflake refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
soniox test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
stability refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
tencent refactor(anthropic): rename experimental_pass_through to pass_through (#43329) 2026-09-26 13:00:50 -07:00
tinyfish refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
together_ai refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
tool_loop feat(harness): add Harness.TOOL_LOOP, a minimal in-process tool-calling loop (#44391) 2026-10-03 17:33:01 +00:00
valkey test: migrate legacy provider tests to tests/unit (wave 2, phase 13) 2026-09-20 14:18:49 +00:00
vercel_ai_gateway test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
vertex_ai refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
volcengine test: stop CI tests from downloading tokenizer files and images (#43257) 2026-09-25 19:27:48 -07:00
voyage refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
wandb test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
watsonx refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
xai refactor(types): replace Any with proven types in 157 files (#44798) 2026-10-06 15:31:54 +00:00
you_com test: migrate phase 14 wave 2 provider tests to tests/unit 2026-09-20 11:01:40 +00:00
zai test: add __init__.py to every tests/unit directory this migration touches 2026-09-20 11:52:09 +00:00
__init__.py test(unit): add package markers to migrated unit test directories 2026-09-20 11:53:14 +00:00
test_cache_control_and_reasoning.py test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
test_file_content_block.py test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
test_file_search_responses.py test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
test_lifecycle_fix.py test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00
test_oss_decision.py feat: add Bespoke Nimble gateway and OSS classifier support (#44246) 2026-10-02 19:37:08 -07:00
test_polling_url_origin_match.py test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199) 2026-09-25 17:10:13 -07:00
test_predibase_transformation.py test: move tests/test_litellm/llms into tests/unit/llms (#43191) 2026-09-25 12:43:23 -07:00