scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-29 06:33:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 200
litellm/tests/unit/llms
History
devin-ai-integration[bot] a2816d7eb7
fix(router): stream /v1/messages lifecycle frames live when no fallback can take over (#43600) 
* fix(router): stream anthropic messages lifecycle frames live when no fallback can take over

The /v1/messages streaming wrapper buffered message_start and
content_block_start until the first content_block_delta and dropped
pings behind buffered frames unconditionally, even for requests no
fallback could ever recover. With adaptive thinking on Bedrock or
Vertex the client saw no bytes for the whole thinking pass and hit
read timeouts.

Buffering now applies only while a fallback can still take over
(generic or refusal chain resolving), and a ping is always forwarded
live since it carries no lifecycle and keeps the connection alive.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(router): mirror every dispatcher fallback path in the anthropic stream gate

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(router): skip already-tried order levels in the anthropic stream gate

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(router): keep a transport-split ping behind buffered lifecycle frames instead of forwarding its head live

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(router): credit the #39566 branch this fix supersedes

Co-authored-by: Radu Swigler <radu.porumba@gmail.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Radu Swigler <radu.porumba@gmail.com>
(cherry picked from commit d46304900f)
2026-10-19 22:40:20 -07:00
..
a2a test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
aiml chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
anthropic fix(router): stream /v1/messages lifecycle frames live when no fallback can take over (#43600) 2026-10-19 22:40:20 -07:00
apiserpent test(unit): add package markers to migrated unit test directories 2026-09-20 11:53:14 +00:00
azure feat(sdk): add litellm.systemone and trim litellm.decisions to the OpenAI shape (#45846) 2026-10-11 01:02:56 +00:00
azure_ai feat(count_tokens)!: configurable token_count_framing and token_count_model, count source in the response (#46777) 2026-10-19 12:35:33 -07:00
base_llm chore(decisions): remove the System One converters and OpenAI spec types left dead by #45214 (#45442) 2026-10-11 01:01:59 +00:00
baseten test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
bedrock fix(count_tokens): narrow stable backport to the October 19 patch 2026-10-19 14:03:42 -07:00
bedrock_mantle chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
black_forest_labs test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
bytez test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
cerebras test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
chat fix(bedrock): honour stream_chunk_size in Invoke streaming (#42686) 2026-09-23 11:25:00 -07:00
chatgpt test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
cloudflare test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
cohere refactor(ocr): remove the Python OCR execution path and require the Rust route (#43081) 2026-09-24 18:18:50 -07:00
cometapi chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
compactifai chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
crusoe test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
custom_httpx test(unit): stop test modules from putting their own directory on sys.path (#43420) 2026-09-26 22:00:16 -07:00
dashscope chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
databricks chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
datarobot test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
deepgram chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
deepinfra chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
deepseek test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
docker_model_runner test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
edenai chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
elevenlabs test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
fal_ai chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
fastcrw test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
featherless_ai chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
fireworks_ai chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
gdc chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
gemini fix(count_tokens): count Anthropic-format requests for Gemini the way /v1/messages inference converts them (#46767) 2026-10-19 12:35:32 -07:00
gigachat test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
github_copilot chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
gradient_ai test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
groq test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
heroku chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
hosted_vllm test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
huggingface chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
inception test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
jina_ai test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
langflow chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
lemonade chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
litellm_proxy fix(spend): attribute CLI session spend to the per-user cli-session alias instead of the hashed session token (#40541) 2026-09-24 18:21:47 -07:00
llamafile test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
lm_studio chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
manus test: migrate phase 9 legacy llm provider tests to tests/unit 2026-09-20 08:13:11 +00:00
meta test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
meta_llama test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
minimax test: migrate phase 9 legacy llm provider tests to tests/unit 2026-09-20 08:13:11 +00:00
mistral chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
modelscope chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
mongodb test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
moonshot chore(cost-map): remove models past their deprecation date (#42435) 2026-09-22 21:19:26 +00:00
nadir chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
nebius chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
neosantara test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
nimble test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
novita test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
nscale test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
nvidia_nim test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
nvidia_riva test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
oci chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
ocr test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
ollama chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
oobabooga test(unit): make every tests/unit directory a package so pytest collection is unique 2026-09-20 11:50:59 +00:00
openai fix(count_tokens): narrow stable backport to the October 19 patch 2026-10-19 14:03:42 -07:00
openai_like chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
openrouter fix(openrouter): bill responses, decisions and pass-through from usage.cost and always allow reasoning params (#45640) [partial: decisions and pass-through billing only] 2026-10-11 01:02:00 +00:00
ovhcloud chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
parallel_ai test: add __init__.py to phase 12 unit test directories 2026-09-20 12:36:15 +00:00
parasail test: add __init__.py to phase 12 unit test directories 2026-09-20 12:36:15 +00:00
pass_through chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
perplexity chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
pg_vector chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
publicai test: add __init__.py to phase 12 unit test directories 2026-09-20 12:36:15 +00:00
ragflow test: add __init__.py to intermediate phase 12 unit test directories 2026-09-20 12:37:21 +00:00
recraft test: add __init__.py to intermediate phase 12 unit test directories 2026-09-20 12:37:21 +00:00
reducto chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
runwayml test: add __init__.py to phase 12 unit test directories 2026-09-20 12:36:15 +00:00
s3_vectors test: add __init__.py to intermediate phase 12 unit test directories 2026-09-20 12:37:21 +00:00
sagemaker chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
sambanova chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
sap chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
scaleway test: add __init__.py to phase 12 unit test directories 2026-09-20 12:36:15 +00:00
snowflake chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
soniox chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
stability test: add __init__.py to intermediate phase 12 unit test directories 2026-09-20 12:37:21 +00:00
strands_decider feat(strands_decider): call a Strands Decider on an AgentCore runtime ARN (#45678) 2026-10-11 01:02:37 +00:00
tencent Merge pull request #42145 from BerriAI/litellm_migrate_tests_p13 2026-09-20 10:50:18 -07:00
tinyfish chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
together_ai test: migrate legacy provider tests to tests/unit (wave 2, phase 13) 2026-09-20 14:18:49 +00:00
valkey test: migrate legacy provider tests to tests/unit (wave 2, phase 13) 2026-09-20 14:18:49 +00:00
vercel_ai_gateway chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
vertex_ai fix(count_tokens): narrow stable backport to the October 19 patch 2026-10-19 14:03:42 -07:00
volcengine chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
voyage test: add __init__.py to every tests/unit directory this migration touches 2026-09-20 11:52:09 +00:00
wandb chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
watsonx test: add __init__.py to every tests/unit directory this migration touches 2026-09-20 11:52:09 +00:00
xai chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
you_com test: migrate phase 14 wave 2 provider tests to tests/unit 2026-09-20 11:01:40 +00:00
zai test: add __init__.py to every tests/unit directory this migration touches 2026-09-20 11:52:09 +00:00
__init__.py test(unit): add package markers to migrated unit test directories 2026-09-20 11:53:14 +00:00
test_cache_control_and_reasoning.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_file_content_block.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_file_search_responses.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_lifecycle_fix.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_polling_url_origin_match.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00
test_predibase_transformation.py chore: backport CI fixes, #43206 and Rust changes to rc/1.104.0 (#43338) 2026-09-26 14:26:59 -07:00