mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-29 06:33:28 +00:00
94e4ebd63d
* test(proxy): cover the max_parallel_requests slot a streamed /v1/messages MCP auto-execute keeps A streamed /v1/messages request that auto-executes MCP tools replays its answer as a stream after its model rounds already logged success. The stream path frees the request's max_parallel_requests slot only on a client disconnect, so after message_stop the slot stays taken and the key's next request answers 429 The test streams such a replay to the end and expects the slot to be free. It fails until the fix lands * fix(proxy): free the max_parallel_requests slot when a replayed /v1/messages MCP stream completes A streamed /v1/messages request that auto-executes MCP tools runs its model rounds without streaming and replays the final answer as a stream. The first round claims the request's only success log, and its limiter callback keeps the slot because the tool call reused it afterwards. The stream path released the slot only on a client disconnect, so after message_stop the key stayed at its max_parallel_requests limit and its next request answered 429 Stream cleanup now releases the stashed slot when the stream completed and its logging object already claimed the success log, which only happens when the stream replays a response that was logged before it started. A real stream still releases through its own success callback * test(mcp): free the key's only parallel slot after a streamed auto-execute ends Run a streamed MCP auto-execute on a key with max_parallel_requests 1 on every LLM surface, then send the next request on that key and expect 200. On main both /v1/messages surfaces answer 429 here, while /v1/responses and /v1/chat/completions already pass * test(proxy): keep only the MCP stream regression tests, since #48628 already frees the slot at stream end * test(proxy): require the MCP stream regression test's response to stream and reach its end event |
||
|---|---|---|
| .. | ||
| _support | ||
| audio_tests | ||
| base_sdk_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| e2e_harness | ||
| fixtures | ||
| harness_e2e | ||
| integration | ||
| rust-python-harness | ||
| test_litellm_rust | ||
| typing | ||
| unit | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _master_key.py | ||
| _openai_record_replay_proxy.py | ||
| _process_helpers.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| _wait_helpers.py | ||
| _ws_vcr.py | ||
| AGENTS.md | ||
| capturing_transport.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_logging.conf | ||
| test_rust_python_harness.py | ||
| VCR_TESTING.md | ||
| white_100x100.png | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/unit
This folder can only run mock tests.