mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-03 07:23:33 +00:00
43e5539e77
The WebSocket relay ran the pre-call hooks with empty data, so the rate limiter never saw a model and skipped a key's model_rpm_limit and model_tpm_limit. The relay already took a model parameter, which only labeled failure logs, so it now puts that model in the hook data too The Deepgram listen and speak routes now pass the model their auth checked the key against. Vertex Live already passed its optional model query param, which its auth also checks, so a session that sends it gets per-model limits too A refused session used to close with 1008 and no reason, and the refusal never reached the failure hooks. The relay now reports a pre-call refusal to post_call_failure_hook the way HTTP pass-through does, and raises the replacement error a failure hook returns, if any The app's WebSocket error handler, which every relay route shares, now closes an accepted socket with a reason. That reason is the standard detail message, or for a rate limit the limit, model and retry time, run through the client redaction helper and cut to fit the close frame. A 429 there now closes with 1013 Try Again Later instead of 1008, matching how an upstream 429 already closes. This changes what the OpenAI, Vertex Live, Deepgram and configured WebSocket pass-through routes send on a refusal after accept. The realtime, live sessions and responses WebSocket routes handle their own errors and are unchanged Resolves LIT-10354 |
||
|---|---|---|
| .. | ||
| _support | ||
| audio_tests | ||
| base_sdk_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| e2e_harness | ||
| fixtures | ||
| harness_e2e | ||
| integration | ||
| rust-python-harness | ||
| test_litellm_rust | ||
| typing | ||
| unit | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _master_key.py | ||
| _openai_record_replay_proxy.py | ||
| _process_helpers.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| _wait_helpers.py | ||
| _ws_vcr.py | ||
| AGENTS.md | ||
| capturing_transport.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_logging.conf | ||
| test_rust_python_harness.py | ||
| VCR_TESTING.md | ||
| white_100x100.png | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/unit
This folder can only run mock tests.