scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-24 05:45:11 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 72
litellm/tests
History
Cabeza 6879090caa fix(google_genai): retry a stream that drops before its first event 
The async generateContent handler returned its streaming iterator as soon as
the response headers arrived, so the first body read happened in the proxy's
data generator, long after the router had recorded the call as a success and
spent its retry budget. A stream that dropped before its first event therefore
reached the client as an in-stream error frame with attempted_retries unset,
and router_settings.num_retries never applied.

Read the first SSE event inside the handler instead, so a drop raises where the
router's retry loop can see it and is mapped to a retriable status. Gemini
sends no lifecycle frames before content, so one event proves the stream is
alive, and the event is replayed to the consumer unchanged. A stream that ends
before its first event stays a valid empty answer and ends the iterator with
its logging intact.

The abandoned response is closed when priming fails, so a dropped stream does
not hold a pooled connection while the retry runs.

The synchronous path has the same shape but is out of scope here: it has no
caller that returns before the first read.

Refs #45457
2026-10-12 15:48:43 +08:00
..
_support fix(params): validate stream_chunk_size once, before any provider call (#43222) 2026-09-26 23:01:20 +00:00
agent_tests/local_only_agent_tests test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
audio_tests test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
base_sdk_tests fix: support OpenAI SDK 3 while retaining SDK 2 compatibility (#44927) 2026-10-09 19:45:50 -07:00
basic_proxy_startup_tests fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
batches_tests test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
benchmarks refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
code_coverage_tests chore(greptile): load nested AGENTS.md files as review context scoped to their directories (#45636) 2026-10-09 13:22:29 -07:00
documentation_tests fix(s3_v2): upload fresh events first, drop terminal failures and hour-old retries by default, opt-in adaptive concurrency (#43022) 2026-09-26 14:58:28 -07:00
e2e fix(bedrock): keep tool_reference results as text on converse (#44590) 2026-10-09 17:52:17 -07:00
e2e_harness docs(agents): ban semicolons and colons in human-facing text (#45684) 2026-10-09 16:36:32 -07:00
guardrails_tests test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
harness_e2e feat: add litellm.agent() to run claude code, codex, opencode and deep agents through the ai gateway (#43885) 2026-10-01 22:27:49 +00:00
image_gen_tests test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
integration fix(proxy): let admins make per-model key/team rate limits a hard 429 instead of falling back (#45459) 2026-10-09 23:10:23 -07:00
litellm_utils_tests test: move offline proxy auth, hook, spend and pass-through tests to tests/unit (#45568) 2026-10-09 09:49:13 -07:00
llm_translation test: fix stale and flaky tests across CircleCI, GHA and Buildkite (#45530) 2026-10-09 11:40:14 -07:00
load_tests fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
local_testing test: move offline anthropic prompt caching tests to tests/unit (#45616) 2026-10-09 10:53:08 -07:00
logging_callback_tests test: move offline proxy auth, hook, spend and pass-through tests to tests/unit (#45568) 2026-10-09 09:49:13 -07:00
mcp_tests feat(mcp): add portable catalog pagination (#44446) 2026-10-06 18:54:50 -07:00
multi_instance_e2e_tests fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
ocr_tests test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
openai_endpoints_tests test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
otel_tests test: replace 61 live logging and otel tests with offline unit and integration coverage (#45362) 2026-10-08 15:55:57 -07:00
pass_through_tests test: delete 34 legacy tests already covered by e2e (#45340) 2026-10-08 09:43:56 -07:00
pass_through_unit_tests test: move offline proxy auth, hook, spend and pass-through tests to tests/unit (#45568) 2026-10-09 09:49:13 -07:00
proxy_admin_ui_tests test: move the unit half of 126 mixed legacy files into tests/unit (#45090) 2026-10-07 14:07:43 -07:00
proxy_behavior refactor(lens): connect LiteLLM to the independent Lens service (#45529) 2026-10-09 16:17:05 -07:00
proxy_e2e_anthropic_messages_tests fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
proxy_migration_tests fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
proxy_security_tests fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
proxy_unit_tests fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
router_unit_tests test: make 77 legacy live tests offline in litellm_utils, router_unit and responses dirs (#45298) 2026-10-08 02:37:55 -07:00
rust-python-harness docs(agents): ban semicolons and colons in human-facing text (#45684) 2026-10-09 16:36:32 -07:00
search_tests test: move 81 legacy live tests in pass-through, spend, batches, audio, search, guardrails, image and ocr dirs offline (#45288) 2026-10-08 02:38:12 -07:00
store_model_in_db_tests fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
test_litellm refactor(lens): connect LiteLLM to the independent Lens service (#45529) 2026-10-09 16:17:05 -07:00
test_litellm_rust fix(python-bridge): preserve callback keyword deletions in native requests (#45692) 2026-10-10 02:36:49 +00:00
typing fix: support OpenAI SDK 3 while retaining SDK 2 compatibility (#44927) 2026-10-09 19:45:50 -07:00
unified_google_tests test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
unit fix(google_genai): retry a stream that drops before its first event 2026-10-12 15:48:43 +08:00
vector_store_tests refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
windows_tests fix(packaging): keep wheel paths under Windows MAX_PATH for Store Python (#43903) 2026-09-30 22:20:36 +00:00
__init__.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
_fake_openai_endpoint_server.py
_flush_vcr_cache.py
_live_test_helpers.py
_master_key.py fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
_openai_record_replay_proxy.py
_process_helpers.py test: count a zombie grandchild as gone in the migrate deploy timeout test (#42570) 2026-09-22 14:59:26 -07:00
_vcr_conftest_common.py fix(tests): stop VCR recording and replaying a test's own localhost upstream (#43346) 2026-09-26 15:49:39 -07:00
_vcr_redis_persister.py
_wait_helpers.py
_ws_vcr.py
AGENTS.md docs(agents): ban semicolons and colons in human-facing text (#45684) 2026-10-09 16:36:32 -07:00
capturing_transport.py test(vcr): guard leaked cassette patches and make injected-transport embedding tests immune (#42542) 2026-09-22 14:20:18 -07:00
eval_swe_bench.py
fake_openai_endpoint.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json
README.MD test: finish the non-proxy half of tests/test_litellm (#43281) 2026-09-25 22:43:41 -07:00
test_debug_warning.py
test_keys.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_litellm_proxy_responses_config.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_logging.conf
test_models.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_new_vector_store_endpoints.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_openai_endpoints.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_proxy_server_non_root.py
test_rust_python_harness.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_service_logger_otel.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_spend_logs.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
test_team_logging.py fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
test_users.py test: move 76 live top-level tests to offline unit and integration coverage (#45352) 2026-10-08 10:33:35 -07:00
white_100x100.png test: stop CI tests from downloading tokenizer files and images (#43257) 2026-09-25 19:27:48 -07:00

README.MD

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/unit

This folder can only run mock tests.