scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-05 08:07:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 136
litellm/tests
History
Yassin Kortam cbb64111fa
fix(proxy): keep serving reads from the read replica when the primary DB is down at startup (#31951) 
* fix(proxy): keep serving reads from the read replica when the primary DB is down at startup

RoutingPrismaWrapper.connect() connected the writer first and let a writer
failure propagate, so a proxy that started during a primary outage ended up
with no Prisma client at all (startup swallows the error under
allow_requests_on_db_unavailable): DB-stored models never loaded and every
inference request failed with 400 Invalid model name, even with a healthy
DATABASE_URL_READ_REPLICA. Workers recycled via MAX_REQUESTS_BEFORE_RESTART
hit this mid-outage and stayed broken for the rest of the outage.

connect() now degrades on a writer-only failure: reads (key auth, DB-stored
model loads) are served by the reader, writes fail at call time, and the DB
health watchdog keeps retrying the writer reconnect, which clears the
degraded flag once the primary recovers. A full outage (both sides down)
still raises as before.

Resolves LIT-4159

* fix(proxy): clear degraded-writer flag when the reconnect probe finds the writer already healthy

The direct-reconnect path returns early when the writer probe succeeds
(engine already reconnected by another path, e.g. an IAM token refresh),
skipping recreate_prisma_client, which was the only runtime path clearing
_writer_unavailable. The stale flag made the watchdog fire reconnect
attempts against a healthy writer on every cooldown cycle until restart.
Clear the flag in the early-return branch and cover it with a regression
test that fails without the change
2026-07-04 12:45:09 -07:00
..
agent_tests feat(a2a): support a2a-sdk 1.x proxy routing for 0.3 and 1.0 agents (#30950) 2026-06-29 09:32:39 +05:30
audio_tests fix(tests): stabilize image-edit VCR cassettes to stop live gpt-image-1 spend (#28110) 2026-05-18 09:15:39 -07:00
basic_proxy_startup_tests build: migrate packaging, CI, and Docker from Poetry to uv (#25007) 2026-04-09 11:46:23 -07:00
batches_tests test(e2e): add live batches suite across providers and routing scenarios (#30958) 2026-07-02 08:05:23 -07:00
benchmarks test(benchmarks): add CodSpeed benchmarks for inference, MCP and A2A hot paths (#31716) 2026-06-30 10:27:12 -07:00
code_coverage_tests fix(otel): hashable scope for _emit_once when guardrail_mode is list (#31262) 2026-06-25 11:59:35 -07:00
documentation_tests feat: add Rust OCR providers (#31272) 2026-06-25 15:12:30 -07:00
e2e feat(proxy): add key-level budget_fallbacks to reroute requests when a per-model budget is exceeded (#31783) 2026-07-03 12:20:12 -07:00
enterprise test(e2e): add live batches suite across providers and routing scenarios (#30958) 2026-07-02 08:05:23 -07:00
guardrails_tests feat: litellm oss 110626 (#30202) 2026-06-11 22:30:26 -07:00
image_gen_tests test(bedrock): switch image gen live test off EOL Titan to Nova Canvas (#31937) 2026-07-01 22:57:29 -07:00
integration feat(sandbox): e2b code execution primitive (#30898) 2026-06-20 16:30:01 -07:00
litellm chore: litellm oss staging (#31185) 2026-06-26 09:17:44 -07:00
litellm-proxy-extras style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
litellm_core_utils Merge branch 'litellm_internal_staging' into litellm_staging_03_22_2026 2026-04-20 19:56:00 +05:30
litellm_utils_tests feat(fireworks_ai): sync chat completions endpoint with full API surface (#30885) 2026-06-20 19:49:07 -07:00
llm_responses_api_testing test(responses): bump deprecated gemini-3-pro-preview to gemini-3.1-pro-preview (#29433) 2026-06-01 09:54:30 -07:00
llm_translation feat(guardrails/headroom): add CCR (compress-cache-retrieve) via agentic loop (#31681) 2026-06-30 19:19:34 -07:00
load_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
local_testing fix(bedrock): drop strict/additionalProperties from toolSpec for Claude Sonnet 4 (#31943) 2026-07-01 23:56:25 -07:00
logging_callback_tests fix(proxy): restore admin key/team callback_vars.turn_off_message_logging override (LIT-3587) (#31905) 2026-07-02 15:01:58 -07:00
mcp_tests feat(mcp): make token_exchange (OBO) production-ready - discovery threading + audit hardening + RFC 9728 challenge (#31622) 2026-07-03 17:12:25 -07:00
multi_instance_e2e_tests
ocr_tests fix(azure_ai): preserve content, tables, and keyValuePairs in doc-intelligence /v1/ocr (#32018) 2026-07-02 22:38:36 -07:00
old_proxy_tests/tests fix: cleanup tests 2026-03-30 16:24:35 -07:00
openai_endpoints_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
otel_tests fix(cli): mint per-session agent credential on lite login (#31072) 2026-06-26 09:05:15 -07:00
pass_through_tests test(pass-through): de-flake vertex spend-log test by routing through the proxy (#31689) 2026-06-30 15:27:48 -07:00
pass_through_unit_tests test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
proxy_admin_ui_tests Revert "chore: remove _experimental/out (#31546)" 2026-07-01 13:25:47 -07:00
proxy_behavior feat(proxy): add AES-256-GCM at-rest credential encryption with versioned format and re-encryption migration (#31215) 2026-06-29 20:14:22 +02:00
proxy_e2e_anthropic_messages_tests Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic (#29847) 2026-06-06 14:33:42 -07:00
proxy_migration_tests test(proxy): stop running real-DB tests in GitHub Actions unit jobs (#29700) 2026-06-04 14:56:02 -07:00
proxy_security_tests test(proxy): stop running real-DB tests in GitHub Actions unit jobs (#29700) 2026-06-04 14:56:02 -07:00
proxy_unit_tests feat(proxy): add key-level budget_fallbacks to reroute requests when a per-model budget is exceeded (#31783) 2026-07-03 12:20:12 -07:00
router_unit_tests fix(vertex/files): stream OpenAI->Vertex batch JSONL uploads (#31036) 2026-06-24 13:19:57 -07:00
scim_tests
search_tests feat(tinyfish): make search provider permissive, attribute errors (#31997) 2026-07-03 10:17:11 -07:00
spend_tracking_tests chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
store_model_in_db_tests style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_litellm fix(proxy): keep serving reads from the read replica when the primary DB is down at startup (#31951) 2026-07-04 12:45:09 -07:00
unified_google_tests fix(google_genai): preserve complete SSE events in Vertex/Gemini image streaming (#30270) 2026-06-12 07:49:30 -07:00
vector_store_tests Revert "chore(tests): migrate Bedrock CI to AWS account 941277531214 (#28728)" (#29326) 2026-05-30 11:26:24 -07:00
windows_tests ci: reproduce default-Windows wheel install to guard MAX_PATH (#29597) 2026-06-03 11:28:08 -07:00
__init__.py
_fake_openai_endpoint_server.py test: point router/completion/triton tests at the local fake OpenAI endpoint (#30900) 2026-06-20 16:20:35 -07:00
_flush_vcr_cache.py tests(vcr): isolate cassette redis to CASSETTE_REDIS_URL 2026-05-01 12:32:59 -07:00
_live_test_helpers.py test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
_openai_record_replay_proxy.py Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic (#29847) 2026-06-06 14:33:42 -07:00
_vcr_conftest_common.py test(vcr): close out the remaining VCR live-call leaks (#29603) 2026-06-03 13:46:43 -07:00
_vcr_redis_persister.py test(vcr): stop refreshing cassette TTL on read so cassettes lapse after 24h (#29784) 2026-06-05 10:22:41 -07:00
eval_swe_bench.py Prompt Compression - add it to the proxy (#25729) 2026-04-20 15:08:00 -07:00
fake_openai_endpoint.py test: point router/completion/triton tests at the local fake OpenAI endpoint (#30900) 2026-06-20 16:20:35 -07:00
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json test: add e2e tests for spend, budgets and llms (#30869) 2026-06-24 15:01:57 -07:00
README.MD
test_anthropic_compaction_usage.py chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
test_budget_management.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_callbacks_on_proxy.py test(callbacks): harden flaky proxy callback-leak detector (#28195) 2026-05-18 16:39:02 -07:00
test_config.py
test_debug_warning.py
test_default_encoding_non_root.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_end_users.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_entrypoint.py
test_fallbacks.py
test_gpt5_azure_temperature_support.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_health.py fix(tests): swap dall-e to gpt-image-1 after openai deprecation 2026-05-12 16:55:18 -07:00
test_keys.py test: harden remaining pass-through CI flakes (image-gen spend poll, ruby assistants timeout) (#30685) 2026-06-17 14:35:47 -07:00
test_litellm_proxy_responses_config.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_logging.conf
test_models.py test: replace test_add_and_delete_models integration test with mock 2026-03-30 21:30:57 -07:00
test_new_vector_store_endpoints.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_openai_endpoints.py Extend the record/replay proxy to chat, embeddings, moderations, rerank, and Anthropic (#29847) 2026-06-06 14:33:42 -07:00
test_organizations.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_otel_thread_leak.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_passthrough_endpoints.py
test_presidio_latency.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_proxy_server_non_root.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_ratelimit.py chore(ci): modernize model references in tests and configs (#27856) 2026-05-15 15:44:28 -07:00
test_resource_cleanup.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_service_logger_otel.py
test_spend_logs.py Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
test_team.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_team_logging.py test: cleanup dead tests 2026-03-28 20:49:02 -07:00
test_team_members.py Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
test_users.py Fix: tag budget reset must drop stale management-cache entry (#27568) 2026-05-10 00:18:55 +00:00

README.MD

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.