scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-28 01:32:17 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 43
litellm/tests
History
mateo-berri a9ad3eaf95 fix(router): add NotFoundErrorRetries so a retry policy can pin 404 retries 
RetryPolicy had no field for 404s, so any policy that set DefaultRetries made
the router retry every 404 across the pool, including OpenAI's 404 on a missing
response id, which arrives as a BadRequestError whose status_code is 404

NotFoundErrorRetries now governs every answer whose status code is 404 whatever
exception class the mapping picked, ahead of the class walk and DefaultRetries.
A 404 without it still falls back to BadRequestErrorRetries for the
BadRequestError shape and then to DefaultRetries, so existing policies keep
their behavior until the new field is set. The Admin UI retry settings tab
gains a NotFoundError (404) row above the catch-all row

Fixes #36896
2026-09-19 16:25:47 -07:00
..
agent_tests fix(a2a): keep the upstream status on card discovery failures and inject the card client in tests 2026-09-16 23:55:57 +00:00
audio_tests
base_sdk_tests test(mcp): add isolated SDK2 dependency compatibility gate 2026-09-17 17:16:47 -07:00
basic_proxy_startup_tests
batches_tests feat(batches): enrich batch cost rows with breakdown, identity, session, and org spend 2026-09-03 17:23:53 -04:00
benchmarks
code_coverage_tests test(e2e): isolate diagnostic reporter subprocess 2026-09-19 15:13:27 -07:00
documentation_tests fix(docs-test): restore line-anchored table regex in router settings check 2026-09-18 20:10:20 +00:00
e2e Merge pull request #41909 from BerriAI/litellm_mcp_oauth_happy_path_e2e 2026-09-19 16:08:51 -07:00
enterprise fix(projects): persist explicit budget cap clears 2026-09-12 13:43:55 -07:00
guardrails_tests Revert "Merge pull request #41220 from BerriAI/litellm_post_call_guardrail_context" 2026-09-19 17:45:15 +00:00
image_gen_tests test: remove fully commented-out test files that collect no tests 2026-09-17 20:05:27 +00:00
integration test(mcp): align live regressions with discovery and error contracts 2026-09-19 13:03:00 -07:00
litellm-proxy-extras test(proxy-extras): move the LITELLM_LOG logger test into the CI-run proxy-extras shard 2026-09-15 22:57:23 +00:00
litellm_utils_tests Merge remote-tracking branch 'origin/main' into litellm_invalid_tool_choice_400 2026-09-19 02:35:29 -07:00
llm_responses_api_testing test: fix five tests left stale by #41311, #41337, #39996 and #41310 2026-09-16 17:47:27 -07:00
llm_translation test(bedrock): expect canonical session tags in the dynamic auth params propagation test 2026-09-18 11:17:34 -07:00
load_tests feat(proxy): per-worker admission control that rejects excess requests with 503 (#39352) 2026-09-03 18:19:04 -07:00
local_testing test(llmguard): move call type alias tests to the mapped enterprise test file 2026-09-19 07:15:51 +00:00
logging_callback_tests test(logging): add azure_spillover to the GCS pub/sub spend-log golden 2026-09-17 18:33:42 -07:00
mcp_tests test(mcp): cover scoped execution and OAuth credential isolation 2026-09-19 12:14:32 -07:00
multi_instance_e2e_tests
ocr_tests test(ocr): replace per-provider OCR test classes with a declarative provider x auth x input matrix 2026-09-18 21:27:03 +00:00
openai_endpoints_tests test(responses): fix stale Anthropic smoke request 2026-09-11 13:56:27 -07:00
otel_tests feat(cli): deprecate the litellm-proxy entrypoint in favour of lite 2026-09-17 14:05:31 -07:00
pass_through_tests fix(mcp): preserve legacy behavior on SDK2 and streamline verification 2026-09-18 22:28:31 -07:00
pass_through_unit_tests test(pass_through): shorten protocol-constrained route docstring 2026-09-17 00:24:29 +00:00
proxy_admin_ui_tests refactor(tests): assign the streamed id and lock poll once instead of rebinding 2026-09-03 13:49:43 -07:00
proxy_behavior fix(proxy): enforce org budget ceilings on /team/update 2026-09-16 15:38:30 -07:00
proxy_e2e_anthropic_messages_tests
proxy_migration_tests fix(bedrock): support aws-sdk-bedrock-runtime 0.10 and 0.11 in the realtime handler 2026-09-17 01:40:52 +00:00
proxy_security_tests
proxy_unit_tests Merge pull request #40243 from zoroyihan7/fix-responses-stream-error-events 2026-09-18 17:29:57 -07:00
router_unit_tests test(router): force the callback to observe the usage stamp before the pre-header increment fails 2026-09-16 20:10:22 +00:00
rust-python-harness test: drop tests that pin provider-owned cost map values 2026-09-18 03:55:51 +00:00
search_tests test: remove fully commented-out test files that collect no tests 2026-09-17 20:05:27 +00:00
spend_tracking_tests
store_model_in_db_tests test(proxy): expect the sanitized unknown-model message in the spend-log error test 2026-09-11 19:18:10 -07:00
test_gateway feat(proxy): offload spend tracking to a pod-local collector sidecar (#40545) 2026-09-10 17:14:13 -07:00
test_litellm fix(router): add NotFoundErrorRetries so a retry policy can pin 404 retries 2026-09-19 16:25:47 -07:00
test_litellm_rust test(rust): pin child interpreters to the parent's litellm and lint for it 2026-09-19 18:17:54 +00:00
unified_google_tests test(unified_google_tests): import ReadOnly from typing_extensions and cover the Vertex global endpoint 2026-09-19 12:15:34 -07:00
vector_store_tests
windows_tests
__init__.py
_fake_openai_endpoint_server.py
_flush_vcr_cache.py
_live_test_helpers.py refactor(test): validate cost-map entries into a typed model 2026-09-16 14:32:08 -07:00
_openai_record_replay_proxy.py
_vcr_conftest_common.py
_vcr_redis_persister.py
_wait_helpers.py
_ws_vcr.py
eval_swe_bench.py
fake_openai_endpoint.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json
README.MD
test_anthropic_compaction_usage.py
test_budget_management.py
test_callbacks_on_proxy.py
test_debug_warning.py
test_default_encoding_non_root.py
test_end_users.py
test_fallbacks.py
test_gpt5_azure_temperature_support.py
test_health.py
test_keys.py
test_litellm_proxy_responses_config.py
test_logging.conf
test_models.py test: address review notes on the chronic-test repairs 2026-09-04 10:18:15 -07:00
test_new_vector_store_endpoints.py
test_openai_endpoints.py test: fix five tests left stale by #41311, #41337, #39996 and #41310 2026-09-16 17:47:27 -07:00
test_organizations.py
test_otel_thread_leak.py
test_presidio_latency.py
test_proxy_server_non_root.py
test_ratelimit.py
test_resource_cleanup.py
test_rust_python_harness.py refactor(rust): remove gateway, config, router, realtime, and trace-parity infrastructure 2026-09-16 16:00:07 +00:00
test_service_logger_otel.py
test_spend_logs.py
test_team.py
test_team_logging.py
test_team_members.py
test_users.py

README.MD

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.