litellm/tests
2026-09-03 10:32:03 -07:00
..
agent_tests test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
audio_tests test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
base_sdk_tests
basic_proxy_startup_tests
batches_tests merge(litellm_internal_staging): reconcile batch observability with per-line resilience 2026-08-24 19:08:40 -04:00
benchmarks
code_coverage_tests Merge pull request #38765 from BerriAI/litellm_ocr_sdk_parity_tests 2026-09-03 10:16:35 -07:00
documentation_tests Merge remote-tracking branch 'origin/main' into litellm_bedrock_messages_disconnect_billing 2026-08-31 08:58:37 -07:00
e2e style: drop explanatory comments from the agent publish fix 2026-09-03 04:27:43 -07:00
enterprise fix(prometheus): bound requested_model label cardinality on client failure paths 2026-09-01 10:34:58 -07:00
guardrails_tests test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
image_gen_tests test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
integration test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
litellm-proxy-extras fix(proxy-extras): only spend a migrate-deploy attempt when a pass made no progress 2026-09-02 23:56:09 -07:00
litellm_utils_tests test(aiohttp): pin NO_PROXY so proxy env cannot hijack the refused-port probe 2026-08-28 12:45:46 -07:00
llm_responses_api_testing fix(guardrails): deliver modify_response block as valid SSE on streaming chat and Responses 2026-08-31 16:01:39 -07:00
llm_translation test: exempt MockTransport request-shape embedding tests from VCR replay 2026-09-01 13:29:12 -07:00
load_tests Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_ruff_dead_test_code 2026-08-24 09:46:56 -07:00
local_testing test(timeout): time out against the local fake endpoint instead of api.openai.com 2026-09-03 09:53:30 -07:00
logging_callback_tests fix(test): drop the duplicate embedding_executor arg in the Bedrock KB fake handler 2026-09-02 23:22:40 -07:00
mcp_tests fix(mcp): scope allow-all servers to virtual keys (#39531) 2026-09-03 10:32:03 -07:00
multi_instance_e2e_tests test: say whether a match= pattern is a regex or a literal (ruff RUF043) 2026-08-21 16:25:33 -07:00
ocr_tests fix(vertex): avoid duplicate DeepSeek OCR model namespace 2026-09-01 14:39:15 -07:00
openai_endpoints_tests test(responses): expect the 404 OpenAI now returns for an unknown model 2026-09-02 18:08:45 -07:00
otel_tests test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
pass_through_tests fix(anthropic_messages): dispatch deferred spend logging when the client disconnects mid-relay 2026-08-31 10:17:27 -07:00
pass_through_unit_tests test(websearch): carry a reasoned test-quality suppression on the router patch 2026-08-31 22:32:05 -07:00
proxy_admin_ui_tests test(access-groups): give each xdist worker its own fixture ids 2026-08-28 09:35:12 -07:00
proxy_behavior fix(proxy): reset a stuck team member's budget (#37971) 2026-08-25 09:50:09 -07:00
proxy_e2e_anthropic_messages_tests
proxy_migration_tests test(proxy-extras): use the modern optional annotation in the deploy budget test 2026-09-02 12:36:29 -07:00
proxy_security_tests
proxy_unit_tests fix(proxy): share per-model budget counters across replicas through the spend counter cache (#39375) 2026-09-02 12:40:59 -07:00
router_unit_tests Merge pull request #39474 from BerriAI/litellm_s3_vectors_query_embedding_executor 2026-09-02 22:23:44 -07:00
rust-python-harness test(ocr): record provider fixtures in the migration harness 2026-09-03 10:16:40 -07:00
sdk_function_trace refactor(python-bridge): split routes and add shared function tracing (#39031) 2026-09-02 16:26:35 -07:00
search_tests fix(search): harden bing_grounding auth, result cap, status, and cost 2026-08-24 12:26:07 -07:00
spend_tracking_tests
store_model_in_db_tests test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
test_litellm fix(mcp): scope allow-all servers to virtual keys (#39531) 2026-09-03 10:32:03 -07:00
unified_google_tests test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
vector_store_tests fix(vector-store): carry request metadata into the Router executor built from the router kwarg 2026-09-02 16:53:05 -07:00
windows_tests test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
__init__.py
_fake_openai_endpoint_server.py test(timeout): time out against the local fake endpoint instead of api.openai.com 2026-09-03 09:53:30 -07:00
_flush_vcr_cache.py
_live_test_helpers.py
_openai_record_replay_proxy.py
_vcr_conftest_common.py
_vcr_redis_persister.py
_wait_helpers.py
_ws_vcr.py
eval_swe_bench.py
fake_openai_endpoint.py
gettysburg.wav
large_text.py
openai_batch_completions.jsonl
pyrightconfig.json
README.MD
test_anthropic_compaction_usage.py
test_budget_management.py
test_callbacks_on_proxy.py test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
test_debug_warning.py
test_default_encoding_non_root.py
test_end_users.py
test_fallbacks.py test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
test_gpt5_azure_temperature_support.py
test_health.py
test_keys.py test: fix staging CI regressions from #38182, #38144, #38265, #37962, and #37969 2026-08-25 23:01:20 -07:00
test_litellm_proxy_responses_config.py
test_logging.conf
test_models.py
test_new_vector_store_endpoints.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_openai_endpoints.py
test_organizations.py
test_otel_thread_leak.py
test_presidio_latency.py
test_proxy_server_non_root.py
test_ratelimit.py test: drop the cwd-relative sys.path.insert calls from the test suite (#37802) 2026-08-22 09:25:58 -07:00
test_resource_cleanup.py
test_rust_python_harness.py Merge pull request #38765 from BerriAI/litellm_ocr_sdk_parity_tests 2026-09-03 10:16:35 -07:00
test_service_logger_otel.py
test_spend_logs.py
test_team.py
test_team_logging.py test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00
test_team_members.py test: reject assertions on a caught error inside except (ruff PT017) 2026-08-21 13:35:08 -07:00
test_users.py test: enforce F811 so a duplicate definition cannot silently replace the first 2026-08-21 12:06:19 -07:00

In total litellm runs 1000+ tests

[02/20/2025] Update:

To make it easier to contribute and map what behavior is tested,

we've started mapping the litellm directory in tests/test_litellm

This folder can only run mock tests.