mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-24 00:52:24 +00:00
65e42526d6
* test(proxy): make two proxy-infra tests independent of sibling-test state Both tests read process-global state that another module in the same xdist worker can change, so they passed or failed on shard scheduling rather than on the behaviour they assert. test_gateway_plus_backend_covers_full_app checked allowlist coverage against the live route table. gateway/main.py trims routes once, inside the lifespan, so it only ever sees what is registered at startup; a lazy feature appends its router on demand afterwards and is never filtered. Those routes cannot be dropped on the floor, but they do enter the assertion the moment a sibling test warms the feature, and 110 of them sit in neither allowlist. Subtract exactly the lazy features this process has loaded, which leaves the assertion at full strength for every eagerly registered route. test_real_proxy_child_auth_privacy_and_body_policy pins prisma_client to a bare object(). litellm.max_budget is a module global that nothing restores between tests; once a sibling leaves it above zero, user_api_key_auth takes the global-spend branch, dereferences prisma_client.db, and the AttributeError surfaces as HTTP 401. Pin max_budget next to the other globals the test already controls. * test(proxy): measure allowlist coverage in a pristine interpreter The previous revision subtracted paths matching a loaded lazy feature's prefixes. Those prefixes are broad enough to swallow eagerly registered routes: 31 of them, including /openai/deployments/*, /access_group/*, /cursor/* and /mcp, which would have made a real allowlist regression invisible. Run the coverage check in a fresh interpreter instead. No lazy feature is loaded there, so the route table is exactly the one gateway/main.py's lifespan trim sees, and nothing has to be subtracted for the result to be deterministic. The probe also reports the lazy modules it loaded and its route count, so an empty uncovered set cannot pass vacuously. Drop _component_paths and the four allowlist constants it used; the probe reproduces the predicate in the child process. |
||
|---|---|---|
| .. | ||
| agent_tests | ||
| audio_tests | ||
| base_sdk_tests | ||
| basic_proxy_startup_tests | ||
| batches_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| enterprise | ||
| guardrails_tests | ||
| image_gen_tests | ||
| integration | ||
| litellm-proxy-extras | ||
| litellm_utils_tests | ||
| llm_responses_api_testing | ||
| llm_translation | ||
| load_tests | ||
| local_testing | ||
| logging_callback_tests | ||
| mcp_tests | ||
| multi_instance_e2e_tests | ||
| ocr_tests | ||
| openai_endpoints_tests | ||
| otel_tests | ||
| pass_through_tests | ||
| pass_through_unit_tests | ||
| proxy_admin_ui_tests | ||
| proxy_behavior | ||
| proxy_e2e_anthropic_messages_tests | ||
| proxy_migration_tests | ||
| proxy_security_tests | ||
| proxy_unit_tests | ||
| router_unit_tests | ||
| rust-python-harness | ||
| search_tests | ||
| spend_tracking_tests | ||
| store_model_in_db_tests | ||
| test_gateway | ||
| test_litellm | ||
| test_litellm_rust | ||
| unified_google_tests | ||
| unit | ||
| vector_store_tests | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _openai_record_replay_proxy.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| _wait_helpers.py | ||
| _ws_vcr.py | ||
| AGENTS.md | ||
| capturing_transport.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_anthropic_compaction_usage.py | ||
| test_budget_management.py | ||
| test_callbacks_on_proxy.py | ||
| test_debug_warning.py | ||
| test_default_encoding_non_root.py | ||
| test_end_users.py | ||
| test_fallbacks.py | ||
| test_gpt5_azure_temperature_support.py | ||
| test_health.py | ||
| test_keys.py | ||
| test_litellm_proxy_responses_config.py | ||
| test_logging.conf | ||
| test_models.py | ||
| test_new_vector_store_endpoints.py | ||
| test_openai_endpoints.py | ||
| test_organizations.py | ||
| test_otel_thread_leak.py | ||
| test_presidio_latency.py | ||
| test_proxy_server_non_root.py | ||
| test_ratelimit.py | ||
| test_resource_cleanup.py | ||
| test_rust_python_harness.py | ||
| test_service_logger_otel.py | ||
| test_spend_logs.py | ||
| test_team.py | ||
| test_team_logging.py | ||
| test_team_members.py | ||
| test_users.py | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.