mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-21 00:21:49 +00:00
bf9c717d77
Two tests in the config/misc management suite were failing every run against the Buildkite e2e stack, and one of them took the rest of the build with it. test_add_allowed_ip_does_not_store_unrelated_config_value posted 127.0.0.1 to /add/allowed_ip. That route sets the live general_settings["allowed_ips"] that auth_utils._check_valid_ip reads before it persists anything, and the check is exact string membership with no CIDR support, so from the moment the POST returns only 127.0.0.1 can reach the proxy. The runner 403s on its very next call, and the deferred /delete/allowed_ip sits behind the same auth dependency, so the cleanup is locked out too and every later test in the build 403s. Build 254's first attempt lost 459 of its 465 failures to that one cascade. There is no safe way to exercise the route against a shared proxy: nothing reports the caller's address as the proxy sees it, so a test cannot allowlist itself first. Move the claim to the route's own TestClient suite, where the auth dependency is overridden and general_settings is per-test, and record the route in the module docstring beside /cache/settings and the Vault override so it is not re-added. save_config's end of the contract was already covered by test_ProxyConfig_save_config_merges_changed_keys_without_copying_file_settings; the new test covers the route's end, that what it hands save_config differs from the loaded config in allowed_ips and nothing else. The unrelated-key probe also only ever worked on one lane: max_parallel_requests was added to tests/e2e/gateway/stage_mirror_ci_config.yml and never to the Buildkite stack's config, where resolve() reports it as "unset" rather than "config". That key is now unused, so drop it again. test_config_update_persists_router_setting_to_get wrote router_settings. num_retries, which both lanes declare in their config file, so the config- ownership work correctly refuses it with a 400. Switch to retry_after, which is declared by neither lane, is accepted by /config/update, and is reported back by GET /router/settings. Verified against a live proxy: max_fallbacks also takes the write but never reads back, so the read-back poll is what picks the key. |
||
|---|---|---|
| .. | ||
| agent_tests | ||
| audio_tests | ||
| base_sdk_tests | ||
| basic_proxy_startup_tests | ||
| batches_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| enterprise | ||
| guardrails_tests | ||
| image_gen_tests | ||
| integration | ||
| litellm-proxy-extras | ||
| litellm_utils_tests | ||
| llm_responses_api_testing | ||
| llm_translation | ||
| load_tests | ||
| local_testing | ||
| logging_callback_tests | ||
| mcp_tests | ||
| multi_instance_e2e_tests | ||
| ocr_tests | ||
| openai_endpoints_tests | ||
| otel_tests | ||
| pass_through_tests | ||
| pass_through_unit_tests | ||
| proxy_admin_ui_tests | ||
| proxy_behavior | ||
| proxy_e2e_anthropic_messages_tests | ||
| proxy_migration_tests | ||
| proxy_security_tests | ||
| proxy_unit_tests | ||
| router_unit_tests | ||
| rust-python-harness | ||
| search_tests | ||
| spend_tracking_tests | ||
| store_model_in_db_tests | ||
| test_gateway | ||
| test_litellm | ||
| test_litellm_rust | ||
| unified_google_tests | ||
| vector_store_tests | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _openai_record_replay_proxy.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| _wait_helpers.py | ||
| _ws_vcr.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_anthropic_compaction_usage.py | ||
| test_budget_management.py | ||
| test_callbacks_on_proxy.py | ||
| test_debug_warning.py | ||
| test_default_encoding_non_root.py | ||
| test_end_users.py | ||
| test_fallbacks.py | ||
| test_gpt5_azure_temperature_support.py | ||
| test_health.py | ||
| test_keys.py | ||
| test_litellm_proxy_responses_config.py | ||
| test_logging.conf | ||
| test_models.py | ||
| test_new_vector_store_endpoints.py | ||
| test_openai_endpoints.py | ||
| test_organizations.py | ||
| test_otel_thread_leak.py | ||
| test_presidio_latency.py | ||
| test_proxy_server_non_root.py | ||
| test_ratelimit.py | ||
| test_resource_cleanup.py | ||
| test_rust_python_harness.py | ||
| test_service_logger_otel.py | ||
| test_spend_logs.py | ||
| test_team.py | ||
| test_team_logging.py | ||
| test_team_members.py | ||
| test_users.py | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.