yuneng-jiang 39a580aa91 test(guardrails): stop five guardrail test files leaking env vars on failure (#37828) ... Onyx, prompt security, hiddenlayer, repelloai and deepkeep all write straight to os.environ and unset again at the bottom of each test. None of the five has a try/finally, so the moment a test fails it returns to the runner with the keys still set and whatever runs next in that worker inherits them. Raising inside test_onyx_guard_with_custom_timeout_from_kwargs on the current files leaves ONYX_API_BASE and ONYX_API_KEY behind; doing the same in test_hiddenlayer_config_saas leaves HIDDENLAYER_API_BASE. Both come back clean after this. 89 raw writes and the hand-rolled deletes become monkeypatch calls. The class-level setup_method and teardown_method pair in the onyx file, sweeping the same three keys twice, becomes one autouse fixture. The sys.path.insert lines and their now-unused imports go too, and litellm.set_verbose = True, which only turned global debug logging on for whatever ran next, is dropped rather than restored. test_onyx_guard_config and test_prompt_security_guard_config asserted nothing at all, so they could only fail by raising. Each now pins what init_guardrails_v2 produces: exactly one guardrail of the right class on litellm.callbacks, carrying the configured name, default_on and hook. The zero-assert tests in the other three are left alone; those are a judgement about each guardrail rather than a mechanical sweep. tests/test_litellm/proxy/guardrails passes at 2873.		2026-08-21 21:29:31 -07:00
..
_experimental/mcp_server	Merge pull request #37824 from BerriAI/litellm_retire_dead_test_mirror	2026-08-21 20:12:56 -07:00
a2a	fix(a2a): reject malformed protocolVersion suffixes while keeping semver prereleases	2026-07-21 13:24:58 -07:00
agent_endpoints	fix(a2a): return SSE (text/event-stream) for message/stream instead of NDJSON (#35037)	2026-08-20 16:19:58 -07:00
analytics_endpoints	feat(ui): split failed requests into their own series on the cache dashboard (#34862)	2026-07-29 09:48:17 -07:00
anthropic_endpoints	fix(proxy): estimate failed-request input tokens on /v1/messages and count system prompts	2026-08-18 14:21:47 -07:00
auth	test: unwind environment writes in tests/test_litellm with monkeypatch (#37806)	2026-08-21 20:28:37 -07:00
batches_endpoints	test(lint): ban blind pytest.raises(Exception) with ruff B017 (#37731)	2026-08-20 18:09:42 -07:00
client	test: say whether a match= pattern is a regex or a literal (ruff RUF043)	2026-08-21 16:25:33 -07:00
common_utils	test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769)	2026-08-20 20:24:49 -07:00
config_resolvers	feat(proxy): add SAML 2.0 SSO for the admin UI (#31429)	2026-07-24 12:51:28 -07:00
credential_endpoints	fix(proxy): return the real status code when a credential update is rejected (#36166)	2026-08-07 08:02:52 +00:00
db	test: say whether a match= pattern is a regex or a literal (ruff RUF043)	2026-08-21 16:25:33 -07:00
discovery_endpoints	test: remove tests that never execute	2026-08-12 10:45:38 -07:00
enterprise_billing	feat(proxy): push-based OTLP billable-request metering for enterprise deployments (#31592)	2026-07-15 12:12:52 -07:00
experimental/mcp_server
fine_tuning_endpoints	fix(proxy): authenticate managed ids before routing	2026-08-08 02:42:34 -07:00
google_endpoints
guardrails	test(guardrails): stop five guardrail test files leaking env vars on failure (#37828)	2026-08-21 21:29:31 -07:00
health_endpoints	test: enforce F811 so a duplicate definition cannot silently replace the first	2026-08-21 12:06:19 -07:00
hooks	test: unwind environment writes in tests/test_litellm with monkeypatch (#37806)	2026-08-21 20:28:37 -07:00
image_endpoints	fix(proxy)!: share one destination check between body and path-supplied model	2026-08-05 15:17:43 -07:00
logging_endpoints
management_endpoints	test(key-management): unwind the global writes the key tests scaffold around (#37822)	2026-08-21 21:09:27 -07:00
management_helpers	test(audit-logs): let monkeypatch own the audit log and s3 callback globals (#37842)	2026-08-21 20:16:10 -07:00
memory	fix(proxy): give proxy_admin_viewer read parity with proxy_admin (#35851)	2026-08-05 18:33:55 +00:00
middleware	feat(proxy): add Amazon Comprehend Medical passthrough provider	2026-08-17 15:44:06 -07:00
ocr_endpoints	fix(ocr): validate body req_format in the proxy endpoint and run its tests in CI	2026-08-17 18:29:35 +00:00
openai_files_endpoint	fix(proxy): scan batch records with the content hooks that are not guardrails (#37786)	2026-08-21 11:20:23 -07:00
pass_through_endpoints	Merge pull request #37878 from BerriAI/litellm_ruff_no_duplicate_definitions	2026-08-21 12:49:43 -07:00
policy_engine	test(callbacks): unwind the callbacks global the policy engine and realtime tests scaffold around (#37826)	2026-08-21 21:19:35 -07:00
prompts	test: enforce F811 so a duplicate definition cannot silently replace the first	2026-08-21 12:06:19 -07:00
proxy_server	test: say whether a match= pattern is a regex or a literal (ruff RUF043)	2026-08-21 16:25:33 -07:00
public_endpoints	feat(cognition): give Cognition its own provider identity	2026-08-20 17:16:53 -07:00
rag_endpoints	fix(rag): track LLM completion usage and spend for /v1/rag/query (#32438)	2026-07-17 17:45:27 +00:00
realtime_endpoints	fix(realtime): bound Vertex credential resolution and make realtime failures loud	2026-08-20 02:02:00 -07:00
response_api_endpoints	fix(responses_api): map bridged chat usage on guardrail-blocked replies	2026-08-14 17:04:27 -07:00
shutdown
spend_tracking	feat(spend): surface per-request auto-router savings to logging callbacks (#37894)	2026-08-21 15:41:35 -07:00
test_configs
types_utils	fix(proxy): resolve router_settings.plugins dotted paths and load plugins from installed packages (#33644)	2026-07-17 11:23:18 -07:00
ui_crud_endpoints	feat(ui): let admins supply a dark-mode variant of their custom logo (#37662)	2026-08-20 12:48:55 -07:00
utils	fix(guardrails): run policy pipelines when the caller sends its own metadata (/v1/messages, Claude Code) (#36889)	2026-08-21 13:04:04 -07:00
vector_store_endpoints	test: run the 30 test files stranded in the second mirror (#37595)	2026-08-20 10:59:43 -07:00
vector_store_files_endpoints	fix(proxy): authenticate managed ids before routing	2026-08-08 02:42:34 -07:00
video_endpoints
__init__.py
conftest.py	test(proxy): keep a leaked llm_router out of the next test in the worker	2026-08-20 23:19:17 -07:00
test_aiohttp_cleanup_closed.py
test_aiohttp_session_recovery.py
test_api_key_masking_in_errors.py
test_audio_speech_prometheus_hooks.py
test_batch_expiry.py
test_batch_metadata_none_fix.py
test_batch_retrieve_bedrock.py
test_batch_x_litellm_model_encoding.py	test(proxy): give the x-litellm-model fallback test deterministic openai env creds	2026-08-17 12:42:57 -07:00
test_blocked_response_usage.py	fix(responses_api): map bridged chat usage on guardrail-blocked replies	2026-08-14 17:04:27 -07:00
test_budget_reservation.py	test: enforce PT012 so a pytest.raises block cannot hide dead assertions (#37748)	2026-08-20 19:36:26 -07:00
test_caching_routes.py
test_chat_completion_metadata.py
test_claude_code_marketplace.py	test: run the 30 test files stranded in the second mirror (#37595)	2026-08-20 10:59:43 -07:00
test_common_request_processing.py	Merge pull request #37878 from BerriAI/litellm_ruff_no_duplicate_definitions	2026-08-21 12:49:43 -07:00
test_component_allowlists.py	feat(proxy): authenticate to Azure Postgres with Microsoft Entra ID tokens	2026-08-20 11:50:16 -07:00
test_conftest.py	test(proxy): stop monkeypatch.undo re-planting fixture-mocked prisma_client	2026-08-13 20:11:06 -07:00
test_cors_config.py
test_custom_proxy.py
test_dynamic_mcp_route.py	fix(proxy): serve aggregate MCP endpoint on bare /mcp instead of 307-redirecting (#34845)	2026-08-14 17:04:32 -07:00
test_empty_model_list.py
test_enforce_user_param.py	test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769)	2026-08-20 20:24:49 -07:00
test_fallback_management_endpoints.py
test_fastapi_offline_routes.py
test_filter_models_by_team_access_group.py
test_health_check_functions.py
test_health_check_max_tokens.py
test_init_litellm_callbacks.py	test: run the 30 test files stranded in the second mirror (#37595)	2026-08-20 10:59:43 -07:00
test_langfuse_passthrough_security.py
test_lazy_openapi_snapshot.py	feat(proxy): add GET /v1/indexes to list vector store indexes (#36289)	2026-08-10 15:09:59 -07:00
test_litellm_pre_call_utils.py	fix(proxy): group Codex turns under one session id (#37895)	2026-08-21 16:04:36 -07:00
test_max_budget_env_var.py
test_mcp_asgi_response.py
test_model_based_routing_files_batches.py
test_model_deprecations_endpoint.py	refactor(proxy): tighten model deprecation typing and cover the endpoint	2026-08-10 22:58:35 +00:00
test_model_dump_with_preserved_fields.py
test_model_id_header_propagation.py	feat(proxy): add x-litellm-model-name response header with deployment model string (#33698)	2026-07-17 20:29:42 -07:00
test_model_info_default_limits.py
test_model_level_guardrails.py	fix(proxy): apply key/team router_settings.model_group_alias (#35486)	2026-08-03 22:09:47 +00:00
test_model_list_healthy_only.py
test_modify_response_streaming_passthrough.py
test_openai_ws_passthrough_routes.py	fix(proxy): close websocket cleanly when OpenAI credentials are missing	2026-08-16 14:40:37 -07:00
test_openapi_schema_validation.py
test_plugin_routes.py
test_pricing_field_strip.py	fix(guardrails): bill all chunks on mid-chunking block, strip client guardrail cost metadata, add cost map schema keys	2026-08-18 14:52:23 -07:00
test_prisma_engine_watchdog.py	test: run the 30 test files stranded in the second mirror (#37595)	2026-08-20 10:59:43 -07:00
test_prisma_migration.py	fix(proxy): fail the standalone prisma migration entrypoint on migration errors (#37692)	2026-08-20 16:20:27 -07:00
test_prometheus_cleanup.py
test_provider_url_destination_guard.py	fix(proxy)!: share one destination check between body and path-supplied model	2026-08-05 15:17:43 -07:00
test_proxy_cli.py	fix(db): apply the configured connection params to the read replica URL (#37691)	2026-08-20 17:23:06 -07:00
test_proxy_logging_hook_detection.py	test: enforce PT012 so a pytest.raises block cannot hide dead assertions (#37748)	2026-08-20 19:36:26 -07:00
test_proxy_server.py	test: say whether a match= pattern is a regex or a literal (ruff RUF043)	2026-08-21 16:25:33 -07:00
test_proxy_types.py	test(proxy): pin the request-body rules proxy/_types.py enforces (#37811)	2026-08-21 20:15:13 -07:00
test_proxy_utils.py	test(proxy): pin what a failed request records as usage and spend (#37813)	2026-08-21 20:15:32 -07:00
test_pyroscope.py
test_read_model_list.py
test_redis_auth_cache_flag.py	test: add six ruff rules that catch tests which cannot fail (#37709)	2026-08-20 14:21:26 -07:00
test_response_model_sanitization.py
test_route_a2a_models.py	fix(proxy): single-row read-through resyncs and reload-race hardening	2026-08-18 21:02:12 -07:00
test_route_llm_request.py	test: enforce PT012 so a pytest.raises block cannot hide dead assertions (#37748)	2026-08-20 19:36:26 -07:00
test_sensitive_route_auth.py
test_shared_health_check.py
test_spend_log_cleanup.py	test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769)	2026-08-20 20:24:49 -07:00
test_swagger_chat_completions.py
test_team_member_update.py
test_team_org_move.py	test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769)	2026-08-20 20:24:49 -07:00
test_tools_allowlist_enforcement.py	fix(responses-bridge): custom tool round-trip and allowlist preservation for Codex CLI (#32258)	2026-07-06 17:34:27 -07:00
test_update_llm_router_resilience.py	feat(complexity_router): custom classifier plugins via classifier_type 'custom' (#37249)	2026-08-18 14:09:19 -07:00