Yucheng Zhu 792f120aef fix(observability): drop the shed mark when a rejection is recovered from ... The mark is set when `ProxyRateLimitError` is constructed, which is the one point every raise site passes through. That also means a rejection the proxy recovers from stays marked: `_pre_call_with_fallbacks` catches the local rate limit and retries against a fallback model, so a provider 429 on that fallback was counted as this pod shedding load. That is the exact conflation the mark exists to prevent, reintroduced through the back door. The mark is now cleared before each fallback attempt, so only a rejection this proxy actually returns counts. The restore before `raise original_exc` states the postcondition at the raise site. Current control flow already satisfies it, because reaching that line means every fallback raised its own `ProxyRateLimitError` and re-marked on construction, so no test distinguishes it. It is kept so the guarantee does not depend on that incidental re-marking.		2026-08-19 18:59:03 -07:00
..
_experimental/mcp_server	fix(mcp): stop reporting failed OpenAPI tool calls as successes (#37496)	2026-08-19 14:26:02 -07:00
a2a	fix(a2a): reject malformed protocolVersion suffixes while keeping semver prereleases	2026-07-21 13:24:58 -07:00
agent_endpoints	fix(proxy): send SSE keepalives on assistants runs and A2A streams (#37368)	2026-08-18 16:22:05 -07:00
analytics_endpoints	feat(ui): split failed requests into their own series on the cache dashboard (#34862)	2026-07-29 09:48:17 -07:00
anthropic_endpoints	fix(proxy): estimate failed-request input tokens on /v1/messages and count system prompts	2026-08-18 14:21:47 -07:00
auth	fix(auth): resolve bare model names against wildcard deployments in model access groups (#37492)	2026-08-19 15:33:29 -07:00
batches_endpoints	Merge pull request #37198 from BerriAI/litellm_lit5660_batches_limit_400	2026-08-17 15:53:46 -07:00
client	feat(cli): add lite login --config-claude to wire Claude Code at login (#37507)	2026-08-19 15:32:14 -07:00
common_utils	feat(observability): expose per-pod request pressure and the enforced concurrency ceiling	2026-08-19 18:59:03 -07:00
config_resolvers	feat(proxy): add SAML 2.0 SSO for the admin UI (#31429)	2026-07-24 12:51:28 -07:00
credential_endpoints	fix(proxy): return the real status code when a credential update is rejected (#36166)	2026-08-07 08:02:52 +00:00
db	fix(observability): report a swallowed Redis outage as an error, not contention	2026-08-19 18:58:37 -07:00
discovery_endpoints	test: remove tests that never execute	2026-08-12 10:45:38 -07:00
enterprise_billing	feat(proxy): push-based OTLP billable-request metering for enterprise deployments (#31592)	2026-07-15 12:12:52 -07:00
experimental/mcp_server
fine_tuning_endpoints	fix(proxy): authenticate managed ids before routing	2026-08-08 02:42:34 -07:00
google_endpoints
guardrails	Merge pull request #37387 from BerriAI/litellm_guardrail_usage_requeue	2026-08-18 16:42:06 -07:00
health_endpoints	feat(proxy): auto-suppress the no-Redis banner for confirmed single-worker deployments	2026-08-14 17:11:05 -07:00
hooks	fix(batches): stop one bad output line from zeroing an entire batch's spend	2026-08-18 20:46:59 -07:00
image_endpoints	fix(proxy)!: share one destination check between body and path-supplied model	2026-08-05 15:17:43 -07:00
logging_endpoints	feat(proxy): add POST /v1/callbacks/logs to replay logging payloads through callbacks (#31134)	2026-06-24 15:25:10 -07:00
management_endpoints	Merge pull request #37515 from BerriAI/litellm_project_key_all_team_models_sentinel	2026-08-19 14:40:34 -07:00
management_helpers	fix(access groups): sync assigned_team_ids from the team write paths (#36825)	2026-08-14 04:45:36 +00:00
memory	fix(proxy): give proxy_admin_viewer read parity with proxy_admin (#35851)	2026-08-05 18:33:55 +00:00
middleware	fix(observability): document exactly the statuses the shed counter counts	2026-08-19 18:59:03 -07:00
ocr_endpoints	fix(ocr): validate body req_format in the proxy endpoint and run its tests in CI	2026-08-17 18:29:35 +00:00
openai_files_endpoint	test(files): hoist shared batch line fixture into one constant	2026-08-19 15:22:43 -07:00
pass_through_endpoints	fix(proxy): log spend for OpenAI passthrough embeddings with unmapped models	2026-08-18 19:48:02 -07:00
policy_engine	fix(guardrails): scan text on /guardrails/apply_guardrail for Azure Content Safety (#36894)	2026-08-17 11:18:40 -07:00
prompts	fix(proxy): give proxy_admin_viewer read parity with proxy_admin (#35851)	2026-08-05 18:33:55 +00:00
proxy_server	fix(proxy): apply db-backed max_batch_file_size_mb on config reload	2026-08-19 15:37:44 -07:00
public_endpoints	fix(ui): distinguish hosted and local vLLM in the provider dropdown	2026-08-14 13:39:37 -07:00
rag_endpoints	fix(rag): track LLM completion usage and spend for /v1/rag/query (#32438)	2026-07-17 17:45:27 +00:00
realtime_endpoints
response_api_endpoints	fix(responses_api): map bridged chat usage on guardrail-blocked replies	2026-08-14 17:04:27 -07:00
shutdown
spend_tracking	refactor(ptu): give the rollup a source-agnostic deployment record (#37501)	2026-08-19 13:51:31 -07:00
test_configs
types_utils	fix(proxy): resolve router_settings.plugins dotted paths and load plugins from installed packages (#33644)	2026-07-17 11:23:18 -07:00
ui_crud_endpoints	feat(ptu): gate PTU flat-cost attribution behind an opt-in env var (#36138)	2026-08-10 12:23:20 -07:00
utils	fix(observability): correct job duration pairing, lock error reporting and the item count	2026-08-19 18:58:37 -07:00
vector_store_endpoints	fix(vector_stores): classify write endpoints before reads on substring collisions	2026-08-14 16:53:48 -07:00
vector_store_files_endpoints	fix(proxy): authenticate managed ids before routing	2026-08-08 02:42:34 -07:00
video_endpoints	test(videos): add 1:1 test file scaffold for videos component paths (#30631)	2026-06-29 09:12:51 +05:30
__init__.py
conftest.py	test(proxy): stop monkeypatch.undo re-planting fixture-mocked prisma_client	2026-08-13 20:11:06 -07:00
test_aiohttp_cleanup_closed.py
test_aiohttp_session_recovery.py
test_api_key_masking_in_errors.py
test_audio_speech_prometheus_hooks.py
test_batch_expiry.py
test_batch_metadata_none_fix.py
test_batch_retrieve_bedrock.py
test_batch_x_litellm_model_encoding.py	test(proxy): give the x-litellm-model fallback test deterministic openai env creds	2026-08-17 12:42:57 -07:00
test_blocked_response_usage.py	fix(responses_api): map bridged chat usage on guardrail-blocked replies	2026-08-14 17:04:27 -07:00
test_budget_reservation.py	feat(proxy): add apply_user_budget_to_team_keys opt-in (#36102)	2026-08-07 15:40:13 +00:00
test_caching_routes.py
test_chat_completion_metadata.py
test_common_request_processing.py	fix(observability): drop the shed mark when a rejection is recovered from	2026-08-19 18:59:03 -07:00
test_component_allowlists.py	fix(gateway): keep the Prometheus /metrics Mount in the gateway route trim (#32317)	2026-07-07 18:36:38 +03:00
test_conftest.py	test(proxy): stop monkeypatch.undo re-planting fixture-mocked prisma_client	2026-08-13 20:11:06 -07:00
test_cors_config.py
test_custom_proxy.py
test_dynamic_mcp_route.py	fix(proxy): serve aggregate MCP endpoint on bare /mcp instead of 307-redirecting (#34845)	2026-08-14 17:04:32 -07:00
test_empty_model_list.py
test_enforce_user_param.py
test_fallback_management_endpoints.py
test_fastapi_offline_routes.py
test_filter_models_by_team_access_group.py
test_health_check_functions.py
test_health_check_max_tokens.py	fix: skip health check for semantic auto_router deployments (#31668)	2026-06-29 19:47:20 -07:00
test_langfuse_passthrough_security.py
test_lazy_openapi_snapshot.py	feat(proxy): add GET /v1/indexes to list vector store indexes (#36289)	2026-08-10 15:09:59 -07:00
test_litellm_pre_call_utils.py	fix(proxy): stop leaking the client_side_timeout marker to providers	2026-08-18 13:37:07 -07:00
test_max_budget_env_var.py
test_mcp_asgi_response.py
test_model_based_routing_files_batches.py
test_model_deprecations_endpoint.py	refactor(proxy): tighten model deprecation typing and cover the endpoint	2026-08-10 22:58:35 +00:00
test_model_dump_with_preserved_fields.py
test_model_id_header_propagation.py	feat(proxy): add x-litellm-model-name response header with deployment model string (#33698)	2026-07-17 20:29:42 -07:00
test_model_info_default_limits.py
test_model_level_guardrails.py	fix(proxy): apply key/team router_settings.model_group_alias (#35486)	2026-08-03 22:09:47 +00:00
test_model_list_healthy_only.py
test_modify_response_streaming_passthrough.py
test_openai_ws_passthrough_routes.py	fix(proxy): close websocket cleanly when OpenAI credentials are missing	2026-08-16 14:40:37 -07:00
test_openapi_schema_validation.py
test_plugin_routes.py	feat: litellm plugin architecture v2 (#30688)	2026-06-20 20:37:22 -07:00
test_pricing_field_strip.py	fix(guardrails): bill all chunks on mid-chunking block, strip client guardrail cost metadata, add cost map schema keys	2026-08-18 14:52:23 -07:00
test_prometheus_cleanup.py
test_provider_url_destination_guard.py	fix(proxy)!: share one destination check between body and path-supplied model	2026-08-05 15:17:43 -07:00
test_proxy_cli.py	feat(proxy): let USE_V2_MIGRATION_RESOLVER select the v2 migration resolver	2026-08-07 22:18:59 -07:00
test_proxy_logging_hook_detection.py	fix(guardrails): scan text on /guardrails/apply_guardrail for Azure Content Safety (#36894)	2026-08-17 11:18:40 -07:00
test_proxy_server.py	fix: gate guardrail read-through to active rows and serialize it with the reload reconcile	2026-08-19 01:46:33 -07:00
test_proxy_types.py	feat(proxy): add project-level ITPM and OTPM quotas	2026-08-05 12:53:22 +05:30
test_proxy_utils.py	fix(proxy): backfill system prompt from the request body when estimating bridged failure tokens	2026-08-18 19:52:31 -07:00
test_pyroscope.py
test_read_model_list.py	feat: add minimal rust router + axum ai-gateway calling router.realtime (2/2) (#31135)	2026-06-23 19:16:34 -07:00
test_redis_auth_cache_flag.py	fix(proxy): share CLI SSO login sessions across workers without enable_redis_auth_cache (#33261)	2026-07-22 10:29:34 -07:00
test_response_model_sanitization.py
test_route_a2a_models.py	fix(proxy): single-row read-through resyncs and reload-race hardening	2026-08-18 21:02:12 -07:00
test_route_llm_request.py	fix(proxy): serialize read-through with reloads, gate db object types	2026-08-18 22:43:44 -07:00
test_sensitive_route_auth.py
test_shared_health_check.py
test_spend_log_cleanup.py	perf(spend-logs): bound retention cleanup so one run cannot saturate the database (#36594)	2026-08-13 18:56:37 -07:00
test_swagger_chat_completions.py
test_team_member_update.py
test_team_org_move.py
test_tools_allowlist_enforcement.py	fix(responses-bridge): custom tool round-trip and allowlist preservation for Codex CLI (#32258)	2026-07-06 17:34:27 -07:00
test_update_llm_router_resilience.py	feat(complexity_router): custom classifier plugins via classifier_type 'custom' (#37249)	2026-08-18 14:09:19 -07:00