Refael Iliaguyev e065a2575b fix(proxy): send a real error event when a /v1/messages stream fails (#41826) ... * fix(proxy): send a real error event when a /v1/messages stream fails When a stream failed halfway through, the proxy wrote the error as a plain `data: {"error": ...}` line with no `event:` in front of it. Anthropic clients pick stream events by that name, so they skip the line and the request looks like it simply stopped with nothing in it. Write the failure as an `event: error` frame with Anthropic's own payload, and take the error type from the status code * fix(proxy): use the shared Anthropic error mapping for the stream error frame The first pass added a third copy of the status to error-type table, and it disagreed with the documented one: 529 came out as `api_error` rather than `overloaded_error`, and 413 as `invalid_request_error` rather than `request_too_large`, which hides the two failures a client can actually act on. Drop that copy and put the frame builder next to the table litellm already keeps in anthropic_interface/exceptions. The bridged adapter path was building the same frame inline, so it uses the shared one now too * fix(proxy): seal a torn SSE frame before the /v1/messages error event An upstream that drops mid-frame leaves the client inside an open event, so the error frame that follows is glued onto the torn data line and the Anthropic SDK raises a JSON decode error instead of an APIStatusError. Close the open frame with a ping event the SDK skips before writing the error event, and add the e2e stream-cut edge with Bedrock, Anthropic boundary, and Anthropic mid-frame legs. * fix(proxy): keep the SSE tail unchanged on a chunk that is not text A serializer that hands a dict or model object through as-is has no bytes the frame tail can learn from, so advance_sse_tail leaves it alone instead of slicing it. * fix(proxy): answer a /v1/messages stream that fails before its first byte as a JSON error carrying its status * test: move the Anthropic error frame tests into tests/unit * test(e2e): cut the upstream stream only after content has been relayed * test(e2e): carry split SSE lines across chunks and always tear a data line mid-frame * test(e2e): find the next data line across a chunk boundary before tearing it --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>		2026-09-25 15:36:35 -07:00
..
_experimental/mcp_server	feat(mcp): configure protocol versions and capability discovery (#43169)	2026-09-25 13:13:52 -07:00
a2a	fix(a2a): reject malformed protocolVersion suffixes while keeping semver prereleases	2026-07-21 13:24:58 -07:00
agent_endpoints	feat(agents): add optional per-agent kill switch webhook (#42841)	2026-09-24 18:26:50 -05:00
analytics_endpoints	fix(proxy): attribute gate-rejected requests to their endpoint in cache analytics (#40824)	2026-09-12 16:03:38 -07:00
anthropic_endpoints	feat(proxy): opt-in litellm_call_id in JSON error bodies (#42391)	2026-09-21 19:17:18 -07:00
auth	fix(jwt): let x-litellm-team-id select DB membership teams when the token also carries a team claim (#43206)	2026-09-25 15:28:14 -07:00
batches_endpoints	fix(spend): attribute CLI session spend to the per-user cli-session alias instead of the hashed session token (#40541)	2026-09-24 18:21:47 -07:00
client	test: deflake fuzzy picker, breached-password HIBP, and MCP stdio timeout tests (rolling deflake 2026-09-22) (#42125)	2026-09-23 08:44:59 -07:00
common_utils	fix(proxy): send a real error event when a /v1/messages stream fails (#41826)	2026-09-25 15:36:35 -07:00
config_resolvers	test: drop two inert type: ignore comments	2026-09-19 14:22:19 -07:00
container_endpoints	fix(containers): page upstream until a non-admin container list fills its limit	2026-09-02 21:26:34 -07:00
credential_endpoints	fix(credentials): answer 409 on a name collision, let PATCH resolve values from model_id	2026-09-15 10:41:43 -07:00
db	fix(proxy): pass team member spend rows as jsonb so a $0 flush cannot poison the pool connection (#43029)	2026-09-24 20:44:56 +00:00
discovery_endpoints	chore(proxy): drop explanatory docstrings from credentials hint helper and tests	2026-09-14 19:29:30 +00:00
enterprise_billing	feat(proxy): push-based OTLP billable-request metering for enterprise deployments (#31592)	2026-07-15 12:12:52 -07:00
experimental/mcp_server	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
fine_tuning_endpoints	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
google_endpoints	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
guardrails	test: move tests/test_litellm/llms into tests/unit/llms (#43191)	2026-09-25 12:43:23 -07:00
health_endpoints	feat(langfuse): migrate the sdk callback to langfuse v4 (#36741)	2026-09-24 23:22:56 -07:00
hooks	fix(spend): attribute CLI session spend to the per-user cli-session alias instead of the hashed session token (#40541)	2026-09-24 18:21:47 -07:00
image_endpoints	ci: move tests/proxy_unit_tests to tests/unit/proxy and run the proxy-db shards from litellm-tests (#42903)	2026-09-24 22:59:11 +00:00
list_api	feat: page the public model hub table off /public/v1/model_hub, keeping every filter (#39691)	2026-09-03 22:36:16 -07:00
logging_endpoints	fix(usage): recover aliases for v1.99 double-hashed spend keys	2026-09-03 14:21:25 +00:00
management_endpoints	fix(sso): gate /sso/debug routes behind ENABLE_SSO_DEBUG, off by default (#43150)	2026-09-25 20:58:17 +02:00
management_helpers	fix(key_management): count team unified access group MCP servers when validating key MCP grants (#41231)	2026-09-24 23:34:15 -07:00
memory	feat(proxy): add a search param to key, memory, audit, and spend log listings	2026-09-03 15:20:13 -07:00
middleware	fix(proxy): release unclaimed budget reservations at request end (#42304)	2026-09-21 19:51:12 -07:00
ocr_endpoints	refactor(ocr): remove the Python OCR execution path and require the Rust route (#43081)	2026-09-24 18:18:50 -07:00
openai_files_endpoint	feat(vertex): native batch JSONL passthrough with cost tracking (#42810)	2026-09-24 12:35:34 -07:00
pass_through_endpoints	fix(responses): fall back on pre-output stream drops, fail truncated streams, honor request_timeout (#43133)	2026-09-25 14:32:41 -07:00
policy_engine	fix(policy_engine): keep inherited parent guardrails when a child policy condition misses (#42548)	2026-09-22 23:27:38 -07:00
prompts	fix(prompts): accept a string prompt_version and carry the viewed environment into code snippets	2026-08-29 12:55:34 -07:00
proxy_server	feat(mcp): configure protocol versions and capability discovery (#43169)	2026-09-25 13:13:52 -07:00
public_endpoints	feat(ui): simplify auto-router setup and clarify feature limits (#42625)	2026-09-22 18:03:32 -07:00
rag_endpoints	test(rag): drop the docstrings from the registered-store ingest tests	2026-09-19 04:10:49 -07:00
realtime_endpoints	refactor(proxy): replace configurable model access denied message with a fixed clean client message	2026-09-16 01:44:36 +00:00
rerank_endpoints	fix(proxy): carry litellm_call_id through endpoint specific error logs and failure responses	2026-09-16 02:29:26 +00:00
response_api_endpoints	fix(proxy): document request body and response schemas for the Responses API in OpenAPI (#42802)	2026-09-24 01:28:34 +00:00
shutdown	refactor(proxy): inject scheduled job shutdown timeouts	2026-09-21 19:37:28 +00:00
spend_tracking	fix(spend): attribute CLI session spend to the per-user cli-session alias instead of the hashed session token (#40541)	2026-09-24 18:21:47 -07:00
test_configs
types_utils	feat(proxy): unified custom_key_policy hook for key generate, update and regenerate	2026-09-12 16:09:24 -07:00
ui_crud_endpoints	feat(proxy): let team admins update member key budgets when enabled (#42555)	2026-09-25 01:54:52 +00:00
utils	fix(proxy): keep deployment labels on cache-hit post_call guardrail rejections (#42780)	2026-09-23 22:48:04 -07:00
vector_store_endpoints	fix(vector_stores): keep config-defined vector stores listed and read-only (#42574)	2026-09-23 04:02:16 +00:00
vector_store_files_endpoints	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
video_endpoints	fix(videos): forward uploaded source file on /v1/videos/edits to the provider	2026-08-24 15:34:21 -07:00
__init__.py
conftest.py	test(proxy): isolate the agent read-through singleton between unknown-agent tests	2026-09-22 01:34:31 +00:00
test__types.py	feat(mcp): configure protocol versions and capability discovery (#43169)	2026-09-25 13:13:52 -07:00
test_aiohttp_cleanup_closed.py
test_aiohttp_session_recovery.py
test_api_key_masking_in_errors.py
test_audio_speech_prometheus_hooks.py	fix(proxy): match /v1/audio/speech content-type to the returned audio format	2026-08-29 13:44:43 -07:00
test_batch_expiry.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_batch_metadata_none_fix.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_batch_retrieve_bedrock.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_batch_x_litellm_model_encoding.py	fix(proxy): apply model grants to unified file and batch ids on batch routes	2026-09-10 18:10:18 -04:00
test_blocked_response_usage.py	fix(guardrails): remove the module-global translation mapping that leaked between tests	2026-09-03 03:28:52 -07:00
test_budget_reservation.py	feat(tokenizer): preserve Python defaults with opt-in Rust dispatch (#42174)	2026-09-22 04:41:11 +00:00
test_bug_report_config.py	feat(proxy): admin-only /debug/report sharing the bug report environment (#42440)	2026-09-22 12:05:23 -07:00
test_caching_routes.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_chat_completion_metadata.py
test_claude_code_marketplace.py	test: run the 30 test files stranded in the second mirror (#37595)	2026-08-20 10:59:43 -07:00
test_collector.py	feat(proxy): offload spend tracking to a pod-local collector sidecar (#40545)	2026-09-10 17:14:13 -07:00
test_common_request_processing.py	fix(proxy): send a real error event when a /v1/messages stream fails (#41826)	2026-09-25 15:36:35 -07:00
test_component_allowlists.py	test(proxy): make two proxy-infra tests independent of sibling-test state (#42581)	2026-09-22 14:58:42 -07:00
test_conftest.py	test(proxy): stop monkeypatch.undo re-planting fixture-mocked prisma_client	2026-08-13 20:11:06 -07:00
test_cors_config.py
test_custom_proxy.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_dynamic_mcp_route.py	fix(proxy): serve aggregate MCP endpoint on bare /mcp instead of 307-redirecting (#34845)	2026-08-14 17:04:32 -07:00
test_empty_model_list.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_enforce_user_param.py	test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769)	2026-08-20 20:24:49 -07:00
test_fallback_management_endpoints.py
test_fastapi_offline_routes.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_filter_models_by_team_access_group.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_health_check_functions.py	fix(health): skip background health check DB writes when the latest-row read fails	2026-09-14 23:03:30 +00:00
test_health_check_max_tokens.py	feat(auto-router): integrate JEV context and usage accounting	2026-09-18 21:38:15 +00:00
test_init_litellm_callbacks.py	test: run the 30 test files stranded in the second mirror (#37595)	2026-08-20 10:59:43 -07:00
test_langfuse_passthrough_security.py
test_lazy_openapi_snapshot.py	refactor(proxy): type the snapshot fragments and wrap a long test line	2026-08-26 14:39:51 -07:00
test_litellm_pre_call_utils.py	fix(spend): attribute CLI session spend to the per-user cli-session alias instead of the hashed session token (#40541)	2026-09-24 18:21:47 -07:00
test_max_budget_env_var.py
test_mcp_asgi_response.py	fix(mcp): surface upstream 401 for token-forwarding MCP servers (#27847)	2026-05-13 12:03:36 -07:00
test_model_based_routing_files_batches.py	test(batches): move orphan tests into tests/test_litellm for CI coverage (#30510)	2026-06-16 10:20:59 -07:00
test_model_deprecations_endpoint.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_model_dump_with_preserved_fields.py
test_model_id_header_propagation.py	feat(proxy): add x-litellm-model-name response header with deployment model string (#33698)	2026-07-17 20:29:42 -07:00
test_model_info_default_limits.py	perf(proxy): serialize /model/info listing once with orjson	2026-09-14 19:47:51 +00:00
test_model_level_guardrails.py	fix(mcp): keep tool attribution on guardrail-blocked REST calls (#42790)	2026-09-23 15:40:56 -07:00
test_model_list_aliases.py	fix(proxy): list key and team model aliases in GET /v1/models (#42908)	2026-09-24 06:41:13 -07:00
test_model_list_callback_filter.py	feat(proxy): let callbacks filter the model listing routes per caller (#43027)	2026-09-24 15:52:44 -07:00
test_model_list_discoverable.py	feat(proxy): honor model_info.discoverable on the model listing endpoints (#42825)	2026-09-23 17:51:16 -07:00
test_model_list_healthy_only.py	perf(proxy): serialize /model/info listing once with orjson	2026-09-14 19:47:51 +00:00
test_modify_response_streaming_passthrough.py	fix(proxy): use e.request_data for logging_obj in ModifyResponseException streaming passthrough (#30800)	2026-06-18 23:29:08 -07:00
test_native_compaction.py	fix(gateway): expose /api/event_logging/batch on the gateway allowlist (#42572)	2026-09-22 14:09:40 -07:00
test_openai_ws_passthrough_routes.py	fix(proxy): keep the token's team model list in the websocket passthrough gate without a database	2026-09-04 19:05:35 -07:00
test_openapi_schema_validation.py	fix(proxy): preserve HTTP operations when injecting WebSocket stubs into OpenAPI schema	2026-05-06 00:28:42 +02:00
test_plugin_routes.py	refactor(proxy): resolve config and DB settings precedence in one SettingsStore	2026-09-17 23:36:27 -07:00
test_pointfive_dashboard_config.py	feat(pointfive): add pointfive to the dashboard logging integrations	2026-09-10 14:02:35 +03:00
test_pointfive_ui_callback.py	feat(pointfive): list pointfive in the proxy callback registry	2026-09-10 14:02:35 +03:00
test_pricing_field_strip.py	fix(proxy): fold litellm_metadata into metadata on chat routes so tag routing sees merged tags	2026-08-29 00:42:43 -07:00
test_prisma_engine_watchdog.py	fix(proxy): recreate the Prisma client when the writer session turns read-only (#40610)	2026-09-10 13:53:42 -07:00
test_prisma_migration.py	fix(proxy): run migrations through python -m prisma when the prisma console script is not on PATH	2026-09-09 18:17:12 -07:00
test_prometheus_cleanup.py	feat(deploy): metrics sidecar and separate metrics port in Helm and Terraform (#40163)	2026-09-08 13:31:07 -07:00
test_prometheus_metrics_server.py	feat(deploy): metrics sidecar and separate metrics port in Helm and Terraform (#40163)	2026-09-08 13:31:07 -07:00
test_provider_url_destination_guard.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_proxy_cli.py	feat(proxy_cli): add --validate_config dry-run flag (#41705)	2026-09-24 17:07:10 -05:00
test_proxy_logging_hook_detection.py	fix(guardrails): store the masked output in spend logs when Presidio masks the response (#42441)	2026-09-22 01:19:22 -07:00
test_proxy_server.py	fix(caching): propagate auth cache invalidation over Redis Cluster via a node-level pub/sub client (#43110)	2026-09-25 07:56:46 -07:00
test_proxy_types.py	fix(router): honor team and key provider weights	2026-09-14 23:31:52 -07:00
test_proxy_utils.py	feat(errors): prefilled GitHub issue link on unmapped internal errors (#42065)	2026-09-21 22:30:41 -07:00
test_pyroscope.py	Implement normalize_nonempty_secret_str function to trim whitespace from secrets and treat empty values as unset. Update proxy_server to use this function for Grafana credentials. Enhance tests to validate the new normalization behavior.	2026-05-04 18:17:31 +00:00
test_read_model_list.py	feat: add minimal rust router + axum ai-gateway calling router.realtime (2/2) (#31135)	2026-06-23 19:16:34 -07:00
test_redis_auth_cache_flag.py	fix(proxy): share per-model budget counters across replicas through the spend counter cache (#39375)	2026-09-02 12:40:59 -07:00
test_response_model_sanitization.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_route_a2a_models.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_route_llm_request.py	test(proxy): isolate the agent read-through singleton between unknown-agent tests	2026-09-22 01:34:31 +00:00
test_route_priority.py	perf(proxy): register liveness and core inference routes first (#40687)	2026-09-11 17:10:38 +00:00
test_sensitive_route_auth.py
test_shared_health_check.py	fix(proxy): derive auto-router health from its underlying models (#38174)	2026-08-26 16:41:54 -07:00
test_spend_log_cleanup.py	fix(proxy): give SpendLogToolIndex its own share of the cleanup budget and log a per-run summary (#41768)	2026-09-25 09:40:20 -07:00
test_swagger_chat_completions.py
test_team_member_update.py	fix(team): schedule membership audit writes after commit and lock the roster on role updates	2026-09-19 22:33:22 +00:00
test_team_org_move.py	test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769)	2026-08-20 20:24:49 -07:00
test_tools_allowlist_enforcement.py	fix(anthropic): close hybrid tool-name allowlist gap and keep native tools through guardrails	2026-08-26 20:37:11 -07:00
test_update_llm_router_resilience.py	fix(proxy): keep the no-model_list guard to absent reads so model_list: [] still evicts	2026-09-16 23:14:52 +00:00
test_zerobus_dashboard_config.py	feat(integrations): add Databricks Zerobus trace logging callback (#42013)	2026-09-25 15:29:23 -07:00