devin-ai-integration[bot] e7cd97c6b6 fix(proxy): release unclaimed budget reservations at request end (#42304) ... * fix(proxy): release unclaimed budget reservations at request end * fix(proxy): release unclaimed budget reservations of websocket sessions too * test(proxy): drop the structural middleware inheritance check * fix(proxy): claim the budget reservation on streaming pass-through before its cost callback The SSE chunk processor hands its success handler to the logging worker after the response, so the request-end release freed the reservation first and left the key unguarded until the worker drained. Claim it at both end-of-stream hand-offs, the immediate enqueue and the coroutine parked for deferred dispatch. Give the xai realtime test double the litellm_params attribute every real Logging object carries, since the wrapper now reads it. * test(pass-through): give the vertex streaming test doubles a litellm_params dict The spec'd Logging mocks in test_vertex_ai_anthropic_streaming_cost_injection.py lacked the instance attribute the chunk processor now reads to claim the budget reservation. Also restores main's _lazy_openapi_snapshot.json: the branch's copy had been regenerated under Python 3.14, which dedents one docstring description that the CI regeneration on Python 3.12 keeps indented, and the PR adds no lazily loaded route, so main's file is the correct one. * fix(pass-through): claim the budget reservation only after its cost callback is enqueued Every pass-through success hand-off stamped callback_bound before handing the coroutine to the logging worker. When that enqueue raised, the reservation stayed claimed with no callback left to reconcile it, so the request-end release skipped it and the reserved cost stayed pinned on the key's counter. Enqueue first, then claim, so a failed hand-off leaves the reservation for the request-end release. --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>		2026-09-21 19:51:12 -07:00
..
_experimental/mcp_server	Merge pull request #41634 from BerriAI/litellm_agent_access_groups	2026-09-21 19:13:46 -05:00
a2a	fix(a2a): reject malformed protocolVersion suffixes while keeping semver prereleases	2026-07-21 13:24:58 -07:00
agent_endpoints	fix(proxy): scope agent permissions to invoking caller	2026-09-21 22:32:32 +00:00
analytics_endpoints	fix(proxy): attribute gate-rejected requests to their endpoint in cache analytics (#40824)	2026-09-12 16:03:38 -07:00
anthropic_endpoints	feat(proxy): opt-in litellm_call_id in JSON error bodies (#42391)	2026-09-21 19:17:18 -07:00
auth	fix(proxy): release unclaimed budget reservations at request end (#42304)	2026-09-21 19:51:12 -07:00
batches_endpoints	chore: merge main into fix/batch-retrieve-model-group	2026-09-19 16:25:39 -07:00
client	fix(cli): serialize footer installs and tolerate unknown versions	2026-09-21 16:48:08 -07:00
common_utils	feat(proxy): opt-in litellm_call_id in JSON error bodies (#42391)	2026-09-21 19:17:18 -07:00
config_resolvers	test: drop two inert type: ignore comments	2026-09-19 14:22:19 -07:00
container_endpoints	fix(containers): page upstream until a non-admin container list fills its limit	2026-09-02 21:26:34 -07:00
credential_endpoints	fix(credentials): answer 409 on a name collision, let PATCH resolve values from model_id	2026-09-15 10:41:43 -07:00
db	Merge pull request #42026 from BerriAI/litellm_user_jwt_savings	2026-09-21 16:41:16 -07:00
discovery_endpoints	chore(proxy): drop explanatory docstrings from credentials hint helper and tests	2026-09-14 19:29:30 +00:00
enterprise_billing	feat(proxy): push-based OTLP billable-request metering for enterprise deployments (#31592)	2026-07-15 12:12:52 -07:00
experimental/mcp_server	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
fine_tuning_endpoints	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
google_endpoints	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
guardrails	test(guardrails): gate the video e2e on a chat probe so a miss starts at most one paid job	2026-09-21 23:20:29 +00:00
health_endpoints	Merge pull request #40814 from BerriAI/litellm_gate_health_services_alert_tests	2026-09-12 21:13:30 -07:00
hooks	Merge pull request #42062 from BerriAI/litellm_pr38499_batch_retrieve_model_group	2026-09-19 18:23:06 -07:00
image_endpoints	Merge remote-tracking branch 'origin/main' into litellm_fix_image_edits_bracketed_alias	2026-09-17 14:49:54 -07:00
list_api	feat: page the public model hub table off /public/v1/model_hub, keeping every filter (#39691)	2026-09-03 22:36:16 -07:00
logging_endpoints	fix(usage): recover aliases for v1.99 double-hashed spend keys	2026-09-03 14:21:25 +00:00
management_endpoints	fix(proxy): drop cost-map metadata echoed back on model save (#41944)	2026-09-21 19:33:16 -07:00
management_helpers	Merge remote-tracking branch 'origin/main' into litellm_jev_autorouter_launch_1789767495	2026-09-19 17:35:42 +00:00
memory	feat(proxy): add a search param to key, memory, audit, and spend log listings	2026-09-03 15:20:13 -07:00
middleware	fix(proxy): release unclaimed budget reservations at request end (#42304)	2026-09-21 19:51:12 -07:00
ocr_endpoints	fix(ocr): validate body req_format in the proxy endpoint and run its tests in CI	2026-08-17 18:29:35 +00:00
openai_files_endpoint	fix(files): keep an explicit target_storage on its old path and refuse litellm_db as a caller choice	2026-09-19 13:20:02 -07:00
pass_through_endpoints	fix(proxy): release unclaimed budget reservations at request end (#42304)	2026-09-21 19:51:12 -07:00
policy_engine	fix(policy_engine): resolve policies once and apply fallback semantics in get_matching_policies	2026-09-21 17:36:03 +00:00
prompts	fix(prompts): accept a string prompt_version and carry the viewed environment into code snippets	2026-08-29 12:55:34 -07:00
proxy_server	feat(proxy): opt-in litellm_call_id in JSON error bodies (#42391)	2026-09-21 19:17:18 -07:00
public_endpoints	test: keep the Qwen brand missing-key tests off the network and split the ChatGPT fields test back out	2026-09-21 12:54:23 -07:00
rag_endpoints	test(rag): drop the docstrings from the registered-store ingest tests	2026-09-19 04:10:49 -07:00
realtime_endpoints	refactor(proxy): replace configurable model access denied message with a fixed clean client message	2026-09-16 01:44:36 +00:00
rerank_endpoints	fix(proxy): carry litellm_call_id through endpoint specific error logs and failure responses	2026-09-16 02:29:26 +00:00
response_api_endpoints	test(proxy): expect 422 for per-model budget rejections on cursor route	2026-09-20 06:10:16 +00:00
shutdown	refactor(proxy): inject scheduled job shutdown timeouts	2026-09-21 19:37:28 +00:00
spend_tracking	fix(proxy): release unclaimed budget reservations at request end (#42304)	2026-09-21 19:51:12 -07:00
test_configs
types_utils	feat(proxy): unified custom_key_policy hook for key generate, update and regenerate	2026-09-12 16:09:24 -07:00
ui_crud_endpoints	Merge remote-tracking branch 'origin/main' into litellm_config_read_source	2026-09-21 15:54:52 -07:00
utils	Merge pull request #41634 from BerriAI/litellm_agent_access_groups	2026-09-21 19:13:46 -05:00
vector_store_endpoints	fix(vector_stores): run the full model grant check on caller-supplied model hints	2026-09-19 00:54:41 -07:00
vector_store_files_endpoints	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
video_endpoints	fix(videos): forward uploaded source file on /v1/videos/edits to the provider	2026-08-24 15:34:21 -07:00
__init__.py
conftest.py	test(proxy): isolate the agent read-through singleton between unknown-agent tests	2026-09-22 01:34:31 +00:00
test__types.py	feat(auth): breached password detection, self-service change-password and forced password reset	2026-09-21 18:48:35 +00:00
test_aiohttp_cleanup_closed.py
test_aiohttp_session_recovery.py
test_api_key_masking_in_errors.py
test_audio_speech_prometheus_hooks.py	fix(proxy): match /v1/audio/speech content-type to the returned audio format	2026-08-29 13:44:43 -07:00
test_batch_expiry.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_batch_metadata_none_fix.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_batch_retrieve_bedrock.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_batch_x_litellm_model_encoding.py	fix(proxy): apply model grants to unified file and batch ids on batch routes	2026-09-10 18:10:18 -04:00
test_blocked_response_usage.py	fix(guardrails): remove the module-global translation mapping that leaked between tests	2026-09-03 03:28:52 -07:00
test_budget_reservation.py	test(proxy): restore pipeline methods on expiring Redis fake after merge	2026-09-20 08:59:46 +00:00
test_caching_routes.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_chat_completion_metadata.py
test_claude_code_marketplace.py	test: run the 30 test files stranded in the second mirror (#37595)	2026-08-20 10:59:43 -07:00
test_collector.py	feat(proxy): offload spend tracking to a pod-local collector sidecar (#40545)	2026-09-10 17:14:13 -07:00
test_common_request_processing.py	feat(proxy): opt-in litellm_call_id in JSON error bodies (#42391)	2026-09-21 19:17:18 -07:00
test_component_allowlists.py	fix(gateway): serve /debug/memory/summary on the data plane so the memory regression test can read each gateway's RSS	2026-09-10 18:12:40 -07:00
test_conftest.py	test(proxy): stop monkeypatch.undo re-planting fixture-mocked prisma_client	2026-08-13 20:11:06 -07:00
test_cors_config.py
test_custom_proxy.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_dynamic_mcp_route.py	fix(proxy): serve aggregate MCP endpoint on bare /mcp instead of 307-redirecting (#34845)	2026-08-14 17:04:32 -07:00
test_empty_model_list.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_enforce_user_param.py	test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769)	2026-08-20 20:24:49 -07:00
test_fallback_management_endpoints.py
test_fastapi_offline_routes.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_filter_models_by_team_access_group.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_health_check_functions.py	fix(health): skip background health check DB writes when the latest-row read fails	2026-09-14 23:03:30 +00:00
test_health_check_max_tokens.py	feat(auto-router): integrate JEV context and usage accounting	2026-09-18 21:38:15 +00:00
test_init_litellm_callbacks.py	test: run the 30 test files stranded in the second mirror (#37595)	2026-08-20 10:59:43 -07:00
test_langfuse_passthrough_security.py	chore(callbacks): guard dynamic integration hosts	2026-04-30 14:27:19 -07:00
test_lazy_openapi_snapshot.py	refactor(proxy): type the snapshot fragments and wrap a long test line	2026-08-26 14:39:51 -07:00
test_litellm_pre_call_utils.py	fix(proxy): forward a client's anthropic-beta and anthropic-version headers to bedrock_mantle	2026-09-19 20:11:14 -07:00
test_max_budget_env_var.py
test_mcp_asgi_response.py	fix(mcp): surface upstream 401 for token-forwarding MCP servers (#27847)	2026-05-13 12:03:36 -07:00
test_model_based_routing_files_batches.py	test(batches): move orphan tests into tests/test_litellm for CI coverage (#30510)	2026-06-16 10:20:59 -07:00
test_model_deprecations_endpoint.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_model_dump_with_preserved_fields.py
test_model_id_header_propagation.py	feat(proxy): add x-litellm-model-name response header with deployment model string (#33698)	2026-07-17 20:29:42 -07:00
test_model_info_default_limits.py	perf(proxy): serialize /model/info listing once with orjson	2026-09-14 19:47:51 +00:00
test_model_level_guardrails.py	fix(proxy): apply key/team router_settings.model_group_alias (#35486)	2026-08-03 22:09:47 +00:00
test_model_list_healthy_only.py	perf(proxy): serialize /model/info listing once with orjson	2026-09-14 19:47:51 +00:00
test_modify_response_streaming_passthrough.py	fix(proxy): use e.request_data for logging_obj in ModifyResponseException streaming passthrough (#30800)	2026-06-18 23:29:08 -07:00
test_openai_ws_passthrough_routes.py	fix(proxy): keep the token's team model list in the websocket passthrough gate without a database	2026-09-04 19:05:35 -07:00
test_openapi_schema_validation.py	fix(proxy): preserve HTTP operations when injecting WebSocket stubs into OpenAPI schema	2026-05-06 00:28:42 +02:00
test_plugin_routes.py	refactor(proxy): resolve config and DB settings precedence in one SettingsStore	2026-09-17 23:36:27 -07:00
test_pointfive_dashboard_config.py	feat(pointfive): add pointfive to the dashboard logging integrations	2026-09-10 14:02:35 +03:00
test_pointfive_ui_callback.py	feat(pointfive): list pointfive in the proxy callback registry	2026-09-10 14:02:35 +03:00
test_pricing_field_strip.py	fix(proxy): fold litellm_metadata into metadata on chat routes so tag routing sees merged tags	2026-08-29 00:42:43 -07:00
test_prisma_engine_watchdog.py	fix(proxy): recreate the Prisma client when the writer session turns read-only (#40610)	2026-09-10 13:53:42 -07:00
test_prisma_migration.py	fix(proxy): run migrations through python -m prisma when the prisma console script is not on PATH	2026-09-09 18:17:12 -07:00
test_prometheus_cleanup.py	feat(deploy): metrics sidecar and separate metrics port in Helm and Terraform (#40163)	2026-09-08 13:31:07 -07:00
test_prometheus_metrics_server.py	feat(deploy): metrics sidecar and separate metrics port in Helm and Terraform (#40163)	2026-09-08 13:31:07 -07:00
test_provider_url_destination_guard.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_proxy_cli.py	fix(proxy): only warn about the deprecated flag when it came from the CLI	2026-09-20 01:22:31 -07:00
test_proxy_logging_hook_detection.py	fix(proxy): keep queued moderation running past a V1 pre_call guardrail	2026-09-18 23:33:29 +00:00
test_proxy_server.py	Merge pull request #42122 from BerriAI/litellm_db_callback_redaction_settings	2026-09-21 16:03:44 -07:00
test_proxy_types.py	fix(router): honor team and key provider weights	2026-09-14 23:31:52 -07:00
test_proxy_utils.py	Merge remote-tracking branch 'origin/main' into litellm_v1_models_alias_metadata	2026-09-18 20:23:06 +00:00
test_pyroscope.py	Implement normalize_nonempty_secret_str function to trim whitespace from secrets and treat empty values as unset. Update proxy_server to use this function for Grafana credentials. Enhance tests to validate the new normalization behavior.	2026-05-04 18:17:31 +00:00
test_read_model_list.py	feat: add minimal rust router + axum ai-gateway calling router.realtime (2/2) (#31135)	2026-06-23 19:16:34 -07:00
test_redis_auth_cache_flag.py	fix(proxy): share per-model budget counters across replicas through the spend counter cache (#39375)	2026-09-02 12:40:59 -07:00
test_response_model_sanitization.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_route_a2a_models.py	test: drop the cwd-relative sys.path.insert calls from the test suite (#37802)	2026-08-22 09:25:58 -07:00
test_route_llm_request.py	test(proxy): isolate the agent read-through singleton between unknown-agent tests	2026-09-22 01:34:31 +00:00
test_route_priority.py	perf(proxy): register liveness and core inference routes first (#40687)	2026-09-11 17:10:38 +00:00
test_sensitive_route_auth.py	chore(proxy): guard sensitive public endpoints	2026-04-30 11:52:47 -07:00
test_shared_health_check.py	fix(proxy): derive auto-router health from its underlying models (#38174)	2026-08-26 16:41:54 -07:00
test_spend_log_cleanup.py	Merge pull request #41213 from BerriAI/litellm_spend_log_cleanup_cancel_outcome	2026-09-21 17:03:25 -07:00
test_swagger_chat_completions.py	fix(proxy): avoid misleading multi-method operation ids	2026-04-30 20:44:14 -07:00
test_team_member_update.py	fix(team): schedule membership audit writes after commit and lock the roster on role updates	2026-09-19 22:33:22 +00:00
test_team_org_move.py	test: require a match= on broad pytest.raises, and drop duplicate parametrize cases (#37769)	2026-08-20 20:24:49 -07:00
test_tools_allowlist_enforcement.py	fix(anthropic): close hybrid tool-name allowlist gap and keep native tools through guardrails	2026-08-26 20:37:11 -07:00
test_update_llm_router_resilience.py	fix(proxy): keep the no-model_list guard to absent reads so model_list: [] still evicts	2026-09-16 23:14:52 +00:00