yucheng-berri c2b3c4b1e4 feat(ptu): accrue flat cost for PTU deployments declared in config.yaml (#37556) ... * feat(ptu): accrue flat cost for PTU deployments declared in config.yaml The flat-cost rollup reads deployments from LiteLLM_ProxyModelTable, and config.yaml models never reach that table by design, so a PTU deployment declared there accrued no flat cost at all while still billing its traffic per token. The provider bills the reservation whichever file declared it. The rollup now also reads the deployments the router holds that no database row owns, identified by db_model, skipping the per-request credential clones that carry original_model_id and reuse their source's PTU config under a fresh id. Registering such a deployment zeroes its pricing, since reserved capacity already pays for the traffic it serves, and leaving a rate unset falls back to the public cost map, which makes the double charge the default rather than an opt-in. The rules both halves apply now live in one module. The rollup's test for what it will charge and the router's test for what to zero have to agree, or a deployment one accepts and the other declines serves its traffic for free. That module also owns the fields the write endpoints already zero, so the two paths cannot drift: tiered_pricing is emptied rather than zeroed because its tiers outrank the rates beside them, the search context table is written zeroed because an absent one means the provider default, and any further rate the deployment itself declares is zeroed alongside the standing set. The prune is bounded to the deployments a run scanned, but only for a run that priced a config-declared deployment. Deciding a row is garbage on staleness alone stays correct while every run derives its charges from the same table, so a database-only run sweeps exactly as it did before; once one host's charges come from a file the others cannot read, a row it never considered is not evidence of anything. Behaviour change worth calling out: a zeroed deployment sorts ahead of an unpriced sibling in QualityRouter's cost tiebreak, where an unset rate previously sorted last. Reserved capacity really is the cheaper choice, but the ordering moves. * refactor(ptu): drop a Final rebind and two redundant isinstance guards The basedpyright budget rejected reassigning a Final in the datetime coercion and two isinstance calls the router entry's own type already guarantees. Filtering the built records rather than the raw entries removes both guards and leaves _router_deployment as the single validator.		2026-08-19 19:08:14 -07:00
..
_experimental/mcp_server	fix(mcp): stop reporting failed OpenAPI tool calls as successes (#37496)	2026-08-19 14:26:02 -07:00
a2a	fix(a2a): reject malformed protocolVersion suffixes while keeping semver prereleases	2026-07-21 13:24:58 -07:00
agent_endpoints	fix(proxy): send SSE keepalives on assistants runs and A2A streams (#37368)	2026-08-18 16:22:05 -07:00
analytics_endpoints	feat(ui): split failed requests into their own series on the cache dashboard (#34862)	2026-07-29 09:48:17 -07:00
anthropic_endpoints	fix(proxy): estimate failed-request input tokens on /v1/messages and count system prompts	2026-08-18 14:21:47 -07:00
auth	fix(auth): resolve bare model names against wildcard deployments in model access groups (#37492)	2026-08-19 15:33:29 -07:00
batches_endpoints	Merge pull request #37198 from BerriAI/litellm_lit5660_batches_limit_400	2026-08-17 15:53:46 -07:00
client	feat(cli): add lite login --config-claude to wire Claude Code at login (#37507)	2026-08-19 15:32:14 -07:00
common_utils	feat(proxy)!: default audit logs on for enterprise licenses (#37518)	2026-08-19 18:49:21 -07:00
config_resolvers	feat(proxy): add SAML 2.0 SSO for the admin UI (#31429)	2026-07-24 12:51:28 -07:00
credential_endpoints	fix(proxy): return the real status code when a credential update is rejected (#36166)	2026-08-07 08:02:52 +00:00
db	Merge pull request #34887 from RayJueWang/litellm_fix_spend_deadlock_retry	2026-08-19 15:53:45 -07:00
discovery_endpoints	test: remove tests that never execute	2026-08-12 10:45:38 -07:00
enterprise_billing	feat(proxy): push-based OTLP billable-request metering for enterprise deployments (#31592)	2026-07-15 12:12:52 -07:00
experimental/mcp_server
fine_tuning_endpoints	fix(proxy): authenticate managed ids before routing	2026-08-08 02:42:34 -07:00
google_endpoints	chore: litellm oss staging160626 (#30527)	2026-06-16 18:23:13 -07:00
guardrails	Merge pull request #37387 from BerriAI/litellm_guardrail_usage_requeue	2026-08-18 16:42:06 -07:00
health_endpoints	feat(proxy): auto-suppress the no-Redis banner for confirmed single-worker deployments	2026-08-14 17:11:05 -07:00
hooks	feat(proxy)!: default audit logs on for enterprise licenses (#37518)	2026-08-19 18:49:21 -07:00
image_endpoints	fix(proxy)!: share one destination check between body and path-supplied model	2026-08-05 15:17:43 -07:00
logging_endpoints	feat(proxy): add POST /v1/callbacks/logs to replay logging payloads through callbacks (#31134)	2026-06-24 15:25:10 -07:00
management_endpoints	feat(proxy)!: default audit logs on for enterprise licenses (#37518)	2026-08-19 18:49:21 -07:00
management_helpers	feat(proxy)!: default audit logs on for enterprise licenses (#37518)	2026-08-19 18:49:21 -07:00
memory	fix(proxy): give proxy_admin_viewer read parity with proxy_admin (#35851)	2026-08-05 18:33:55 +00:00
middleware	feat(proxy): add Amazon Comprehend Medical passthrough provider	2026-08-17 15:44:06 -07:00
ocr_endpoints	fix(ocr): validate body req_format in the proxy endpoint and run its tests in CI	2026-08-17 18:29:35 +00:00
openai_files_endpoint	test(files): hoist shared batch line fixture into one constant	2026-08-19 15:22:43 -07:00
pass_through_endpoints	fix(vertex_ai): resolve passthrough serving location in the logging cost recompute	2026-08-19 17:24:44 -07:00
policy_engine	fix(guardrails): scan text on /guardrails/apply_guardrail for Azure Content Safety (#36894)	2026-08-17 11:18:40 -07:00
prompts	fix(proxy): give proxy_admin_viewer read parity with proxy_admin (#35851)	2026-08-05 18:33:55 +00:00
proxy_server	fix(proxy): initialize the secret manager before resolving os.environ config references (#37544)	2026-08-19 17:00:26 -07:00
public_endpoints	fix(ui): distinguish hosted and local vLLM in the provider dropdown	2026-08-14 13:39:37 -07:00
rag_endpoints	fix(rag): track LLM completion usage and spend for /v1/rag/query (#32438)	2026-07-17 17:45:27 +00:00
realtime_endpoints	chore: litellm oss 170626 (#30637)	2026-06-17 21:11:12 -07:00
response_api_endpoints	fix(responses_api): map bridged chat usage on guardrail-blocked replies	2026-08-14 17:04:27 -07:00
shutdown
spend_tracking	feat(ptu): accrue flat cost for PTU deployments declared in config.yaml (#37556)	2026-08-19 19:08:14 -07:00
test_configs
types_utils	fix(proxy): resolve router_settings.plugins dotted paths and load plugins from installed packages (#33644)	2026-07-17 11:23:18 -07:00
ui_crud_endpoints	feat(ptu): gate PTU flat-cost attribution behind an opt-in env var (#36138)	2026-08-10 12:23:20 -07:00
utils	fix(proxy): retry end-user spend updates on Postgres deadlock instead of dropping them	2026-08-19 15:20:59 -07:00
vector_store_endpoints	fix(vector_stores): classify write endpoints before reads on substring collisions	2026-08-14 16:53:48 -07:00
vector_store_files_endpoints	fix(proxy): authenticate managed ids before routing	2026-08-08 02:42:34 -07:00
video_endpoints	test(videos): add 1:1 test file scaffold for videos component paths (#30631)	2026-06-29 09:12:51 +05:30
__init__.py
conftest.py	test(proxy): stop monkeypatch.undo re-planting fixture-mocked prisma_client	2026-08-13 20:11:06 -07:00
test_aiohttp_cleanup_closed.py
test_aiohttp_session_recovery.py
test_api_key_masking_in_errors.py
test_audio_speech_prometheus_hooks.py	fix(proxy): extend response headers hook to streaming, TTS, image gen, and pass-through (#24232)	2026-06-09 22:10:23 +02:00
test_batch_expiry.py
test_batch_metadata_none_fix.py
test_batch_retrieve_bedrock.py
test_batch_x_litellm_model_encoding.py	test(proxy): give the x-litellm-model fallback test deterministic openai env creds	2026-08-17 12:42:57 -07:00
test_blocked_response_usage.py	fix(responses_api): map bridged chat usage on guardrail-blocked replies	2026-08-14 17:04:27 -07:00
test_budget_reservation.py	feat(proxy): add apply_user_budget_to_team_keys opt-in (#36102)	2026-08-07 15:40:13 +00:00
test_caching_routes.py	Litellm oss staging 040626 (#29671)	2026-06-04 11:07:20 -07:00
test_chat_completion_metadata.py
test_common_request_processing.py	fix(proxy): send SSE keepalives while a slow upstream is still silent (#37322)	2026-08-18 14:43:01 -07:00
test_component_allowlists.py	fix(gateway): keep the Prometheus /metrics Mount in the gateway route trim (#32317)	2026-07-07 18:36:38 +03:00
test_conftest.py	test(proxy): stop monkeypatch.undo re-planting fixture-mocked prisma_client	2026-08-13 20:11:06 -07:00
test_cors_config.py
test_custom_proxy.py
test_dynamic_mcp_route.py	fix(proxy): serve aggregate MCP endpoint on bare /mcp instead of 307-redirecting (#34845)	2026-08-14 17:04:32 -07:00
test_empty_model_list.py
test_enforce_user_param.py
test_fallback_management_endpoints.py
test_fastapi_offline_routes.py
test_filter_models_by_team_access_group.py
test_health_check_functions.py
test_health_check_max_tokens.py	fix: skip health check for semantic auto_router deployments (#31668)	2026-06-29 19:47:20 -07:00
test_langfuse_passthrough_security.py
test_lazy_openapi_snapshot.py	feat(proxy): add GET /v1/indexes to list vector store indexes (#36289)	2026-08-10 15:09:59 -07:00
test_litellm_pre_call_utils.py	fix(proxy): stop leaking the client_side_timeout marker to providers	2026-08-18 13:37:07 -07:00
test_max_budget_env_var.py
test_mcp_asgi_response.py
test_model_based_routing_files_batches.py	test(batches): move orphan tests into tests/test_litellm for CI coverage (#30510)	2026-06-16 10:20:59 -07:00
test_model_deprecations_endpoint.py	refactor(proxy): tighten model deprecation typing and cover the endpoint	2026-08-10 22:58:35 +00:00
test_model_dump_with_preserved_fields.py
test_model_id_header_propagation.py	feat(proxy): add x-litellm-model-name response header with deployment model string (#33698)	2026-07-17 20:29:42 -07:00
test_model_info_default_limits.py	fix(proxy): align /v1/model/info with router deployments (#30025)	2026-06-10 19:38:21 -07:00
test_model_level_guardrails.py	fix(proxy): apply key/team router_settings.model_group_alias (#35486)	2026-08-03 22:09:47 +00:00
test_model_list_healthy_only.py	feat: litellm oss 110626 (#30202)	2026-06-11 22:30:26 -07:00
test_modify_response_streaming_passthrough.py	fix(proxy): use e.request_data for logging_obj in ModifyResponseException streaming passthrough (#30800)	2026-06-18 23:29:08 -07:00
test_openai_ws_passthrough_routes.py	fix(proxy): close websocket cleanly when OpenAI credentials are missing	2026-08-16 14:40:37 -07:00
test_openapi_schema_validation.py
test_plugin_routes.py	feat: litellm plugin architecture v2 (#30688)	2026-06-20 20:37:22 -07:00
test_pricing_field_strip.py	fix(guardrails): bill all chunks on mid-chunking block, strip client guardrail cost metadata, add cost map schema keys	2026-08-18 14:52:23 -07:00
test_prometheus_cleanup.py
test_provider_url_destination_guard.py	fix(proxy)!: share one destination check between body and path-supplied model	2026-08-05 15:17:43 -07:00
test_proxy_cli.py	feat(proxy): let USE_V2_MIGRATION_RESOLVER select the v2 migration resolver	2026-08-07 22:18:59 -07:00
test_proxy_logging_hook_detection.py	fix(guardrails): scan text on /guardrails/apply_guardrail for Azure Content Safety (#36894)	2026-08-17 11:18:40 -07:00
test_proxy_server.py	fix: gate guardrail read-through to active rows and serialize it with the reload reconcile	2026-08-19 01:46:33 -07:00
test_proxy_types.py	feat(proxy): add project-level ITPM and OTPM quotas	2026-08-05 12:53:22 +05:30
test_proxy_utils.py	fix(proxy): strip client standard_logging_object before the failure logging handler	2026-08-19 16:53:02 -07:00
test_pyroscope.py
test_read_model_list.py	feat: add minimal rust router + axum ai-gateway calling router.realtime (2/2) (#31135)	2026-06-23 19:16:34 -07:00
test_redis_auth_cache_flag.py	fix(proxy): share CLI SSO login sessions across workers without enable_redis_auth_cache (#33261)	2026-07-22 10:29:34 -07:00
test_response_model_sanitization.py
test_route_a2a_models.py	fix(proxy): single-row read-through resyncs and reload-race hardening	2026-08-18 21:02:12 -07:00
test_route_llm_request.py	fix(proxy): serialize read-through with reloads, gate db object types	2026-08-18 22:43:44 -07:00
test_sensitive_route_auth.py
test_shared_health_check.py
test_spend_log_cleanup.py	perf(spend-logs): bound retention cleanup so one run cannot saturate the database (#36594)	2026-08-13 18:56:37 -07:00
test_swagger_chat_completions.py
test_team_member_update.py	feat(ui): add budget duration to edit team member form (#29717)	2026-06-06 17:24:55 -07:00
test_team_org_move.py
test_tools_allowlist_enforcement.py	fix(responses-bridge): custom tool round-trip and allowlist preservation for Codex CLI (#32258)	2026-07-06 17:34:27 -07:00
test_update_llm_router_resilience.py	feat(complexity_router): custom classifier plugins via classifier_type 'custom' (#37249)	2026-08-18 14:09:19 -07:00