Commit graph

53166 commits

Author SHA1 Message Date
kerry
61ac4f5739 test(e2e): add scripted-provider cost calculation suite
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 23:03:56 +00:00
yassin
c8a2d8c349 feat(proxy): add LiteLLM_DailyGlobalSpend key-free rollup for the usage dashboard
Adds a daily spend table without api_key or user_id, written atomically alongside
LiteLLM_DailyUserSpend from the batched writer, reconciled from history by a
scheduled job that advances a marker in LiteLLM_Config, and read by the key-free
arm of the aggregated usage query once the marker covers the requested range.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 23:00:36 +00:00
yassin
c007fb9928 fix(proxy): skip alias rewrite only for dispatched pass-through handlers
Match the pass-through skip to what FastAPI actually dispatched (the user-defined
endpoint marker or a provider handler's {endpoint:path} param) instead of the
mapped route prefixes, which also cover native routes such as /openai/v1/responses
and /cursor/chat/completions. Wrap the added test lines to the 120-column limit.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 23:00:08 +00:00
Mateo Wang
6bc821492b
Merge pull request #40476 from BerriAI/litellm_codex_model_catalog_sync
feat(cli): sync Codex /model picker from proxy /v1/models in lite codex
2026-09-15 15:59:04 -07:00
yassin
557c237b81 test(proxy-extras): move the LITELLM_LOG logger test into the CI-run proxy-extras shard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:57:23 +00:00
Yuneng Jiang
56685ddeae
Merge remote-tracking branch 'origin/main' into litellm_/release-version-bump-940c92 2026-09-15 15:56:29 -07:00
Yassin Kortam
0bb95e298c
Merge pull request #41309 from BerriAI/litellm_playground_custom_request_headers 2026-09-15 15:56:22 -07:00
Yuneng Jiang
818fd7bbb1
Merge remote-tracking branch 'origin/main' into litellm_/release-version-bump-940c92 2026-09-15 15:56:18 -07:00
Yassin Kortam
9d75cdd502
Merge pull request #41304 from BerriAI/litellm_lit1698_openai_system_messages_first 2026-09-15 15:56:13 -07:00
kerry
082f02bd73 refactor(model_info): drop Gemini vertex routing rule, keep baseline to certain capabilities
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:55:29 +00:00
yassin
b50a22b370 fix(proxy): restrict /nvidia_nim route to NIM-backed model groups and inject router in tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:55:14 +00:00
Yassin Kortam
9bb83fdaea
Merge pull request #41281 from BerriAI/litellm_lit_7417_jwt_key_mapping_issuer_scope
fix(jwt-auth): scope JWT key mappings by issuer to prevent cross-issuer collisions
2026-09-15 15:51:50 -07:00
yassin
fd90eeb3c6 test(proxy): cover disjoint-method db/yaml pass-through entries on a shared path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:50:29 +00:00
shivam
a654034f6a test(ci): allow completion_cost recursion for mixed-tier WS pricing split
The recursion is bounded to depth 1: split parts each carry a single
service_tier, so the recursive call's partition has one key and the split
helper returns None.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:50:28 +00:00
kerry
c94a4f8696 test: restore xai reported-cost passthrough tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:49:57 +00:00
kerry
024a887521 feat(model_info): add Gemini fallback generalization rules (vertex routing + 2.5+ family baseline)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:49:35 +00:00
Yuneng Jiang
ad74c3ede7
bump: litellm-enterprise 0.1.67 -> 0.1.68, litellm-proxy-extras 0.4.97 -> 0.4.98, litellm 1.102.0 -> 1.103.0 2026-09-15 15:48:54 -07:00
yassin
91dde59bc3 refactor(proxy): validate request metadata before reconstructing key rate-limit view
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:48:47 +00:00
Yassin Kortam
eec954bf9a
Merge pull request #41296 from BerriAI/litellm_models_table_url_state
feat(ui): persist Models table search, filters, sort and page in the URL
2026-09-15 15:46:27 -07:00
Yassin Kortam
94a6dfaf52
Merge pull request #41307 from BerriAI/litellm_passthrough_deployment_model_id
fix(passthrough): attribute Vertex passthrough successes to the resolved router deployment
2026-09-15 15:44:38 -07:00
yassin
d4d8cc9092 fix(proxy): apply access denied message to JWT paths, sanitize denial log, await dashboard saves
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:44:03 +00:00
shivam
65132ef2e9 fix(cost): price native Responses WebSocket turns at their returned service_tier
The logging object built for aresponses_websocket sessions dropped the
service_tier carried by each billable response.completed/response.incomplete
event, so sessions were priced at the default tier. The logging object now
carries the tier when the session is single-tier, and completion_cost splits
mixed-tier sessions into one object per tier before pricing. Fixes #41299

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:42:58 +00:00
yassin
b13cf21fd1 fix(vertex_ai): bill Gemini Omni Interactions usage and Veo sampleCount on passthrough
Route Vertex and Gemini /interactions create responses through the shared
Interactions usage transformation so SpendLogs carry per-modality tokens and
cost instead of zeros, and carry parameters.sampleCount into Veo usage as
video_count so duration-based video cost is multiplied by the number of
generated videos

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:42:27 +00:00
Mateo Wang
6ee1d72ae8
Merge pull request #40872 from BerriAI/litellm_bedrock_sanitize_tool_use_id
fix(bedrock): sanitize client tool_call ids to Bedrock toolUseId constraints
2026-09-15 15:41:39 -07:00
yassin
be082013f7 fix(ui): label the response time chart tooltip with the series name
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:38:55 +00:00
kerry
357e0fca8e chore: merge origin/main into test cleanup
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:36:58 +00:00
yassin
db0f06d153 fix(proxy): skip team model tpm accounting when key owns model tpm limit
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:35:48 +00:00
yassin
ebb9a3ceeb chore(ui): regenerate schema.d.ts for the /nvidia_nim passthrough route
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:34:59 +00:00
yassin
c48723f210 fix(proxy): let a db pass-through entry override the yaml entry on the same path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:34:35 +00:00
Mateo Wang
c3222ec110
Merge pull request #41171 from BerriAI/litellm_converted_stream_spend_tracking
fix(logging): track spend for streams a deployment hook converted to non-streaming
2026-09-15 15:34:13 -07:00
yassin
5d6e367d56 fix(proxy): keep alias rewrite off pass-through bodies and auth-merged params
Skip router_settings.model_group_alias resolution on registered pass-through
routes, rebuild the rewritten body from the cached client payload instead of the
auth-enriched request_data, and centralize the resolved-scope sentinel in
litellm/constants.py

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:32:11 +00:00
mateo-berri
69e9edb91a test(responses): fold the empty-choices regression tests into the mapped streaming iterator test file
Some checks failed
ai-gateway image / ai-gateway release image (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
2026-09-15 15:31:38 -07:00
kerry
f6ef0a24e0 fix(tests): restore rerank response import
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:28:23 +00:00
mateo-berri
fa116a0ecc Merge remote-tracking branch 'origin/main' into litellm_bedrock_sanitize_tool_use_id 2026-09-15 15:27:55 -07:00
yassin
1b6e76af16 fix(proxy): resolve LITELLM_LOG for uvicorn at startup and restore logger state in tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:27:24 +00:00
yassin
168607cc3f fix(proxy): register /nvidia_nim/ with the lazy passthrough loader
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:26:49 +00:00
yassin
754e87fa67 fix(ui): keep gateway auth and content-type headers ahead of playground custom headers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:24:13 +00:00
mateo-berri
f22c7d274a Merge remote-tracking branch 'origin/main' into litellm_bedrock_rag_retrieval_filter
Some checks failed
ai-gateway image / ai-gateway release image (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
# Conflicts:
#	litellm/rag/main.py
#	tests/test_litellm/rag/test_main.py
2026-09-15 15:21:57 -07:00
mateo-berri
a6b0d4266f Merge origin/main into litellm_lit4767_empty_choices_streaming_guard
Takes main's Anthropic Messages adapter iterator, transformation, and
combined-chunk tests as-is: #35314 already skips choiceless chunks at the
top of both adapter loops, so the adapter-side guards this branch carried
are superseded. The Responses bridge guards stay
2026-09-15 15:20:16 -07:00
yassin
5b0fa89056 feat(ui): show average response time per model in usage model activity
Roll request_duration_ms of successful, non-internal requests into the
daily spend tables as total_response_time_ms plus timed_requests, expose
both through the daily activity endpoints, and derive the average in the
Usage -> Model Activity view of the Admin UI

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:16:12 +00:00
kerry
2586b21893 test: keep tests that survive correct cost-map updates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:16:08 +00:00
yassin
9261a72eb7 test(passthrough): cover Vertex router resolution without a matching deployment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:14:44 +00:00
yassin
c05af7a12f feat(ui): add custom request headers to the API Playground
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:12:06 +00:00
yassin
6c8b9a7b05 feat(keys): filter /key/list by active, expired, revoked or deleted status and serve deleted keys from /key/info
Persist and expose the lifecycle of API keys so spend, audit and FinOps
workflows can still resolve a key after it is revoked, expires or is
deleted.

/key/list?status= now accepts active, expired and revoked next to the
existing deleted value. revoked means blocked=true, expired means not
blocked with a past expiry, active is the rest, so the three values
partition the live key table. deleted keeps reading the
LiteLLM_DeletedVerificationToken archive.

/key/info falls back to that archive when the key is no longer in the
live table, running the same owner/team/org authorization check, and
every response now carries a derived status field. The hashed token is
still stripped.

The Virtual Keys page gets a Status filter (URL-persisted) and a Deleted
badge that shows when and by whom the key was deleted.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:12:01 +00:00
yassin
c3e937b845 docs(proxy): describe the single UNION ALL aggregate statement in the endpoint docstring
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:11:20 +00:00
yassin
69be041da7 feat(proxy): add /nvidia_nim passthrough route for NIM object detection and OCR /v1/infer
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:11:08 +00:00
yassin
af4a0b4bc3 fix(proxy): keep yaml pass-through endpoints visible to auth after db overlay
Resolves LIT-2053

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:10:42 +00:00
yassin
460a0128c9 refactor(proxy): drop docstrings from key model limit helpers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:08:39 +00:00
yassin
bae2bf003e fix(proxy): resolve router_settings.model_group_alias before key/team model auth
Key and team router_settings.model_group_alias aliases were resolved only after the key/team model allowlist checks ran, so a key allowed the alias target was denied when it requested the alias. Resolve the alias during auth and rewrite the request body to the target before the allowlist checks. The alias the client sent is kept in the request scope so the response model still echoes it.

Resolves LIT-3054

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:06:55 +00:00
Yassin Kortam
226b1e1bb9
Merge pull request #41288 from BerriAI/litellm_langsmith_preserve_events_during_flush
fix(langsmith): keep events appended during an in-flight flush instead of clearing them
2026-09-15 15:06:36 -07:00