Commit graph

53219 commits

Author SHA1 Message Date
Mateo Wang
a056370e1e
Merge pull request #41179 from IToSSc/feat/TASK-2BK38Y-model-prices-docs
feat: add aihubmix provider pricing entries
2026-09-15 16:09:05 -07:00
yuneng-jiang
fa9177d5f2
Merge pull request #41321 from BerriAI/litellm_/release-version-bump-940c92
chore: bump litellm-enterprise 0.1.67 -> 0.1.68, litellm-proxy-extras 0.4.97 -> 0.4.98, litellm 1.102.0 -> 1.103.0
2026-09-15 16:06:39 -07:00
kerry
e64371062c test: remove test files left empty by the cleanup
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 23:04:40 +00:00
kerry
61ac4f5739 test(e2e): add scripted-provider cost calculation suite
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 23:03:56 +00:00
yassin
c8a2d8c349 feat(proxy): add LiteLLM_DailyGlobalSpend key-free rollup for the usage dashboard
Adds a daily spend table without api_key or user_id, written atomically alongside
LiteLLM_DailyUserSpend from the batched writer, reconciled from history by a
scheduled job that advances a marker in LiteLLM_Config, and read by the key-free
arm of the aggregated usage query once the marker covers the requested range.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 23:00:36 +00:00
yassin
c007fb9928 fix(proxy): skip alias rewrite only for dispatched pass-through handlers
Match the pass-through skip to what FastAPI actually dispatched (the user-defined
endpoint marker or a provider handler's {endpoint:path} param) instead of the
mapped route prefixes, which also cover native routes such as /openai/v1/responses
and /cursor/chat/completions. Wrap the added test lines to the 120-column limit.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 23:00:08 +00:00
Mateo Wang
6bc821492b
Merge pull request #40476 from BerriAI/litellm_codex_model_catalog_sync
feat(cli): sync Codex /model picker from proxy /v1/models in lite codex
2026-09-15 15:59:04 -07:00
yassin
557c237b81 test(proxy-extras): move the LITELLM_LOG logger test into the CI-run proxy-extras shard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:57:23 +00:00
Yuneng Jiang
56685ddeae
Merge remote-tracking branch 'origin/main' into litellm_/release-version-bump-940c92 2026-09-15 15:56:29 -07:00
Yassin Kortam
0bb95e298c
Merge pull request #41309 from BerriAI/litellm_playground_custom_request_headers 2026-09-15 15:56:22 -07:00
Yuneng Jiang
818fd7bbb1
Merge remote-tracking branch 'origin/main' into litellm_/release-version-bump-940c92 2026-09-15 15:56:18 -07:00
Yassin Kortam
9d75cdd502
Merge pull request #41304 from BerriAI/litellm_lit1698_openai_system_messages_first 2026-09-15 15:56:13 -07:00
kerry
082f02bd73 refactor(model_info): drop Gemini vertex routing rule, keep baseline to certain capabilities
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:55:29 +00:00
yassin
b50a22b370 fix(proxy): restrict /nvidia_nim route to NIM-backed model groups and inject router in tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:55:14 +00:00
Yassin Kortam
9bb83fdaea
Merge pull request #41281 from BerriAI/litellm_lit_7417_jwt_key_mapping_issuer_scope
fix(jwt-auth): scope JWT key mappings by issuer to prevent cross-issuer collisions
2026-09-15 15:51:50 -07:00
yassin
fd90eeb3c6 test(proxy): cover disjoint-method db/yaml pass-through entries on a shared path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:50:29 +00:00
shivam
a654034f6a test(ci): allow completion_cost recursion for mixed-tier WS pricing split
The recursion is bounded to depth 1: split parts each carry a single
service_tier, so the recursive call's partition has one key and the split
helper returns None.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:50:28 +00:00
kerry
c94a4f8696 test: restore xai reported-cost passthrough tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:49:57 +00:00
kerry
024a887521 feat(model_info): add Gemini fallback generalization rules (vertex routing + 2.5+ family baseline)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:49:35 +00:00
Yuneng Jiang
ad74c3ede7
bump: litellm-enterprise 0.1.67 -> 0.1.68, litellm-proxy-extras 0.4.97 -> 0.4.98, litellm 1.102.0 -> 1.103.0 2026-09-15 15:48:54 -07:00
yassin
91dde59bc3 refactor(proxy): validate request metadata before reconstructing key rate-limit view
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:48:47 +00:00
Yassin Kortam
eec954bf9a
Merge pull request #41296 from BerriAI/litellm_models_table_url_state
feat(ui): persist Models table search, filters, sort and page in the URL
2026-09-15 15:46:27 -07:00
Yassin Kortam
94a6dfaf52
Merge pull request #41307 from BerriAI/litellm_passthrough_deployment_model_id
fix(passthrough): attribute Vertex passthrough successes to the resolved router deployment
2026-09-15 15:44:38 -07:00
yassin
d4d8cc9092 fix(proxy): apply access denied message to JWT paths, sanitize denial log, await dashboard saves
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:44:03 +00:00
shivam
65132ef2e9 fix(cost): price native Responses WebSocket turns at their returned service_tier
The logging object built for aresponses_websocket sessions dropped the
service_tier carried by each billable response.completed/response.incomplete
event, so sessions were priced at the default tier. The logging object now
carries the tier when the session is single-tier, and completion_cost splits
mixed-tier sessions into one object per tier before pricing. Fixes #41299

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:42:58 +00:00
yassin
b13cf21fd1 fix(vertex_ai): bill Gemini Omni Interactions usage and Veo sampleCount on passthrough
Route Vertex and Gemini /interactions create responses through the shared
Interactions usage transformation so SpendLogs carry per-modality tokens and
cost instead of zeros, and carry parameters.sampleCount into Veo usage as
video_count so duration-based video cost is multiplied by the number of
generated videos

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:42:27 +00:00
Mateo Wang
6ee1d72ae8
Merge pull request #40872 from BerriAI/litellm_bedrock_sanitize_tool_use_id
fix(bedrock): sanitize client tool_call ids to Bedrock toolUseId constraints
2026-09-15 15:41:39 -07:00
yassin
be082013f7 fix(ui): label the response time chart tooltip with the series name
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:38:55 +00:00
kerry
357e0fca8e chore: merge origin/main into test cleanup
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:36:58 +00:00
yassin
db0f06d153 fix(proxy): skip team model tpm accounting when key owns model tpm limit
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:35:48 +00:00
yassin
ebb9a3ceeb chore(ui): regenerate schema.d.ts for the /nvidia_nim passthrough route
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:34:59 +00:00
yassin
c48723f210 fix(proxy): let a db pass-through entry override the yaml entry on the same path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:34:35 +00:00
Mateo Wang
c3222ec110
Merge pull request #41171 from BerriAI/litellm_converted_stream_spend_tracking
fix(logging): track spend for streams a deployment hook converted to non-streaming
2026-09-15 15:34:13 -07:00
yassin
5d6e367d56 fix(proxy): keep alias rewrite off pass-through bodies and auth-merged params
Skip router_settings.model_group_alias resolution on registered pass-through
routes, rebuild the rewritten body from the cached client payload instead of the
auth-enriched request_data, and centralize the resolved-scope sentinel in
litellm/constants.py

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:32:11 +00:00
mateo-berri
69e9edb91a test(responses): fold the empty-choices regression tests into the mapped streaming iterator test file
Some checks failed
ai-gateway image / ai-gateway release image (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
2026-09-15 15:31:38 -07:00
kerry
f6ef0a24e0 fix(tests): restore rerank response import
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:28:23 +00:00
mateo-berri
fa116a0ecc Merge remote-tracking branch 'origin/main' into litellm_bedrock_sanitize_tool_use_id 2026-09-15 15:27:55 -07:00
yassin
1b6e76af16 fix(proxy): resolve LITELLM_LOG for uvicorn at startup and restore logger state in tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:27:24 +00:00
yassin
168607cc3f fix(proxy): register /nvidia_nim/ with the lazy passthrough loader
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:26:49 +00:00
yassin
754e87fa67 fix(ui): keep gateway auth and content-type headers ahead of playground custom headers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:24:13 +00:00
mateo-berri
f22c7d274a Merge remote-tracking branch 'origin/main' into litellm_bedrock_rag_retrieval_filter
Some checks failed
ai-gateway image / ai-gateway release image (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
# Conflicts:
#	litellm/rag/main.py
#	tests/test_litellm/rag/test_main.py
2026-09-15 15:21:57 -07:00
mateo-berri
a6b0d4266f Merge origin/main into litellm_lit4767_empty_choices_streaming_guard
Takes main's Anthropic Messages adapter iterator, transformation, and
combined-chunk tests as-is: #35314 already skips choiceless chunks at the
top of both adapter loops, so the adapter-side guards this branch carried
are superseded. The Responses bridge guards stay
2026-09-15 15:20:16 -07:00
yassin
5b0fa89056 feat(ui): show average response time per model in usage model activity
Roll request_duration_ms of successful, non-internal requests into the
daily spend tables as total_response_time_ms plus timed_requests, expose
both through the daily activity endpoints, and derive the average in the
Usage -> Model Activity view of the Admin UI

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:16:12 +00:00
kerry
2586b21893 test: keep tests that survive correct cost-map updates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:16:08 +00:00
yassin
9261a72eb7 test(passthrough): cover Vertex router resolution without a matching deployment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:14:44 +00:00
yassin
c05af7a12f feat(ui): add custom request headers to the API Playground
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:12:06 +00:00
yassin
6c8b9a7b05 feat(keys): filter /key/list by active, expired, revoked or deleted status and serve deleted keys from /key/info
Persist and expose the lifecycle of API keys so spend, audit and FinOps
workflows can still resolve a key after it is revoked, expires or is
deleted.

/key/list?status= now accepts active, expired and revoked next to the
existing deleted value. revoked means blocked=true, expired means not
blocked with a past expiry, active is the rest, so the three values
partition the live key table. deleted keeps reading the
LiteLLM_DeletedVerificationToken archive.

/key/info falls back to that archive when the key is no longer in the
live table, running the same owner/team/org authorization check, and
every response now carries a derived status field. The hashed token is
still stripped.

The Virtual Keys page gets a Status filter (URL-persisted) and a Deleted
badge that shows when and by whom the key was deleted.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:12:01 +00:00
yassin
c3e937b845 docs(proxy): describe the single UNION ALL aggregate statement in the endpoint docstring
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:11:20 +00:00
yassin
69be041da7 feat(proxy): add /nvidia_nim passthrough route for NIM object detection and OCR /v1/infer
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:11:08 +00:00
yassin
af4a0b4bc3 fix(proxy): keep yaml pass-through endpoints visible to auth after db overlay
Resolves LIT-2053

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 22:10:42 +00:00