Commit graph

45353 commits

Author SHA1 Message Date
mateo
6f7050c05f docs: make an unfilled feature request attempt visibly empty and attested
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-11 10:40:57 -07:00
mateo
471bb834ba docs: rename the feature request flow lists to before/after this feature
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-11 10:40:57 -07:00
mateo
c8f3854833 docs: require a user flow and a stuck-at proof in feature requests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-11 10:40:57 -07:00
devin-ai-integration[bot]
657c4c2b1e
Merge pull request #36498 from BerriAI/litellm_issue_template_user_flow_proof
docs: require a user flow and live-proxy proof in bug reports
2026-08-11 10:40:52 -07:00
mateo-berri
d608948eef Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_decrease_anys_fable5
# Conflicts:
#	litellm/proxy/common_request_processing.py
#	litellm/proxy/pass_through_endpoints/streaming_handler.py
2026-08-11 10:39:02 -07:00
Mateo Wang
02b0ee7608
Merge pull request #36503 from BerriAI/litellm_fix_openai_passthrough_stream_cost
fix(proxy): inject streaming usage cost on openai passthrough streams
2026-08-11 10:06:12 -07:00
tin-berri
dca7ba18d4
feat(ui): show models under each tier in routing benchmark chart (#36291)
* feat(ui): show models under each tier in routing benchmark chart

- Add TierTurnsChart: donut chart showing turns per complexity tier with
  tier-assigned models listed below each tier name in the legend
- Only complexity routers show models; quality routers show tier name + %
  (quality tiers don't pin specific models)
- Change 'Estimated spend at highest-cost model' wording to 'highest-tier'
  to clarify it's the most capable tier's estimated cost, not just the
  single-highest model

Closes LIT-5302

* fix(ui): use categorical colors for tier donut, trim redundant turn count

- Tier donut chart now uses a dedicated categorical palette instead of
  SEQUENTIAL_COLOR_RAMP, which is a blue monochrome gradient meant for
  magnitude series, not distinct categories.
- Space out the tier legend rows (gap-3 -> gap-6) for readability.
- Drop the turn count from "avg saved per session" since Routing by
  tier already shows the total turns.

Co-Authored-By: Claude <noreply@anthropic.com>

* test(ui): drop prohibited explanatory comments in TierTurnsChart test

Per repo convention against source comments; the test name and
assertions already communicate the scenario. Addresses Greptile review.

Co-Authored-By: Claude <noreply@anthropic.com>

* Remove dead modulo from color index in TierTurnsChart

The colors array is built with length equal to slices.length, so idx % colors.length
is always a no-op in the render loop. Simplify to idx for clarity.

* fix(ui): wrap CostOptimizationView tests in QueryClientProvider

The tests render CostOptimizationView which uses useCan() → useIsOrgAdmin() →
useOrganizations() and useDailyActivityRange(), both of which call React Query's
useQuery(). Without QueryClientProvider wrapping the render, React Query throws
'No QueryClient set' error.

Also mock the required networking calls (organizationListCall, userDailyActivityCall)
to prevent spurious network errors in test runs.

All 7 tests now pass (CostOptimizationView + CostOptimizationView.activity).

* style(ui): format test files and extract object literal to fix linting

- Format CostOptimizationView.test.tsx with prettier
- Extract getToolSpend mock response to named variable to satisfy eslint
- Pass frontend-lint checks

* fix(ui): hoist mockToolSpendResponse into vi.hoisted to fix test initialization

Extracting the response object to a named variable violated hoisting rules:
vi.mock() factories are evaluated at hoisting time before regular const
declarations. Move mockToolSpendResponse into vi.hoisted() block.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-11 09:41:55 -07:00
mateo
ad1ff19195 chore: say real $ instead of $$$ in the bug report proof rules
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-11 16:31:19 +00:00
mateo
10d4213453 chore: drop the bug report proof attestation and tighten its wording
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-11 16:25:52 +00:00
Mateo Wang
0a41b19e5b
chore: make it more brief 2026-08-11 08:33:22 -07:00
mateo-berri
262d1b4ca0 fix: remove over-strict stream logging validator, restore base seam behavior 2026-08-11 08:22:57 -07:00
mateo-berri
df5425675e fix(schema): declare supports_tool_search in the model prices schemas 2026-08-11 15:16:31 +00:00
mateo-berri
e5ac4e0068 chore(typing): clear 1.6k basedpyright Any errors across 56 files
reportAny 16720 -> 15482 and reportExplicitAny 5689 -> 5316 with real types only: no casts, no ignores, no new Any. Whole-tree basedpyright drops 2173 diagnostics with zero per-rule or per-file regressions. Budgets ratcheted: basedpyright -2173, ruff-strict -188, type-discipline -55
2026-08-11 06:47:39 -07:00
Devin AI
20c0e0dc91 fix(model_prices): add provider-announced deprecation dates for Bedrock, Mistral, Cohere, Gemini models
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-11 13:13:25 +00:00
william-xue
c8655c3825 fix(proxy): track streamed passthrough Responses cost 2026-08-11 18:30:42 +08:00
mateo-berri
7d4488d2e8 refactor(bedrock): read tool search support from the model map
Record supports_tool_search on the Bedrock Claude entries in both cost
map files and have _supports_tool_search_on_bedrock read it first via
the provider-resolved capability lookup, keeping the name patterns as a
fallback for ARNs and ids the map cannot resolve. Threads the flag
through ModelInfoBase and drops a dated remark from the pattern list
2026-08-11 10:06:04 +00:00
ansh-agrawal
67c4eb86b1 feat(proxy): add opt-in flag to require rpm/tpm for project models (create + update) 2026-08-11 15:22:56 +05:30
mateo-berri
0f41365c34 fix(bedrock): forward output_config effort for application inference profile ARNs 2026-08-11 09:11:22 +00:00
ozolam
141281b1d6 ci: retrigger CI 2026-08-11 11:22:12 +03:00
ozolam
a33378bdc1 fix(claude-code): fix prettier formatting and remove unused import 2026-08-11 11:22:12 +03:00
ozolam
05c91aa5f2 fix(claude-code): correct skill install command and marketplace setup UX
- formatInstallCommand now produces /plugin install {name}@litellm instead of /plugin marketplace add {source}
- extraKnownMarketplaces snippet fixed: source must be a nested object not a flat string; the flat string caused Claude Code to reject the settings file
- marketplace key renamed from my-org to litellm to match the name the proxy returns in marketplace.json
- setup tab now shows /plugin marketplace add command as primary option with settings.json as secondary
- usage tab now shows a hint to run /plugin marketplace update litellm when a plugin is not found
2026-08-11 11:22:12 +03:00
mateo-berri
e7c8cff3b7 fix(proxy): preserve crlf line endings when injecting streamed usage cost 2026-08-11 00:45:32 -07:00
Kunal Nayyar
526fc9eab1 fix(ui): title validation errors correctly instead of Rate Limit Exceeded
The /model/new endpoint returns a 400 validation error (type: validation_error)
when 'rpm and tpm must be set to a positive value when enforce_rpm_tpm_on_model_add
is enabled in general_settings' but the frontend's titleFor() keyword matcher
mistitled it as 'Rate Limit Exceeded' because the message contains 'rpm'/'tpm'
substrings, which matched the generic rate-limit keyword check before the more
specific validation check could catch it.

Add "'enforce_rpm_tpm_on_model_add' is enabled" to VALIDATION_MATCH so this
message is classified as a Validation Error, matching the actual HTTP 400
validation_error the backend already returns. A narrow match on the setting
name (rather than the generic "must be set when") avoids overriding the
status-based classification of unrelated 401s, e.g. the PKCE
'GENERIC_CLIENT_ID must be set when PKCE is enabled' error.
2026-08-11 13:03:55 +05:30
Kunal Nayyar
65eae963a7 feat(proxy): opt-in enforce rpm/tpm when adding a model
Add general_settings toggle 'enforce_rpm_tpm_on_model_add' (default false).
When true, /model/new rejects a model whose rpm or tpm is missing or not a
positive value, so the Admin UI Add Model form surfaces a 400 validation
error instead of silently storing an unbounded model (or one with a
zero/negative limit that would exclude it from routing).
2026-08-11 13:03:55 +05:30
Souravrajvi0
e3da917e67 fix(proxy): parse form-encoded video edit/extension bodies after auth
Fixes #36487

video_edit, video_extension, and video_remix called request.body() after
user_api_key_auth had already parsed multipart/form bodies via
_read_request_body(), causing RuntimeError Stream consumed and 500s for
OpenAI SDK clients.

Use _read_request_body consistently and normalize bare-string or JSON-string
video references from form fields into video_id.
2026-08-11 06:29:07 +00:00
mubashir1osmani
b0fac57fe4
fix(email): stop duplicate legacy invitation email and fix its onboarding link (#36455) 2026-08-10 23:26:27 -07:00
Mateo Wang
1a8cd8a078
Merge pull request #34290 from eugene-yao-zocdoc/litellm_fix_anthropic_midturn_interjections
fix(anthropic): preserve midturn system corrections
2026-08-10 22:55:09 -07:00
Mateo Wang
c4656d806c
Merge pull request #36160 from daleselaji-dev/codex/bedrock-s3-credentials-36155
fix(bedrock): use deployment credentials for AWS requests
2026-08-10 22:46:37 -07:00
mateo-berri
1b488f7c2f fix(proxy): ban caller-supplied aws identity selectors in request bodies 2026-08-10 22:05:49 -07:00
mateo-berri
1819c97c27 style: format streaming handler 2026-08-10 21:50:43 -07:00
mateo-berri
b9b200b348 fix(anthropic): keep tool exchanges intact around midturn system write-back 2026-08-10 21:43:42 -07:00
mateo-berri
938396ef90 fix(proxy): recognize crlf sse frame boundaries in passthrough reassembly 2026-08-10 21:37:07 -07:00
mateo-berri
929ee52b87 fix(bedrock): preserve adaptive thinking effort through the /v1/messages bridge
Claude Code drives Opus 4.7 with thinking {"type": "adaptive"} plus
output_config {"effort": "max"}. The anthropic-to-openai adapter
forwarded thinking verbatim for Claude models but dropped output_config,
and Bedrock Converse streams zero reasoningContent blocks for adaptive
thinking without an effort tier. Forward the effort subset of
output_config for Bedrock targets, accept it in the converse supported
params, and map it with the model's effort ceiling applied. Re-enable
the skipped e2e compat cell that catches this
2026-08-11 03:23:16 +00:00
mateo-berri
b5eed5e526 test(bedrock): cover the aws request param merge guard in the unit suite 2026-08-10 20:08:20 -07:00
mateo-berri
46fb1cd514 fix(proxy): reassemble fragmented SSE frames and inject logging dependency 2026-08-10 20:07:57 -07:00
mateo
5643a59aa4 fix: move bug report guidance out of prefilled values so required means filled
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-11 02:58:21 +00:00
mateo-berri
426b909447 fix(proxy): inject streaming usage cost on openai passthrough streams 2026-08-10 19:56:34 -07:00
mateo
7a17735473 fix(triage): treat an unfilled bug report scaffold as missing proof
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-11 02:56:13 +00:00
mateo
f8a5d6a6a1 docs: make an unfilled bug report proof visibly empty and attested
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-11 02:49:57 +00:00
mateo-berri
b63ba63655 fix(router): preserve aws session token and role params in deployment credential resolution 2026-08-10 19:46:41 -07:00
mateo-berri
bc98c67028 fix(bedrock): send tool-search beta header for Haiku 4.5 on Invoke /v1/messages
Bedrock InvokeModel rejects tool_search_tool_* tool types unless the
request body carries the tool-search-tool-2025-10-19 beta. The model
allowlist gating that beta omitted Haiku 4.5 (and Opus 4.7, supported
since launch per live verification), so every tool-search request on
those models got a Bedrock 400. Add both to the allowlist and re-enable
the e2e compat cell that caught it.
2026-08-11 02:46:27 +00:00
mateo
48fa4a0f06 fix(ui): treat router redis as configured for the no-redis banner
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-11 02:40:14 +00:00
mateo-berri
78dd81c014 Merge origin/litellm_internal_staging into litellm_fix_anthropic_midturn_interjections 2026-08-10 19:36:35 -07:00
mateo
550682d5f8 docs: redact only sensitive env vars in bug report proof
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-11 02:36:11 +00:00
mateo-berri
79accda090 fix(bedrock): add text block to converse user messages carrying documents 2026-08-11 02:35:49 +00:00
mateo
7653532e89 docs: spell out secret redaction in the bug report proof
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-11 02:35:44 +00:00
mateo
7e8faf9267 docs: ask bug reports for the config and version behind the proof
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-11 02:33:10 +00:00
Devin AI
0cd28c5c40 chore(dashscope): drop cost map regression test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-11 02:32:46 +00:00
tin-berri
6f36bee6ba
feat(ui): deployment affinity toggle for the auto-router (#36302)
Some checks failed
Unit Tests: Proxy Auth & Key Management / proxy-auth (push) Waiting to run
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Proxy API Endpoints / proxy-endpoints (push) Waiting to run
Unit Tests: Proxy API Endpoints / proxy-server (push) Waiting to run
Unit Tests: Proxy Infrastructure / proxy-infra (push) Waiting to run
Unit Tests: Proxy Legacy Tests / auth-and-jwt (push) Waiting to run
Unit Tests: Proxy Legacy Tests / key-generation (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-config (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-response-and-misc (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-server (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-server-extras (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-token-counter (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-user-auth-and-spend (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-utils (push) Waiting to run
Unit Tests: Responses, Caching & Types / responses-caching-types (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
2026-08-10 19:31:20 -07:00
mateo-berri
f72ddedf39 fix(bedrock): keep s3_region_name authoritative over merged deployment region 2026-08-10 19:28:05 -07:00