Commit graph

43876 commits

Author SHA1 Message Date
Mateo Wang
60459c60b4
Merge pull request #36181 from BerriAI/litellm_fix_batch_group_fallback
fix(router): keep batch fallbacks inside the model group that owns the file
2026-08-10 09:39:15 -07:00
mateo-berri
efd98bb1b4
fix(ci): bound setup steps so pytest always gets its full budget
The summed job deadline alone did not protect the test budget. Setup that
overran its allowance still ate into pytest's window, which is the same
failure this change set out to remove, just with more headroom.

Every step before pytest now carries its own ceiling, and their sum is the
`setup-timeout-minutes` default. Setup can no longer overrun into the test
budget without failing its own step first, and a slow setup step now reports
as a red step naming itself rather than a cancelled shard whose tests passed.

Model the workflow YAML the guard reads with Pydantic instead of bare dicts,
so the shapes it depends on are validated once at the boundary. A workflow
that does not parse is now reported as a finding rather than a traceback.
2026-08-10 16:22:07 +00:00
mateo-berri
6a83a84f31
ci: cache Prisma CLI and engine binaries, split test timeout from setup
`prisma generate` runs `npm install prisma@<version>` whenever the
prisma-client-py binary cache directory has no CLI entrypoint, pulling ~85 MB
of query and schema engines over the network. Every workflow pointed
PRISMA_BINARY_CACHE_DIR at `${{ runner.temp }}/prisma-cache`, which GitHub
wipes and recreates per job, so that cache was empty on every job of every
run and the download was never avoidable.

The download is normally a few seconds and occasionally minutes. On one
proxy-db run it took 5m18s on a single shard against 3.8s on its eleven
siblings, which pushed the job past its 15 minute timeout and cancelled a
shard whose tests were at 99% and all passing.

Leave PRISMA_BINARY_CACHE_DIR unset so the binaries land in the
prisma-client-py default, which is already keyed by prisma and engine
version, and restore both that path and the @prisma/engines staging cache
through a shared composite action.

Job timeouts also counted setup against the test budget. `timeout-minutes`
now bounds the pytest step, with a separate allowance for checkout,
dependency install, and client generation, so slow setup shows up as a slow
job instead of a cancelled test run.

check_prisma_binary_cache.py guards all three invariants: no workflow
reintroduces the override, every job that generates the client restores the
cache, and the version the action greps out of uv.lock still resolves.
2026-08-10 16:10:54 +00:00
Mateo Wang
33d7e3f862
Merge pull request #36386 from BerriAI/litellm_decrease_anys_fable4 2026-08-10 09:04:27 -07:00
mateo
69def0545d refactor(model_prices): keep fallback_generalizations last in the registry
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-10 16:01:41 +00:00
mateo-berri
8febd8cafb chore(typing): restore one-line ToolParam comment 2026-08-10 07:17:06 -07:00
Devin AI
cb0c96bf46 feat(model_prices): add missing OpenAI transcription, Anthropic Mythos, Gemini robotics streaming and Mistral models
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-10 14:14:37 +00:00
mateo
b5b663fd8b fix(xai): bill the above-200k tier at exactly 200k prompt tokens
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-10 14:13:05 +00:00
Devin AI
76586a8268 fix(model_prices): drop unverified deprecation dates, correct gemini embedding and anthropic opus 4.1
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-10 13:21:33 +00:00
Devin AI
9456564b3c fix(model_prices): refresh deprecation dates and xAI pricing from provider docs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-10 13:11:55 +00:00
devin-ai-integration
1b2430b8b6 fix(bedrock): report uploaded size in the FileObject returned by managed batch uploads 2026-08-10 10:25:37 +00:00
mateo-berri
5ee0b1f5db chore(typing): remove 914 basedpyright Any errors across 16 hotspot files
Whole-tree basedpyright drops from 147,728 to 146,543 errors (reportAny
-670, reportExplicitAny -244) with no rule increasing repo-wide or in
any file. TypedDicts, Mapping/Sequence views, Protocols, and precise
helper return types replace Any; no casts, ignores, or runtime changes.
Budgets ratcheted down by the fixed amounts.
2026-08-10 01:24:40 -07:00
Yang Yang
1705f86d50 fix(responses): add int tokens before summing usage totals 2026-08-09 21:36:47 -07:00
Mateo Wang
f6b9518ddb
Merge pull request #36092 from BerriAI/devin_ai_fix_openai_passthrough_files_route_36086
Some checks are pending
Unit Tests: LLM Provider Transformations / All Other Providers (push) Waiting to run
Unit Tests: MCP, Secrets, Containers & Misc / misc (push) Waiting to run
Unit Tests: Proxy Auth & Key Management / proxy-auth (push) Waiting to run
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Proxy API Endpoints / proxy-endpoints (push) Waiting to run
Unit Tests: Proxy API Endpoints / proxy-server (push) Waiting to run
Unit Tests: Proxy Infrastructure / proxy-infra (push) Waiting to run
Unit Tests: Proxy Legacy Tests / auth-and-jwt (push) Waiting to run
Unit Tests: Proxy Legacy Tests / key-generation (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-config (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-response-and-misc (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-server (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-server-extras (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-token-counter (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-user-auth-and-spend (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-utils (push) Waiting to run
Unit Tests: Responses, Caching & Types / responses-caching-types (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
fix(proxy): stop /{provider}/v1/files from capturing /openai_passthrough
2026-08-09 12:05:20 -07:00
mateo-berri
bd719c21dc test(passthrough): annotate route match scope as Final 2026-08-09 11:49:20 -07:00
Devin AI
4664523d3c build(deps): bump pypdf to 6.15.0 to clear osv-scan
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-09 13:09:28 +00:00
Yuneng Jiang
d554450f62
test(ui): drop test commentary and assert the normalized org-admin denial 2026-08-08 21:20:32 -07:00
Yuneng Jiang
00600c1af7
test(proxy): guard management_v1 against fastapi names removed in supported releases 2026-08-08 21:15:56 -07:00
Yuneng Jiang
e3d3177ff1
style(ui): drop narration comments from the usage gating tests
Both restated what the test name and the surrounding setup already say, so
they were maintenance cost without explanatory value. The reasoning they
carried lives in the commit that added the gates.
2026-08-08 21:12:27 -07:00
yuneng-jiang
ecba48dd7c
Merge pull request #35773 from HuanQian571/litellm_fix_management_v1_get_flat_params
fix(proxy): restore management_v1 query-param validation under fastapi>=0.140.7
2026-08-08 21:09:13 -07:00
Yang Yang
fc102b5f1a fix(xai): replace setattr with assignments for B010 2026-08-08 20:56:03 -07:00
Yang Yang
da69b5bfe8 fix(xai): satisfy type-discipline gate on web search billing 2026-08-08 20:42:32 -07:00
Yuneng Jiang
2502ee4a2a
fix(ui): gate policy and prompt lookups on an admin capability
/policies/list and /prompts/list are default-deny for internal_user, but the
Virtual Keys create/edit flow, the Teams forms and the Playground called them
on mount, so every internal user landing on the dashboard fired two requests
that 401. Add viewPolicies and viewPrompts to the capability map and use them
to gate the nav entry, the form field and the fetch together, following the
pattern from the Tool Policies migration. Non-admins now see no policy or
prompt selector at all rather than an empty dropdown.
2026-08-08 20:34:56 -07:00
Yuneng Jiang
6a540a1bf8
fix(ui): gate organization and agent usage views behind capabilities
The Usage page admits internal users because their own usage view works,
but the entity breakdown selector inside it also offered Organization
Usage, so picking it fired /organization/daily/activity and collected a
401. Neither that route nor /agent/daily/activity appears in any non-admin
route list, so both are default-deny. The team breakdown leaked the second
one too: it fetches agent activity unconditionally to fill its Top Agents
card, which 401s for the same roles.

Adds viewOrganizationUsage and viewAgentUsage to the existing capability
map and points the selector option, the page section, and the fetch's
enabled flag at the same capability, so a role that cannot call the
endpoint never sees the breakdown and never issues the request. The team
and tag breakdowns, which internal users can read, are untouched, and the
default Usage view was already one of those.
2026-08-08 20:17:12 -07:00
Yuneng Jiang
30c4898de9
fix(ui): hide admin-only Logs tabs from roles that cannot call their endpoints
The Logs nav entry is open to internal users so they can read their own
request logs, but the page rendered all four tabs unconditionally. Audit
Logs calls GET /audit and Deleted Teams calls GET /v2/team/list?status=deleted,
neither of which an internal user is permitted to call, so the page fired
requests that came back 401.

Gate both tabs on new viewAuditLogs / viewDeletedTeams capabilities, using
the same CAPABILITY_ROLES map and useCan hook introduced for Tool Policies.
Hiding a tab drops its panel from the tree entirely, so the request is never
issued rather than issued and rejected.

Selecting a tab also mapped index 0 to "request logs" and every other index
to "audit logs", which activated the audit panel whenever a user opened
Deleted Keys or Deleted Teams. Derive the active tab from the visible tab
list instead, so the mapping survives tabs being filtered out.
2026-08-08 20:16:22 -07:00
Devin AI
25144fc03c chore: retrigger ci after docs merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-09 03:11:12 +00:00
Yang Yang
21f742041a fix(xai): type tool usage details helpers without Any 2026-08-08 20:05:43 -07:00
Devin AI
15a6664171 fix(search): keep signed auth headers out of logging callbacks
Log the pre-signing headers in the search pre_call hook so SigV4 and bearer Authorization values are never handed to user-configured logger callbacks, and tighten sign_request's annotations.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-09 02:17:44 +00:00
Devin AI
b1d77bb5db style: ruff format agentcore search transformation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-09 01:36:33 +00:00
Devin AI
07f14617f6 feat(search): add Amazon Bedrock AgentCore web search provider
Adds 'agentcore' as a search provider backed by an AgentCore Gateway MCP web-search target, usable from litellm.search()/`/search` and as a websearch_interception backend. Supports SigV4 (AWS_IAM gateways) and bearer tokens (CUSTOM_JWT gateways) via a new BaseSearchConfig.sign_request hook.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-09 01:32:21 +00:00
mateo-berri
933c18b21c Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_batch_group_fallback
# Conflicts:
#	litellm/router_utils/fallback_event_handlers.py
#	tests/test_litellm/router_utils/test_fallback_event_handlers.py
2026-08-08 18:04:18 -07:00
mateo-berri
85c1b5d04a Merge remote-tracking branch 'origin/litellm_internal_staging' into devin_ai_fix_openai_passthrough_files_route_36086
Some checks failed
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
2026-08-08 18:01:59 -07:00
Mateo Wang
e9d1ea59fb
Merge pull request #36326 from BerriAI/litellm_files_list_has_more_scoped
Some checks are pending
Unit Tests: LLM Provider Transformations / All Other Providers (push) Waiting to run
Unit Tests: MCP, Secrets, Containers & Misc / misc (push) Waiting to run
Unit Tests: Proxy Auth & Key Management / proxy-auth (push) Waiting to run
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Proxy API Endpoints / proxy-endpoints (push) Waiting to run
Unit Tests: Proxy API Endpoints / proxy-server (push) Waiting to run
Unit Tests: Proxy Infrastructure / proxy-infra (push) Waiting to run
Unit Tests: Proxy Legacy Tests / auth-and-jwt (push) Waiting to run
Unit Tests: Proxy Legacy Tests / key-generation (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-config (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-response-and-misc (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-server (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-server-extras (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-token-counter (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-user-auth-and-spend (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-utils (push) Waiting to run
Unit Tests: Responses, Caching & Types / responses-caching-types (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
fix(proxy): report has_more false on caller-scoped file list pages
2026-08-08 17:56:10 -07:00
mateo-berri
82662dc104 fix(proxy): report has_more false on caller-scoped file list pages 2026-08-08 17:28:43 -07:00
Mateo Wang
6eaeab8eae
Merge pull request #36273 from BerriAI/litellm_dbless_hook_registration
fix(proxy): skip prisma-dependent hooks when no database is attached
2026-08-08 17:25:23 -07:00
Mateo Wang
1f01f19457
Merge pull request #36093 from BerriAI/devin_ai_fix_file_list_cursor_leak_36087
fix(proxy): scope file list pagination cursors to the caller
2026-08-08 17:22:30 -07:00
Yang Yang
4a536098e1 style: ruff format xAI cost calculator and responses transform 2026-08-08 16:58:25 -07:00
Mateo Wang
b205bf9f4b
Merge pull request #36301 from BerriAI/litellm_claude_md_comment_exceptions
docs: rewrite the CLAUDE.md comment rule with explicit exceptions
2026-08-08 16:54:16 -07:00
Yang Yang
749a8b0701 fix(xai): keep chat Usage through Responses completions bridge
xAI already converts Responses usage to chat Usage so web_search_calls survive
cost tracking. The chat completions bridge then re-ran the Responses usage
transform and crashed on missing input_tokens. Pass through already-chat Usage
and chat-shaped dumps instead
2026-08-08 16:49:09 -07:00
mateo-berri
2112422c71 test(managed-files): read the scoped page id from the row's unified_file_id 2026-08-08 16:34:26 -07:00
mateo-berri
508e0dbb35 Merge remote-tracking branch 'origin/litellm_internal_staging' into devin_ai_fix_file_list_cursor_leak_36087 2026-08-08 16:31:33 -07:00
Yang Yang
a9277b4b6e style: black format xAI cost calculator tests 2026-08-08 16:22:06 -07:00
Yang Yang
74100989a2 revert: remove xAI-specific web search gate from shared cost tracking
Gate web search like OpenAI (output/annotations/web_search_requests).
xAI uses server_side_tool_usage_details only for per-call cost math, with
web_search_requests mirrored in llms/xai for existing gate compatibility.
2026-08-08 16:22:06 -07:00
Yang Yang
8ad0a57387 style: drop unused pytest import in xAI responses tests 2026-08-08 16:21:49 -07:00
Yang Yang
478118ac36 test(xai): expand cost_calculator coverage for web search helpers
Add unit tests for apply_server_side_tool_usage_details_to_usage edge
cases and model_info-driven web_search per-call pricing fallbacks.
2026-08-08 16:21:49 -07:00
Yang Yang
c03a6076ce test(xai): cover Responses tool usage attach helpers
Add unit tests for server_side_tool_usage_details extraction/attach and
streaming completed-event pass-through in XAIResponsesAPIConfig.
2026-08-08 16:21:49 -07:00
Yang Yang
ea493543d7 fix(xai): attach tool usage details on Responses stream terminal events
Apply server_side_tool_usage_details on completed/incomplete/failed
streaming events so stream=true web_search is billed like non-stream.
2026-08-08 16:21:49 -07:00
Yang Yang
7aaa9358aa fix(xai): read web_search per-call rate from model_info
Use search_context_cost_per_query from the model cost map (with $5/1k
fallback) so web search billing can change via pricing JSON updates.
2026-08-08 16:21:49 -07:00
Yang Yang
8687d7372a fix(xai): gate web search cost on server_side_tool_usage_details
Treat positive web_search_calls as a web-search signal in built-in tool
cost gating, and mirror counts onto prompt_tokens_details.web_search_requests
when attaching xAI tool usage details so charges are not skipped.
2026-08-08 16:21:49 -07:00
Yang Yang
ea98d8e116 fix(xai): read tool usage details from ResponseAPIUsage extras
Also inspect model_extra when attaching server_side_tool_usage_details
for Responses cost tracking.
2026-08-08 16:21:06 -07:00