Commit graph

42657 commits

Author SHA1 Message Date
Mateo Wang
60459c60b4
Merge pull request #36181 from BerriAI/litellm_fix_batch_group_fallback
fix(router): keep batch fallbacks inside the model group that owns the file
2026-08-10 09:39:15 -07:00
mateo-berri
efd98bb1b4
fix(ci): bound setup steps so pytest always gets its full budget
The summed job deadline alone did not protect the test budget. Setup that
overran its allowance still ate into pytest's window, which is the same
failure this change set out to remove, just with more headroom.

Every step before pytest now carries its own ceiling, and their sum is the
`setup-timeout-minutes` default. Setup can no longer overrun into the test
budget without failing its own step first, and a slow setup step now reports
as a red step naming itself rather than a cancelled shard whose tests passed.

Model the workflow YAML the guard reads with Pydantic instead of bare dicts,
so the shapes it depends on are validated once at the boundary. A workflow
that does not parse is now reported as a finding rather than a traceback.
2026-08-10 16:22:07 +00:00
mateo-berri
6a83a84f31
ci: cache Prisma CLI and engine binaries, split test timeout from setup
`prisma generate` runs `npm install prisma@<version>` whenever the
prisma-client-py binary cache directory has no CLI entrypoint, pulling ~85 MB
of query and schema engines over the network. Every workflow pointed
PRISMA_BINARY_CACHE_DIR at `${{ runner.temp }}/prisma-cache`, which GitHub
wipes and recreates per job, so that cache was empty on every job of every
run and the download was never avoidable.

The download is normally a few seconds and occasionally minutes. On one
proxy-db run it took 5m18s on a single shard against 3.8s on its eleven
siblings, which pushed the job past its 15 minute timeout and cancelled a
shard whose tests were at 99% and all passing.

Leave PRISMA_BINARY_CACHE_DIR unset so the binaries land in the
prisma-client-py default, which is already keyed by prisma and engine
version, and restore both that path and the @prisma/engines staging cache
through a shared composite action.

Job timeouts also counted setup against the test budget. `timeout-minutes`
now bounds the pytest step, with a separate allowance for checkout,
dependency install, and client generation, so slow setup shows up as a slow
job instead of a cancelled test run.

check_prisma_binary_cache.py guards all three invariants: no workflow
reintroduces the override, every job that generates the client restores the
cache, and the version the action greps out of uv.lock still resolves.
2026-08-10 16:10:54 +00:00
Mateo Wang
33d7e3f862
Merge pull request #36386 from BerriAI/litellm_decrease_anys_fable4 2026-08-10 09:04:27 -07:00
mateo
69def0545d refactor(model_prices): keep fallback_generalizations last in the registry
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-10 16:01:41 +00:00
mateo-berri
8febd8cafb chore(typing): restore one-line ToolParam comment 2026-08-10 07:17:06 -07:00
Devin AI
cb0c96bf46 feat(model_prices): add missing OpenAI transcription, Anthropic Mythos, Gemini robotics streaming and Mistral models
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-10 14:14:37 +00:00
mateo
b5b663fd8b fix(xai): bill the above-200k tier at exactly 200k prompt tokens
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-10 14:13:05 +00:00
Devin AI
76586a8268 fix(model_prices): drop unverified deprecation dates, correct gemini embedding and anthropic opus 4.1
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-10 13:21:33 +00:00
Devin AI
9456564b3c fix(model_prices): refresh deprecation dates and xAI pricing from provider docs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-10 13:11:55 +00:00
mateo-berri
5ee0b1f5db chore(typing): remove 914 basedpyright Any errors across 16 hotspot files
Whole-tree basedpyright drops from 147,728 to 146,543 errors (reportAny
-670, reportExplicitAny -244) with no rule increasing repo-wide or in
any file. TypedDicts, Mapping/Sequence views, Protocols, and precise
helper return types replace Any; no casts, ignores, or runtime changes.
Budgets ratcheted down by the fixed amounts.
2026-08-10 01:24:40 -07:00
Mateo Wang
f6b9518ddb
Merge pull request #36092 from BerriAI/devin_ai_fix_openai_passthrough_files_route_36086
Some checks are pending
Unit Tests: LLM Provider Transformations / All Other Providers (push) Waiting to run
Unit Tests: MCP, Secrets, Containers & Misc / misc (push) Waiting to run
Unit Tests: Proxy Auth & Key Management / proxy-auth (push) Waiting to run
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Proxy API Endpoints / proxy-endpoints (push) Waiting to run
Unit Tests: Proxy API Endpoints / proxy-server (push) Waiting to run
Unit Tests: Proxy Infrastructure / proxy-infra (push) Waiting to run
Unit Tests: Proxy Legacy Tests / auth-and-jwt (push) Waiting to run
Unit Tests: Proxy Legacy Tests / key-generation (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-config (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-response-and-misc (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-server (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-server-extras (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-token-counter (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-user-auth-and-spend (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-utils (push) Waiting to run
Unit Tests: Responses, Caching & Types / responses-caching-types (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
fix(proxy): stop /{provider}/v1/files from capturing /openai_passthrough
2026-08-09 12:05:20 -07:00
mateo-berri
bd719c21dc test(passthrough): annotate route match scope as Final 2026-08-09 11:49:20 -07:00
Devin AI
4664523d3c build(deps): bump pypdf to 6.15.0 to clear osv-scan
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-09 13:09:28 +00:00
Yuneng Jiang
d554450f62
test(ui): drop test commentary and assert the normalized org-admin denial 2026-08-08 21:20:32 -07:00
Yuneng Jiang
00600c1af7
test(proxy): guard management_v1 against fastapi names removed in supported releases 2026-08-08 21:15:56 -07:00
Yuneng Jiang
e3d3177ff1
style(ui): drop narration comments from the usage gating tests
Both restated what the test name and the surrounding setup already say, so
they were maintenance cost without explanatory value. The reasoning they
carried lives in the commit that added the gates.
2026-08-08 21:12:27 -07:00
yuneng-jiang
ecba48dd7c
Merge pull request #35773 from HuanQian571/litellm_fix_management_v1_get_flat_params
fix(proxy): restore management_v1 query-param validation under fastapi>=0.140.7
2026-08-08 21:09:13 -07:00
Yuneng Jiang
2502ee4a2a
fix(ui): gate policy and prompt lookups on an admin capability
/policies/list and /prompts/list are default-deny for internal_user, but the
Virtual Keys create/edit flow, the Teams forms and the Playground called them
on mount, so every internal user landing on the dashboard fired two requests
that 401. Add viewPolicies and viewPrompts to the capability map and use them
to gate the nav entry, the form field and the fetch together, following the
pattern from the Tool Policies migration. Non-admins now see no policy or
prompt selector at all rather than an empty dropdown.
2026-08-08 20:34:56 -07:00
Yuneng Jiang
6a540a1bf8
fix(ui): gate organization and agent usage views behind capabilities
The Usage page admits internal users because their own usage view works,
but the entity breakdown selector inside it also offered Organization
Usage, so picking it fired /organization/daily/activity and collected a
401. Neither that route nor /agent/daily/activity appears in any non-admin
route list, so both are default-deny. The team breakdown leaked the second
one too: it fetches agent activity unconditionally to fill its Top Agents
card, which 401s for the same roles.

Adds viewOrganizationUsage and viewAgentUsage to the existing capability
map and points the selector option, the page section, and the fetch's
enabled flag at the same capability, so a role that cannot call the
endpoint never sees the breakdown and never issues the request. The team
and tag breakdowns, which internal users can read, are untouched, and the
default Usage view was already one of those.
2026-08-08 20:17:12 -07:00
Yuneng Jiang
30c4898de9
fix(ui): hide admin-only Logs tabs from roles that cannot call their endpoints
The Logs nav entry is open to internal users so they can read their own
request logs, but the page rendered all four tabs unconditionally. Audit
Logs calls GET /audit and Deleted Teams calls GET /v2/team/list?status=deleted,
neither of which an internal user is permitted to call, so the page fired
requests that came back 401.

Gate both tabs on new viewAuditLogs / viewDeletedTeams capabilities, using
the same CAPABILITY_ROLES map and useCan hook introduced for Tool Policies.
Hiding a tab drops its panel from the tree entirely, so the request is never
issued rather than issued and rejected.

Selecting a tab also mapped index 0 to "request logs" and every other index
to "audit logs", which activated the audit panel whenever a user opened
Deleted Keys or Deleted Teams. Derive the active tab from the visible tab
list instead, so the mapping survives tabs being filtered out.
2026-08-08 20:16:22 -07:00
mateo-berri
933c18b21c Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_batch_group_fallback
# Conflicts:
#	litellm/router_utils/fallback_event_handlers.py
#	tests/test_litellm/router_utils/test_fallback_event_handlers.py
2026-08-08 18:04:18 -07:00
mateo-berri
85c1b5d04a Merge remote-tracking branch 'origin/litellm_internal_staging' into devin_ai_fix_openai_passthrough_files_route_36086
Some checks failed
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
2026-08-08 18:01:59 -07:00
Mateo Wang
e9d1ea59fb
Merge pull request #36326 from BerriAI/litellm_files_list_has_more_scoped
Some checks are pending
Unit Tests: Proxy Infrastructure / proxy-infra (push) Waiting to run
Unit Tests: Proxy Legacy Tests / key-generation (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-config (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-response-and-misc (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-server (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-server-extras (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-token-counter (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-user-auth-and-spend (push) Waiting to run
Unit Tests: LLM Provider Transformations / All Other Providers (push) Waiting to run
Unit Tests: MCP, Secrets, Containers & Misc / misc (push) Waiting to run
Unit Tests: Proxy Auth & Key Management / proxy-auth (push) Waiting to run
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests: Proxy API Endpoints / proxy-endpoints (push) Waiting to run
Unit Tests: Proxy API Endpoints / proxy-server (push) Waiting to run
Unit Tests: Proxy Legacy Tests / auth-and-jwt (push) Waiting to run
Unit Tests: Proxy Legacy Tests / proxy-utils (push) Waiting to run
Unit Tests: Responses, Caching & Types / responses-caching-types (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
fix(proxy): report has_more false on caller-scoped file list pages
2026-08-08 17:56:10 -07:00
mateo-berri
82662dc104 fix(proxy): report has_more false on caller-scoped file list pages 2026-08-08 17:28:43 -07:00
Mateo Wang
6eaeab8eae
Merge pull request #36273 from BerriAI/litellm_dbless_hook_registration
fix(proxy): skip prisma-dependent hooks when no database is attached
2026-08-08 17:25:23 -07:00
Mateo Wang
1f01f19457
Merge pull request #36093 from BerriAI/devin_ai_fix_file_list_cursor_leak_36087
fix(proxy): scope file list pagination cursors to the caller
2026-08-08 17:22:30 -07:00
Mateo Wang
b205bf9f4b
Merge pull request #36301 from BerriAI/litellm_claude_md_comment_exceptions
docs: rewrite the CLAUDE.md comment rule with explicit exceptions
2026-08-08 16:54:16 -07:00
mateo-berri
2112422c71 test(managed-files): read the scoped page id from the row's unified_file_id 2026-08-08 16:34:26 -07:00
mateo-berri
508e0dbb35 Merge remote-tracking branch 'origin/litellm_internal_staging' into devin_ai_fix_file_list_cursor_leak_36087 2026-08-08 16:31:33 -07:00
yucheng-berri
efc4e6f28c
fix(batches): keep batch state in sync on a poll without claiming attribution (#34456)
A poll of a Vertex passthrough batch wrote nothing to the managed-object row,
so status and file_object stayed frozen at the create-time snapshot and
GET /v1/batches served a stale status and an empty output file id for the life
of the batch. Only the create may claim a batch, but every observation of one
may refresh its state.

store_unified_object_id takes create_if_missing, which the poll clears: it
refreshes status and file_object through update_many, and leaves a row that is
absent absent rather than creating one owned by the observer, since created_by
and team_id are written by whoever reaches the create branch. The update payload
is now shared with the upsert so it cannot drift into writing api_key,
request_tags, created_by or team_id.

The passthrough identity re-assertion that was previously part of this PR ships
separately in #36121, so this PR keeps only the batch attribution work.

The creating key owns user_api_key_alias only when it actually has one. Guarding
the overwrite on the presence of a key rather than on a resolved alias nulled the
field out for every key generated without key_alias, and for any key rotated or
deleted before its batch finished, losing the creating user's alias that the spend
row previously carried. The guard now matches the team-alias line below it.
2026-08-08 16:01:47 -07:00
Mateo Wang
805fc49776
chore: mention AI slop reason 2026-08-08 14:29:36 -07:00
yuneng-jiang
27d2fa8481
Merge pull request #36297 from BerriAI/litellm_/release-ui-build-528a42
chore: rebuild Admin UI bundle for the 2026-08-08 release
2026-08-08 14:21:24 -07:00
Mateo Wang
1f8964a4c4
chore: handwrite the rule 2026-08-08 14:20:13 -07:00
mateo
6e6e0d662b docs: frame the comment rule around AI slop and allow TODO/FIXME
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-08 21:07:40 +00:00
mateo
abbad8ad52 docs: limit the comment exception to tool-read directives
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-08 21:04:43 +00:00
mateo
8812debeff docs: allow functional comments as an exception in CLAUDE.md
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-08 20:58:30 +00:00
Yuneng Jiang
84d63cbdcf
chore: update Next.js build artifacts (2026-08-08 20:14 UTC, node v24.19.0) 2026-08-08 13:14:45 -07:00
yuneng-jiang
97a59c8c90
Merge pull request #36293 from BerriAI/litellm_fix_circleci_88641_outdated_tests
test: repair stale CircleCI contracts
2026-08-08 13:08:22 -07:00
tin-berri
e35ee4e5fa
feat(router): independent, default-on deployment affinity for the auto-router (#36146) 2026-08-08 13:02:29 -07:00
Mateo Wang
554f065361
Merge pull request #36296 from BerriAI/litellm_claude_md_descending_importance
docs: clarify guideline priority ordering in CLAUDE.md
2026-08-08 13:01:38 -07:00
mateo
ff5f8132d1 docs: clarify guideline priority ordering in CLAUDE.md
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-08 19:55:37 +00:00
Yuneng Jiang
1a40a67394
fix: stabilize generated user role ordering 2026-08-08 12:54:23 -07:00
Mateo Wang
334e6dabf4
Merge pull request #36295 from BerriAI/litellm_remove_pre_commit_rule
chore: remove pre-commit rule
2026-08-08 12:44:48 -07:00
Shivam Rawat
7b89b3a29f
Merge pull request #35708 from BerriAI/devin_ai_lit_5033_websearch_interception_spend
fix(websearch_interception): bill intercepted searches to the calling key
2026-08-08 12:41:31 -07:00
devin-ai-integration[bot]
12aeb53aec
fix(otel): mark v2 server spans as failed for pre-call errors (#34546)
* fix(otel): mark v2 server spans as failed for pre-call errors (LIT-4780)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(otel): authenticate malformed-body requests before rejecting them (LIT-4780)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(auth): cover malformed-body rejection when auth error is recovered

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(auth): skip authorization for a request whose body never parsed

Deferring the parse failure ran the full auth phase, including budget reservation, whose reserved amount is only released by the endpoint's post call path; the endpoint never runs, so malformed requests leaked reservations and locked a budgeted key out. Authorization now runs only when the body parsed, and a parse failure with a rejected key keeps returning the 400 it returned before.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: shivam <shivam@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-08 12:40:00 -07:00
Mateo Wang
4150248095
chore: remove pre-commit rule
some users do not use make pre-commit as it is a multi-minute process. I personally use it but I want users themselves to decide whether to pre-commit before each commit or not, based on what works best for them
2026-08-08 12:38:23 -07:00
devin-ai-integration[bot]
cfd64d45a8
fix(ui): show team BYOK models in team fallback settings (#36241)
* fix(ui): show team BYOK models in team fallback settings

Team router settings loaded fallback options from /model_group/info, which resolves models without a team, so a team's own BYOK deployments were never selectable in its own fallback config. Load the team-scoped listing when a team id is present.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): ignore stale team model responses in router settings

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(ui): use react-query for fallback model listing in router settings accordion

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
2026-08-08 19:28:57 +00:00
Yuneng Jiang
0d7f7c689a
test: repair stale CircleCI contracts 2026-08-08 12:19:29 -07:00
Mateo Wang
8b16ee1dc2
Merge pull request #36277 from BerriAI/litellm_make_check_fallback
build(lint): rename make pre-commit to make check with a working-tree fallback
2026-08-08 12:08:34 -07:00