Commit graph

43309 commits

Author SHA1 Message Date
Cursor Agent
98fae6845a
test(ui): filter the Default model pin combobox by typing, not by virtualized index
The 'default model pin' tests picked the pin out of the antd Select by title, which
only works while the pin is inside the (virtualized) options list's visible viewport.
Adding a third bundled preset in #37068 grew the mocked model set, pushed the pin
below the viewport, and reddened the tests on staging even though nothing about the
production pin behaviour changed. Typing filters the Select down to the pin so the
match is not sensitive to where the pin lands in the list.

Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com>
2026-08-17 18:16:33 +00:00
ryan-crabbe-berri
9d40cd4df7
feat(ui): link key info header to its user, creator, team, and organization (#37187)
* feat(ui): link key info header to its user, creator, team, and organization

The key info page showed the owning user and creator as plain text and never surfaced the team or organization at all, so walking from a key to its parent entities meant copying ids into other pages. The header now renders User and Created By as links to the user detail page, and gains a far-right column with Team and Organization links (alias when known, id otherwise). Client-side navigation logic shared by BadgeLink and IdentityCell moves into a reusable EntityLink so all entity links behave the same

* test(ui): mock next/navigation in VirtualKeysTable test

KeyInfoView now renders EntityLink, which calls useRouter, so the table test that opens the key detail needs the app router mocked
2026-08-17 10:17:28 -07:00
tin-berri
3c3ada9af0
feat(ui): add Lite mixed-provider auto-router preset (#37068)
* feat(ui): add Lite mixed-provider auto-router preset

* feat(ui): disable classifier context window in Lite preset
2026-08-17 10:07:01 -07:00
tin-berri
5ecc6af541
fix(UI): add default model pin to complexity router UI (#36615)
* Add default model pin to complexity router UI

A complexity router's default model was only ever derived from the tiers, so
operators had no way to point the fallback at a model that is not first in the
Simple or Medium tier. Adds a Default Model select that records an explicit pin.

The pin is stored in complexity_router_config.default_model, which the backend
already reads, and mirrored onto complexity_router_default_model on save. Both
paths resolve through one helper that mirrors init_complexity_router_deployment:
a pin wins, otherwise MEDIUM or SIMPLE. Recording the pin in the config keeps it
distinguishable from a derived value, so a pin that happens to match the tiers
survives a round trip instead of being read back as tier tracking.

The edit modal only requires one non-empty tier, so a router with models in
COMPLEX alone can reach save with nothing the backend would pick. That now
blocks with an inline message rather than saving a router that raises at init.

* fix(UI): probe the pinned default model in the auto router connection test

The connection test built its targets from the tiers and the embedding model
only, so a Default Model pin outside every tier was never reached and a green
result could hide an unreachable default. model_info_view had already hand
rolled the dedupe and append locally, so the rule moved into
buildAutoRouterTestTargets and both call sites now share it.

* fix(ui): mirror backend precedence when resolving a complexity router default

The edit modal only recognized a pin stored in complexity_router_config.default_model,
so a router whose default lived solely in litellm_params.complexity_router_default_model
lost it on the next save. That field cannot be trusted outright either: before this PR
every save wrote a tier-derived value into it, so treating any value as a pin would
freeze legacy routers away from their tiers. Hydration now takes the config marker as
authoritative and falls back to litellm_params only when it diverges from what the tiers
alone derive, which is only reachable through an external API or config write.

Test Connection had the mirror-image bug: it fell back to complexity_router_config.default_model,
a UI-only marker init_complexity_router_deployment never reads, so it could probe a model
the router would never call. It now follows router.py exactly: litellm_params, else pure
tier-derivation.

Also reword a tooltip that hardcoded the Default Model select's position on the page, and
document the dual write and the create-vs-edit validation asymmetry.
2026-08-17 10:04:23 -07:00
Mateo Wang
2bc87ec3cc
Merge pull request #34067 from MUSE-CODE-SPACE/fix/batch-logging-null-output-file
fix(batches): don't crash logging when a completed batch has no output file
2026-08-17 10:01:02 -07:00
Mateo Wang
41c3133d0e
Merge pull request #34087 from ArjunPakhan/fix/bedrock-cancel-batch
fix(batches): support AWS Bedrock batch cancellation via `StopModelInvocationJob`
2026-08-17 10:00:57 -07:00
Mateo Wang
a1644eaf84
Merge pull request #36392 from BerriAI/devin_ai_fix_bedrock_batch_file_bytes_36388
fix(bedrock): report uploaded size in the FileObject returned by managed batch uploads
2026-08-17 10:00:53 -07:00
Mateo Wang
cfb2eba7f9
Merge pull request #36151 from LHMQ878/fix/36088-openai-ws-passthrough
fix(proxy): register WebSocket passthrough for OpenAI prefixes
2026-08-17 10:00:49 -07:00
Mateo Wang
1dbed6eb60
Merge pull request #37073 from BerriAI/litellm_decrease_anys_fable_round2 2026-08-17 09:40:40 -07:00
mateo-berri
5965648547 fix(proxy): close websocket cleanly when OpenAI credentials are missing 2026-08-16 14:40:37 -07:00
mateo-berri
4ba9d6b136 fix(proxy): expose url join helper at module level for websocket route 2026-08-16 14:29:21 -07:00
mateo-berri
81aefe4b3c Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_pr36151_ws_passthrough
# Conflicts:
#	litellm/proxy/pass_through_endpoints/pass_through_endpoints.py
#	tests/test_litellm/proxy/pass_through_endpoints/test_pass_through_endpoints.py
2026-08-16 14:21:57 -07:00
mateo-berri
862f33bbaa fix(proxy): negotiate client subprotocol on OpenAI websocket passthrough 2026-08-16 14:11:12 -07:00
mateo-berri
a258b2b130 fix(proxy): harden OpenAI websocket passthrough
- decode upstream first frame as utf-8 instead of ascii
- reject model-restricted keys at connect to match HTTP model enforcement
- log the actual request path for /openai_passthrough traffic
2026-08-16 13:51:50 -07:00
mateo-berri
5764c0daed Merge fork updates, keep typed kwargs and status-based cancel idempotency
# Conflicts:
#	litellm/llms/bedrock/batches/handler.py
2026-08-16 13:44:35 -07:00
mateo-berri
1b13957776 fix(bedrock): treat ConflictException on stop as idempotent cancel 2026-08-16 13:44:27 -07:00
Arjun Pakhan
4ff4e12557 fix(bedrock): add missing kwargs type annotations and refine conflict handling 2026-08-16 17:00:21 +00:00
Arjun Pakhan
88ac6ae63b style(bedrock): format handler.py with ruff to fix CI linting 2026-08-16 16:37:30 +00:00
Arjun Pakhan
ba11eebb2c fix(bedrock): handle ConflictException during idempotent batch cancel 2026-08-16 16:06:06 +00:00
mateo-berri
255dad6716 chore(typing): drop 1.3k basedpyright errors across 30 Any hotspot files
Replace Any-typed seams with real types in files carrying the highest
remaining reportAny/reportExplicitAny density after #34745: the proxy
server and its utils, the router, the streaming handler and chunk builder,
litellm_logging, the redis cache, the MCP db/tool-registry/spend-writer
layer, the anthropic pass-through adapters and guardrail translation, the
lasso and presidio guardrail hooks, the azure_ai agents handler, the
management endpoints (keys, users, ui_sso, model access groups, config
override, MCP, projects), the responses MCP handlers, response polling
background streaming, and the containers and vector stores mains

No casts, no type: ignore, no noqa, no new suppressions, and no Any
annotations that were not already at base. Whole-tree basedpyright:
reportAny 14,610 -> 14,009, reportExplicitAny 5,100 -> 4,780, total
144,743 -> 143,471, with no rule increasing repo-wide or in any file.
Budgets ratcheted: basedpyright -1,272 across 48 rules, ruff-strict -85,
type-discipline -37
2026-08-16 03:56:02 +00:00
yuneng-jiang
973329e986
Merge pull request #37069 from BerriAI/litellm_/frosty-goldwasser-93a2a1
Some checks failed
Code Quality Checks / code-quality (push) Has been cancelled
UI Unit Tests / ui-unit-tests (push) Has been cancelled
CI Coverage / assert-ci-coverage (push) Has been cancelled
Unit Tests: Core Utilities / core-utils (push) Has been cancelled
Publish basedpyright base counts / publish (push) Has been cancelled
GitHub Actions Security Analysis / zizmor (push) Has been cancelled
Unit Tests: Documentation Validation / documentation (push) Has been cancelled
Unit Tests: Enterprise, Google GenAI & Routing / enterprise-routing (push) Has been cancelled
Unit Tests: Integrations (Callbacks & Logging) / integrations (push) Has been cancelled
Unit Tests: LLM Provider Transformations / Vertex AI (push) Has been cancelled
Unit Tests: LLM Provider Transformations / All Other Providers (push) Has been cancelled
Unit Tests: MCP, Secrets, Containers & Misc / misc (push) Has been cancelled
Unit Tests: Proxy Auth & Key Management / proxy-auth (push) Has been cancelled
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Has been cancelled
Unit Tests: Proxy API Endpoints / proxy-endpoints (push) Has been cancelled
Unit Tests: Proxy API Endpoints / proxy-server (push) Has been cancelled
Unit Tests: Proxy Infrastructure / proxy-infra (push) Has been cancelled
Unit Tests: Responses, Caching & Types / responses-caching-types (push) Has been cancelled
Unit Tests: Proxy DB Operations / auth-checks (push) Has been cancelled
Unit Tests: Proxy DB Operations / budgets (push) Has been cancelled
Unit Tests: Proxy DB Operations / custom-logging (push) Has been cancelled
Unit Tests: Proxy DB Operations / db-and-spend (push) Has been cancelled
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Has been cancelled
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Has been cancelled
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Has been cancelled
Unit Tests: Proxy DB Operations / key-generation (push) Has been cancelled
Unit Tests: Proxy DB Operations / logging-misc (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-runtime (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-server-core (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-utils (push) Has been cancelled
test(e2e/ui): assert the log drawer chevrons by their lucide classes
2026-08-15 17:59:12 -07:00
Yuneng Jiang
de2b220c36
test(e2e/ui): assert the log drawer chevrons by their lucide classes
The log details drawer moved off Ant Design in 03d2b16bc, so its section
header renders lucide ChevronUp/ChevronDown rather than antd's UpOutlined
and DownOutlined. The collapse test still waited on .anticon-up and
.anticon-down, which no longer exist anywhere under view_logs, so it
failed on every run and burned all three attempts identically.

Point the three assertions at .lucide-chevron-up and .lucide-chevron-down,
matching how the dashboard's other suites address lucide icons.
2026-08-15 17:50:40 -07:00
yuneng-jiang
ae8afec7c1
Merge pull request #37066 from BerriAI/litellm_/release-ui-build-1e15d9
chore: rebuild Admin UI bundle from litellm_internal_staging
2026-08-15 17:38:54 -07:00
mateo-berri
904ff9efa7 Merge branch 'litellm_internal_staging' into devin_ai_fix_bedrock_batch_file_bytes_36388
Some checks failed
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Resolves the transform_create_file_response conflict by keeping the
_uploaded_object_size handoff over the response Content-Length read,
and adds the rebind-ok justification LIT011 now requires for the
upload-size litellm_params handoff after the base budget ratcheted.
2026-08-15 17:31:52 -07:00
mateo-berri
4114f907ea fix(bedrock): reraise cancel validation errors for non-terminal jobs, allow bedrock in acancel_batch typing 2026-08-15 17:25:16 -07:00
Yuneng Jiang
b3077c9dc0
chore: update Next.js build artifacts (2026-08-16 00:22 UTC, node v24.19.0) 2026-08-15 17:22:48 -07:00
yuneng-jiang
992a8123ac
Merge pull request #37010 from BerriAI/litellm_shadcn_next_0814
fix(ui): de-duplicate the reset budget option and polish shadcn surfaces
2026-08-15 17:18:25 -07:00
mateo-berri
57e946f279 Merge origin/litellm_internal_staging into fix/bedrock-cancel-batch 2026-08-15 17:18:08 -07:00
yuneng-jiang
91aee78e78
Merge pull request #37065 from BerriAI/litellm_/nice-wilson-9fbed6
test(e2e): assert provider error shape instead of pinned prose
2026-08-15 17:15:30 -07:00
Mateo Wang
dddee7d848
Merge pull request #37063 from BerriAI/litellm_pr_template_proof_format
docs(github): proof-of-fix section shows only the latest run as Before/After with nested cases
2026-08-15 17:13:12 -07:00
yuneng-jiang
1968562733
Merge pull request #37059 from BerriAI/litellm_/circleci-pipeline-triage-9b92e5
test: unstick the suites CircleCI is failing on
2026-08-15 17:12:46 -07:00
Yuneng Jiang
4df93c713c
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/nice-wilson-9fbed6 2026-08-15 16:59:36 -07:00
Mateo Wang
26113da7dd
Merge pull request #37057 from BerriAI/litellm_claude_md_gate_slot_locks
docs(claude): tell agents to let heavy gates queue for machine-wide slots
2026-08-15 16:55:47 -07:00
mateo-berri
7b60504cc8 docs(github): numbered steps are required under every proof case 2026-08-15 16:51:24 -07:00
Mateo Wang
db5c846919 chore: add more clarity 2026-08-15 16:44:31 -07:00
Mateo Wang
082469ff3d
chore: make it more concise 2026-08-15 16:41:00 -07:00
mateo-berri
83335c2f39 docs(github): proof-of-fix template shows only the latest run as Before/After with nested cases 2026-08-15 16:37:21 -07:00
Mateo Wang
13d94ec546
Merge pull request #36869 from BerriAI/litellm_lit002_typeddict_dict_literals
feat(lint): exempt TypedDict-annotated dict literals from LIT002
2026-08-15 16:35:24 -07:00
devin-ai-integration[bot]
74a1beda77
fix(panw_prisma_airs): scan tool call args as plain text, not a tool_event (#37038)
* fix(panw_prisma_airs): scan tool call args as plain text, not a tool_event

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(panw_prisma_airs): type the tool call argument extractor

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(panw_prisma_airs): cover tool call error fallback and dict masking paths

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(panw_prisma_airs): scan tool names with args and tolerate custom tool calls

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(panw_prisma_airs): scan tool call arguments that arrive already parsed

The tool call slice types arguments as a string, so a client posting parsed JSON
failed validation and the whole tool call, name included, read as unscannable and
was skipped without ever reaching AIRS. The OpenAI request path forwards
client-supplied tool_calls verbatim, so that shape is reachable.

Coerce non-string arguments instead of rejecting them, so the content is scanned.

* fix(panw_prisma_airs): route tool-block masked data by scan side, not by key name

Merging #37036 (already on staging) with this PR produces no conflict and a
silent bug. #37036 withholds prompt_masked_data on response-side tool blocks,
which was right while tool calls went out as a request-side tool_event: AIRS
reported the model's arguments under that key. This PR scans tool calls as
ordinary prompt/response text, so the side of the scan now decides which key
holds what. The model's arguments arrive under response_masked_data, already
covered by _CLIENT_HIDDEN_SCAN_FIELDS, and prompt_masked_data goes back to
being the caller's own input -- one of the audit fields LIT-5638 asks for.

Left as merged, a response-side tool block drops that field with nothing to
flag it.

- Tool-path block branch calls _build_error_detail without also_hide
- also_hide parameter removed; after this change it has no callers
- Regression test asserts both directions: model output withheld, caller
  input preserved. It fails against the auto-merged combination.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(panw_prisma_airs): a wrong-typed tool name must not suppress the scan

_ToolCallFunctionSlice types name as str, and _get_tool_call_function turns any
ValidationError into (None, None), which _scan_tool_calls_for_guardrail reads as
an unscannable tool call and skips. So a client posting "name": 123 keeps its
arguments off the wire to AIRS entirely -- no error, no log, no block. The
OpenAI request path forwards client tool_calls verbatim, so this is reachable by
any caller holding a valid key.

_coerce_arguments already existed for exactly this failure mode on the sibling
field. Widening it to cover name closes the gap:

  name='transfer_funds'   AIRS called: 1x   args scanned: True
  name=123 (int)          AIRS called: 0x   args scanned: False   <- before
  name=123 (int)          AIRS called: 1x   args scanned: True    <- after

Reported by Cursor Bugbot on fd9f6396e5.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Yucheng Zhu <yucheng@berri.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-15 23:14:43 +00:00
Mateo Wang
a41c45a4bf
Merge pull request #37050 from BerriAI/litellm_batch_cost_accounted_once
fix(batches): account a managed batch's cost exactly once
2026-08-15 16:11:15 -07:00
Yuneng Jiang
481ab07ca7
test(e2e): skip the bedrock web search cell the stack cannot provision
This cell needs the websearch_interception callback and a declared search
backend, both listed in its own module docstring. The ephemeral e2e stack
ships neither, so the request falls through to the bedrock transformation
and takes the by-design 400 that tells you to enable interception.

The cell has never been green here: the error path merged about an hour and
a half before the cell did, and the last full suite to pass predates the
cell entirely. Skip it with the reason recorded so the run reports honestly
instead of carrying a permanent red, and unskip once the stack ships the
config the docstring already spells out.
2026-08-15 16:09:02 -07:00
Yuneng Jiang
bf93483b95
test(e2e): assert provider error shape instead of pinned prose
Both providers reworded the error strings these two cells pinned, so the
suite went red without any behavior changing. Anthropic's auth error is now
"API key is invalid." rather than "invalid x-api-key", and OpenAI rejects an
empty upload with "This model does not support the format you provided.",
which names neither "file" nor "audio".

Assert the durable shape instead. The otel cell pins the machine-readable
authentication_error type plus a non-empty message, and the embedded JSON
still has to parse, which is what proves the attribute survived untruncated.
The transcription cell pins that the 400 relays the provider's own rejection
and is typed as a client input error, so a regression that swallows the
provider reason or returns a 500 still fails.
2026-08-15 16:08:52 -07:00
mateo-berri
bb1c3366cf Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_batch_cost_accounted_once
# Conflicts:
#	tests/test_litellm/proxy/openai_files_endpoint/test_files_common_utils.py
2026-08-15 15:56:11 -07:00
Yuneng Jiang
45ea07cdb3
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/circleci-pipeline-triage-9b92e5 2026-08-15 15:54:13 -07:00
Mateo Wang
9a96b9327c
Merge pull request #37058 from BerriAI/litellm_passthrough_accept_encoding
fix(passthrough): stop forwarding client Accept-Encoding upstream
2026-08-15 15:49:46 -07:00
Yuneng Jiang
6de68f18f8
test: refresh three suites that drifted from the code they cover
The google ai studio responses test still asserted tools == [], but the
transformation now pops empty tools and tool_choice before calling
completion, so assert the keys are absent.

test_openai_endpoints pinned claude-3-sonnet-20240229, which Bedrock has
retired; move it to us.anthropic.claude-sonnet-4-5-20250929-v1:0.

The AssemblyAI EU passthrough test depended on a credential that no longer
resolves in CI, and the US path plus the bad-key case already cover the
route; drop it rather than keep a permanently red case.
2026-08-15 15:37:09 -07:00
Yuneng Jiang
edc1d65558
test(e2e/ui): address the migrated dashboard controls by role, not antd classes
The playground, logs drawer and AI Hub modal moved off antd, so the specs
that reached for .ant-select, .ant-drawer-content, .ant-modal and
.ant-radio-button-wrapper no longer match anything and time out.

Address the same controls through their accessible role and name instead,
which holds across the component library swap and reads closer to what a
user does.
2026-08-15 15:37:09 -07:00
mateo-berri
90493a217f fix(passthrough): protect accept-encoding from x-pass- forwarding 2026-08-15 15:34:28 -07:00
devin-ai-integration[bot]
d7d10be063
fix(guardrails): return the full PANW AIRS scan response on blocked requests (#37036)
* fix(guardrails): return the full PANW AIRS scan response on blocked requests

The blocked-request error detail was assembled from a hardcoded allowlist, so audit fields like prompt_detection_details, prompt_masked_data, source, transaction_id and session_id never reached the client even though AIRS returned them.

Resolves LIT-5638

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(guardrails): drop redundant comment in AIRS error detail

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(panw_prisma_airs): withhold response_masked_data from the blocked-response error

The full AIRS passthrough also reached the response-side block path, where
response_masked_data carries the model's own generation. That branch is only
reached when mask_response_content is False, so the operator had explicitly
declined to deliver that text, and the error body handed it back anyway.

Withhold response_masked_data from the client-visible detail. prompt_masked_data
stays: it is the caller's own input and one of the fields the ticket asks for.

Every other AIRS field, including prompt_detection_details, source,
transaction_id and session_id, is unchanged.

* fix(panw_prisma_airs): withhold generated tool args from response-side blocks

_scan_tool_calls_for_guardrail calls AIRS with is_response=False because
tool_event is request-side in the AIRS schema, so AIRS returns the scanned
tool arguments under prompt_masked_data. When the tool calls being scanned
are the model's own output, that key holds generated content, and the
_CLIENT_HIDDEN_SCAN_FIELDS default (response_masked_data, empty on this
path) does not cover it. With the default mask_response_content=False the
block branch then shipped the model's masked tool arguments in the 400 --
the same content channel this PR closed for response_masked_data.

_build_error_detail takes an extra_hidden_fields argument so the withholding
stays in one place, and the tool-call block branch passes prompt_masked_data
when is_response is True. Request-side blocks are unchanged and still carry
prompt_masked_data, which is what LIT-5638 asks for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* style(panw_prisma_airs): apply ruff format

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Yucheng Zhu <yucheng@berri.ai>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-15 15:31:23 -07:00
tin-berri
540caa6574
feat(ui): direction picker and reverse-mode display for shadow evals (#36994)
* feat(ui): direction picker and reverse-mode display for shadow evals

* fix(ui): include configured model groups in the shadow eval baseline picker
2026-08-15 15:30:28 -07:00