Commit graph

45043 commits

Author SHA1 Message Date
yuneng-jiang
b9a267c693
refactor(ui): migrate the teams form graph off antd Form onto react-hook-form (#37417)
* test(ui): pin the teams create and update payloads before the form migration

The teams graph (Teams.tsx, TeamInfo.tsx and the MetadataKeyValueFields child they
share) is next for the antd Form to react-hook-form migration, and its submit payload
is a function of which collapsible sections the user happened to open. Nine sections
across the two files use the shadcn Collapsible, none of them passes keepMounted, and
Base UI unmounts the closed branch, so a closed section registers nothing and its keys
never reach the request body.

That matters beyond parity. /team/update reads the body with exclude_unset, so an
omitted key is never written, while an explicitly null team member budget key reaches
clear_team_member_budget_fields and nulls max_budget, budget_duration, rpm_limit and
tpm_limit on the shared budget row. antd cannot reach that today because the field is
unregistered rather than null. A port that seeds those fields or coalesces on the way
into the payload would turn a save with the section never opened into a silent clear.

The coverage that shipped with the team modal reached one of the four gating sections
on the create side and asserted key sets rather than the request body, so a null where
antd sent undefined would have passed. These cases assert both the raw payload and its
JSON round trip with toStrictEqual, which is what separates absent from null from
undefined, and they cover every gating section on both screens.

Also pinned, because each is a live behaviour a port can quietly change:

- the create path sends max_budget, tpm_limit and rpm_limit as strings, while
  team_member_budget arrives as a number through its normalize prop
- an invalid secret manager config blocks the create with its rule message suppressed
  by the item's help prop, so nothing is shown to the user
- the disable global guardrails switch is inert for a non premium user
- a value typed into a section survives collapsing and re-expanding it

Verified by adding keepMounted to all nine panels, which is the change a porter reaches
for on noticing that fields go missing: 35 of 118 went red, including every one of these
cases. The files were restored byte identical afterwards.

No production file changes here. 145 tests pass across the three files.

* refactor(ui): migrate the teams form graph off antd Form onto react-hook-form

Teams.tsx and TeamInfo.tsx were the last large antd `Form` graph in the
dashboard. Both now use `useZodForm` + `FormField`, with the shared
`MetadataKeyValueFields` child converted to a `useFieldArray`.

antd only returns the mounted registered fields from `onFinish`, so a
closed collapsible contributed no keys at all. react-hook-form keeps
unmounted values in the store (and `shouldUnregister: true` would lose
them on re-expand), so both forms project the submitted values through
the currently mounted section list before handing them to the existing
payload builders. Closed sections therefore still produce absent keys
rather than nulls, which matters at /team/update where an explicit null
clears the shared budget row.

Widgets that had no shadcn equivalent are replaced with the existing
shared ones: SearchSelect for the organization pickers, MultiSelect for
default member models, TagsInput for guardrails/policies, and a new
GuardrailsSelect for the grouped global/other guardrail dropdown.

* refactor(ui): forward the field ref to NumericalInput in the teams forms

staging turned NumericalInput into a forwardRef, so the teams graph can stop
dropping the react-hook-form ref on the floor.

* test(ui): pin the capability gate, required rules and guardrail kill switch

A mutation run over the ported teams forms found five survivors the payload
cases did not reach: the viewPolicies gate on both forms, the team name rule
on both forms, the guardrail kill switch resync, and the number coercion on a
typed model rate limit. Six cases close them.
2026-08-18 22:37:04 -07:00
Mateo Wang
559310f077
Merge pull request #37423 from BerriAI/litellm_fix_thinking_bool_crash
fix: accept bool thinking param instead of crashing with AttributeError
2026-08-18 22:30:14 -07:00
Mateo Wang
d6afe728aa
Merge pull request #37365 from BerriAI/litellm_lit_5690_failed_request_token_counts
fix(proxy): record estimated input tokens in spend logs for failed dispatched requests
2026-08-18 22:29:51 -07:00
HarryZhou
2d4e6afe1c fix(guardrails): inspect responses reasoning content and summary text 2026-08-19 12:57:43 +08:00
HarryZhou
de95372dfb fix(responses-bridge): type-safe reasoning_content assignment in merge pass 2026-08-19 12:57:43 +08:00
HarryZhou
438c1850fe fix(responses-bridge): satisfy type-discipline budget in reasoning merge 2026-08-19 12:57:43 +08:00
HarryZhou
5911124f1d fix(responses-bridge): satisfy ruff strict-rule budget in reasoning merge 2026-08-19 12:57:43 +08:00
HarryZhou
3a77556dc1 fix(responses-bridge): preserve reasoning merge order when assistant already has reasoning_content 2026-08-19 12:57:43 +08:00
HarryZhou
b6ee13803d fix(responses-bridge): preserve reasoning input items as reasoning_content 2026-08-19 12:57:42 +08:00
mateo-berri
3a2728a42f test: derive vertex batch cost expectation from the cost map
The gemini 3.6 flash batch rates landed at half the standard rates in
94a29e0708, so the hardcoded standard-rate expectation started failing
on staging and red-lit misc / Run tests on every PR.
2026-08-18 21:18:16 -07:00
Mateo Wang
8941f2a622
Merge pull request #37424 from BerriAI/litellm_lit_5788_managed_file_fallback_pin
fix(router): keep acreate_file fallbacks inside the requested model group
2026-08-18 21:07:12 -07:00
mateo-berri
ac2db91b06 fix(proxy): single-row read-through resyncs and reload-race hardening
Resync registry misses with single-row DB fetches (guardrail by unique
name, agent by unique id or name, model by name then id) instead of
full-table loads, and bound them with a global budget of 20 resyncs per
5s window per registry that fails closed without negative-caching the
key.

Access group create/update now trust the reconcile outcome snapshot
captured under the reload lock instead of a post-lock router read, so a
concurrent reconcile can no longer surface a false degraded-serving 500.

Router.upsert_deployment restores the previously served deployment when
the replacement add fails under ignore_invalid_deployments, so a bad
update no longer silently drops a healthy deployment from serving.
2026-08-18 21:02:12 -07:00
mateo
6f4844bfd9 refactor(cost): trim streamed cost helper docstrings to the non-obvious bits
Some checks failed
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-19 03:58:24 +00:00
mateo
5d5dc4523f fix(cost): price streamed Messages usage via calculate_usage and the logging obj
Streamed `/v1/messages` `usage.cost` disagreed with the cost the logging callback recorded in three ways: `input_tokens` was read as the whole prompt total, but Anthropic reports it excluding cache tokens, so the non-cached input went unbilled on cache hits; the `cache_creation` 5m/1h split was dropped, billing 1h writes at the 5m rate; and costing by model name alone ignored the deployment's custom pricing, so a negotiated discount still streamed sticker price.

Anthropic usage now goes through `AnthropicConfig.calculate_usage`, the same transformation the non-streaming path uses, and the chunk is priced through the call's logging object when there is one so it inherits `custom_pricing`, `custom_llm_provider`, `base_model` and `router_model_id`, falling back to `completion_cost` by model name.

`calculate_usage` only reads its `usage_object`, so it now takes a `Mapping`.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-19 03:56:32 +00:00
Mateo Wang
9cb3cf7bef
Merge pull request #37425 from BerriAI/litellm_fix_passthrough_embeddings_unmapped_spend
Some checks are pending
CodSpeed Benchmarks / benchmarks (push) Waiting to run
Unit Tests: Enterprise, Google GenAI & Routing / enterprise-routing (push) Waiting to run
Unit Tests: Integrations (Callbacks & Logging) / integrations (push) Waiting to run
Unit Tests: LLM Provider Transformations / All Other Providers (push) Waiting to run
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy API Endpoints / proxy-endpoints (push) Waiting to run
Unit Tests: Proxy API Endpoints / proxy-server (push) Waiting to run
Unit Tests: Proxy Infrastructure / proxy-infra (push) Waiting to run
Unit Tests: Responses, Caching & Types / responses-caching-types (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
Publish basedpyright base counts / publish (push) Waiting to run
Code Quality Checks / code-quality (push) Waiting to run
UI Unit Tests / ui-unit-tests (push) Waiting to run
Unit Tests: Core Utilities / core-utils (push) Waiting to run
Unit Tests: Documentation Validation / documentation (push) Waiting to run
Unit Tests: LLM Provider Transformations / Vertex AI (push) Waiting to run
Unit Tests: MCP, Secrets, Containers & Misc / misc (push) Waiting to run
Unit Tests: Proxy Auth & Key Management / proxy-auth (push) Waiting to run
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
fix(proxy): log spend for OpenAI passthrough embeddings with unmapped models
2026-08-18 20:53:57 -07:00
mateo-berri
2e1d407711 test(e2e): pin the tag-routing denial to its actual cause
The strict-denial pin only asserted a 401, so any unrelated 401 (a bad key, a deleted key) would have kept it green while tag routing silently broke. The harness now keeps the 401 response body, the way it already does for 429s, and the pin asserts the tag-routing denial message.
2026-08-18 20:49:12 -07:00
mateo-berri
608d749983 fix(batches): stop one bad output line from zeroing an entire batch's spend 2026-08-18 20:46:59 -07:00
yuneng-jiang
c94d692864
fix(ui): highlight the first member search match so Enter picks it (#37429)
* fix(ui): highlight the first member search match so Enter picks it

Moving the modal off antd Form swapped antd Select for Base UI Combobox, which
highlights nothing until an arrow key moves the cursor. Typing an email and
pressing Enter therefore selected no one, and the form submitted anyway, so
/team/member_add went out with member.user_email undefined.

autoHighlight="always" restores the behaviour antd had. The Enter key still
does not submit the form, which the existing test continues to cover.

* fix(ui): keep the always-highlight prop past the type check

Combobox.Root re-declares autoHighlight as boolean while the AriaCombobox it
wraps types it as boolean | "always". Only "always" highlights a list this
component filters server-side, since the plain flag highlights on Base UI's own
filtering pass, which filter={null} turns off.

Verified with next build, the same type check that failed in CI.
2026-08-19 03:31:23 +00:00
mateo-berri
dfc30e6b4f Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_replica_registry_read_through
# Conflicts:
#	litellm/proxy/management_endpoints/model_access_group_management_endpoints.py
#	ruff.toml
#	tests/test_litellm/proxy/test_route_llm_request.py
2026-08-18 20:29:09 -07:00
yuneng-jiang
bcead282e2
test: move the remaining live groq call sites off the retired llama models (#37426)
The earlier sweep only caught the conformance suite in tests/llm_translation.
Groq retired llama-3.1-8b-instant alongside llama-3.3-70b-versatile, and four
tests under tests/local_testing still call them for real, so litellm_router_testing
and both local_testing shards 404 with model_not_found.

Only the sites that leave the process move. The chunk fixtures in
test_stream_chunk_builder, and the cost and routing tests that never open a
socket, keep the old ids because the string is data there, not a request.
2026-08-18 20:26:17 -07:00
Mateo Wang
16bc32fa23
Merge pull request #33842 from BerriAI/litellm_fix_bedrock_malformed_toolcall_18667
fix(bedrock): degrade gracefully on malformed tool-call arguments
2026-08-18 20:11:55 -07:00
yucheng-berri
71044bf5ea
feat(otel): attribute Prisma database spans to PostgreSQL instead of localhost (#36595)
* feat(otel): attribute Prisma database spans to PostgreSQL instead of localhost

Prisma reaches PostgreSQL through a query engine on loopback, so transport
instrumentation attributes database waits to localhost and operators cannot
tell the work is PostgreSQL or correlate it with the database's own metrics.

Datastore service spans now carry db.system.name, db.system, db.operation.name
and, for PostgreSQL, server.address, server.port and db.namespace derived from
DATABASE_URL, and are emitted as CLIENT spans. Only host, port, database and
schema are read, so no credential reaches an exporter. Endpoint attributes are
omitted when a read replica is configured, because routing is decided per Prisma
call underneath the span.

* fix(otel): reject a mis-split DSN authority and name socket-only databases

An unencoded '/' in the password truncates the URL authority, so urlparse
reports the username as the host and the password tail as the database, which
put credential material in db.namespace. Postgres drivers reject that DSN
outright, so the only safe reading is no endpoint at all.

A hostless 'postgresql:///litellm' is a valid local-socket DSN that Prisma
accepts, and it now yields db.namespace with no server address rather than
nothing. The default schema is matched case-insensitively, since an unquoted
PostgreSQL identifier folds and one deployment must yield one namespace.

* fix(otel): keep a non-default schema in db.namespace

Prisma quotes the schema name, so a DSN with ?schema=PUBLIC provisions a
second schema alongside public rather than reusing it. Observed on a live
proxy: the PUBLIC schema came up with its own 70 tables next to public's 78,
and a key created under one was not visible under the other.

Case-folding the two into a single namespace therefore reported two different
schemas as the same database, which is the misattribution this feature exists
to remove. Match the default literally.

* fix(otel): reject any DSN whose userinfo fell outside the authority

An unencoded '#' or '?' in the password sends the tail to the fragment or
query, leaving the path empty, so the marker check on the database segment
never fired and urlparse's hostname (the database username) was exported as
server.address.

The stranded userinfo '@' is the general tell for every mis-split, so guard on
that instead of enumerating the characters that cause it.

* fix(otel): allow an at-sign inside a well-formed DSN query

The previous guard rejected any DSN whose userinfo at-sign fell outside the
netloc, which also caught libpq parameters that legitimately carry one, so
?application_name=svc@prod and ?user=admin@company.com lost their endpoint
attributes.

Discriminate instead: a PostgreSQL DSN never has a fragment, its database name
cannot hold an unencoded at-sign or slash, and an at-sign in the query is only
suspicious when the query did not parse as parameters.

* fix(otel): resolve the database endpoint per span instead of once per process

The endpoint was cached for the process lifetime on the premise that
DATABASE_URL is deployment-static. It is not. The RDS IAM refresh rebuilds the
URL from DATABASE_HOST/PORT/NAME/SCHEMA on every rotation, the reconnect path
re-reads DATABASE_URL, and the DB-backed environment_variables config overlay
sets arbitrary keys post-startup with no blocklist covering DATABASE_*. A
process that had genuinely failed over kept exporting the old server.

Read the environment per span, which is also what Prisma connects with, so the
span can no longer name a different server than the one serving the query;
get_secret_str consulted a secret manager first and could diverge from it. Only
the parse is memoized, keyed on the URL.

* fix(otel): reject a question-mark mis-split whose tail parses as parameters

A '?' in a password strands the rest of the authority in the query, and that
tail can still parse as key=value, so testing only for an unparseable query let
the login through as server.address. One spelling hijacked the host= parameter
and put the password suffix there directly.

A legitimate at-sign in a query always follows a database path, and a
'?'-mis-split never leaves one, so require both.

* refactor(otel): drop the DSN parse cache that retained rotated credentials

The cache was keyed on the full DATABASE_URL, so up to eight complete DSNs,
each carrying a password or a retired IAM token, stayed referenced for the
process lifetime and outlived the rotation that replaced them. Nothing reached a
span, but a heap dump or crash report would have surfaced them.

Parsing costs about four microseconds against a span emission that costs orders
of magnitude more, so the cache bought nothing worth that.

* fix(otel): avoid a set construction the tightened LIT002 budget rejects

* fix(otel): refuse an ambiguous DSN authority instead of guessing at it

A password holding both an unencoded slash and a query-like tail defeated all
three shape checks: the slash left a clean path carrying the password
remainder, the query still parsed as parameters, and no fragment survived. The
login went out as server.address, the password's leading digits as server.port
and the rest as db.namespace.

A DSN whose at-sign sits in a query parameter is indistinguishable from that
mis-split by any property of the parse; both leave no userinfo, a host, a port
and a path. Guessing wrong publishes a credential fragment, so the ambiguity
now resolves to refusing the endpoint. Such a DSN loses server.address and
db.namespace and keeps the rest of the span; percent-encoding the at-sign
restores them.

Also honour port= alongside host=, which libpq pairs and this read ignored.

* docs(otel): fix a spliced sentence and a stale cache claim in db_endpoint

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-18 20:08:43 -07:00
Mateo Wang
1de398d9aa
Merge pull request #37366 from BerriAI/litellm_e2e_auto_router_regression_pins
fix(router): honor key-level tag filtering in pre-routing and pin auto-router e2e regressions
2026-08-18 20:06:49 -07:00
Mateo Wang
822cd4c4ea
Merge pull request #37377 from BerriAI/devin_ai_lit5757_dashscope_nested_cache_creation
fix(types): map nested prompt_tokens_details.cache_creation_input_tokens to cache_write_tokens
2026-08-18 20:02:17 -07:00
mateo-berri
b7f4f531b3 test(router): type the acreate_file fallback test helpers 2026-08-18 19:57:24 -07:00
mateo-berri
54cc988a9e test: drop restating comment and wrap long call in thinking tests 2026-08-18 19:55:22 -07:00
yuneng-jiang
a799351a5f
refactor(ui): extract the create-key payload builder out of create_key_button (#37397)
* refactor(ui): extract the create-key payload builder out of create_key_button

handleCreate built the POST /key/generate body inline by mutating the object antd
handed it, across roughly 170 lines mixing form values, twelve pieces of React
state and five sources that all fold into object_permission. Nothing could assert
that shape without rendering the whole modal.

The construction now lives in createKeyPayload.ts as a pure function returning a
tagged union, with 46 unit tests that run in single digit milliseconds and pin
whole payloads with toStrictEqual, following the createServerPayload pattern the
dashboard CLAUDE.md documents.

Behaviour is unchanged. The extraction was checked against the pre-extraction
handler with a differential harness over 50 input combinations, comparing the
built object, its key order and its serialised bytes, and eight mutations of the
new module were each confirmed to fail the committed tests.

* refactor(ui): align the key payload builder to the programme's entry-condition spec

Folds the duplicate-alias guard and the endpoint choice into the builder, so
KeyPayloadResult now carries three variants and handleCreate holds no payload
decision of its own. The two failure branches keep their original position
around toast.info and setIsModalVisible, and the builder checks the alias before
the agent selection, so the observable order is unchanged.

Pins the serialised wire shape as well as the object. The closed form registers
team_id at null through initialValue while organization_id has no initialValue
and stays undefined, so an untouched create sends seven of its eight keys.
Opening Optional Settings takes the object to 23 keys and the wire to nine. Both
directions of the definedness contract are now covered: undefined must not
become null, and null must not be dropped.

The earlier fixture fed a team_id the closed form cannot produce and pinned a
six-key wire as a result.

* refactor(ui): fold the service-account metadata write into its only caller

Removes assignServiceAccountId as a separate helper so there is no mutating
function available for reuse, which was the substance of the review finding. The
write now sits two lines below the JSON.parse that produced the value, so it is
visibly local and cannot reach a caller-owned object.

The write itself stays. Metadata has no validation rules, so a user can submit a
JSON body that parses to a primitive or an array. On a primitive the property
write raises a TypeError and the existing catch surfaces an error toast, and on
an array it leaves the array intact through JSON.stringify. A spread coerces
both to plain objects instead, silently creating a key from input the form
rejects today. Two tests pin those cases and go red against the spread.
2026-08-18 19:55:03 -07:00
mateo-berri
f8c8b41bf5 fix(proxy): backfill system prompt from the request body when estimating bridged failure tokens 2026-08-18 19:52:31 -07:00
yuneng-jiang
0289e3fb00
test: point the live gemini and groq conformance suites at models that still exist (#37422)
gemini/imagen-4.0-generate-001 now 404s with "no longer available, please
update your code to use models/gemini-3.1-flash-image", and
groq/llama-3.3-70b-versatile reached its deprecation_date of 2026-08-16, which
is the day the llm_translation and image_gen jobs went red.

Only the two live call sites move. The remaining references to the old ids sit
in offline transformation tests, where the string is just a routing key and no
request leaves the process.
2026-08-19 02:50:46 +00:00
mateo-berri
81914ebc31 fix(proxy): log spend for OpenAI passthrough embeddings with unmapped models 2026-08-18 19:48:02 -07:00
mateo-berri
2a1c21b72d fix(router): keep acreate_file fallbacks inside the requested model group
A file uploaded through Router.acreate_file lands in the account of the
deployment that stored it, so a cross-group fallback silently stores the
file with the wrong provider and every later batch or fine-tuning call
against the returned id permanently fails. Extend the provider-scoped
fallback pin that already covers input_file_id and training_file to file
creation, so the original provider error surfaces instead.
2026-08-18 19:46:45 -07:00
mateo-berri
138c77023a fix: accept bool thinking param instead of crashing with AttributeError
litellm.completion(thinking=True) crashed pre-network in is_thinking_enabled
with a retryable APIConnectionError ('bool' object has no attribute 'get'),
so the router burned retries on a deterministic failure and proxy clients got
a traceback instead of a usable response.

validate_and_fix_thinking_param now coerces thinking=True to the enabled dict
with the default medium budget and drops thinking=False, and the remaining
dict-assuming thinking accessors (base config, bedrock converse, deepseek)
guard with isinstance so raw bools can never crash a transform.
2026-08-18 19:44:31 -07:00
yuneng-jiang
f27d88bd13
test(ui): repoint the e2e locators at the post-antd form controls (#37421)
* test(ui): repoint the e2e locators at the post-antd form controls

Nine Playwright specs went red after the tremor and antd removals, none of
them because the product broke. The specs selected on markup those libraries
owned: tremor's TextInput stamped data-testid="base-input", the antd toast
facade rendered .ant-notification, and the team member modal's email field was
an .ant-select. Removing the libraries deleted those hooks silently.

Repoint each onto a user-facing locator that survives the next migration:
getByLabel for the key name, the MCP tool argument and the two cache pricing
fields, getByRole("combobox") for the team member email search, and the toast
container for the add-model success message.

The pricing fields needed a source change to be reachable at all. antd's
Form.Item used to assign the field name as the input id and tie the label to
it; the react-hook-form rewrite renders FieldLabel with no htmlFor and lets
FormField generate an opaque control id, so both cache inputs lost their
accessible name and could only be told apart by placeholder, which they share.
Pin the id back to the field name and point the label at it.

* fix(ui): let FormField own the pricing field label instead of hand-rendering one

The previous commit gave the cache cost inputs an accessible name by rendering
a FieldLabel with htmlFor next to the FormField. FieldLabel forwards Label's
props, and Label only accepts children, so next build failed type checking.

FormField already renders a label wired to the control id it generates, and
FormField.test.tsx covers that association, so passing label through is both
type-safe and less markup. The read-only branch keeps its plain FieldLabel,
which has no control to point at.
2026-08-19 02:41:05 +00:00
yuneng-jiang
1bc1a5ed81
refactor(ui): migrate the key edit form off Ant Design onto react-hook-form (#37398)
* refactor(ui): migrate the key edit form off Ant Design onto react-hook-form

The key editor was the last antd Form in the key flow. antd's store decided
the saved payload implicitly: onFinish reported whatever mounted Form.Items
happened to be registered, so a control could stop feeding the request
without anything failing.

The form now runs on react-hook-form with the dashboard's own field
primitives, and the payload is projected explicitly in keyEditFormValues so
every saved key is written out by name. That reproduces the old payload
exactly, including the keys antd sent holding undefined and the two
role-gated keys it dropped entirely when the field was not mounted.

The existing suite is kept as the contract and passes unedited apart from
the selects, whose queries moved from Ant Design class selectors to roles
and labels. MultiSelect now forwards a per-option disabled flag, which the
Models select needs to keep greying out individual models once the
all-proxy-models sentinel is picked.

* fix(ui): keep the key edit prompts control gated behind premium

The antd control carried disabled={!premiumUser} and a tooltip saying
prompts by key are premium. The port kept the premium placeholder but
dropped the gate, so a non-premium admin could type a prompt and have the
whole save rejected by the endpoint.

TagsInput had no disabled prop at all, which is why the gate could not
survive the port; it now takes one and passes it to the combobox. An audit
of every disabled expression against the antd original shows this was the
only gate lost.

Also pins the two payload keys that are assembled in the submit handler
from React state rather than bound fields, budget_fallbacks and
tag_rpm_limit. A field-driven suite cannot see them, and tag_rpm_limit's
only previous appearance was an empty map that reads the same whether the
assignment works or is deleted. The cases were written against the antd
implementation by another lane and are added unchanged.
2026-08-18 19:40:37 -07:00
yuneng-jiang
cb5f158023
test(ui): characterize the create key form payload contract (#37405)
The create key suite mocked antd wholesale and swapped in a fake form
store whose onFinish spread every value it had ever seen, which is the
inverse of what rc-field-form does: real antd reports only the mounted
registered fields, so anything inside a closed collapsible is absent
from the body rather than present and null. Fourteen children were
stubbed to () => null on top of that, so the suite passed no matter
what the form would actually submit and could not protect a port off
antd.

The integration test now renders the real tree and stubs only the
network boundary, and a submit payload contract block pins the exact
body with toStrictEqual so null, undefined and absent stay distinct.
It covers all sections closed, Optional Settings alone, all open, and
each nested section opened by itself, plus the collapse and re-expand
path that antd's store survives and a shouldUnregister port would not.

The genuinely unit-level logic, fetchTeamModels and fetchUserModels,
moves to a plain unit test that mocks modelAvailableCall alone.

getOpenAPISchema stays reachable through a stubbed global fetch rather
than the networking module mock, because networking imports jsonFields
from check_openapi_schema and that module imports getOpenAPISchema back,
so importOriginal binds the real export through the cycle.
2026-08-18 19:40:23 -07:00
yuneng-jiang
63f740f511
test(ui): pin the MCP server edit save payload before the form migration (#37404)
`mcp_server_edit` builds its update payload from `validateFields()`, so the
request body is whatever antd had mounted at the moment Save was pressed.
Nothing asserted that body, and the existing suite has no `toStrictEqual`
anywhere, so a key appearing, disappearing, or arriving as null instead of
undefined was invisible.

This adds a characterization net that drives the real component tree and
captures the exact object handed to `updateMCPServer`. Every case is pinned
with `toStrictEqual`, which is the only matcher that separates the three
states a key can be in: absent, present-as-undefined, and present-with-value.
That distinction is the whole point here, since a gated-off field and a
mounted-but-empty field currently differ, and `entra_obo` versus plain
`token_exchange` is exactly that difference in the wild.

19 cases cover the transports, all ten auth types, the credential subsets each
one contributes, and the tool allowlist gate. Values were captured from the
component as it behaves today rather than written from reading the source, so
the file records current behaviour instead of intended behaviour

A six-mutant battery run against the payload builder kills 6/6, each run
executing all 19 tests: leaking an audit field, deleting a binding, removing a
gate, turning a flat key into a lodash path, swapping undefined for null, and
swapping an empty-array default for null

Two explicit assignments in the payload turn out to be dead. `alias` and
`mcp_access_groups` are both already supplied by the `...restValues` spread
above them, so removing either line changes nothing. Left in place here and
noted for the payload extraction that follows
2026-08-18 19:40:02 -07:00
yuneng-jiang
6ecad76dec
test: allow protocol-constrained pass-through routes to declare fewer methods (#37415)
The pass-through method contract asserted that every route accepts GET, POST,
PUT, DELETE and PATCH. That holds for routes fronting a REST API, where
narrowing the set turns a request the upstream would have served into a 405,
but it does not hold for Amazon Comprehend Medical: it speaks AWS JSON 1.1,
which is POST-only, and its handler signs a POST upstream and requires a JSON
object body. Registering the other four verbs there would only produce routes
that always fail body parsing, so the route is POST-only on purpose and the
blanket assertion is the part that is wrong.

Carry a narrow exemption list instead of dropping the check. Exempted routes
are still pinned to an exact method set, so widening or narrowing one fails,
and a companion test asserts every exempted path is still registered so the
list cannot outlive the routes it covers.
2026-08-18 19:38:05 -07:00
yuneng-jiang
153b205d3e
test: build redaction and batch limiter fixtures the way production does (#37416)
Two suites broke because they stood in for production objects with stand-ins
that no longer answer the same way.

The redaction test faked a ResponsesAPIResponse and then reassigned
builtins.isinstance so the fake would pass the type check. Redaction now gates
on a tuple of accepted types, and the patched isinstance only recognised the
bare class, so the fake fell through to the generic branch and the assertions
ran against a plain dict. Building a real ResponsesAPIResponse drops the
builtins patch entirely and exercises the same type gate production takes.

The batch rate limiter tests constructed _PROXY_BatchRateLimiter with
parallel_request_limiter=None even though the parameter is not optional. That
stayed harmless until the output-token estimate started reading the limiter,
which turned it into an AttributeError. Inject the limiter the proxy injects,
sharing one InternalUsageCache the way _add_proxy_hooks does.
2026-08-18 19:37:50 -07:00
yuneng-jiang
7ac764970b
fix(proxy): return no rows when the aggregated activity entity filter is empty (#37414)
The aggregated daily-activity queries build their WHERE clause as raw SQL, and
an empty entity list rendered as `"team_id" IN ()`, which Postgres rejects with
`syntax error at or near ")"`. Callers reach that state normally: a caller
without admin view and no explicit team_ids has its scope resolved to the teams
it belongs to, so anyone belonging to no teams, an org admin for instance, sent
an empty list and got a 500 back from /team/daily/activity/aggregated.

The paginated endpoint hands the same empty list to Prisma, which renders
`in: []` and matches nothing, so it kept returning 200 with an empty result set.
Emit FALSE for the empty case so the raw-SQL path lands on the same answer,
mirroring what the api_key filter a few lines below already does.

The fix covers both aggregated queries at once because they share one WHERE
clause builder.
2026-08-18 19:37:32 -07:00
Mateo Wang
5f6a14dbb7
Merge pull request #37419 from BerriAI/litellm_fix_ocr_tests_req_format
test(ocr): update Azure DI supported-params assertion for req_format
2026-08-18 19:37:20 -07:00
mateo-berri
818886cfa1 fix(bedrock): raise non-retryable BadRequestError on unconvertible tool calls 2026-08-18 19:24:18 -07:00
mateo-berri
05e32e9fee test(ocr): update Azure DI supported-params assertion for req_format 2026-08-18 19:21:46 -07:00
mateo-berri
fe8c353435 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_bedrock_malformed_toolcall_18667 2026-08-18 19:16:41 -07:00
ryan-crabbe-berri
ec6f1c1a56
refactor(ui): codemod the antd Tooltips outside form files onto the shadcn atom (#37402)
* refactor(ui): codemod the antd Tooltips outside form files onto the shadcn atom

The atoms/Tooltip wrapper gains an optional side and renders its children without a popup when there is no content, which covers antd's placement prop and its title={undefined} escape hatch. A TypeScript-AST codemod then rewrites every antd Tooltip that does not sit in a file still using antd Form, since those files get their primitives swapped as part of the react-hook-form migration.

* refactor(ui): fold the atoms Tooltip into the ui tooltip primitive as SimpleTooltip
2026-08-19 02:14:37 +00:00
mateo-berri
89fcdc30d9 chore(typing): ratchet lint budgets down by the errors this branch fixed
basedpyright -1283 across 48 rules, ruff-strict -115, type-discipline -99
2026-08-19 02:13:49 +00:00
mateo-berri
86e7bcaf54 fix(websearch): keep _inject_native_blocks untyped rather than dodge the write
Threading a TypeVar through the helper makes the fallback attribute write
unprovable, and routing it through setattr to quiet that only trades one
diagnostic for a bugbear violation. Leave the seam as it was
2026-08-19 02:00:51 +00:00
Mateo Wang
807e1da4af
Merge pull request #32448 from ChenluJi/feat/tinyfish-search-headers-and-extras
feat(tinyfish): surface response headers + top-level response extras
2026-08-18 18:53:02 -07:00
yuneng-jiang
5012d11a18
refactor(ui): style the logging settings from semantic tokens (#37385)
team/LoggingSettings.tsx carried 34 hardcoded palette classes and
common_components/PremiumLoggingSettings.tsx another 9, so both render
light-only regardless of theme. Map the neutrals onto foreground,
muted-foreground, muted and border, the red affordances onto destructive,
and swap the hand-rolled chips for the shadcn Badge primitive.

The three event-type options carried decorative green, red and blue dots.
The design system has no success or info token, so the dots are dropped
and the option labels, which already say "Success Only", "Failure Only"
and "Success & Failure", carry the meaning on their own.

This is groundwork, not a visible change: nothing in the dashboard ever
applies the .dark class today, so the dark palette is unreachable. The
files no longer hardcode colour and will follow the theme once one exists.
2026-08-18 18:52:41 -07:00
mateo-berri
7602c5ea72 fix(responses): keep websocket response.create pass-through semantics
Typing the managed-responses call kwargs as dict[str, object] forced an
isinstance filter on the popped model and previous_response_id, which turned a
malformed client value from a loud downstream failure into a silent fallback to
the connection's model. Keep those two seams and the metadata mapping as they
were so the frame still fails the way it always did
2026-08-19 01:45:42 +00:00
Srivatsa03
cd7cdb3e3a fix(cost): stop double-billing cached tokens that overlap a modality
Providers report cached_tokens and image_tokens as overlapping subsets of
prompt_tokens rather than a disjoint partition, so a request whose images
were served from cache paid for them twice, once at the cache-read rate and
again at the image or input rate. The synthetic case in the issue came out
at 109e-6 against a correct 39e-6. Clamp each modality to the part of the
request the cache did not already cover, so the billed components still sum
to prompt_tokens

Fixes #37281
2026-08-18 20:44:45 -05:00