Commit graph

43684 commits

Author SHA1 Message Date
Yuneng Jiang
770a4feb8d
refactor(ui): extract the MCP server edit payload into a pure module
`handleSave` was 280 lines that parsed stdio JSON, filtered credentials,
resolved half a dozen conditional flags, built the request body, and then made
the network call, all inside one try block reading ten closure variables. None
of it could be tested without rendering the whole modal and pressing Save.

The payload building moves to `editServerPayload.ts` as `buildEditServerPayload`,
mirroring the `createServerPayload.ts` module that already sits beside it: a
pure function taking form values, UI state, and the server being edited, and
returning a tagged union rather than throwing. The six ways it can fail are now
values in that union, and the component maps them to the same toasts it showed
before via an exhaustive switch, so no message text changes.

`AUTH_TYPES_REQUIRING_CREDENTIALS` and the static-header reducer were duplicated
between the create and edit paths; the extracted module imports the create
module's copies instead. The credential filter and stdio parser stay separate,
because the edit versions genuinely differ: edit writes an explicit null to
clear a blanked admin-config key, and stringifies stdio args and env values
where create passes them through

Behaviour is unchanged, and the payload net added earlier is what proves it:
all 20 of its cases still pass against the extracted builder without any
edit to their expectations.

40 unit tests cover the new module in single-digit milliseconds, reaching the
rejection paths and parsing branches that were previously only reachable
through a full render. A ten-mutant battery kills 10/10, each mutant taking
down exactly one test, which is the profile worth having: the tests are
targeted rather than broadly overlapping

One of those mutants is worth noting. Removing the access-group name
normalisation survives the integration net but dies here, because the antd
Select hands the form plain strings and the unit test does not. That mapping is
dead only while that control stays; it is covered now either way
2026-08-18 22:06:44 -07:00
Yuneng Jiang
198ea7e85f
test(ui): cover object-shaped MCP access groups in the save payload net
The net had no case where `mcp_access_groups` arrives as objects rather than
strings, which left the normalising `.map()` in the payload builder untested.

Adding the case shows the normalisation is currently a no-op: the payload
carries the group names either way, because the antd Select hands
`validateFields()` plain strings before the mapping ever runs. The line is
dead today for that reason, not because the spread above it duplicates the
value

Worth pinning rather than deleting. The form migration replaces that Select,
and a replacement control that yields objects would make the mapping
load-bearing again. This case is what would catch it
2026-08-18 21:53:57 -07:00
Yuneng Jiang
f20443b5e0
Merge branch 'litellm_internal_staging' of github.com:BerriAI/litellm into litellm_mcp_graph_a 2026-08-18 21:49:58 -07:00
Mateo Wang
8941f2a622
Merge pull request #37424 from BerriAI/litellm_lit_5788_managed_file_fallback_pin
fix(router): keep acreate_file fallbacks inside the requested model group
2026-08-18 21:07:12 -07:00
Mateo Wang
9cb3cf7bef
Merge pull request #37425 from BerriAI/litellm_fix_passthrough_embeddings_unmapped_spend
Some checks are pending
CodSpeed Benchmarks / benchmarks (push) Waiting to run
Unit Tests: Enterprise, Google GenAI & Routing / enterprise-routing (push) Waiting to run
Unit Tests: Integrations (Callbacks & Logging) / integrations (push) Waiting to run
Unit Tests: LLM Provider Transformations / All Other Providers (push) Waiting to run
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy API Endpoints / proxy-endpoints (push) Waiting to run
Unit Tests: Proxy API Endpoints / proxy-server (push) Waiting to run
Unit Tests: Proxy Infrastructure / proxy-infra (push) Waiting to run
Unit Tests: Responses, Caching & Types / responses-caching-types (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
Publish basedpyright base counts / publish (push) Waiting to run
Code Quality Checks / code-quality (push) Waiting to run
UI Unit Tests / ui-unit-tests (push) Waiting to run
Unit Tests: Core Utilities / core-utils (push) Waiting to run
Unit Tests: Documentation Validation / documentation (push) Waiting to run
Unit Tests: LLM Provider Transformations / Vertex AI (push) Waiting to run
Unit Tests: MCP, Secrets, Containers & Misc / misc (push) Waiting to run
Unit Tests: Proxy Auth & Key Management / proxy-auth (push) Waiting to run
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
fix(proxy): log spend for OpenAI passthrough embeddings with unmapped models
2026-08-18 20:53:57 -07:00
yuneng-jiang
c94d692864
fix(ui): highlight the first member search match so Enter picks it (#37429)
* fix(ui): highlight the first member search match so Enter picks it

Moving the modal off antd Form swapped antd Select for Base UI Combobox, which
highlights nothing until an arrow key moves the cursor. Typing an email and
pressing Enter therefore selected no one, and the form submitted anyway, so
/team/member_add went out with member.user_email undefined.

autoHighlight="always" restores the behaviour antd had. The Enter key still
does not submit the form, which the existing test continues to cover.

* fix(ui): keep the always-highlight prop past the type check

Combobox.Root re-declares autoHighlight as boolean while the AriaCombobox it
wraps types it as boolean | "always". Only "always" highlights a list this
component filters server-side, since the plain flag highlights on Base UI's own
filtering pass, which filter={null} turns off.

Verified with next build, the same type check that failed in CI.
2026-08-19 03:31:23 +00:00
yuneng-jiang
bcead282e2
test: move the remaining live groq call sites off the retired llama models (#37426)
The earlier sweep only caught the conformance suite in tests/llm_translation.
Groq retired llama-3.1-8b-instant alongside llama-3.3-70b-versatile, and four
tests under tests/local_testing still call them for real, so litellm_router_testing
and both local_testing shards 404 with model_not_found.

Only the sites that leave the process move. The chunk fixtures in
test_stream_chunk_builder, and the cost and routing tests that never open a
socket, keep the old ids because the string is data there, not a request.
2026-08-18 20:26:17 -07:00
Mateo Wang
16bc32fa23
Merge pull request #33842 from BerriAI/litellm_fix_bedrock_malformed_toolcall_18667
fix(bedrock): degrade gracefully on malformed tool-call arguments
2026-08-18 20:11:55 -07:00
yucheng-berri
71044bf5ea
feat(otel): attribute Prisma database spans to PostgreSQL instead of localhost (#36595)
* feat(otel): attribute Prisma database spans to PostgreSQL instead of localhost

Prisma reaches PostgreSQL through a query engine on loopback, so transport
instrumentation attributes database waits to localhost and operators cannot
tell the work is PostgreSQL or correlate it with the database's own metrics.

Datastore service spans now carry db.system.name, db.system, db.operation.name
and, for PostgreSQL, server.address, server.port and db.namespace derived from
DATABASE_URL, and are emitted as CLIENT spans. Only host, port, database and
schema are read, so no credential reaches an exporter. Endpoint attributes are
omitted when a read replica is configured, because routing is decided per Prisma
call underneath the span.

* fix(otel): reject a mis-split DSN authority and name socket-only databases

An unencoded '/' in the password truncates the URL authority, so urlparse
reports the username as the host and the password tail as the database, which
put credential material in db.namespace. Postgres drivers reject that DSN
outright, so the only safe reading is no endpoint at all.

A hostless 'postgresql:///litellm' is a valid local-socket DSN that Prisma
accepts, and it now yields db.namespace with no server address rather than
nothing. The default schema is matched case-insensitively, since an unquoted
PostgreSQL identifier folds and one deployment must yield one namespace.

* fix(otel): keep a non-default schema in db.namespace

Prisma quotes the schema name, so a DSN with ?schema=PUBLIC provisions a
second schema alongside public rather than reusing it. Observed on a live
proxy: the PUBLIC schema came up with its own 70 tables next to public's 78,
and a key created under one was not visible under the other.

Case-folding the two into a single namespace therefore reported two different
schemas as the same database, which is the misattribution this feature exists
to remove. Match the default literally.

* fix(otel): reject any DSN whose userinfo fell outside the authority

An unencoded '#' or '?' in the password sends the tail to the fragment or
query, leaving the path empty, so the marker check on the database segment
never fired and urlparse's hostname (the database username) was exported as
server.address.

The stranded userinfo '@' is the general tell for every mis-split, so guard on
that instead of enumerating the characters that cause it.

* fix(otel): allow an at-sign inside a well-formed DSN query

The previous guard rejected any DSN whose userinfo at-sign fell outside the
netloc, which also caught libpq parameters that legitimately carry one, so
?application_name=svc@prod and ?user=admin@company.com lost their endpoint
attributes.

Discriminate instead: a PostgreSQL DSN never has a fragment, its database name
cannot hold an unencoded at-sign or slash, and an at-sign in the query is only
suspicious when the query did not parse as parameters.

* fix(otel): resolve the database endpoint per span instead of once per process

The endpoint was cached for the process lifetime on the premise that
DATABASE_URL is deployment-static. It is not. The RDS IAM refresh rebuilds the
URL from DATABASE_HOST/PORT/NAME/SCHEMA on every rotation, the reconnect path
re-reads DATABASE_URL, and the DB-backed environment_variables config overlay
sets arbitrary keys post-startup with no blocklist covering DATABASE_*. A
process that had genuinely failed over kept exporting the old server.

Read the environment per span, which is also what Prisma connects with, so the
span can no longer name a different server than the one serving the query;
get_secret_str consulted a secret manager first and could diverge from it. Only
the parse is memoized, keyed on the URL.

* fix(otel): reject a question-mark mis-split whose tail parses as parameters

A '?' in a password strands the rest of the authority in the query, and that
tail can still parse as key=value, so testing only for an unparseable query let
the login through as server.address. One spelling hijacked the host= parameter
and put the password suffix there directly.

A legitimate at-sign in a query always follows a database path, and a
'?'-mis-split never leaves one, so require both.

* refactor(otel): drop the DSN parse cache that retained rotated credentials

The cache was keyed on the full DATABASE_URL, so up to eight complete DSNs,
each carrying a password or a retired IAM token, stayed referenced for the
process lifetime and outlived the rotation that replaced them. Nothing reached a
span, but a heap dump or crash report would have surfaced them.

Parsing costs about four microseconds against a span emission that costs orders
of magnitude more, so the cache bought nothing worth that.

* fix(otel): avoid a set construction the tightened LIT002 budget rejects

* fix(otel): refuse an ambiguous DSN authority instead of guessing at it

A password holding both an unencoded slash and a query-like tail defeated all
three shape checks: the slash left a clean path carrying the password
remainder, the query still parsed as parameters, and no fragment survived. The
login went out as server.address, the password's leading digits as server.port
and the rest as db.namespace.

A DSN whose at-sign sits in a query parameter is indistinguishable from that
mis-split by any property of the parse; both leave no userinfo, a host, a port
and a path. Guessing wrong publishes a credential fragment, so the ambiguity
now resolves to refusing the endpoint. Such a DSN loses server.address and
db.namespace and keeps the rest of the span; percent-encoding the at-sign
restores them.

Also honour port= alongside host=, which libpq pairs and this read ignored.

* docs(otel): fix a spliced sentence and a stale cache claim in db_endpoint

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-18 20:08:43 -07:00
Mateo Wang
1de398d9aa
Merge pull request #37366 from BerriAI/litellm_e2e_auto_router_regression_pins
fix(router): honor key-level tag filtering in pre-routing and pin auto-router e2e regressions
2026-08-18 20:06:49 -07:00
Mateo Wang
822cd4c4ea
Merge pull request #37377 from BerriAI/devin_ai_lit5757_dashscope_nested_cache_creation
fix(types): map nested prompt_tokens_details.cache_creation_input_tokens to cache_write_tokens
2026-08-18 20:02:17 -07:00
mateo-berri
b7f4f531b3 test(router): type the acreate_file fallback test helpers 2026-08-18 19:57:24 -07:00
yuneng-jiang
a799351a5f
refactor(ui): extract the create-key payload builder out of create_key_button (#37397)
* refactor(ui): extract the create-key payload builder out of create_key_button

handleCreate built the POST /key/generate body inline by mutating the object antd
handed it, across roughly 170 lines mixing form values, twelve pieces of React
state and five sources that all fold into object_permission. Nothing could assert
that shape without rendering the whole modal.

The construction now lives in createKeyPayload.ts as a pure function returning a
tagged union, with 46 unit tests that run in single digit milliseconds and pin
whole payloads with toStrictEqual, following the createServerPayload pattern the
dashboard CLAUDE.md documents.

Behaviour is unchanged. The extraction was checked against the pre-extraction
handler with a differential harness over 50 input combinations, comparing the
built object, its key order and its serialised bytes, and eight mutations of the
new module were each confirmed to fail the committed tests.

* refactor(ui): align the key payload builder to the programme's entry-condition spec

Folds the duplicate-alias guard and the endpoint choice into the builder, so
KeyPayloadResult now carries three variants and handleCreate holds no payload
decision of its own. The two failure branches keep their original position
around toast.info and setIsModalVisible, and the builder checks the alias before
the agent selection, so the observable order is unchanged.

Pins the serialised wire shape as well as the object. The closed form registers
team_id at null through initialValue while organization_id has no initialValue
and stays undefined, so an untouched create sends seven of its eight keys.
Opening Optional Settings takes the object to 23 keys and the wire to nine. Both
directions of the definedness contract are now covered: undefined must not
become null, and null must not be dropped.

The earlier fixture fed a team_id the closed form cannot produce and pinned a
six-key wire as a result.

* refactor(ui): fold the service-account metadata write into its only caller

Removes assignServiceAccountId as a separate helper so there is no mutating
function available for reuse, which was the substance of the review finding. The
write now sits two lines below the JSON.parse that produced the value, so it is
visibly local and cannot reach a caller-owned object.

The write itself stays. Metadata has no validation rules, so a user can submit a
JSON body that parses to a primitive or an array. On a primitive the property
write raises a TypeError and the existing catch surfaces an error toast, and on
an array it leaves the array intact through JSON.stringify. A spread coerces
both to plain objects instead, silently creating a key from input the form
rejects today. Two tests pin those cases and go red against the spread.
2026-08-18 19:55:03 -07:00
yuneng-jiang
0289e3fb00
test: point the live gemini and groq conformance suites at models that still exist (#37422)
gemini/imagen-4.0-generate-001 now 404s with "no longer available, please
update your code to use models/gemini-3.1-flash-image", and
groq/llama-3.3-70b-versatile reached its deprecation_date of 2026-08-16, which
is the day the llm_translation and image_gen jobs went red.

Only the two live call sites move. The remaining references to the old ids sit
in offline transformation tests, where the string is just a routing key and no
request leaves the process.
2026-08-19 02:50:46 +00:00
mateo-berri
81914ebc31 fix(proxy): log spend for OpenAI passthrough embeddings with unmapped models 2026-08-18 19:48:02 -07:00
mateo-berri
2a1c21b72d fix(router): keep acreate_file fallbacks inside the requested model group
A file uploaded through Router.acreate_file lands in the account of the
deployment that stored it, so a cross-group fallback silently stores the
file with the wrong provider and every later batch or fine-tuning call
against the returned id permanently fails. Extend the provider-scoped
fallback pin that already covers input_file_id and training_file to file
creation, so the original provider error surfaces instead.
2026-08-18 19:46:45 -07:00
yuneng-jiang
f27d88bd13
test(ui): repoint the e2e locators at the post-antd form controls (#37421)
* test(ui): repoint the e2e locators at the post-antd form controls

Nine Playwright specs went red after the tremor and antd removals, none of
them because the product broke. The specs selected on markup those libraries
owned: tremor's TextInput stamped data-testid="base-input", the antd toast
facade rendered .ant-notification, and the team member modal's email field was
an .ant-select. Removing the libraries deleted those hooks silently.

Repoint each onto a user-facing locator that survives the next migration:
getByLabel for the key name, the MCP tool argument and the two cache pricing
fields, getByRole("combobox") for the team member email search, and the toast
container for the add-model success message.

The pricing fields needed a source change to be reachable at all. antd's
Form.Item used to assign the field name as the input id and tie the label to
it; the react-hook-form rewrite renders FieldLabel with no htmlFor and lets
FormField generate an opaque control id, so both cache inputs lost their
accessible name and could only be told apart by placeholder, which they share.
Pin the id back to the field name and point the label at it.

* fix(ui): let FormField own the pricing field label instead of hand-rendering one

The previous commit gave the cache cost inputs an accessible name by rendering
a FieldLabel with htmlFor next to the FormField. FieldLabel forwards Label's
props, and Label only accepts children, so next build failed type checking.

FormField already renders a label wired to the control id it generates, and
FormField.test.tsx covers that association, so passing label through is both
type-safe and less markup. The read-only branch keeps its plain FieldLabel,
which has no control to point at.
2026-08-19 02:41:05 +00:00
yuneng-jiang
1bc1a5ed81
refactor(ui): migrate the key edit form off Ant Design onto react-hook-form (#37398)
* refactor(ui): migrate the key edit form off Ant Design onto react-hook-form

The key editor was the last antd Form in the key flow. antd's store decided
the saved payload implicitly: onFinish reported whatever mounted Form.Items
happened to be registered, so a control could stop feeding the request
without anything failing.

The form now runs on react-hook-form with the dashboard's own field
primitives, and the payload is projected explicitly in keyEditFormValues so
every saved key is written out by name. That reproduces the old payload
exactly, including the keys antd sent holding undefined and the two
role-gated keys it dropped entirely when the field was not mounted.

The existing suite is kept as the contract and passes unedited apart from
the selects, whose queries moved from Ant Design class selectors to roles
and labels. MultiSelect now forwards a per-option disabled flag, which the
Models select needs to keep greying out individual models once the
all-proxy-models sentinel is picked.

* fix(ui): keep the key edit prompts control gated behind premium

The antd control carried disabled={!premiumUser} and a tooltip saying
prompts by key are premium. The port kept the premium placeholder but
dropped the gate, so a non-premium admin could type a prompt and have the
whole save rejected by the endpoint.

TagsInput had no disabled prop at all, which is why the gate could not
survive the port; it now takes one and passes it to the combobox. An audit
of every disabled expression against the antd original shows this was the
only gate lost.

Also pins the two payload keys that are assembled in the submit handler
from React state rather than bound fields, budget_fallbacks and
tag_rpm_limit. A field-driven suite cannot see them, and tag_rpm_limit's
only previous appearance was an empty map that reads the same whether the
assignment works or is deleted. The cases were written against the antd
implementation by another lane and are added unchanged.
2026-08-18 19:40:37 -07:00
yuneng-jiang
cb5f158023
test(ui): characterize the create key form payload contract (#37405)
The create key suite mocked antd wholesale and swapped in a fake form
store whose onFinish spread every value it had ever seen, which is the
inverse of what rc-field-form does: real antd reports only the mounted
registered fields, so anything inside a closed collapsible is absent
from the body rather than present and null. Fourteen children were
stubbed to () => null on top of that, so the suite passed no matter
what the form would actually submit and could not protect a port off
antd.

The integration test now renders the real tree and stubs only the
network boundary, and a submit payload contract block pins the exact
body with toStrictEqual so null, undefined and absent stay distinct.
It covers all sections closed, Optional Settings alone, all open, and
each nested section opened by itself, plus the collapse and re-expand
path that antd's store survives and a shouldUnregister port would not.

The genuinely unit-level logic, fetchTeamModels and fetchUserModels,
moves to a plain unit test that mocks modelAvailableCall alone.

getOpenAPISchema stays reachable through a stubbed global fetch rather
than the networking module mock, because networking imports jsonFields
from check_openapi_schema and that module imports getOpenAPISchema back,
so importOriginal binds the real export through the cycle.
2026-08-18 19:40:23 -07:00
yuneng-jiang
63f740f511
test(ui): pin the MCP server edit save payload before the form migration (#37404)
`mcp_server_edit` builds its update payload from `validateFields()`, so the
request body is whatever antd had mounted at the moment Save was pressed.
Nothing asserted that body, and the existing suite has no `toStrictEqual`
anywhere, so a key appearing, disappearing, or arriving as null instead of
undefined was invisible.

This adds a characterization net that drives the real component tree and
captures the exact object handed to `updateMCPServer`. Every case is pinned
with `toStrictEqual`, which is the only matcher that separates the three
states a key can be in: absent, present-as-undefined, and present-with-value.
That distinction is the whole point here, since a gated-off field and a
mounted-but-empty field currently differ, and `entra_obo` versus plain
`token_exchange` is exactly that difference in the wild.

19 cases cover the transports, all ten auth types, the credential subsets each
one contributes, and the tool allowlist gate. Values were captured from the
component as it behaves today rather than written from reading the source, so
the file records current behaviour instead of intended behaviour

A six-mutant battery run against the payload builder kills 6/6, each run
executing all 19 tests: leaking an audit field, deleting a binding, removing a
gate, turning a flat key into a lodash path, swapping undefined for null, and
swapping an empty-array default for null

Two explicit assignments in the payload turn out to be dead. `alias` and
`mcp_access_groups` are both already supplied by the `...restValues` spread
above them, so removing either line changes nothing. Left in place here and
noted for the payload extraction that follows
2026-08-18 19:40:02 -07:00
yuneng-jiang
6ecad76dec
test: allow protocol-constrained pass-through routes to declare fewer methods (#37415)
The pass-through method contract asserted that every route accepts GET, POST,
PUT, DELETE and PATCH. That holds for routes fronting a REST API, where
narrowing the set turns a request the upstream would have served into a 405,
but it does not hold for Amazon Comprehend Medical: it speaks AWS JSON 1.1,
which is POST-only, and its handler signs a POST upstream and requires a JSON
object body. Registering the other four verbs there would only produce routes
that always fail body parsing, so the route is POST-only on purpose and the
blanket assertion is the part that is wrong.

Carry a narrow exemption list instead of dropping the check. Exempted routes
are still pinned to an exact method set, so widening or narrowing one fails,
and a companion test asserts every exempted path is still registered so the
list cannot outlive the routes it covers.
2026-08-18 19:38:05 -07:00
yuneng-jiang
153b205d3e
test: build redaction and batch limiter fixtures the way production does (#37416)
Two suites broke because they stood in for production objects with stand-ins
that no longer answer the same way.

The redaction test faked a ResponsesAPIResponse and then reassigned
builtins.isinstance so the fake would pass the type check. Redaction now gates
on a tuple of accepted types, and the patched isinstance only recognised the
bare class, so the fake fell through to the generic branch and the assertions
ran against a plain dict. Building a real ResponsesAPIResponse drops the
builtins patch entirely and exercises the same type gate production takes.

The batch rate limiter tests constructed _PROXY_BatchRateLimiter with
parallel_request_limiter=None even though the parameter is not optional. That
stayed harmless until the output-token estimate started reading the limiter,
which turned it into an AttributeError. Inject the limiter the proxy injects,
sharing one InternalUsageCache the way _add_proxy_hooks does.
2026-08-18 19:37:50 -07:00
yuneng-jiang
7ac764970b
fix(proxy): return no rows when the aggregated activity entity filter is empty (#37414)
The aggregated daily-activity queries build their WHERE clause as raw SQL, and
an empty entity list rendered as `"team_id" IN ()`, which Postgres rejects with
`syntax error at or near ")"`. Callers reach that state normally: a caller
without admin view and no explicit team_ids has its scope resolved to the teams
it belongs to, so anyone belonging to no teams, an org admin for instance, sent
an empty list and got a 500 back from /team/daily/activity/aggregated.

The paginated endpoint hands the same empty list to Prisma, which renders
`in: []` and matches nothing, so it kept returning 200 with an empty result set.
Emit FALSE for the empty case so the raw-SQL path lands on the same answer,
mirroring what the api_key filter a few lines below already does.

The fix covers both aggregated queries at once because they share one WHERE
clause builder.
2026-08-18 19:37:32 -07:00
Mateo Wang
5f6a14dbb7
Merge pull request #37419 from BerriAI/litellm_fix_ocr_tests_req_format
test(ocr): update Azure DI supported-params assertion for req_format
2026-08-18 19:37:20 -07:00
mateo-berri
818886cfa1 fix(bedrock): raise non-retryable BadRequestError on unconvertible tool calls 2026-08-18 19:24:18 -07:00
mateo-berri
05e32e9fee test(ocr): update Azure DI supported-params assertion for req_format 2026-08-18 19:21:46 -07:00
mateo-berri
fe8c353435 Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_fix_bedrock_malformed_toolcall_18667 2026-08-18 19:16:41 -07:00
ryan-crabbe-berri
ec6f1c1a56
refactor(ui): codemod the antd Tooltips outside form files onto the shadcn atom (#37402)
* refactor(ui): codemod the antd Tooltips outside form files onto the shadcn atom

The atoms/Tooltip wrapper gains an optional side and renders its children without a popup when there is no content, which covers antd's placement prop and its title={undefined} escape hatch. A TypeScript-AST codemod then rewrites every antd Tooltip that does not sit in a file still using antd Form, since those files get their primitives swapped as part of the react-hook-form migration.

* refactor(ui): fold the atoms Tooltip into the ui tooltip primitive as SimpleTooltip
2026-08-19 02:14:37 +00:00
Mateo Wang
807e1da4af
Merge pull request #32448 from ChenluJi/feat/tinyfish-search-headers-and-extras
feat(tinyfish): surface response headers + top-level response extras
2026-08-18 18:53:02 -07:00
yuneng-jiang
5012d11a18
refactor(ui): style the logging settings from semantic tokens (#37385)
team/LoggingSettings.tsx carried 34 hardcoded palette classes and
common_components/PremiumLoggingSettings.tsx another 9, so both render
light-only regardless of theme. Map the neutrals onto foreground,
muted-foreground, muted and border, the red affordances onto destructive,
and swap the hand-rolled chips for the shadcn Badge primitive.

The three event-type options carried decorative green, red and blue dots.
The design system has no success or info token, so the dots are dropped
and the option labels, which already say "Success Only", "Failure Only"
and "Success & Failure", carry the meaning on their own.

This is groundwork, not a visible change: nothing in the dashboard ever
applies the .dark class today, so the dark palette is unreachable. The
files no longer hardcode colour and will follow the theme once one exists.
2026-08-18 18:52:41 -07:00
Yuneng Jiang
9df4cafcfb
test(ui): pin the MCP server edit save payload before the form migration
`mcp_server_edit` builds its update payload from `validateFields()`, so the
request body is whatever antd had mounted at the moment Save was pressed.
Nothing asserted that body, and the existing suite has no `toStrictEqual`
anywhere, so a key appearing, disappearing, or arriving as null instead of
undefined was invisible.

This adds a characterization net that drives the real component tree and
captures the exact object handed to `updateMCPServer`. Every case is pinned
with `toStrictEqual`, which is the only matcher that separates the three
states a key can be in: absent, present-as-undefined, and present-with-value.
That distinction is the whole point here, since a gated-off field and a
mounted-but-empty field currently differ, and `entra_obo` versus plain
`token_exchange` is exactly that difference in the wild.

19 cases cover the transports, all ten auth types, the credential subsets each
one contributes, and the tool allowlist gate. Values were captured from the
component as it behaves today rather than written from reading the source, so
the file records current behaviour instead of intended behaviour

A six-mutant battery run against the payload builder kills 6/6, each run
executing all 19 tests: leaking an audit field, deleting a binding, removing a
gate, turning a flat key into a lodash path, swapping undefined for null, and
swapping an empty-array default for null

Two explicit assignments in the payload turn out to be dead. `alias` and
`mcp_access_groups` are both already supplied by the `...restValues` spread
above them, so removing either line changes nothing. Left in place here and
noted for the payload extraction that follows
2026-08-18 18:44:54 -07:00
Mateo Wang
41de13aa0c
Merge pull request #28501 from epistoteles/add-databricks-model-pricing
fix(databricks): add cost map entries for 14 newer Databricks models
2026-08-18 18:40:35 -07:00
mateo-berri
490079e7df test: cover nested cache_creation_input_tokens in responses bridge and spend logs 2026-08-18 18:38:30 -07:00
tin-berri
f6eaca9069
fix(mcp): serve token-forwarding servers when oauth discovery fails (#37399)
true_passthrough and oauth_delegate forward the caller's own bearer and mint
nothing, so their sessions consume no discovered OAuth endpoint. The discovery
completeness gate still failed them closed with a 503 raised before the upstream
was ever contacted, which the tools/list fan-out swallowed into HTTP 200 with an
empty tool list. Any upstream that publishes no RFC 9728 metadata, including
every OpenAPI-backed internal API, was permanently unusable.

A failed discovery is now fatal only to servers whose OAuth flow the gateway
runs itself. Discovery still runs for the forwarding modes, so /authorize,
/token and /register keep resolving their endpoints, and each keeps its own 400
when they are missing.

MCPServer.is_client_forwarded_token now owns the mode pair that five call sites
spelled inline, one of which had already named it is_client_forwarded_mode.
2026-08-19 01:32:41 +00:00
Mateo Wang
704cc41f28
Merge pull request #37388 from BerriAI/litellm_lit_5718_mcp_tool_bound_to_server
fix(mcp): bind tool existence check to the selected server
2026-08-18 18:22:00 -07:00
yuneng-jiang
2ed354f62b
bump: litellm-enterprise 0.1.56 -> 0.1.57, litellm-proxy-extras 0.4.86 -> 0.4.87, litellm 1.98.0 -> 1.99.0 (#37395)
enterprise/ and litellm-proxy-extras/ both changed between main and staging, so each gets a PATCH bump. The 1.98.0 line already graduated with v1.98.0-rc.1, so this promotion opens the 1.99.0 line and litellm takes its MINOR bump.

uv.lock re-resolved against the three new versions; the exclude-newer timestamp moves because the lock uses a rolling P3D window
2026-08-18 18:14:31 -07:00
ryan-crabbe-berri
4493c826e7
fix(logging): close three secret-leak paths in verbose logging (#37391)
* fix(logging): close three secret-leak paths in verbose logging

The AWS credential pattern was the only key-name matcher in secret_redaction
that skipped optional quotes, so quoted dict-repr values leaked. Fold the three
AWS key names into the shared key-name alternation instead.

SecretRedactionFilter only scrubs str record attributes, so a dict/list/set
passed through extra={...} reached the formatter unredacted. Redact at the
formatter boundary so no value shape can bypass it.

log_raw_request_response wrote the request curl command to metadata["raw_request"]
unredacted, returned an unmasked raw_request_api_base, and fell back to dumping
model_call_details whenever api_base was empty.

* Update litellm/_logging.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix(logging): redact JSON log values without breaking the document

JsonFormatter redacted the serialized JSON, so a secret-named member
collapsed from '"api_key": "sk-..."' to a bare REDACTED token and the
line stopped parsing as JSON.

Redact before serialization instead: safe_dumps takes an optional
value_transform hook (default None, so all other callers are unchanged)
and redact_structured_value collapses only the value, leaving the key
and surrounding structure intact.

JsonFormatter now emits "api_key": "REDACTED" where the formatter unit
test expected the already-masked "sk**********". That test bypasses
SecretRedactionFilter, which in production collapses the pair before any
formatter runs, so the assertion is updated to match real behavior.

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-08-19 01:03:34 +00:00
mateo-berri
24fc3f721c fix(databricks): match the claude 4.6 context limits to Anthropic's published values 2026-08-18 17:55:45 -07:00
ryan-crabbe-berri
869a8cd984
fix(vector_stores): stop leaking stored credentials in direct search debug logs (#37373)
* fix(vector_stores): stop leaking stored credentials in direct search debug logs

Direct vector store providers (RESP datastores like Valkey) have no HTTP
request to echo, so both search handlers called `logging_obj.pre_call` with
no `api_base`. The logging helper treats an empty `api_base` as "nothing to
render" and falls back to `str(self.model_call_details)`, which carries the
resolved `litellm_params`: the stored `valkey_password` and the embedding
config's `api_key` among them.

The stdout logger's regex redaction hid this, but `pre_call` also writes the
same string to `litellm_params["metadata"]["raw_request"]`, which ships
unredacted to every logging callback (Langfuse, OTel, etc.).

Pass a synthetic `<provider>://<vector_store_id>` endpoint plus an explicit
`request_str` so the debug output describes the call instead of dumping call
details, and fold the duplicated sync/async blocks into one helper so the
sanitized descriptor cannot drift between them.

* fix(vector_stores): type direct search query as Sequence[str]

The new helper's list[str] annotation pushed LIT001 over its
type-discipline ceiling. Sequence is the read-only shape the helper
actually needs, and list[str] still satisfies it at both call sites.
2026-08-18 17:46:59 -07:00
yuneng-jiang
b19d59be09
refactor(ui): move the model info edit form off antd Form (#37392)
* test(ui): characterize the model info and cache control submit payloads

Pins the antd behaviour these forms have today, ahead of moving them onto
react-hook-form: the full model info PATCH body, the sticky touched-field
semantics that decide which pricing keys ship, the mounted-only cache control
keys, and the string-typed injection point index.

* refactor(ui): move the model info edit form off antd Form

The deployment edit form on the model info view now runs on react-hook-form
with a zod resolver and shadcn controls, extracted into ModelInfoEditForm so
the view keeps the payload builder and the form keeps the fields.

Cache control injection points become a presentational value/onChange child,
which lets the model info view host it through react-hook-form while the add
model form keeps hosting it through antd. That child never wrote to a real
store on either side: it registered under cache_control_points while both
parents read cache_control_injection_points, so its form prop was inert.

antd marks a field touched on change and never clears it, and neither
touchedFields nor dirtyFields reproduces that, so the four pricing keys that
gate on it track first change explicitly.

The PTU rules move from antd validator wrappers to pure predicates that both
surfaces share, since the add model form still feeds the wrappers to its own
antd form.

* refactor(ui): trim comments and type the model record prop on the edit form

Cuts the explanatory comments that the house rules do not allow, keeping
only the three that record non-obvious library behaviour plus the eslint
directive, and narrows the modelData prop to the two fields the form reads.

Corrects the claim in 5b7ecede4e that the cache control child registered
the wrong key. At the staging tip the Form.List registration is on
cache_control_injection_points and is live, which is why this PR rehosts
it into advanced_settings. The dead part is the three
getFieldValue("cache_control_points") readers, whose key nothing
registers, so updateCacheControlPoints dereferences undefined and the
caught error reaches the console on every role, index or remove change.
2026-08-18 17:40:49 -07:00
ryan-crabbe-berri
bb8324c119
refactor(ui): drop @tremor/react and the theming scaffolding it needed (#37394)
The last tremor component import left the dashboard when the primitive
sweep merged, so the package, its v3 compatibility shim, its @theme token
block and the palette safelist it needed at runtime all have no consumer.

Removing the safelist is what shrinks the shipped stylesheet: tremor built
class names at runtime, so Tailwind had to emit every bg/text/border/ring/
stroke/fill utility across 22 palettes and 11 shades in case one was used.
Nothing in the app constructs a class name that way any more, so the
scanner finds every utility on its own.

The date-fns overrides pin also goes. It only existed because tremor and
react-day-picker@8 peered on date-fns 3 while Base UI wanted 4, and the
lockfile still resolves a single hoisted 4.4.0 without it.
2026-08-18 17:38:12 -07:00
yuneng-jiang
564ea1cf73
feat(ui): add success, warning and info status tokens (#37393)
The dashboard had no shared tokens for non-destructive status colours, so
components reached for raw Tailwind shades instead. Add --success, --warning
and --info alongside the existing --destructive, in both :root and .dark, and
register them in @theme inline so the usual utilities resolve.

Light values are picked for legibility as foreground text rather than by
copying a fixed shade number. Tailwind's ramps are not perceptually aligned
across hues, so amber-600 and green-600 sit at 66.6% and 62.7% lightness and
fail WCAG AA on white (3.19:1 and 3.22:1). green-700, amber-700 and blue-600
land at 52.7%, 55.5% and 54.6%, the same band as --destructive at 57.7%, and
clear AA. Dark mode uses the -400 shades, matching --destructive.

The .dark values are populated even though nothing can apply that class yet.
They are the artifact the later theme switch work will turn on.

Alert moves its info and warning variants onto the tokens. The tint is /5
rather than /10 because /10 drops both below AA. The error variant keeps its
existing shades: it involves no new token, and its current 9.21:1 is better
than anything the token form would give it.
2026-08-18 17:36:10 -07:00
mateo-berri
9018a95037 test(mcp): build fixture mapping state without in-place mutation 2026-08-18 17:35:40 -07:00
devin-ai-integration[bot]
3f15dc3287
fix(mcp): attach per-user BYOK credential when listing tools for non-oauth2 auth types (#34787)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-08-18 17:10:00 -07:00
mateo-berri
4285ffd82b fix(databricks): drop the minimal reasoning effort flag from the claude-opus-4-6 entry 2026-08-18 17:09:17 -07:00
yuneng-jiang
c33b3a32a6
feat(bedrock): add a config toggle to disable agent-runtime pass-through (#37386)
* feat(bedrock): add a config toggle to disable agent-runtime pass-through

The /bedrock pass-through dispatches agents, knowledge bases, flows, rerank,
retrieveAndGenerate, generateQuery and optimize-prompt to bedrock-agent-runtime,
so an operator who only wants to expose model invoke and converse has no way to
narrow that surface

Adds general_settings.disable_bedrock_agent_runtime_passthrough. When set, those
routes are rejected with a 403 before credentials are fetched or the request is
signed. Plain bedrock-runtime model pass-through is unaffected, and the setting
defaults to off, so existing deployments behave exactly as before

The branch is inverted to an early return for the non-agent-runtime case so the
toggle can reject outright instead of falling through to model extraction, which
would surface a confusing 400 about an unparseable model

* style(bedrock): drop redundant docstrings from the agent-runtime toggle
2026-08-18 17:05:40 -07:00
mateo-berri
75a569b3e2 fix(databricks): sync the packaged cost map backup with the new Databricks entries 2026-08-18 17:04:25 -07:00
Mateo Wang
f570af9fcf
Merge branch 'litellm_internal_staging' into add-databricks-model-pricing 2026-08-18 16:55:02 -07:00
Mateo Wang
55777d0e80
Merge pull request #35110 from shivijain2323/feature/bedrock-mantle-quota-project-itr1
feat(proxy): add project-level ITPM and OTPM quotas
2026-08-18 16:54:33 -07:00
Mateo Wang
054aefce0d
Merge pull request #37387 from BerriAI/litellm_guardrail_usage_requeue
fix(guardrails): requeue usage rollup rows dropped after retry exhaustion
2026-08-18 16:42:06 -07:00