The server detail page's Tool Testing Playground gated its browser-held
token handling on the legacy PKCE-passthrough shape, so a
true_passthrough or oauth_delegate server listed tools unauthenticated
and surfaced 'Failed to fetch MCP tools' with no way to authorize. The
playground now treats both modes as browser-held-token servers: it
reads the sessionStorage token established by the create/edit
browser-only Authorize, forwards it via the x-mcp-{alias}-authorization
header, evicts it on a 401, and shows its own Authorize gate when the
token is absent.
That gate's flow uses the gateway's relayed authorize/register/token
endpoints with the real server id, which previously 400ed for anything
but oauth2. Those endpoints now also accept the client-forwarded token
modes (the minted token is upstream-audienced and browser-held; DCR
persistence stays off on this path), and registry builds run the same
RFC 9728/8414 endpoint discovery for these modes that oauth2 rows get,
since their rows never store an authorization_url.
true_passthrough and oauth_delegate persist no upstream credentials, so
the create/edit forms had no way to preview tools or configure the tool
allowlist: tools/list went upstream unauthenticated and came back 401.
This reuses the existing OAuth authorize machinery in browser-only mode
for those two auth types: the admin authorizes against the upstream
(DCR/PKCE, with optional client credentials for IdPs without dynamic
registration), the token lands in sessionStorage exactly like the
legacy PKCE-passthrough path, and the tools preview forwards it via the
per-server x-mcp-{alias}-authorization header, which the passthrough
resolver arm already accepts. Nothing is written to the server row or
the per-user credential store; the create payload keeps excluding
credentials for these auth types via AUTH_TYPES_REQUIRING_CREDENTIALS.
The tools preview endpoint now also extracts the Authorization header
for the two new auth types so the browser-held token reaches the
passthrough arm during create-time previews.
Adds the two client-forwarded token modes to the MCP server create and
edit form auth dropdowns, and shows a warning when true_passthrough is
selected: the gateway performs no admission auth for that server, so
callers reach the upstream without a LiteLLM key and per-key/per-team
rate limits and spend tracking do not apply. The warning is a shared
component so the two forms cannot drift on the copy.
When an IdP denies SSO access it redirects back to /sso/callback with
error and error_description query params and no code param. The callback
previously fell through to the provider token exchange, which failed
with a generic "'code' parameter was not found in callback request"
400 that hides the real denial reason. Raise a 401 that surfaces the
IdP's error and description instead.
Ported from #26640 with conflicts resolved against current staging
The v2 OTel integration stamped litellm-specific error details as
error.code, error.stack_trace, and error.llm_provider, squatting on the
semconv-owned error.* namespace. They now live at
litellm.provider.error.code, litellm.provider.error.stack_trace, and
litellm.provider.error.llm_provider alongside the other vendor-extension
keys. error.type and error.message stay on the semconv keys.
* fix(guardrails): walk Responses-API text taxonomy in shared content helpers
Every guardrail sharing litellm/proxy/guardrails/_content_utils.py silently
drops all text on the /v1/responses path. AIM turns it into a loud 422 (
{"error":"No messages in the request"}); every other guardrail (Lakera v2,
Cato, Lasso, Repello, IBM, Azure Content Safety, enterprise secret
detection) scans an empty payload and lets the request through unscanned.
Three defects, all in _content_utils.py:
1. _iter_text_parts_in_content recognised only part.type == "text", but the
Responses API uses input_text (request) and output_text (assistant).
2. _coerce_input_to_messages gated on "every item has a role key"; any
Responses input list containing a function_call or function_call_output
item failed the check and was wrapped as one opaque blob.
3. build_inspection_messages forwarded any role through, including a bare
tool role missing tool_call_id, which validators like AIM's /fw/v1/analyze
reject with a schema error.
Fix walks the actual Responses item taxonomy (message, function_call,
function_call_output, bare content parts and strings), recognises
{text, input_text, output_text} everywhere, and coerces any role outside
{system, user, assistant} to user in the outbound inspection payload.
* style: ruff-format changed guardrail files
* test(guardrails): cover function_call_output string form; drop em-dash in new docstring
* fix(guardrails): map function_call_output straight to user role
Avoids ever materialising a schema-invalid bare tool message. The
downstream role-safety coercion in build_inspection_messages still
guards genuinely caller-supplied non-standard roles (developer,
function, custom values); add a regression test covering that path
so the coercion has real coverage after this simplification.
* test(guardrails): pin chat-completions tool-role coercion in build_inspection_messages
* docs(test): soften AIM-specific claims in LIT-4294 test docstrings
Ryan's review flagged that several test docstrings assert AIM's
/fw/v1/analyze validates + rejects specific schema violations. That
behavior is customer-reported in the LIT-4294 writeup, not directly
verified by us. Rephrase to attribute the AIM 422 to the customer's
writeup and describe the underlying constraint as the OpenAI chat
schema; any downstream API that validates against that schema rejects
the same shape.
* refactor(guardrails): move unsupported-role coercion into AIM only
The generic coercion in build_inspection_messages collapsed any role
outside {system, user, assistant} to user for every caller of the
helper. Combined with the pre-existing apply_redacted_messages_back
write-back behavior in Lakera/AIM/Cato, that turned a loud OpenAI 400
on chat-completions tool-message masking into a silent semantic
corruption of the outbound request (role tool with tool_call_id got
rewritten to bare role user, dropping the assistant + tool_calls
sibling).
AIM specifically requires the coercion because its /fw/v1/analyze
validates the payload against the OpenAI chat schema; other guardrails
either do not validate roles or do their own reconstruction. Move the
coercion to AimGuardrail._build_aim_inspection_messages so the shared
helper keeps caller roles intact and no new cross-guardrail role
corruption is introduced. The pre-existing apply_redacted_messages_back
structural flatten remains as separate follow-up work.
function_call_output items still synthesise role user in the shared
helper because they have no natural role field, which is a different
concern from coercing a caller-supplied role.
* refactor(guardrails): preserve role fidelity in shared _content_utils
Shared inspection helpers should extract text and preserve semantic
role signals; role coercion for third-party schema safety stays inside
the guardrail that needs it (AIM).
Three shared-helper changes:
- Bare content-part dicts (input_text/output_text) with an explicit role
keep it; only role-less parts default to user.
- Responses message items already had their role preserved; the
behavior is now covered by an explicit test.
- function_call_output items default to role tool (semantic equivalent
of the chat-completions tool message shape) instead of role user, so
Responses and chat completions produce symmetric inspection payloads.
A caller-supplied role on the item is still preserved.
AIM's schema-safe coercion in _build_aim_inspection_messages already
handles the resulting role tool: it collapses to user before the POST
to /fw/v1/analyze so AIM's OpenAI-schema validator does not reject the
bare tool message (no tool_call_id can survive the flatten). Added a
regression test in test_aim.py covering that path.
The in-workflow regex list was hard to maintain and, because it runs on pull_request, could be modified by the same PR it inspects. Path gating for the OSS daily branches now lives in repository branch protection settings, so this workflow keeps only the OSS-safe checks: the hardcoded-secret test and ruff
Bedrock Converse supports cachePoint ttl (1h GA for Claude 4.5+), and
_get_cache_point_block maps cache_control.ttl -> cachePoint.ttl, but the
model parameter its allow-list gate requires was only threaded through
the system-message path. Every message-level path either called
_get_cache_point_block without model= (8 call sites in
_bedrock_converse_messages_pt / _pt_async) or hardcoded
CachePointBlock(type="default") (tool-result blocks and
_convert_to_bedrock_tool_call_invoke), so a requested 1h ttl silently
degraded to the 5-minute default - exactly on the conversation-tail
breakpoint that long-running agents need to survive tool calls longer
than 5 minutes.
- pass model= at the 8 _get_cache_point_block call sites
- tool-result blocks: capture the cache_control dict (was a boolean)
and route through _get_cache_point_block so ttl survives
- _convert_to_bedrock_tool_call_invoke: accept optional model and route
per-tool-call cache_control through _get_cache_point_block
Completes the ttl support added for system messages (#19848, #20326):
message-level cache_control now behaves identically.
Note: message-level cache_control on a content-less assistant message
emits no cachePoint at all today; that pre-existing gap is orthogonal
to ttl and left out of scope (per-tool-call placement covers it).
Co-authored-by: Arash <arashne@glia-ai.com>
* refactor(ui): point invitation links at the dedicated /onboarding route
Invitation and reset-password links were built as /ui?invitation_id=..., which lands on the dashboard index and renders the onboarding form inline. They now point at the standalone /ui/onboarding route, so the index no longer has to special-case invitations. Old links keep working unchanged; the index still renders onboarding inline for ?invitation_id until the migration closeout removes that branch.
Updates the three generators (the enterprise email builder, bulk user create, and the invitation/reset-password modal) and extracts the modal's URL building into a pure, unit-tested buildOnboardingUrl
Refs LIT-3687
* refactor(ui): guard buildOnboardingUrl against a missing invitation id
Return "" instead of emitting an invitation_id=undefined link when the id is not yet available, matching the existing empty-baseUrl guard. Placed after the SSO branch so the SSO link, which does not use the id, is unaffected
Refs LIT-3687
An org admin updating a team budget from the Hub UI was rejected with 401, because the route gate only recognizes an org admin when the request body carries organization_id while the UI sends team_id. For /team/update, resolve the target team's organization_id from team_id before the gate runs, so an org admin of the team's own org clears the org-scoped branch without the client passing organization_id. Team admins and cross-org admins stay denied at the gate, and callers that already pass organization_id are unaffected, so the existing /team/update authorization matrix is unchanged
* fix(rerank): log optional_rerank_params at debug not info to avoid leaking request content
* test(rerank): exercise sync rerank path so coverage counts the log line
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(vertex): forward realtime health check params
* refactor(vertex): resolve realtime health check params via VertexBase helpers
Address review feedback on the vertex param forwarding: pass model_params
through to _realtime_health_check and resolve vertex credentials, project,
and location inside the vertex_ai branch using the existing
VertexBase.safe_get_vertex_ai_* helpers, so provider-specific key extraction
no longer lives in litellm_core_utils and dict-typed vertex_credentials are
supported
* test(vertex): move realtime health check test to mapped unit test path
codecov/patch reported the vertex branch of _realtime_health_check as
uncovered because tests/litellm_utils_tests is not part of the unit test
groups that upload coverage. Move the test into
tests/test_litellm/litellm_core_utils/test_health_check_helpers.py, which
the core-utils group runs, keeping the same end-to-end assertions through
litellm.ahealth_check
---------
Co-authored-by: Aleksandr Liadov <72351793+AleksandrLiadov@users.noreply.github.com>
The connect gate resolved x-mcp-{alias}-authorization by matching the raw
lowercased alias/server_name/name only, but dashboard clients send
x-mcp-{sanitize_mcp_alias_for_header(alias)}-authorization, and egress resolves
those through lookup_mcp_server_auth_in_headers, which also tries the sanitized
alias. So a per-server token bound with a sanitized alias (e.g. alias 'pt-server'
arriving as header key 'pt_server') was forwarded at egress but still triggered a
preemptive 401 at connect. _client_has_per_server_auth_header now resolves through
the same lookup_mcp_server_auth_in_headers egress uses, so connect and egress
agree on which header names match.
The listing fan-out withholds the request-wide Authorization from a
true_passthrough / oauth_delegate server when another server in scope also
consumes it, so one bearer is not replayed across upstreams. The later
server.extra_headers copy loop did not honor that decision: a server listing
Authorization in extra_headers would re-copy the withheld bearer from
raw_headers. The withhold decision is now computed once and applied to both
the forwarding branch and the extra_headers loop.
The preemptive 401 for true_passthrough and oauth_delegate only inspected the
request-wide Authorization, so a caller who bound the upstream token via the
per-server x-mcp-{alias}-authorization header (the required shape in a
multi-server aggregate, where the request-wide Authorization is withheld) was
spuriously 401'd at initialize even though egress already honors that header.
The gate now recognizes the per-server header for both modes via a shared
helper, mode-correctly: true_passthrough treats any Authorization or the
per-server header as the upstream token, oauth_delegate keeps requiring a
distinct x-litellm-api-key so a lone Authorization consumed for admission is
never mistaken for an upstream token. The preemptive raise is also gated to
single-server scopes so a multi-server aggregate degrades gracefully instead
of one missing token 401-ing the whole connect.
For the client-forwarded token modes the gateway relays the caller's
Authorization to the upstream, so a request carrying more than one
Authorization header would make which token is forwarded ambiguous (the
ASGI header list collapses to last-wins) and could diverge from what
admission inspected. Multiple Authorization headers is malformed for
bearer auth anyway (RFC 9110: not a comma-combinable field), so the
ingress header converter now fails closed with a 400 instead of silently
keeping one. Applies to every MCP request, not just passthrough.
* feat(ui): add shared table_cells kit and convert logs columns
DateCell, MoneyCell, IdCell and StatusBadge consolidate the duplicated
per-table cell implementations behind one component each. The logs page
columns are the reference conversion; the dead auditLogColumns export
(superseded by audit_logs.tsx) is removed with it
* refactor(ui): consolidate table cells onto the shared table_cells kit
106 cell sites across 44 table files converge onto DateCell, MoneyCell,
IdCell and StatusBadge, replacing 8 date formats, 6 spend formats, 7 id
truncation strategies and 6 status badge styles with one implementation
each. Badge now forwards refs so Base UI tooltip triggers composed over
it can attach (they previously never opened under React 18). TimeCell
is deleted; its two consumers now render DateCell
* fix(ui): suppress cost tooltip for zero spend and drop dead getStatusBadge param
The logs Cost tooltip showed the raw $0 over a "-" cell for zero or
null spend (pre-existing, surfaced by review); the tooltip now only
renders when there is a real amount. healthCheckColumns no longer takes
the unused getStatusBadge callback and its dead definition is removed
* fix(ui): restyle StatusBadge as tinted pill matching the prior antd Tag look
* fix(ui): keep StatusBadge fully rounded like the other kit pills
Staging recently added the local eslint rules no-large-inline-object-arg and
no-long-condition-chain and tightened no-nested-ternary to an error. After
merging staging, the license-banner code tripped them: the banner's tiered
description was a nested ternary (now an error), and two option objects were
passed inline (adding budget debt). Extract the description into an
early-return helper, and hoist the useQuery options and the date-format options
into named constants. No behavior change; keeps the inline-object-arg count at
the committed baseline rather than bumping it
* fix(guardrails): forward grayswan scan id header
* test(guardrails): cover grayswan scan id forwarding
* fix(guardrails): prevent overwriting existing metadata headers when extracting scan id
* test(guardrails): cover header merging logic
* chore(guardrails): fix formatting
* test(guardrails): enforce case preservation
* chore(guardrails): corrected grayswan type annotations
* fix(guardrails): sanitized grayswan header metadata
* test(guardrails): covered grayswan logging headers
* fix(guardrails): guard grayswan header lookup against None and drop dead comment
- Fall back to {} when proxy_server_request is explicitly None so
request_data.get(...).get('headers') never raises AttributeError.
- Remove the commented-out user_api_key_auth pop; it was inert and
greptile called it out as ambiguous.
---------
Co-authored-by: Theodore Drzewinski <93957989+tediferJones@users.noreply.github.com>
The layout test renders DashboardShell without a QueryClientProvider and mocks
DebugWarningBanner to null for exactly that reason. The new LicenseExpiryBanner
also uses a React Query hook, so it needs the same treatment; without it the
test threw "No QueryClient set". Runtime is unaffected: the app mounts a
QueryClientProvider above the layout (DebugWarningBanner already relies on it)
* feat(ui): add eslint rules for nested ternaries, large inline object args, and long condition chains
Adds three dashboard lint rules to keep new code readable. Nested ternaries
are banned outright via the built-in no-nested-ternary, with the 265 existing
occurrences grandfathered in eslint-suppressions.json so only new ones fail.
Two custom rules ship as a small local plugin under scripts/eslint-rules:
no-large-inline-object-arg flags object literals with 4+ properties passed
straight into a call, nudging toward a named variable, and no-long-condition-chain
flags boolean expressions that combine 4+ conditions, nudging toward a named
boolean. Both are warnings tracked on the existing budget ratchet
(eslint-budgets.json + eslint-metrics.json) with headroom above the current
counts, so they ratchet down over time rather than freezing a baseline. Both
thresholds are configurable rule options and covered by RuleTester unit tests.
* fix(ui): scope no-long-condition-chain to boolean operators, not nullish
Greptile flagged that the rule counted nullish-coalescing chains the same as
&&/|| chains, so a 4-part `a ?? b ?? c ?? d` fallback surfaced "Boolean
expression combines 4 conditions", which is inaccurate since a `??` fallback
is value defaulting, not a condition. Restrict the visitor to && / || nodes so
`??` chains are treated as leaves, while a boolean chain nested inside a `??`
is still caught. Drops 6 miscounted occurrences (240 -> 234).
* chore(ui): sync lint metrics and suppressions with staging
Merge advanced the base branch, adding one no-large-inline-object-arg
occurrence (508 -> 509) and making one grandfathered react-hooks suppression
stale. Regenerate eslint-metrics.json and prune the suppression so the
budget/drift gate passes.
* chore(ui): sync lint metrics with staging
Merge advanced the base, adding four no-large-inline-object-arg occurrences
(509 -> 513). Regenerate eslint-metrics.json so the drift gate passes.
UsageIndicator was fetching /health/license through its own useEffect while
the new expiry banner fetches the same endpoint via useLicenseInfo, so an admin
with the usage widget open made two identical calls per page load. Point
UsageIndicator at useLicenseInfo too; both callers now share one React Query
cache entry, collapsing it back to a single request. The null/error semantics
are preserved (data ?? null matches the previous catch-to-null), and license
errors never fed the widget's error state before either
* test(ui): characterize DataTable behavior before shadcn reskin
Pins the shared view_logs DataTable contract with library-agnostic
queries ahead of the tremor-to-shadcn table migration: loading and
empty states, TanStack column defs with custom cell renderers,
onRowClick payload, both expansion render paths (colspan sub-component
and sibling child rows), the getRowCanExpand gate, and client-side
sorting on and off. These must pass unchanged after the reskin.
* refactor(ui): reskin shared DataTable from tremor onto shadcn table primitives
Swaps the view_logs DataTable's presentational layer from @tremor/react
to the in-repo components/ui/table primitives and hardens the seam that
every later table migration copies:
- getRowId is injected instead of hardcoded to request_id through an
any cast; identity defaults to the row index and the logs page now
passes request_id explicitly, keeping expansion state attached to the
right row across refetch reorders
- one expansion render path: renderChildRows had zero consumers and is
removed; renderSubComponent (colspan cell) is the single path
- the four consumers passing dead no-op renderSubComponent and
getRowCanExpand boilerplate drop it
- loading and empty defaults become generic (Loading... / No results)
instead of log-specific
The characterization tests from the previous commit pass unchanged
except the dead child-rows path test, replaced by a reorder-stability
test for injected getRowId plus coverage of the new generic defaults.
First tremor removal of the tables track; view_logs/table.tsx no longer
imports @tremor/react.
* test(ui): assert child rows hidden before expansion in DataTable test
* fix(ui): suppress row hover on DataTable placeholder rows
* feat(ui): polish DataTable with skeleton loading, header band, and numeric column alignment
* feat(ui): shape DataTable skeletons per column and keep stale rows during refetch
* revert(ui): drop DataTable skeleton loading, restore text loading row
* fix(ui): clip DataTable to its rounded wrapper and right-align Duration/TTFT values
The v2 emitter has never stamped error.message / error.code /
error.stack_trace / error.llm_provider as span attributes; only error.type
reached the wire. Backends that flatten span attributes into label
indexes (Elastic APM labels.error_*, Datadog span tags) lost these
four fields when v2 became the active integration on v1.90+ for
otel_v2-flagged deployments. The pre-existing exception span event
carrying the full message (LIT-3758) is unchanged; the message now
rides both places at once, matching v1s shape.
SpanError grows three optional detail fields; _parse_error threads
them from StandardLoggingPayloadErrorInformation; the emitters error
branch stamps them via a new module-level helper, guarded per field so
guardrail-shape errors are not polluted with empty attributes. New
semconv constants mirror open_inference.ErrorAttributes byte-for-byte,
so v1 and v2 consumers read the same keys.
Regression tests extend the mapped test files under
tests/test_litellm/integrations/otel/. pytest reports 243 passed.
Surfaces a persistent, tiered banner under the dashboard navbar when an airgapped enterprise license is close to expiring: an amber, session-dismissible warning within 30 days, a non-dismissible red alert within 7 days, and a non-dismissible red banner once the date has passed. It reads the existing /health/license endpoint, so no backend change is needed, and is driven strictly by expiration_date; community and remote-validated instances that report no date show nothing. Shared day-count math is extracted to licenseUtils so the banner and the existing UsageIndicator widget stay in sync
* fix(health): bridge litellm_metadata into logging object in _batch_health_check
* Update litellm/litellm_core_utils/health_check_helpers.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix(health): address review - share single metadata copy, conditional api_base, add tests
- Only set api_base in litellm_params when a value actually exists;
providers like bedrock/vertex/gemini resolve it implicitly and an
empty string overwrites their resolution.
- Use a single .copy() for both metadata and litellm_metadata to
prevent downstream drift between the two references.
- Add 6 unit tests covering metadata bridging, api_base omission,
guard conditions, and dispatch routing.
Signed-off-by: pramod <pramod.b@pfizer.com>
* refactor(health): use update_from_kwargs helper for metadata bridge
Collapses the manual metadata/litellm_metadata plumbing in
_batch_health_check into a single update_from_kwargs call, matching
how the sibling batch/image/rerank/ocr surfaces bridge metadata onto
the pre-injected logging object. Drops the bare Dict typing and the
inline comment, and switches the tests to assert against the helper.
---------
Signed-off-by: pramod <pramod.b@pfizer.com>
Co-authored-by: pramod <pramod.b@pfizer.com>
Co-authored-by: Pramod B <155433727+BPRMD18@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
