* fix(guardrails): forward grayswan scan id header
* test(guardrails): cover grayswan scan id forwarding
* fix(guardrails): prevent overwriting existing metadata headers when extracting scan id
* test(guardrails): cover header merging logic
* chore(guardrails): fix formatting
* test(guardrails): enforce case preservation
* chore(guardrails): corrected grayswan type annotations
* fix(guardrails): sanitized grayswan header metadata
* test(guardrails): covered grayswan logging headers
* fix(guardrails): guard grayswan header lookup against None and drop dead comment
- Fall back to {} when proxy_server_request is explicitly None so
request_data.get(...).get('headers') never raises AttributeError.
- Remove the commented-out user_api_key_auth pop; it was inert and
greptile called it out as ambiguous.
---------
Co-authored-by: Theodore Drzewinski <93957989+tediferJones@users.noreply.github.com>
* feat(ui): add eslint rules for nested ternaries, large inline object args, and long condition chains
Adds three dashboard lint rules to keep new code readable. Nested ternaries
are banned outright via the built-in no-nested-ternary, with the 265 existing
occurrences grandfathered in eslint-suppressions.json so only new ones fail.
Two custom rules ship as a small local plugin under scripts/eslint-rules:
no-large-inline-object-arg flags object literals with 4+ properties passed
straight into a call, nudging toward a named variable, and no-long-condition-chain
flags boolean expressions that combine 4+ conditions, nudging toward a named
boolean. Both are warnings tracked on the existing budget ratchet
(eslint-budgets.json + eslint-metrics.json) with headroom above the current
counts, so they ratchet down over time rather than freezing a baseline. Both
thresholds are configurable rule options and covered by RuleTester unit tests.
* fix(ui): scope no-long-condition-chain to boolean operators, not nullish
Greptile flagged that the rule counted nullish-coalescing chains the same as
&&/|| chains, so a 4-part `a ?? b ?? c ?? d` fallback surfaced "Boolean
expression combines 4 conditions", which is inaccurate since a `??` fallback
is value defaulting, not a condition. Restrict the visitor to && / || nodes so
`??` chains are treated as leaves, while a boolean chain nested inside a `??`
is still caught. Drops 6 miscounted occurrences (240 -> 234).
* chore(ui): sync lint metrics and suppressions with staging
Merge advanced the base branch, adding one no-large-inline-object-arg
occurrence (508 -> 509) and making one grandfathered react-hooks suppression
stale. Regenerate eslint-metrics.json and prune the suppression so the
budget/drift gate passes.
* chore(ui): sync lint metrics with staging
Merge advanced the base, adding four no-large-inline-object-arg occurrences
(509 -> 513). Regenerate eslint-metrics.json so the drift gate passes.
* test(ui): characterize DataTable behavior before shadcn reskin
Pins the shared view_logs DataTable contract with library-agnostic
queries ahead of the tremor-to-shadcn table migration: loading and
empty states, TanStack column defs with custom cell renderers,
onRowClick payload, both expansion render paths (colspan sub-component
and sibling child rows), the getRowCanExpand gate, and client-side
sorting on and off. These must pass unchanged after the reskin.
* refactor(ui): reskin shared DataTable from tremor onto shadcn table primitives
Swaps the view_logs DataTable's presentational layer from @tremor/react
to the in-repo components/ui/table primitives and hardens the seam that
every later table migration copies:
- getRowId is injected instead of hardcoded to request_id through an
any cast; identity defaults to the row index and the logs page now
passes request_id explicitly, keeping expansion state attached to the
right row across refetch reorders
- one expansion render path: renderChildRows had zero consumers and is
removed; renderSubComponent (colspan cell) is the single path
- the four consumers passing dead no-op renderSubComponent and
getRowCanExpand boilerplate drop it
- loading and empty defaults become generic (Loading... / No results)
instead of log-specific
The characterization tests from the previous commit pass unchanged
except the dead child-rows path test, replaced by a reorder-stability
test for injected getRowId plus coverage of the new generic defaults.
First tremor removal of the tables track; view_logs/table.tsx no longer
imports @tremor/react.
* test(ui): assert child rows hidden before expansion in DataTable test
* fix(ui): suppress row hover on DataTable placeholder rows
* feat(ui): polish DataTable with skeleton loading, header band, and numeric column alignment
* feat(ui): shape DataTable skeletons per column and keep stale rows during refetch
* revert(ui): drop DataTable skeleton loading, restore text loading row
* fix(ui): clip DataTable to its rounded wrapper and right-align Duration/TTFT values
The v2 emitter has never stamped error.message / error.code /
error.stack_trace / error.llm_provider as span attributes; only error.type
reached the wire. Backends that flatten span attributes into label
indexes (Elastic APM labels.error_*, Datadog span tags) lost these
four fields when v2 became the active integration on v1.90+ for
otel_v2-flagged deployments. The pre-existing exception span event
carrying the full message (LIT-3758) is unchanged; the message now
rides both places at once, matching v1s shape.
SpanError grows three optional detail fields; _parse_error threads
them from StandardLoggingPayloadErrorInformation; the emitters error
branch stamps them via a new module-level helper, guarded per field so
guardrail-shape errors are not polluted with empty attributes. New
semconv constants mirror open_inference.ErrorAttributes byte-for-byte,
so v1 and v2 consumers read the same keys.
Regression tests extend the mapped test files under
tests/test_litellm/integrations/otel/. pytest reports 243 passed.
* fix(health): bridge litellm_metadata into logging object in _batch_health_check
* Update litellm/litellm_core_utils/health_check_helpers.py
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix(health): address review - share single metadata copy, conditional api_base, add tests
- Only set api_base in litellm_params when a value actually exists;
providers like bedrock/vertex/gemini resolve it implicitly and an
empty string overwrites their resolution.
- Use a single .copy() for both metadata and litellm_metadata to
prevent downstream drift between the two references.
- Add 6 unit tests covering metadata bridging, api_base omission,
guard conditions, and dispatch routing.
Signed-off-by: pramod <pramod.b@pfizer.com>
* refactor(health): use update_from_kwargs helper for metadata bridge
Collapses the manual metadata/litellm_metadata plumbing in
_batch_health_check into a single update_from_kwargs call, matching
how the sibling batch/image/rerank/ocr surfaces bridge metadata onto
the pre-injected logging object. Drops the bare Dict typing and the
inline comment, and switches the tests to assert against the helper.
---------
Signed-off-by: pramod <pramod.b@pfizer.com>
Co-authored-by: pramod <pramod.b@pfizer.com>
Co-authored-by: Pramod B <155433727+BPRMD18@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
On re-auth against a server with a persisted DCR client, register_client_with_server
short-circuits and returns a placeholder client_secret ("dummy") that the browser
echoes back to /token. exchange_token_with_server overrode the caller's client_id
with the persisted one but still fell back to the caller's secret when the server
had none stored, so a persisted public PKCE client (which has no secret) was paired
with the literal string "dummy" and the IdP rejected the exchange with 401 on every
re-authorization; the proxy surfaced that as a 500. First connects and brand-new
servers worked because a real DCR registration ran and no placeholder existed.
Resolve the secret from the server whenever the server's client_id wins, so a
secretless public client sends no client_secret at all
key_alias can become the secret name used by external secret manager
integrations (HashiCorp Vault, CyberArk Conjur) when store_virtual_keys is
enabled. Add raise_if_unsafe_secret_name, a shared validation check applied
unconditionally before a secret name reaches either integration or the
/key/generate, /key/update, and /key/regenerate API boundary, independent
of the existing enable_key_alias_format_validation opt-in flag.
Also hardens the Vault URL builder to percent-encode reserved characters
in secret_name (preserving "/" and "@"), and switches the Conjur policy
body to a real YAML serializer instead of raw string interpolation.
An MCP tool call that completes with CallToolResult.isError=true correctly
returns HTTP 200 per the MCP spec, but the shared post-call logging helper
always fired async_success_handler, so the standard logging payload carried
status=success and OTel (whose _parse_error only marks ERROR on
status=failure) showed green spans for failed tools.
The helper now checks the result after async_post_mcp_tool_call_hook runs
(guardrails may flip isError there) and routes error results to the failure
path: success gates are consumed so the @client wrapper cannot enqueue a
success log, failure_handler and async_failure_handler fire with a new
MCPToolResultError carrying the tool's first text content, and
post_call_failure_hook records the failure the same way raised exceptions
already do. Raised exceptions never reach the helper, so no double failure
logging. HTTP wire behavior is unchanged
Resolves LIT-4081
* test(realtime): record and replay websocket traffic in redis vcr cassettes
* style(realtime): ruff-format ws-vcr harness
* fix(realtime): warn instead of silently disabling ws-vcr when the redis client cannot be built
* fix(mcp): drop the cached per-user OAuth token when the credential row changes
The v2 authorization_code chain Cached(Refreshing(V2PerUserTokenStore)) caches a positive token
until its expires_at (or 300s without one), and CachedOAuthTokenStore.invalidate had no callers,
so a re-authorization or revocation wrote the DB while egress kept serving the replaced token
from the in-process cache until its TTL. LazyPerUserOAuthTokenStore now exposes invalidate,
MCPServerManager threads it to the write side, and the three credential write sites (the OAuth
callback, the Tools-tab persist endpoint, and the revoke endpoint) drop the cache entry after
the row changes. The v2 refresher's own persist stays untouched; RefreshingTokenStore already
feeds the rotated token back into the cache in the same fetch
* test(mcp): pin cache invalidation on the revoke already-gone branch
Greptile's review flagged that only the happy-path delete asserted the invalidate; a refactor
moving the call inside the try block would silently skip the cache drop when the row was
already deleted by a concurrent request while the cache still held the revoked token. The new
test fails on exactly that mutation
* test(mcp): cover invalidate on the redis-backed lazy store path
Codecov flagged the redis fast path of LazyPerUserOAuthTokenStore.invalidate as unexercised;
the existing invalidate tests only ran the no-redis chain. The new test builds the redis chain
via a fetch and asserts a subsequent invalidate reaches the same store instance without a
rebuild
Add a tag_rpm_limit field to virtual keys so each request tag gets its own independent RPM counter on the v3 rate limiter. A key configured with per-tag limits tracks each tag/group separately, and requests whose tag has no configured limit fall back to the key-level limit. Includes the dashboard UI to manage per-tag limits on key create and edit.
Resolves LIT-3147
* feat(ui): expose MCP max_concurrent_requests in server create and edit forms
The proxy has enforced a per-server outbound tool-call concurrency cap
(max_concurrent_requests) across every MCP egress path since #31641, and the
management API has accepted the field on create and update all along, but the
dashboard offered no way to set it. Add an optional Max Concurrent Requests
input to the MCP server create and edit forms; it applies to every auth type
and transport, so it renders unconditionally rather than gated on auth mode.
Clearing the field on edit sends null so the stored limit is unset.
Also rebuild the per-server semaphore when the configured limit changes.
Previously the semaphore was created once per server_id and never resized, so
an edited limit only took effect after a proxy restart even though the new
value was persisted and reloaded into the registry.
* feat(ui): mark MCP max concurrent requests field label as optional
* test(ui): stop OBO create-form tests from timing out on CI
The token-exchange payload test and the Entra scope-required test filled five
text fields with user.type, which dispatches a full keystroke sequence per
character; every input event runs the antd form onValuesChange handler and
re-renders the whole CreateMCPServer tree, roughly 120 renders per test. As
the form grew the two tests reached 8s and 18s locally, which crosses the 30s
vitest timeout on slower CI containers; ui_unit_tests failed twice this way.
Switch the plain text fields to fireEvent.change (one input event per field),
matching the existing stdio test pattern. Both tests assert form output, not
keystroke behavior, and now run in about 3s each.
* ci: skip unit test workflows when only docs or ui files change
Mirror the CircleCI backend path filter (.circleci/scripts/classify_changes.sh)
in the GitHub Actions unit test workflows by adding paths-ignore for ui/**,
docs/**, *.md and *.mdx to every test-unit-*.yml pull_request trigger
* ci: drop docs/** from unit test paths-ignore since the folder no longer exists
* ci(responses): bound azure shell tool e2e call and enforce per-test timeout
The azure variant of test_responses_api_shell_tool always makes a live
Azure call (its skip outcome means no VCR cassette is ever persisted).
When Azure held the connection instead of answering, the call sat on
litellm's 6000s responses deadline until CircleCI killed the whole job
via no_output_timeout after 15m of silence (job 2013288).
Bound the e2e call at 90s and skip on litellm.Timeout, matching the
existing InternalServerError and BadRequestError skips, and give the
llm_responses_api_testing job the same pytest-timeout guard the
llm_translation_testing job already uses so no single hung test can
consume the 15m no-output window again.
* test(responses): drop job-level pytest timeout, keep shell tool 90s bound
* fix(passthrough): stream non-sse passthrough responses instead of buffering in memory
Non-SSE passthrough responses were fully read into proxy memory (content = await response.aread()) before the first byte reached the client. For large non-JSON bodies such as Anthropic batch results jsonl files this ballooned proxy RSS to a multiple of the file size and produced near-total TTFB dead air, letting intermediaries kill the silent connection and truncate the download.
The upstream request is now sent with httpx stream semantics and the buffering decision is made from the response headers: application/json (and +json) bodies plus upstream errors keep the buffered behavior since spend logging, guardrails and managed-id rewriting inspect them, while every other 2xx body is relayed as a StreamingResponse that iterates upstream bytes without accumulating them, preserving status code and headers (including x-litellm-*) and firing the success-handler logging with response_body=None once the stream completes.
* fix(passthrough): log client disconnects mid-stream and derive test client cache key from production code
* test(passthrough): intercept AsyncClient.send in legacy passthrough tests and assert final wire params
* test(passthrough): fail with a clear assert when the passthrough client cache scan misses
* fix(utils): resolve bedrock regional inference profiles to regional pricing in get_model_info (LIT-4056)
* test(register_model): use a triple provider prefix as the unresolvable-key fixture
get_model_info now resolves bedrock/bedrock/... like a routing prefix, so the
double-prefix fixture stopped exercising the register_model fallback path.
Lock the new double-prefix resolution in as a model-info regression test
* fix(passthrough): stop request params from clobbering merged target query params
* fix(passthrough): rewrite managed ids in query params before folding them into the URL
* fix(ui): scope key models dropdown options to the key's team
A teamless key no longer offers the all-team-models option in the create and
edit forms; the backend expands that sentinel to the full proxy model list when
no team is attached, which is rarely what the user intended. A team key no
longer surfaces the all-proxy-models sentinel that leaks in verbatim when the
team's own model list carries it; the dropdown keeps All Team Models plus the
team's individual models.
Adds browser coverage to the management e2e suite: playwright (an optional
dependency behind importorskip) drives the proxy-served dashboard at /ui,
asserts the dropdown options a real user sees for teamless and team keys on
both create and edit, and walks the create modal end to end, reading the
persisted key back through /key/info.
* fix(ui): offer all-proxy-models on teamless keys in the models dropdown
A teamless key has no team allowlist to inherit, so the dropdown now offers All
Proxy Models in place of All Team Models on both the create and edit forms, with
the same exclusive-selection handling. Component and browser e2e tests updated to
pin the swapped option pair; the teamless create case now also walks the modal end
to end and reads the persisted key back through /key/info.
* test(ui): update no-team key creation spec to pick All Proxy Models
The create modal no longer offers All Team Models without a team; the teamless
path now offers All Proxy Models, which is what this spec exercises
* fix(ui): gate All Team Models on the team object being loaded
When a key has a team_id but the teams prop does not yet include the matching team, availableModels stays empty and the models dropdown rendered All Team Models on its own with nothing to compare against. Gate the option on the team object being present so it only appears once team models are known, and add a regression test for the loading state
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): filter all-proxy-models from teamless model fetch in key edit form
The teamless fetch path stored modelAvailableCall results without excludeProxyWideSentinel, so an all-proxy-models entry in the response rendered a second option colliding with the hardcoded All Proxy Models sentinel. Apply the same filter used on the team path and add a regression test asserting the sentinel option is not duplicated
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Mubashir Osmani <mubashir@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Root cause: PR #30867 removed request-time os.environ/ expansion in
BaseAWSLLM.get_credentials. That is only safe if config-load pre-resolves
os.environ/ refs so the value reaching get_credentials is already the real
secret. The YAML config path has always done this. The DB-load path
(ProxyConfig._resolve_db_litellm_param) only re-expanded keys in a hardcoded
whitelist (_DB_LITELLM_PARAM_ENV_REF_KEYS) plus short-circuited env-ref
resolution entirely for team-scoped rows. PR #32256 extended that whitelist
to 18 keys to unblock a customer whose Bedrock model with aws_role_name:
os.environ/BEDROCK_ASSUME_ROLE_ARN broke on v1.90+, but the whitelist is
structurally fragile: every future auth field breaks the same way until
someone remembers to add it
Fix: remove the whitelist and the team-scope short-circuit. The DB-load
resolver now expands os.environ/ on every string field, matching the YAML
path. Trust boundary stays on the write side: only PROXY_ADMIN can create
team_id=None rows, only team admins of a team can create rows scoped to
that team, and the request-body vector is still blocked by
_BANNED_REQUEST_BODY_PARAMS. Team-scoped rows now resolve env refs — this
is a deliberate LIT-3831 threat-model expansion trusting team admins for
env-var reads
Regression tests in tests/test_litellm/proxy/proxy_server/test_proxy_config.py:
- test_ProxyConfig__add_deployment_resolves_env_refs_after_db_decrypt pins
admin-scoped rows resolve every field (previously api_base stayed literal)
- test_ProxyConfig__add_deployment_resolves_team_env_refs pins team rows
resolve env refs (previously stayed literal)
- test_ProxyConfig__add_deployment_resolves_env_refs_on_arbitrary_field pins
the no-whitelist invariant against a made-up field name
- test_ProxyConfig__add_deployment_resolves_env_refs_for_aws_bedrock_auth_params
(from #32256) still passes
- Path B counterparts (decrypt_model_list_from_db) mirror the above
Left as followups (not fixed here):
- /model/info and /v2/model/info still echo resolved values for fields not
in the current pop-list (aws_role_name, aws_sts_endpoint, api_base, etc.).
Fix is to extend remove_sensitive_info_from_deployment; separate PR
- Master-key rotation reads DB rows via decrypt_model_list_from_db which
now resolves universally, so rotation collapses env-refs into hardcoded
values. Pre-existing bug for the 6 previously-whitelisted fields; wider
surface after this PR. Separate PR
* feat(mcp): let users select the entra_obo token_exchange profile in the UI and API
The backend token_exchange arm supports two wire dialects via token_exchange_profile
("rfc8693" default, or "entra_obo" for Microsoft Entra's On-Behalf-Of, the RFC 7523
jwt-bearer grant), but it could only be set through config.yaml. This surfaces it to the
create/update REST API and the dashboard so an admin can create an entra_obo server there,
completing the parity started in the parent PR for the other token-exchange fields.
token_exchange_profile becomes a dedicated column on LiteLLM_MCPServerTable, mirroring the
sibling fields: it is added to the request models, read column-first in
build_mcp_server_from_table with the credentials-blob as a back-compat fallback and a
default of rfc8693, and carried through both runtime-to-table builders so registry
round-trips preserve it. It is a non-secret dialect selector, so it is not scrubbed from
non-admin or virtual-key responses.
In the dashboard a Profile dropdown (RFC 8693 vs Microsoft Entra OBO) is added to the
token-exchange section. Entra OBO carries the target resource in the scope, so selecting it
makes the scope required and hints the api://<app-id>/.default form, while audience and
subject_token_type (which that dialect ignores) are hidden.
* fix(mcp): extend the blob-to-column lift and non-admin scrubbing to token_exchange_profile
token_exchange_profile gets the same storage contract as the other three
token-exchange settings: the column is authoritative, a blob copy is the legacy
shape — lifted into the column on every write and stripped from the stored
blob — and switching auth_type away from token exchange clears it
(_AUTH_FLOW_SCOPED_FIELDS). Both restricted-view sanitizers scrub it for
uniformity, and the edit form's auth-switch payload nulling includes it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(mcp): assert every token-exchange setting is configurable via config.yaml
Pins the config surface: token_exchange_endpoint, audience, subject_token_type
and token_exchange_profile load from top-level config keys onto the built
server and through to the resolver spec; omitted keys resolve to their
documented defaults (RFC 8693 subject token type, rfc8693 profile), and
token_exchange servers need no oauth2_flow.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* test(ui): pin that the token-exchange fields render only for the oauth2_token_exchange auth type
No form section asserted the visibility contract: the token-exchange fields
(Token Exchange Endpoint, Audience, Subject Token Type) must appear when
'OAuth Token Exchange (OBO)' is selected and for no other auth type. Assert
hidden under plain OAuth, shown under token exchange, hidden again after
switching to API Key.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test(ui): assert the stdio transport switch unmounts the token-exchange fields
The create form gates the whole Authentication section on non-stdio transport,
so selecting OAuth Token Exchange (OBO) and then switching to stdio removes the
token-exchange fields (and their required-credential rules, which antd does not
validate while unmounted). Pin that sequence so the section-level gate cannot
regress silently.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The token_exchange (OBO) branch of _call_regular_mcp_tool built its coroutine
by calling _obo_call_tool_with_retry directly, outside the
_limit_outbound_concurrency context manager that the regular branch uses. OBO
tool calls (and the internal re-mint retry, which issues a second upstream
call_tool) therefore bypassed the per-server max_concurrent_requests semaphore,
so an authenticated caller could run unlimited concurrent tool calls against an
OBO MCP server despite an admin-configured limit.
Wrap the OBO coroutine in _limit_outbound_concurrency the same way the regular
path does, holding one permit across the initial call, the on-401 re-mint, and
the retry, so OBO calls honor the configured cap.
