Selecting a semantic (or any non-Redis-KV) response cache left
redis_usage_cache unset, silently downgrading cross-pod rate limits,
parallel-request limits, spend coordination, and the pod lock manager
to per-pod in-memory state. Fall back to a standalone RedisCache built
from REDIS_* environment variables, mirroring the existing
use_redis_transaction_buffer escape hatch, which now shares the same
helper.
Resolves LIT-3861
Addresses review nits on the sidebar shell. The Meter primitive now owns its bar color through a tone variant, so the usage card passes a tone instead of a bg-* class, and the near-limit tone goes back to 80% to match the previous usage indicator. The enterprise usage card is now explicitly gated on an active license, so it never renders for an unlicensed proxy rather than relying on seat/team data being null. The collapsed-rail control and the sidebar menu button render through the shadcn and Base UI button primitives instead of raw button elements; the menu button gains render/nativeButton passthrough while its group toggles stay native buttons, so the nav link leaves keep their link semantics. Adds a regression test that the card stays hidden without a license even when seat limits exist
The backup file is used by tests; the root model_prices_and_context_window.json
is what gets published to the pricing URL and loaded by the proxy at runtime.
Without this, the proxy would continue resolving supports_reasoning via the
provider-level fallback and returning true for Gemini image generation models.
Also covers vertex_ai/gemini-3-pro-image and vertex_ai/gemini-3.1-flash-image
(non-preview variants) and gemini/gemini-3.1-flash-image which exist only in
the root JSON.
vertex_ai/gemini-2.5-flash-image, vertex_ai/gemini-3-pro-image-preview,
vertex_ai/gemini-3.1-flash-image-preview, gemini/gemini-3-pro-image-preview,
and gemini/gemini-3.1-flash-image-preview were missing supports_reasoning
entries; _supports_factory then fell through to the vertex_ai provider-level
config which returns true, causing requests with reasoning_effort to be sent
to an API that rejects them.
The guard-main-branch error messages and the contributor docs still
pointed people at litellm_oss_staging. Redirect them to the current
daily OSS branch (litellm_oss_daily_YYYY_MM_DD), a fresh one of which
is cut each weekday, so contributors should target the most recent
Hoisting every role system entry into the top-level system field mutates
the cache prefix whenever a client such as Claude Code appends a new
mid-conversation system message, invalidating the prompt cache for the
entire message history on Bedrock Invoke. Bedrock only rejects a system
entry at messages.0, so hoist just the leading run and forward the rest
in place
A correctly signed JWT whose user_id or server_id claim was an empty string
passed claims validation but raised ValidationError from the EnvelopeIdentity
constructor inside open_envelope, breaking its never-raises guarantee. The
claims model now mirrors the identity's min_length constraints, so any claim
set that validates also constructs, and the empty-identity case maps to
MalformedPayload like every other bad claim shape.
Pure, unwired module: mints and opens the single client-held bearer that
carries both a litellm identity and the encrypted upstream OAuth grant with
zero server-side storage. HS256 JWT signing (same approach as the BYOK
session bearer) plus the existing encrypt_value/decrypt_value symmetric
helpers, with all key material and the clock injected as parameters. Opening
returns typed frozen error values (not_an_envelope, bad_signature, expired,
malformed_payload, decrypt_failed); minting rejects envelopes over
MAX_ENVELOPE_BYTES with a typed error instead of truncating. Error values
and reprs never carry token material.
The lint job lived in test-litellm-ui-build.yml (workflow name "UI Build
Check") next to the build job, so its check surfaced as the misleading
"UI Build Check / frontend-lint" even though it does prettier, eslint, lint
budgets, and knip, not building. Split it into test-litellm-ui-lint.yml
(name "UI Lint") so the check reads "UI Lint / frontend-lint". The build
workflow keeps only build-ui; the lint job (including the knip step) moves
over unchanged.
Note for whoever manages branch protection: this renames the lint required-
check context from "UI Build Check / frontend-lint" to
"UI Lint / frontend-lint"; update the required-check entry so PRs don't strand.
* refactor(ui): colocate the skills view, keeping the AIHub-shared skill components
Group-1 split for the skills (claude_code_plugins) segment. It is a file-plus-
folder combo where part of the folder is shared with AIHub and the public model
hub: the skill types, skill_detail, MakeSkillPublicForm, and helpers stay in
@/components/claude_code_plugins (imported by AIHub's ModelHubTable and
SkillHubDashboard, plus public_model_hub, networking, and skill_hub_table_columns),
while the plugins-management view moves to skills/_components.
claude_code_plugins.tsx becomes skills/_components/index.tsx; add_plugin_form and
plugin_table move alongside it. The moved files' imports of the retained shared
files become @/components/claude_code_plugins paths, other escaping imports are
absolutized, and lint suppressions are re-keyed for moved files only (the staying
components keep their src/components keys). The shared components did not move, so
their consumers are untouched. No behavior change.
* refactor(ui): name the agents/guardrails view files instead of index.tsx
The moved view files were renamed to _components/index.tsx, which reads like a
barrel of re-exports rather than the single view component each actually is.
Rename them to match their default export (AgentsPanel.tsx, GuardrailsPanel.tsx)
and their colocated tests, and point page.tsx at the named file.
login and internalUserIdentity used an unscoped "Virtual Keys" text check that now also matches the top-bar breadcrumb, and adminSettings navigated via antd menuitem roles the shadcn nav no longer emits. Scope these to the sidebar's complementary landmark and drive the Settings group by role (expand the button, then click the child link), matching the fixes already applied to the migration and navigation specs
For MCP servers with auth_type=oauth2 + delegate_auth_to_upstream=true, a
client-supplied upstream token that the upstream rejects was masked: the
upstream 401 raised during tools/list is absorbed by the list handler, so on a
single-server route a rejected token became HTTP 200 with an empty tool list.
Clients showed "0 tools" instead of re-authenticating, and monitoring never saw
an unauthorized signal.
Extend the connect-time preflight _check_passthrough_upstream_auth to probe
delegate-auth servers with the caller's bare Authorization bearer, reusing the
existing _probe_upstream_auth and the RFC 6750 challenge builder, so a rejected
token fails the connect with 401 + WWW-Authenticate error="invalid_token" and a
compliant client re-runs the upstream OAuth flow.
The bare Authorization header is a valid upstream token only when admission took
the delegate bypass, so the delegate target is resolved through
get_mcp_server_by_name (the same resolver admission uses) rather than the wider
allowed-server prefix/access-group matching. A name that reaches a delegate
server only via server_id or an access group is admitted as a real LiteLLM key,
so probing it would leak that key upstream; requiring the admission-resolver
match closes that gap. The probe is gated to single-server routes (matching the
OBO preflight), keyed to the caller's authorized set by server_id, and the
challenge echoes the requested name so aliased routes get the same
resource_metadata URL as the tokenless preemptive challenge. Tokenless requests
keep flowing to the preemptive discovery challenge unchanged.
Resolves LIT-4194
The dashboard lint-budgets step ran check-lint-budgets.mjs in --check mode,
which fails and tells you to run `npm run lint:metrics` and re-stage by hand.
Add a --write mode that rewrites eslint-metrics.json from the same eslint
report, and have pre-commit use it, then flag drift via git diff so you
re-stage; this mirrors how the block below regenerates schema.d.ts. CI keeps
using --check, so it still fails on a stale committed metrics file.
The redesigned sidebar + header shell is now scoped to the ai-gateway dashboard. Agent-control-plane (plugin) mode falls back to the original full-width Navbar, which carries the account menu, so preferences and sign-out stay reachable there; Chat and the public model hub already use that Navbar
Update the e2e specs the antd Menu -> shadcn nav swap broke. Sidebar items are now links/buttons rather than antd menuitems, and the top bar renders a breadcrumb whose current-page item is also a "Virtual Keys" link, so the migration and navigation smoke checks now scope their sidebar lookups to the complementary landmark and expand the new collapsible groups by data-slot. Also harden the collapsed-rail unit test to assert the icon and link rather than the CSS-hidden label
knip was producing garbage locally and was never wired into CI, so nobody
trusted it. Two structural problems: it silently degrades when deps are
missing (a partial worktree install flagged all 436 test files as unused),
and its config had blind spots that surfaced as false positives.
Fixes so a knip run means something:
- Register every playwright config (serverRootPath + migration variants), not
just the main one. serverRootPath.config.ts is invoked via --config in
test_server_root_path.yml, which knip can't see; it was falsely flagged as
an unused file
- Treat src/components/ui/** as entry points. These are shadcn design-system
primitives, intentionally part of the palette before every one is consumed;
knip was flagging not-yet-used ones (e.g. select.tsx) as dead files and
their sub-exports as unused. Marking the directory as the design-system
surface is the correct fix, not deleting components someone is about to use
- Declare @ant-design/icons as a direct dependency. It was imported in ~198
files but only resolved via antd hoisting, so every one showed up as an
"unlisted dependency"
- Add an explicit vitest plugin block so test-file classification no longer
rides on auto-detection
- Stage severities via rules: gate the now-clean categories (files,
dependencies, unlisted, unresolved) as errors and keep exports/types/
duplicates as warnings, so CI enforces what's at zero today while the
remaining findings ratchet down in follow-ups
- Run npm run knip in the frontend-lint CI job, which installs with npm ci so
it never sees a partial tree
knip now exits 0 with the gated categories clean
* fix(e2e): wire batch provider secrets for docker and k8s
Point batch deployments at the credential field names and os.environ refs
the gateway actually resolves from process env (compose .env or EKS secret
mounts). Missing secrets skip instead of failing red so a red run means a
product bug. Mirror S3 bucket env aliases in docker-compose for provider_fallback
* fix(e2e): drop batch provider_env unit tests
The batches suite is live e2e only; no monkeypatch or unit-level tests
* fix: batch credentials, provider list, and team db lookup
Keep object-storage fields through CredentialLiteLLMParams and resolve
os.environ/ refs when reading deployment credentials so Vertex/Bedrock
batch file uploads see bucket and AWS keys from K8s/docker env
Skip managed batch list when the request is provider-scoped so
/{provider}/v1/batches list works instead of 500
Force DB on check_db_only team lookups and stop masking non-404 errors
as "team doesn't exist"
Drop e2e runner-side skip helpers; hard-fail on missing gateway secrets
* fix: tag reseed, team window spend, and remaining e2e flakes
Reseed spend:tag counters from LiteLLM_TagTable so cold redis still
enforces after the spend writer flushes
When applying post-call cost to team multi-window counters, load the
team from the DB if it is missing from the management cache so window
spend is not dropped on cache misses
Harden cold-counter reseed e2e (namespace-aware keys, burst success,
poll). Give tag budget more headroom. Retry /key/update on redis DNS
blips. Ensure NLTK punkt_tab is present for pipecat realtime audio
* revert: drop product code changes; e2e-only scope
Reverts all litellm/ and unit-test product edits. This branch is limited
to tests/e2e per contributor instruction
* fix(e2e): harden batch list and team member setup races
provider_fallback list falls back when managed batches reject provider
filtering. Team create waits for /team/info and member_add retries on
transient team-not-found so split control-plane lag does not red the suite
* fix(e2e): remove .env.example
Leave local .env and docker-compose env wiring as the secret source
* fix(e2e): wire files_settings and faster budget rescheduler for compose
OpenAI/Azure batch file uploads need files_settings; budget reset e2e needs a
short rescheduler window. Drop unsupported bedrock-encoded create_batch cells,
tolerate bedrock file.bytes=0, and surface team-info wait failures instead of
hanging silently
* chore(e2e): strip verbose comments from batch capabilities
* fix(e2e): assert managed list fallback before provider_fallback skip
When provider-scoped list is rejected, still fetch the unfiltered list and
check the envelope. Only skip membership when the id is a raw
provider_fallback batch that managed list cannot index
* refactor(ui): colocate agents and guardrails views, keeping shared selectors and types
The last two group-1 colocation splits. Both are file-plus-folder combos: the
page view is a top-level file (agents.tsx / guardrails.tsx) sitting beside a
supporting folder of the same name, and part of that folder is shared.
agents: agents/types (imported by the agents hook) stays in @/components/agents;
agents.tsx and the rest of the agents/ folder move into agents/_components (the
view file becomes _components/index.tsx).
guardrails: GuardrailSelector (imported by the Playground, the key edit view,
and the agents add-guardrail form) and types stay in @/components/guardrails;
guardrails.tsx and the other 47 folder files (including the tool_permission,
custom_code, content_filter, and llm_judge subfolders) move into
guardrails/_components.
Moved files' imports of the retained shared files become @/components paths,
other escaping relative imports are absolutized against @/, and moved test
files have their from/vi.mock/vi.importActual paths rewritten to match. Lint
suppressions are re-keyed for moved files only (the staying selector/types keep
their src/components keys). The shared selectors did not move, so their external
consumers are untouched. No behavior change.
* fix(ui): move the orphaned agents/guardrails view tests and drop dead GuardrailItem
Greptile follow-ups on the agents/guardrails colocation:
- agents.tsx and guardrails.tsx moved to their _components/index.tsx, but their
sibling test files (src/components/agents.test.tsx, guardrails.test.tsx) were
left behind still importing ./agents and ./guardrails, which broke a full
vitest run. Move them to the matching _components/index.test.tsx and rewrite
their view import to ./index, folder mocks to local ./ siblings, and
networking to @/components/networking.
- Remove the unused GuardrailItem interface and its GuardrailDefinitionLocation
import from the guardrails view (dead code carried over from before the move;
state is typed as Guardrail[]).
* fix(spend-logs): honor store_prompts_in_spend_logs for guardrail_information (LIT-4314)
_get_spend_logs_metadata passed guardrail_information entries through
verbatim, so guardrail hooks that echo the LLM request into
guardrail_response leaked the raw prompt into LiteLLM_SpendLogs.metadata
regardless of store_prompts_in_spend_logs. This mirrored the pre-existing
gap for the other prompt-carrying fields (vector_store_request_metadata,
error_information, etc.), which already sanitize via
_should_store_prompts_and_responses_in_spend_logs.
Add _sanitize_guardrail_information_for_spend_logs alongside the other
per-field sanitizers and wire it into _get_spend_logs_metadata. When the
flag is False the sanitizer replaces guardrail_request and
guardrail_response with REDACTED_BY_LITELM_STRING while preserving every
other typed field on the entry (name, provider, mode, status, timings,
action, violation_categories, risk_score, masked_entity_count, ...) so
guardrail dashboards keep working. When the flag is True (or the field
is None) the entries pass through unchanged.
Widen StandardLoggingGuardrailInformation.guardrail_request from
Optional[dict] to Optional[Union[dict, str]] so the redacted sentinel
satisfies the TypedDict without needing a cast; guardrail_response
already accepted str.
Regression tests cover the three cases (flag=False redacts,
flag=True passes through, None passes through) plus an end-to-end
get_logging_payload path that fails if the wire-in at line 139 is
reverted.
* chore(spend-logs): review nits (one-shot dict build, scrub identifier in tests)
- _redact_prompt_fields_in_guardrail_entry now returns the redacted
dict in one expression instead of seed-then-mutate (TYPE-3)
- swap the illustrative guardrail_name in the new test fixtures for
a generic 'demo-echo-guard' identifier
* chore(spend-logs): only redact guardrail prompt fields when caller supplied them
Greptile P2: the sanitizer was unconditionally writing REDACTED_BY_LITELM
into both guardrail_request and guardrail_response on the copy, so entries
that never carried one of those fields (e.g. a guardrail that only emits
a guardrail_response) came out with a phantom guardrail_request key added.
Guard both assignments with an in-check so the output shape is stable.
Add a mutation-checked regression test that fails if either guard is
removed.
* fix(spend-logs): also redact match_details and classification in guardrail_information
The initial LIT-4314 fix redacted guardrail_request and guardrail_response,
but two other typed fields on StandardLoggingGuardrailInformation also
carry raw prompt content when a first-party guardrail populates them:
- litellm_content_filter/content_filter.py:1676 sets classification =
dict(CompetitorIntentDetection), whose evidence[*].match is a substring
taken directly from the user's normalized prompt (see
litellm_content_filter/competitor_intent/base.py:184-194).
- block_code_execution/block_code_execution.py:571 sets match_details =
guardrail_response = [dict(d) for d in detections], where detections
carry the fenced-code-block content extracted from the user's message.
Reproduced live against localhost:4000 with store_prompts_in_spend_logs
false and a custom guardrail passing tracing_detail with both fields:
before this commit the raw prompt shows up in metadata.guardrail_information[0]
under match_details and classification; after, both are the sentinel.
Widen the two TypedDict fields to Optional[Union[..., str]] so the
sentinel string satisfies the schema without a cast, and consolidate
the redaction set into a tuple so future prompt-carrying additions are
one-line changes.
* fix(spend-logs): normalize non-list guardrail_information shapes in sanitizer
xecguard's logging hook (xecguard.py:246) assigns a bare dict to
standard_logging_object['guardrail_information'] instead of a list,
violating the typed contract Optional[List[StandardLoggingGuardrailInformation]].
Without defensive normalization, _sanitize_guardrail_information_for_spend_logs
iterates the dict's string keys and _redact_prompt_fields_in_guardrail_entry
raises TypeError on {**'guardrail_name'}, which get_logging_payload's
downstream update_database catches with a broad except and silently drops
the entire spend-log write for that request.
Normalize a bare-dict input to a single-item list at the sanitizer's
entry point, and skip any non-dict entries defensively (matching OTEL's
existing isinstance filter at opentelemetry.py:1751-1753 for the same
field). Downstream readers already model this defensively; make the
spend-log write path match.
The root cause is xecguard's writer, not the sanitizer. That is being
tracked as a separate ticket; this PR keeps xecguard-enabled deploys
from silently losing spend logs when store_prompts_in_spend_logs=false.
* fix(types): declare guardrail Union members str-first to avoid poisoning typing cache
CPython's typing module caches Union[...] order-insensitively (first-
construction wins), and litellm/types/utils.py has no 'from __future__
import annotations', so its unions are constructed eagerly at import
time -- before any proxy model. Declaring guardrail_request,
classification, and match_details with dict-first ordering seeds the
typing cache with a dict-first tuple, and later proxy models that
declare custom_llm_provider / model_aliases / vertex_credentials as
Optional[Union[str, dict]] pick up the same dict-first object.
Downstream, Pydantic's get_args() then reports anyOf in dict-first
order, FastAPI emits the OpenAPI accordingly, and 'npm run gen:api'
produces a schema.d.ts diff on unrelated fields, tripping the schema-
sync CI check.
Behaviorally identical in Python and at the wire; the flip only reorders
the union members so the first construction matches how the codebase
had always declared these unions, and 'npm run gen:api' now produces a
zero diff against the committed schema.d.ts.
