* fix(proxy): match list/dict guardrail_mode in compliance mode checks
* test(compliance): cover ComplianceChecker guardrail_mode shapes (str/list/dict/None)
* fix(proxy): trust only Mode.default in compliance mode matching (ignore tag overrides)
* fix(proxy): match dict guardrail_mode only when every branch runs in mode (no false-compliant)
* fix(proxy): treat multi-mode guardrail_mode as unresolved (no false-compliant)
The list branch previously counted a guardrail configured with mode:
[pre_call, post_call] under every listed mode. But when the writer cannot
infer the concrete hook that fired (apply_guardrail invocations), the raw
list is logged, and an image-only request that only reaches the post-call
path still records both modes. That let a pre_call compliance check pass on
a request that only ran post_call.
Match the tightened dict semantics: a list now counts for mode only when
every listed mode equals mode. Same trade-off (under-report instead of
false-COMPLIANT). Speculative set support is dropped (spend logs are
JSON-serialized, sets do not cross the wire).
Tests updated to reflect the tightened list semantics, deduplicated (single
TestModeMatching class), and shortened. The invariant is now expressed as
a computed check: True implies every branch runs in the matched mode.
---------
Co-authored-by: Marton Schneider <marton@schneider.co.nl>
Render a general_settings.coordination_redis block into the litellm-helm
proxy config when the bundled Redis is enabled, gated on a new
redis.coordination.enabled value and skipped when the user already
supplies their own block. Sentinel deployments render sentinel_nodes and
service_name rather than a host/port pair.
Also fixes litellm.redis.serviceName, which gated its sentinel branch on
standalone architecture. The bundled Redis subchart only serves sentinel
in replication mode, and renders no master Service there, so REDIS_HOST
pointed at a Service that never existed for every sentinel user.
Documents the coordination redis in the componentized chart and in the
terraform modules, whose existing REDIS_* exports now feed it directly.
Adds helm-unittest coverage for both charts' redis wiring, which had none
* fix(proxy): build redis usage cache from REDIS_* env when cache backend is not Redis
Selecting a semantic (or any non-Redis-KV) response cache left
redis_usage_cache unset, silently downgrading cross-pod rate limits,
parallel-request limits, spend coordination, and the pod lock manager
to per-pod in-memory state. Fall back to a standalone RedisCache built
from REDIS_* environment variables, mirroring the existing
use_redis_transaction_buffer escape hatch, which now shares the same
helper.
Resolves LIT-3861
* feat(proxy): configure the coordination redis independently of the response cache
Adds general_settings.coordination_redis, an explicit block for the Redis
the proxy uses for cross-pod rate limits, parallel-request limits, spend
tracking, the pod lock manager, and shared health checks. Resolution order
is the explicit block, then a plain-Redis response-cache backend, then the
REDIS_* environment. Cluster and sentinel targets are supported, and a
cluster target now builds a RedisClusterCache so cluster-aware consumers
take the cluster path.
Admins can configure it from the Caching page of the dashboard via
/coordination_redis/settings, which reports which source is in effect,
redacts credentials on read, and offers a connection test. Settings saved
there are read back at startup so they take effect on restart.
Also fixes redis client construction so an explicitly configured host
outranks REDIS_URL in the environment. Previously the url branch stripped
the caller's host and port, so an explicit block, or a connection test
typed into the dashboard, silently targeted whatever REDIS_URL named
* fix(ui): move coordination_redis_settings into renamed _components directory
---------
Co-authored-by: Yucheng Zhu <yucheng@berri.ai>
* feat(otel): emit the gen_ai.client.operation.exception event on failed LLM calls
The GenAI semantic conventions record failures of a GenAI client operation as
a log-based event named gen_ai.client.operation.exception, carrying the
exception.type / exception.message / exception.stacktrace trio at severity
WARN and correlated to the failed span. OTel v2 never emitted it: a failed LLM
call produced only the deprecated error.* span attributes, a generic exception
span event without a stacktrace, and the stacktrace under the vendor key
litellm.provider.error.stack_trace.
Build the logs pipeline (LoggerProvider + console/OTLP log exporters mirroring
the metrics plumbing) and record the event behind the enable_events flag, which
until now was defined but consumed nowhere. An operator-configured LoggerProvider
global is reused so the events ride their existing logs pipeline; an explicit
NoOpLoggerProvider global is honored as an opt-out and builds no recorder at all.
The existing span-side error surface (error.type, error.message, the exception
span event, and the litellm.provider.error.* detail keys) is untouched for
backwards compatibility.
* fix(otel): always ride the semconv-required exception pair on the GenAI event
Filtering the event attributes on truthiness conflated "absent" with "empty",
so an empty exception.type or exception.message would have been dropped, leaving
an event with neither semconv-required field. Build the attributes so the pair is
unconditional and only the recommended stacktrace is omitted when the payload
carries none.
* docs(otel): document the events plumbing module in the package README
* test(otel): cover the log exporter selection and logs endpoint normalization
The new logs plumbing had no coverage for exporter-kind selection, the
console fallback for an unrecognized kind, the /v1/logs signal-path rewriting
that lets one OTEL_ENDPOINT serve every signal, or the simple-vs-batch
processor split.
Selecting a semantic (or any non-Redis-KV) response cache left
redis_usage_cache unset, silently downgrading cross-pod rate limits,
parallel-request limits, spend coordination, and the pod lock manager
to per-pod in-memory state. Fall back to a standalone RedisCache built
from REDIS_* environment variables, mirroring the existing
use_redis_transaction_buffer escape hatch, which now shares the same
helper.
Resolves LIT-3861
Addresses review nits on the sidebar shell. The Meter primitive now owns its bar color through a tone variant, so the usage card passes a tone instead of a bg-* class, and the near-limit tone goes back to 80% to match the previous usage indicator. The enterprise usage card is now explicitly gated on an active license, so it never renders for an unlicensed proxy rather than relying on seat/team data being null. The collapsed-rail control and the sidebar menu button render through the shadcn and Base UI button primitives instead of raw button elements; the menu button gains render/nativeButton passthrough while its group toggles stay native buttons, so the nav link leaves keep their link semantics. Adds a regression test that the card stays hidden without a license even when seat limits exist
The guard-main-branch error messages and the contributor docs still
pointed people at litellm_oss_staging. Redirect them to the current
daily OSS branch (litellm_oss_daily_YYYY_MM_DD), a fresh one of which
is cut each weekday, so contributors should target the most recent
Hoisting every role system entry into the top-level system field mutates
the cache prefix whenever a client such as Claude Code appends a new
mid-conversation system message, invalidating the prompt cache for the
entire message history on Bedrock Invoke. Bedrock only rejects a system
entry at messages.0, so hoist just the leading run and forward the rest
in place
A correctly signed JWT whose user_id or server_id claim was an empty string
passed claims validation but raised ValidationError from the EnvelopeIdentity
constructor inside open_envelope, breaking its never-raises guarantee. The
claims model now mirrors the identity's min_length constraints, so any claim
set that validates also constructs, and the empty-identity case maps to
MalformedPayload like every other bad claim shape.
Pure, unwired module: mints and opens the single client-held bearer that
carries both a litellm identity and the encrypted upstream OAuth grant with
zero server-side storage. HS256 JWT signing (same approach as the BYOK
session bearer) plus the existing encrypt_value/decrypt_value symmetric
helpers, with all key material and the clock injected as parameters. Opening
returns typed frozen error values (not_an_envelope, bad_signature, expired,
malformed_payload, decrypt_failed); minting rejects envelopes over
MAX_ENVELOPE_BYTES with a typed error instead of truncating. Error values
and reprs never carry token material.
The lint job lived in test-litellm-ui-build.yml (workflow name "UI Build
Check") next to the build job, so its check surfaced as the misleading
"UI Build Check / frontend-lint" even though it does prettier, eslint, lint
budgets, and knip, not building. Split it into test-litellm-ui-lint.yml
(name "UI Lint") so the check reads "UI Lint / frontend-lint". The build
workflow keeps only build-ui; the lint job (including the knip step) moves
over unchanged.
Note for whoever manages branch protection: this renames the lint required-
check context from "UI Build Check / frontend-lint" to
"UI Lint / frontend-lint"; update the required-check entry so PRs don't strand.
* refactor(ui): colocate the skills view, keeping the AIHub-shared skill components
Group-1 split for the skills (claude_code_plugins) segment. It is a file-plus-
folder combo where part of the folder is shared with AIHub and the public model
hub: the skill types, skill_detail, MakeSkillPublicForm, and helpers stay in
@/components/claude_code_plugins (imported by AIHub's ModelHubTable and
SkillHubDashboard, plus public_model_hub, networking, and skill_hub_table_columns),
while the plugins-management view moves to skills/_components.
claude_code_plugins.tsx becomes skills/_components/index.tsx; add_plugin_form and
plugin_table move alongside it. The moved files' imports of the retained shared
files become @/components/claude_code_plugins paths, other escaping imports are
absolutized, and lint suppressions are re-keyed for moved files only (the staying
components keep their src/components keys). The shared components did not move, so
their consumers are untouched. No behavior change.
* refactor(ui): name the agents/guardrails view files instead of index.tsx
The moved view files were renamed to _components/index.tsx, which reads like a
barrel of re-exports rather than the single view component each actually is.
Rename them to match their default export (AgentsPanel.tsx, GuardrailsPanel.tsx)
and their colocated tests, and point page.tsx at the named file.
login and internalUserIdentity used an unscoped "Virtual Keys" text check that now also matches the top-bar breadcrumb, and adminSettings navigated via antd menuitem roles the shadcn nav no longer emits. Scope these to the sidebar's complementary landmark and drive the Settings group by role (expand the button, then click the child link), matching the fixes already applied to the migration and navigation specs
The dashboard lint-budgets step ran check-lint-budgets.mjs in --check mode,
which fails and tells you to run `npm run lint:metrics` and re-stage by hand.
Add a --write mode that rewrites eslint-metrics.json from the same eslint
report, and have pre-commit use it, then flag drift via git diff so you
re-stage; this mirrors how the block below regenerates schema.d.ts. CI keeps
using --check, so it still fails on a stale committed metrics file.
The redesigned sidebar + header shell is now scoped to the ai-gateway dashboard. Agent-control-plane (plugin) mode falls back to the original full-width Navbar, which carries the account menu, so preferences and sign-out stay reachable there; Chat and the public model hub already use that Navbar
Update the e2e specs the antd Menu -> shadcn nav swap broke. Sidebar items are now links/buttons rather than antd menuitems, and the top bar renders a breadcrumb whose current-page item is also a "Virtual Keys" link, so the migration and navigation smoke checks now scope their sidebar lookups to the complementary landmark and expand the new collapsible groups by data-slot. Also harden the collapsed-rail unit test to assert the icon and link rather than the CSS-hidden label
knip was producing garbage locally and was never wired into CI, so nobody
trusted it. Two structural problems: it silently degrades when deps are
missing (a partial worktree install flagged all 436 test files as unused),
and its config had blind spots that surfaced as false positives.
Fixes so a knip run means something:
- Register every playwright config (serverRootPath + migration variants), not
just the main one. serverRootPath.config.ts is invoked via --config in
test_server_root_path.yml, which knip can't see; it was falsely flagged as
an unused file
- Treat src/components/ui/** as entry points. These are shadcn design-system
primitives, intentionally part of the palette before every one is consumed;
knip was flagging not-yet-used ones (e.g. select.tsx) as dead files and
their sub-exports as unused. Marking the directory as the design-system
surface is the correct fix, not deleting components someone is about to use
- Declare @ant-design/icons as a direct dependency. It was imported in ~198
files but only resolved via antd hoisting, so every one showed up as an
"unlisted dependency"
- Add an explicit vitest plugin block so test-file classification no longer
rides on auto-detection
- Stage severities via rules: gate the now-clean categories (files,
dependencies, unlisted, unresolved) as errors and keep exports/types/
duplicates as warnings, so CI enforces what's at zero today while the
remaining findings ratchet down in follow-ups
- Run npm run knip in the frontend-lint CI job, which installs with npm ci so
it never sees a partial tree
knip now exits 0 with the gated categories clean
* fix(e2e): wire batch provider secrets for docker and k8s
Point batch deployments at the credential field names and os.environ refs
the gateway actually resolves from process env (compose .env or EKS secret
mounts). Missing secrets skip instead of failing red so a red run means a
product bug. Mirror S3 bucket env aliases in docker-compose for provider_fallback
* fix(e2e): drop batch provider_env unit tests
The batches suite is live e2e only; no monkeypatch or unit-level tests
* fix: batch credentials, provider list, and team db lookup
Keep object-storage fields through CredentialLiteLLMParams and resolve
os.environ/ refs when reading deployment credentials so Vertex/Bedrock
batch file uploads see bucket and AWS keys from K8s/docker env
Skip managed batch list when the request is provider-scoped so
/{provider}/v1/batches list works instead of 500
Force DB on check_db_only team lookups and stop masking non-404 errors
as "team doesn't exist"
Drop e2e runner-side skip helpers; hard-fail on missing gateway secrets
* fix: tag reseed, team window spend, and remaining e2e flakes
Reseed spend:tag counters from LiteLLM_TagTable so cold redis still
enforces after the spend writer flushes
When applying post-call cost to team multi-window counters, load the
team from the DB if it is missing from the management cache so window
spend is not dropped on cache misses
Harden cold-counter reseed e2e (namespace-aware keys, burst success,
poll). Give tag budget more headroom. Retry /key/update on redis DNS
blips. Ensure NLTK punkt_tab is present for pipecat realtime audio
* revert: drop product code changes; e2e-only scope
Reverts all litellm/ and unit-test product edits. This branch is limited
to tests/e2e per contributor instruction
* fix(e2e): harden batch list and team member setup races
provider_fallback list falls back when managed batches reject provider
filtering. Team create waits for /team/info and member_add retries on
transient team-not-found so split control-plane lag does not red the suite
* fix(e2e): remove .env.example
Leave local .env and docker-compose env wiring as the secret source
* fix(e2e): wire files_settings and faster budget rescheduler for compose
OpenAI/Azure batch file uploads need files_settings; budget reset e2e needs a
short rescheduler window. Drop unsupported bedrock-encoded create_batch cells,
tolerate bedrock file.bytes=0, and surface team-info wait failures instead of
hanging silently
* chore(e2e): strip verbose comments from batch capabilities
* fix(e2e): assert managed list fallback before provider_fallback skip
When provider-scoped list is rejected, still fetch the unfiltered list and
check the envelope. Only skip membership when the id is a raw
provider_fallback batch that managed list cannot index
* refactor(ui): colocate agents and guardrails views, keeping shared selectors and types
The last two group-1 colocation splits. Both are file-plus-folder combos: the
page view is a top-level file (agents.tsx / guardrails.tsx) sitting beside a
supporting folder of the same name, and part of that folder is shared.
agents: agents/types (imported by the agents hook) stays in @/components/agents;
agents.tsx and the rest of the agents/ folder move into agents/_components (the
view file becomes _components/index.tsx).
guardrails: GuardrailSelector (imported by the Playground, the key edit view,
and the agents add-guardrail form) and types stay in @/components/guardrails;
guardrails.tsx and the other 47 folder files (including the tool_permission,
custom_code, content_filter, and llm_judge subfolders) move into
guardrails/_components.
Moved files' imports of the retained shared files become @/components paths,
other escaping relative imports are absolutized against @/, and moved test
files have their from/vi.mock/vi.importActual paths rewritten to match. Lint
suppressions are re-keyed for moved files only (the staying selector/types keep
their src/components keys). The shared selectors did not move, so their external
consumers are untouched. No behavior change.
* fix(ui): move the orphaned agents/guardrails view tests and drop dead GuardrailItem
Greptile follow-ups on the agents/guardrails colocation:
- agents.tsx and guardrails.tsx moved to their _components/index.tsx, but their
sibling test files (src/components/agents.test.tsx, guardrails.test.tsx) were
left behind still importing ./agents and ./guardrails, which broke a full
vitest run. Move them to the matching _components/index.test.tsx and rewrite
their view import to ./index, folder mocks to local ./ siblings, and
networking to @/components/networking.
- Remove the unused GuardrailItem interface and its GuardrailDefinitionLocation
import from the guardrails view (dead code carried over from before the move;
state is typed as Guardrail[]).
