* feat(router): add LLM-based classifier option to complexity router
Adds classifier_type: "heuristic" | "llm" to complexity_router_config.
When set to "llm", the router calls a configured model (e.g. a small
model like haiku) via structured output to pick the complexity tier,
falling back to the existing regex/keyword scorer on any error, empty
response, or unparseable output.
* feat(ui): add classifier_type option to complexity router UI, fix edit flow
Adds an "Advanced: Classification Method" section to ComplexityRouterConfig
with a heuristic/LLM toggle, revealing a classifier model picker and timeout
when LLM is selected.
Also fixes the auto router edit modal, which never rendered the complexity
router UI at all (it only handled the semantic router), and the "Edit Auto
Router" button visibility check, which was gated on auto_router_config and
never matched complexity router deployments.
* fix(router): attribute classifier calls to caller, raise default timeout
Forwards the original request's litellm_metadata into the classifier's
acompletion call. Without it, the proxy's cost-tracking gate sees no
user_api_key/team_id/user_id and silently drops spend logging and budget
accounting for every classifier call, letting an authenticated user rack
up unaccounted provider spend via repeated requests.
Also raises the default classifier timeout from 400ms to 3000ms (400ms
undershoots real LLM latency and would silently degrade to the heuristic
scorer on most requests) and corrects the module/class docstrings, which
still claimed zero external API calls after the llm classifier path was
added.
* fix(ci): resolve ruff strict-budget and frontend-lint failures
- Use PEP 585 generics (dict/tuple/list) in the new aclassify/_classify_with_llm
signatures instead of typing.Dict/Tuple/List, and suppress BLE001 on the
intentionally broad except in aclassify's fallback path with a reason.
- Fix prettier formatting in ComplexityRouterConfig.tsx.
- Regenerate eslint-metrics.json (was stale after the classifier UI changes).
* fix(ci): regenerate stale eslint-metrics.json
* fix(router): strip parent budget reservation from classifier metadata
The classifier's internal acompletion call previously forwarded the
parent request's full litellm_metadata, including its budget
reservation (user_api_key_budget_reservation / user_api_key_auth).
That reservation belongs to the routed completion the classifier is
deciding on, not to the classifier call itself, so it's now stripped
while key/team attribution fields are still forwarded for spend
logging.
A trailing slash on --base-url (or LITELLM_PROXY_URL) produced
double-slash URLs like https://host//sso/cli/start, which 404s. Normalize
once in the CLI's top-level group callback so every subcommand benefits.
* fix(proxy): match list/dict guardrail_mode in compliance mode checks
* test(compliance): cover ComplianceChecker guardrail_mode shapes (str/list/dict/None)
* fix(proxy): trust only Mode.default in compliance mode matching (ignore tag overrides)
* fix(proxy): match dict guardrail_mode only when every branch runs in mode (no false-compliant)
* fix(proxy): treat multi-mode guardrail_mode as unresolved (no false-compliant)
The list branch previously counted a guardrail configured with mode:
[pre_call, post_call] under every listed mode. But when the writer cannot
infer the concrete hook that fired (apply_guardrail invocations), the raw
list is logged, and an image-only request that only reaches the post-call
path still records both modes. That let a pre_call compliance check pass on
a request that only ran post_call.
Match the tightened dict semantics: a list now counts for mode only when
every listed mode equals mode. Same trade-off (under-report instead of
false-COMPLIANT). Speculative set support is dropped (spend logs are
JSON-serialized, sets do not cross the wire).
Tests updated to reflect the tightened list semantics, deduplicated (single
TestModeMatching class), and shortened. The invariant is now expressed as
a computed check: True implies every branch runs in the matched mode.
---------
Co-authored-by: Marton Schneider <marton@schneider.co.nl>
Render a general_settings.coordination_redis block into the litellm-helm
proxy config when the bundled Redis is enabled, gated on a new
redis.coordination.enabled value and skipped when the user already
supplies their own block. Sentinel deployments render sentinel_nodes and
service_name rather than a host/port pair.
Also fixes litellm.redis.serviceName, which gated its sentinel branch on
standalone architecture. The bundled Redis subchart only serves sentinel
in replication mode, and renders no master Service there, so REDIS_HOST
pointed at a Service that never existed for every sentinel user.
Documents the coordination redis in the componentized chart and in the
terraform modules, whose existing REDIS_* exports now feed it directly.
Adds helm-unittest coverage for both charts' redis wiring, which had none
* fix(proxy): build redis usage cache from REDIS_* env when cache backend is not Redis
Selecting a semantic (or any non-Redis-KV) response cache left
redis_usage_cache unset, silently downgrading cross-pod rate limits,
parallel-request limits, spend coordination, and the pod lock manager
to per-pod in-memory state. Fall back to a standalone RedisCache built
from REDIS_* environment variables, mirroring the existing
use_redis_transaction_buffer escape hatch, which now shares the same
helper.
Resolves LIT-3861
* feat(proxy): configure the coordination redis independently of the response cache
Adds general_settings.coordination_redis, an explicit block for the Redis
the proxy uses for cross-pod rate limits, parallel-request limits, spend
tracking, the pod lock manager, and shared health checks. Resolution order
is the explicit block, then a plain-Redis response-cache backend, then the
REDIS_* environment. Cluster and sentinel targets are supported, and a
cluster target now builds a RedisClusterCache so cluster-aware consumers
take the cluster path.
Admins can configure it from the Caching page of the dashboard via
/coordination_redis/settings, which reports which source is in effect,
redacts credentials on read, and offers a connection test. Settings saved
there are read back at startup so they take effect on restart.
Also fixes redis client construction so an explicitly configured host
outranks REDIS_URL in the environment. Previously the url branch stripped
the caller's host and port, so an explicit block, or a connection test
typed into the dashboard, silently targeted whatever REDIS_URL named
* fix(ui): move coordination_redis_settings into renamed _components directory
---------
Co-authored-by: Yucheng Zhu <yucheng@berri.ai>
* feat(otel): emit the gen_ai.client.operation.exception event on failed LLM calls
The GenAI semantic conventions record failures of a GenAI client operation as
a log-based event named gen_ai.client.operation.exception, carrying the
exception.type / exception.message / exception.stacktrace trio at severity
WARN and correlated to the failed span. OTel v2 never emitted it: a failed LLM
call produced only the deprecated error.* span attributes, a generic exception
span event without a stacktrace, and the stacktrace under the vendor key
litellm.provider.error.stack_trace.
Build the logs pipeline (LoggerProvider + console/OTLP log exporters mirroring
the metrics plumbing) and record the event behind the enable_events flag, which
until now was defined but consumed nowhere. An operator-configured LoggerProvider
global is reused so the events ride their existing logs pipeline; an explicit
NoOpLoggerProvider global is honored as an opt-out and builds no recorder at all.
The existing span-side error surface (error.type, error.message, the exception
span event, and the litellm.provider.error.* detail keys) is untouched for
backwards compatibility.
* fix(otel): always ride the semconv-required exception pair on the GenAI event
Filtering the event attributes on truthiness conflated "absent" with "empty",
so an empty exception.type or exception.message would have been dropped, leaving
an event with neither semconv-required field. Build the attributes so the pair is
unconditional and only the recommended stacktrace is omitted when the payload
carries none.
* docs(otel): document the events plumbing module in the package README
* test(otel): cover the log exporter selection and logs endpoint normalization
The new logs plumbing had no coverage for exporter-kind selection, the
console fallback for an unrecognized kind, the /v1/logs signal-path rewriting
that lets one OTEL_ENDPOINT serve every signal, or the simple-vs-batch
processor split.
Selecting a semantic (or any non-Redis-KV) response cache left
redis_usage_cache unset, silently downgrading cross-pod rate limits,
parallel-request limits, spend coordination, and the pod lock manager
to per-pod in-memory state. Fall back to a standalone RedisCache built
from REDIS_* environment variables, mirroring the existing
use_redis_transaction_buffer escape hatch, which now shares the same
helper.
Resolves LIT-3861
Addresses review nits on the sidebar shell. The Meter primitive now owns its bar color through a tone variant, so the usage card passes a tone instead of a bg-* class, and the near-limit tone goes back to 80% to match the previous usage indicator. The enterprise usage card is now explicitly gated on an active license, so it never renders for an unlicensed proxy rather than relying on seat/team data being null. The collapsed-rail control and the sidebar menu button render through the shadcn and Base UI button primitives instead of raw button elements; the menu button gains render/nativeButton passthrough while its group toggles stay native buttons, so the nav link leaves keep their link semantics. Adds a regression test that the card stays hidden without a license even when seat limits exist
The guard-main-branch error messages and the contributor docs still
pointed people at litellm_oss_staging. Redirect them to the current
daily OSS branch (litellm_oss_daily_YYYY_MM_DD), a fresh one of which
is cut each weekday, so contributors should target the most recent
Hoisting every role system entry into the top-level system field mutates
the cache prefix whenever a client such as Claude Code appends a new
mid-conversation system message, invalidating the prompt cache for the
entire message history on Bedrock Invoke. Bedrock only rejects a system
entry at messages.0, so hoist just the leading run and forward the rest
in place
A correctly signed JWT whose user_id or server_id claim was an empty string
passed claims validation but raised ValidationError from the EnvelopeIdentity
constructor inside open_envelope, breaking its never-raises guarantee. The
claims model now mirrors the identity's min_length constraints, so any claim
set that validates also constructs, and the empty-identity case maps to
MalformedPayload like every other bad claim shape.
Pure, unwired module: mints and opens the single client-held bearer that
carries both a litellm identity and the encrypted upstream OAuth grant with
zero server-side storage. HS256 JWT signing (same approach as the BYOK
session bearer) plus the existing encrypt_value/decrypt_value symmetric
helpers, with all key material and the clock injected as parameters. Opening
returns typed frozen error values (not_an_envelope, bad_signature, expired,
malformed_payload, decrypt_failed); minting rejects envelopes over
MAX_ENVELOPE_BYTES with a typed error instead of truncating. Error values
and reprs never carry token material.
The lint job lived in test-litellm-ui-build.yml (workflow name "UI Build
Check") next to the build job, so its check surfaced as the misleading
"UI Build Check / frontend-lint" even though it does prettier, eslint, lint
budgets, and knip, not building. Split it into test-litellm-ui-lint.yml
(name "UI Lint") so the check reads "UI Lint / frontend-lint". The build
workflow keeps only build-ui; the lint job (including the knip step) moves
over unchanged.
Note for whoever manages branch protection: this renames the lint required-
check context from "UI Build Check / frontend-lint" to
"UI Lint / frontend-lint"; update the required-check entry so PRs don't strand.
* refactor(ui): colocate the skills view, keeping the AIHub-shared skill components
Group-1 split for the skills (claude_code_plugins) segment. It is a file-plus-
folder combo where part of the folder is shared with AIHub and the public model
hub: the skill types, skill_detail, MakeSkillPublicForm, and helpers stay in
@/components/claude_code_plugins (imported by AIHub's ModelHubTable and
SkillHubDashboard, plus public_model_hub, networking, and skill_hub_table_columns),
while the plugins-management view moves to skills/_components.
claude_code_plugins.tsx becomes skills/_components/index.tsx; add_plugin_form and
plugin_table move alongside it. The moved files' imports of the retained shared
files become @/components/claude_code_plugins paths, other escaping imports are
absolutized, and lint suppressions are re-keyed for moved files only (the staying
components keep their src/components keys). The shared components did not move, so
their consumers are untouched. No behavior change.
* refactor(ui): name the agents/guardrails view files instead of index.tsx
The moved view files were renamed to _components/index.tsx, which reads like a
barrel of re-exports rather than the single view component each actually is.
Rename them to match their default export (AgentsPanel.tsx, GuardrailsPanel.tsx)
and their colocated tests, and point page.tsx at the named file.
login and internalUserIdentity used an unscoped "Virtual Keys" text check that now also matches the top-bar breadcrumb, and adminSettings navigated via antd menuitem roles the shadcn nav no longer emits. Scope these to the sidebar's complementary landmark and drive the Settings group by role (expand the button, then click the child link), matching the fixes already applied to the migration and navigation specs
The dashboard lint-budgets step ran check-lint-budgets.mjs in --check mode,
which fails and tells you to run `npm run lint:metrics` and re-stage by hand.
Add a --write mode that rewrites eslint-metrics.json from the same eslint
report, and have pre-commit use it, then flag drift via git diff so you
re-stage; this mirrors how the block below regenerates schema.d.ts. CI keeps
using --check, so it still fails on a stale committed metrics file.
The redesigned sidebar + header shell is now scoped to the ai-gateway dashboard. Agent-control-plane (plugin) mode falls back to the original full-width Navbar, which carries the account menu, so preferences and sign-out stay reachable there; Chat and the public model hub already use that Navbar
Update the e2e specs the antd Menu -> shadcn nav swap broke. Sidebar items are now links/buttons rather than antd menuitems, and the top bar renders a breadcrumb whose current-page item is also a "Virtual Keys" link, so the migration and navigation smoke checks now scope their sidebar lookups to the complementary landmark and expand the new collapsible groups by data-slot. Also harden the collapsed-rail unit test to assert the icon and link rather than the CSS-hidden label
knip was producing garbage locally and was never wired into CI, so nobody
trusted it. Two structural problems: it silently degrades when deps are
missing (a partial worktree install flagged all 436 test files as unused),
and its config had blind spots that surfaced as false positives.
Fixes so a knip run means something:
- Register every playwright config (serverRootPath + migration variants), not
just the main one. serverRootPath.config.ts is invoked via --config in
test_server_root_path.yml, which knip can't see; it was falsely flagged as
an unused file
- Treat src/components/ui/** as entry points. These are shadcn design-system
primitives, intentionally part of the palette before every one is consumed;
knip was flagging not-yet-used ones (e.g. select.tsx) as dead files and
their sub-exports as unused. Marking the directory as the design-system
surface is the correct fix, not deleting components someone is about to use
- Declare @ant-design/icons as a direct dependency. It was imported in ~198
files but only resolved via antd hoisting, so every one showed up as an
"unlisted dependency"
- Add an explicit vitest plugin block so test-file classification no longer
rides on auto-detection
- Stage severities via rules: gate the now-clean categories (files,
dependencies, unlisted, unresolved) as errors and keep exports/types/
duplicates as warnings, so CI enforces what's at zero today while the
remaining findings ratchet down in follow-ups
- Run npm run knip in the frontend-lint CI job, which installs with npm ci so
it never sees a partial tree
knip now exits 0 with the gated categories clean
