* feat(auto-router): make reminder marker pair configurable
Some harnesses inject internal context using their own marker pair
instead of Claude Code's <system-reminder>/</system-reminder>
convention, and some send it as a separate follow-up user message
rather than inline with the ask. Both cases fall out of the same root
cause: the router's marker-matching is hardcoded, so foreign markers
never strip to empty and the reminder-only turn wins "newest human
ask" selection instead of being skipped.
Add an optional reminder_markers field to ComplexityRouterConfig so
operators can override the (open, close) pair via proxy config, with
the existing skip-when-empty selection logic handling both cases once
the markers match.
* test(auto-router): drop unsolicited comments from the reminder-markers regression test
Per Greptile review on #35874: no comments unless explicitly requested.
A three-segment token presented while `general_settings.enable_jwt_auth` is
unset is never treated as JWT-shaped, so it falls through to the virtual-key
path and is rejected for not starting with 'sk-'. That reads as a missing
key in the verification table and sends the operator off to inspect virtual
keys, when the real cause is one missing config line. The rejection now
names `enable_jwt_auth`, appended to the existing text so the Prometheus
invalid-key filter and the admin UI keep matching what they match today.
The hint claims only that the key is JWT-shaped. Segment count cannot tell a
JWT from any other dotted credential, so asserting the key IS a JWT would
swap one confident misdiagnosis for a narrower one.
The enterprise gate on that same path raised a bare `ValueError`, which the
terminal handler turns into a 401. Every sibling enterprise gate answers
403, and a 401 tells the client to retry with a better credential, which no
credential can satisfy while the install is unlicensed. It now raises a 403
`ProxyException` like the SSO gate does.
For a backend-only commit, staging the regenerated schema.d.ts newly
satisfies the ui file triggers, so the folder-wide dashboard lint
budgets run locally for the first time and CI's frontend-lint job
(budgets plus knip) activates on the PR. Those can only fail from
pre-existing dashboard-tree state, never from the regenerated file,
but the guidance should say so instead of implying a re-run is
always redundant.
The stale-types failure already writes the regenerated schema.d.ts to the
working tree, and staging it cannot introduce a new failure: the file is
listed in .prettierignore and the eslint config ignores, so no lint pass
sees it, and gen:api derives it purely from the Python proxy code, so a
second regeneration is a no-op. The only reason left to re-run is when
other checks also failed, so say exactly that in the script message and
CLAUDE.md instead of prescribing an unconditional re-run.
The template tests hardcoded the model names the presets happened to ship
with, so editing autorouter_presets.json to name newer models turned every
preset red in the fixtures and hung six waitFor calls
* feat(ui): add Test Routing to the auto router create form
Route a test prompt through the complexity-router config on screen before the router
is saved, showing the model it lands on and the same decision trace the Logs page renders.
Adds POST /auto_router/test_routing, which classifies with the live pre-routing hook and
sends nothing to the routed model.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): reset the routing test modal on reopen and expose /auto_router on the UI backend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): enforce caller model access and key budget on the routing test's classifier call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: tin <tin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This reverts commit dcb4e5033c.
The suites landed without the proof-of-fix and QA runbook the PR body
itself flagged as outstanding, so the coverage they claim is unverified
against a live proxy
* fix(zscaler_ai_guard): return 400 on guardrail block
* fix(zscaler_ai_guard): don't log error on intentional BLOCK
A BLOCK is expected guardrail behavior, not a failure. Before this
fix, raising HTTPException inside the try block caused the generic
except to log it as "Failed to apply guardrail", producing spurious
error-level noise for every normal block event.
Added except HTTPException: raise before the generic handler (matching
the existing pattern in make_zscaler_ai_guard_api_call), and a
regression test that asserts logger.error is not called on a BLOCK.
---------
Co-authored-by: yucheng-berri <yucheng@berri.ai>
* fix(claude-code): make skill registration create-only with a PUT update route
POST /claude-code/plugins upserted by name, so re-registering an existing
name silently overwrote the stored skill's source and metadata. The "Add
New Skill" UI button posts here, so a name collision clobbered a different
skill with no signal to the user.
Make POST create-only: it returns 409 if the name already exists, with a
unique-violation guard mapping the find-then-create race to the same 409.
Add an explicit PUT /claude-code/plugins/{plugin_name} for updates (404 if
the name is missing). PUT is a full replace and documents that omitted
fields reset to their defaults, so UpdatePluginRequest defaults version to
None instead of fabricating the create-time 1.0.0.
The shared mutable fields move to a PluginSpec base; RegisterPluginRequest
keeps its name and its generated schema unchanged, UpdatePluginRequest
carries no name. Regenerated the dashboard types and the lazy openapi
snapshot for the new route.
Resolves LIT-4110
* fix(ui): surface the proxy error detail so the skill 409 conflict is legible
The add-skill form rendered the raw HTTPException envelope on failure
because deriveErrorMessage did not unwrap an object-shaped detail
({"detail": {"error": ...}}), so the new create-only 409 reached the user
as a JSON blob. Unwrap object-shaped detail at the client layer, which
covers every handler that returns detail={"error": ...}, and surface the
resulting message verbatim on the form instead of burying it under a
generic prefix.
* refactor(claude-code): replace blind excepts in plugin mutations with typed handling
Narrow register_plugin's create-conflict guard from a broad 'except Exception'
+ isinstance dance to a direct 'except UniqueViolationError', using an Exception
subclass sentinel (not None) as the prisma-absent fallback so the sentinel can be
caught directly. Drop update_plugin's outer 'except Exception -> 500' wrapper so
HTTPExceptions propagate on their own and unexpected DB errors surface as FastAPI's
default 500 rather than echoing str(e). Keeps the BLE001 strict-rule budget green.
* fix(claude-code): restore structured 500 handling on update_plugin via typed PrismaError catch
Flattening update_plugin to satisfy the no-blind-except rule dropped its error
wrapper entirely, so a data-layer failure (e.g. a dropped DB connection) would
skip the intentional verbose_proxy_logger.exception call and degrade the response
from the endpoint's structured {"error": ...} body to FastAPI's default
{"detail": "Internal Server Error"}, inconsistent with every sibling route.
Wrap update_plugin in 'except PrismaError' instead of the blind 'except Exception'
the other routes use: it logs and returns the structured 500 for real DB failures
while letting genuine code bugs surface rather than masking them as 'Update failed',
and stays off the BLE001 budget. Add a regression test that a PrismaError during
the update maps to a structured 500.
* fix(claude-code): import prisma error types at function level to satisfy LIT009
* refactor(claude-code): typed plugin mutation responses and lint gate fixes
Return RegisterPluginResponse models from POST and PUT instead of ad-hoc
dicts, declare them as response_model so the OpenAPI schema and dashboard
types carry the real response shape, build the stored manifest via
model_dump, and drop update_plugin's unused auth parameter (the route
dependency already enforces auth). Keeps the LIT002/B008/UP045 budgets at
their ratcheted ceilings after merging litellm_internal_staging
Generic SigV4 double-encodes the canonical URI while S3 canonicalizes the wire path with single encoding, so any object key containing a character that percent-encodes (a team alias, key alias or s3_path with a space) was signed over %2520 while the request carried %20; S3 recomputed a different signature and answered 403.
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng <yucheng@berri.ai>
The 12 GB NODE_OPTIONS setting lived only in the Makefile export and the
CI env line, so any hand-run gate pipeline forgot it and node OOMed at
the ~4 GB default after 80 seconds, with || true feeding the gate empty
output. The gate now spawns basedpyright itself for both the head and
base passes, appends the heap flag last so it wins node's last-flag-wins
resolution while preserving other caller flags, and fails loudly on
crash exit codes instead of reading them as zero errors.
This reverts commit 66bc70365f and the
follow-up 2-line type fix a6d4654261 (#35706), which only retyped a
signature #35492 introduced.
Closing evicted litellm-owned clients breaks every object that fetches
get_async_httpx_client once in __init__ and holds the handler for the
life of the process: 40 guardrail classes plus the pagerduty and email
callbacks. Once the cache entry is evicted (TTL 3600s or the 200-entry
size cap) and the 900s grace passes, the held client is closed and every
subsequent request through it fails with RuntimeError: Cannot send a
request, as the client has been closed. On a production deployment with
a default-on guardrail this surfaced as every request 500ing roughly 75
minutes after boot.
The connection-reclaim goal of #35492 can re-land once handlers survive
their inner client being closed.
The proxy serves POST /openai/v1/responses alongside /responses and
/v1/responses, but only the latter two were in API_ROUTE_TO_CALL_TYPES.
UnifiedLLMGuardrails.async_post_call_success_hook resolves the call type
from request_route, so on the alias it resolved to None and returned the
response unscanned; model output reached the client with post-call
guardrails never running. The key and team tool allowlist was unenforced
on the same alias for the same reason.
Register the alias family in API_ROUTE_TO_CALL_TYPES and in
LiteLLMRoutes.openai_routes, mirroring how the /openai/v1/realtime
aliases are registered, and log a warning at the two points where the
unified guardrail skips post-call scanning so a future unmapped route is
visible instead of silent.
The Responses block of API_ROUTE_TO_CALL_TYPES moves from list to tuple
literals because the LIT002 budget rejects net-new mutable-collection
construction; the map is read-only, so it is now typed as a Mapping of
Sequence and the budgets ratchet down accordingly.
e2e_ui_testing and e2e_ui_testing_server_root_path run on
cimg/python:3.12-browsers, the one UI executor whose image supplies Node
rather than taking it from a cimg/node tag. That image ships Node 24.14.0,
which bundles npm 11.9.0, so both lanes have failed EBADENGINE against the
engines floor added in #35801. Every Node 24 release through 24.14.0 bundles
an npm below 11.10.0, so engines.node also rises to 24.14.1 (npm 11.11.0),
the first release where the two floors agree
The pinned install goes into /opt/node with /opt/node/bin prepended to PATH
instead of unpacking over /usr/local. On this image /usr/local already holds
npm 11.9.0, and extracting the tarball on top of it merges the two trees into
an npm that reports 11.17.0 and then exits 1 on npm ci printing no error text
at all, which is a worse failure than the one being fixed
The install moves into a reusable install_node command so the version and its
checksum have one home, shared with proxy_pass_through_endpoint_tests, and the
command refuses to run when it disagrees with ui/litellm-dashboard/.nvmrc. A
lane drifting off the version the rest of the toolchain uses is what produced
this failure, so that mismatch now stops the job instead of surfacing later as
an install error
The e2e node_modules cache key moves to v4 because the saved trees were built
by the old npm
* feat(ui): add template picker to the Add Auto Router flow
Add Auto Router now opens straight into name + an optional Template
dropdown (Anthropic/OpenAI model-family presets or Custom). A preset
prefills the full complexity-router config and collapses the Detailed
Configuration section to a one-line tier summary; choosing Custom (or
nothing yet) leaves it expanded, and a caller can toggle it manually
at any point. A preset option greys out with the specific missing
model(s) named when the caller lacks a model it needs, or while the
model list is loading or failed to load.
Prefill and submit-gating logic live in testable pure functions
(buildPresetPrefill, getReferencedModelsError) rather than inline in
the component, per the dashboard's own testing guidance.
* refactor(ui): memoize presetAvailability
Consistency with the other memoized derived values it closes over
(availableModelSet, presets). Negligible perf impact with two
presets today, but keeps the pattern uniform as more get added.
* refactor(ui): drop pointless useMemo around getAllPresets()
getAllPresets() already returns a stable module-level array
reference; wrapping it in useMemo added React machinery for
something that can't change.
* refactor(ui): hoist presets to module scope
getAllPresets() was still being called from inside the component
body on every render even after dropping the useMemo wrapper.
Resolving it once at module load, alongside PRESETS' own
module-level initialization in autorouter_presets.ts, is the
actually-clean version of the previous fix.
* fix(ui): collapse Detailed Configuration by default
It was defaulting to expanded before any template was chosen, so
the modal still opened onto the full tier/classifier form instead
of just Name + Template. Custom still auto-expands it, and a
preset still collapses it after prefilling.
* fix(ui): list Custom Configuration last in the Template dropdown
Custom is the escape hatch, not the headline choice, so the bundled
presets now come first with Custom listed after them.
Also lets the collapsed Detailed Configuration summary wrap onto
its own line(s) instead of sharing a line with the section label
and truncating mid-model-name.
* feat(ui): match preset models across "-"/"." version separators
Admins spell version numbers inconsistently (claude-sonnet-4-5 vs
claude-sonnet-4.5), so a preset's hardcoded name and a caller's
registered one can refer to the same model while differing only in
that punctuation. getMissingModels (and therefore presetAvailability
and the submit-blocking check) now treats the two as equivalent.
Applying a preset writes the caller's actual registered spelling
into the tiers, not the preset's literal string, since the caller
may only have the dotted (or hyphenated) form and never the other
one - buildPresetPrefill now takes the available-models set for
this rewrite. Two different model names never collide; only the
separator within one version number does.
* fix(ui): re-check referenced models inside submitRecommendedRouter
submitBlockedReason disables the button for a stale/missing model
reference, but Form's onFinish (wired to the same handler) fires on
a real form submission regardless of the button's own disabled
state. The other four blocking checks already re-validate inside
submitRecommendedRouter for this exact reason; this one was missing
it, so a router could still be created referencing a model no
longer in availableModelSet.
Found by Bugbot.
* Update autorouter_presets.json
The vendored provider pinned google.golang.org/grpc v1.79.2 alongside a set of
golang.org/x modules that govulncheck reports as reachable from plugin.Serve.
Raising grpc to v1.82.1 and golang.org/x/text to v0.39.0 pulls the remainder up
through minimal version selection and leaves govulncheck reporting no findings
Only go.mod and go.sum move here, no provider source is touched. gofmt, go vet,
go build and go test all pass at the new versions
* fix(proxy): apply key_alias/key_hash filters to all /key/list visibility branches
The filters previously lived only in the own-keys OR branch, so a team admin's admin-team branch matched every team key and the Key Alias filter in the Virtual Keys UI appeared broken. Both filters are now global AND conditions alongside team_id/project_id/access_group_id/agent_id, narrowing every visibility branch while leaving unfiltered visibility unchanged.
* chore: drop new explanatory comments flagged by review
* chore: restore schema.d.ts to base enum order
