Adds the pieces needed for the 3-stage public rollout:
- red 'not ready' label paired with 'ready for review'; the review gate
reconciles the pair on every run and reconsider flips it on reopen
- lite mode (AGENT_SHIN_MODE=lite): the review gate never closes and
instead posts a one-time 'closes start in 7 days' notice linking the
policy blog post (AGENT_SHIN_POLICY_URL); labels still reconcile
- stricter reconsider for closed PRs: requires a Greptile confidence
score of at least 4/5 (missing score fails with a comment pointing at
@greptileai) and bypasses the linked-issue short-circuit so the QA
evidence rubric always applies; a pass reopens and tags ready
- reconsider on an OPEN PR now runs the review gate so the label pair
flips immediately instead of waiting for the daily sweep
- PR rubric tightened: bug fixes need before AND after evidence; command
proof from a custom script must include the script source (collapsible
section or linked gist)
- default triage model bumped to gpt-5.6-luna
Reintroduces triage_pr_with_llm.yml and review_gate.yml, removed in #30784
because they ran on pull_request_target. Both now run only on
workflow_dispatch (plus the review gate's daily schedule sweep), so fork
authors cannot fire them and no privileged context is ever exposed to a
fork-controlled event. Instant reaction to fork PR events is provided by
the agent-shin GitHub App bridge, which dispatches these workflows with
the PR number when AGENT_SHIN_ENABLED is true.
* feat(router): soft-floor adaptive mode for complexity router
Let complexity_router_config.adaptive=true Thompson-sample across the
union of tier pools with a tier-distance penalty, and wire the existing
adaptive post-call bandit so mis-tiered requests can still recover.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(router): reattach adaptive hooks for hybrid complexity
Finalize was wiping every AdaptiveRouterPostCallHook and only
re-registering standalone auto_router/adaptive_router deployments,
so complexity adaptive=true never received bandit updates.
Co-authored-by: Cursor <cursoragent@cursor.com>
* chore(router): drop unnecessary hybrid docstrings
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(router): attribute adaptive feedback
Credit user reactions to the model that produced the previous response while keeping current-response signals on the serving model
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(router): tune hybrid cold defaults
Use the cost-weighted policy that beat equal-pool complexity in the full bakeoff, and make the committed harness compare identical tier pools
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(router): preserve hybrid cold quality floor
Sample only unobserved models in the classified tier until feedback exists, then apply adaptive scoring without mis-penalizing models shared across tiers
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(router): bound feedback context cache
Cap retained session feedback so unique session IDs cannot exhaust router memory
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(router): preserve exhaustion signals
Include tool-result exhaustion in adaptive feedback and clear strict lint regressions blocking CI
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor(router): remove stale owner cache
Remove obsolete attribution state, tighten the embedded router type, and keep the test diff focused on adaptive behavior
Co-authored-by: Cursor <cursoragent@cursor.com>
* refactor(router): centralize hook cleanup
Use the callback manager to discover and remove adaptive hooks across every registered callback list
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat(router): add Router(plugins=[...]) routing-plugin pipeline
Runs a sequence of user-supplied plugins before the routing decision is
made. Each plugin reads/mutates a RoutingContext (messages, candidate
models, metadata, signals); the narrowed candidate list is enforced when
picking a deployment, raising rather than silently falling back if a
plugin narrows to zero candidates.
Prototype for the routing-plugin pipeline discussed in #32168.
* fix(router): use ruff-modern typing, add raw/structured messages to RoutingContext
- Use dict/list/X|None instead of Dict/List/Optional in new code, staying
within the ruff strict-rule budget ratchet
- Extract the guardrail-translation message normalization ComplexityRouter
already had into a shared resolve_structured_messages() helper
(litellm_core_utils/prompt_templates/factory.py), reused by
ComplexityRouter and the new routing-plugin pipeline instead of
duplicating it
- RoutingContext now exposes both raw_messages (as received) and
structured_messages (normalized across chat completions / Anthropic
messages / Responses API), mirroring CustomGuardrail.apply_guardrail's
pattern, per review feedback on #32972
- Add direct unit tests for _run_routing_plugins and
_filter_by_routing_plugin_candidates (router_code_coverage gate requires
every router.py function be called by name somewhere in tests/)
* fix(test): rename to test_router_routing_plugins.py
router_code_coverage.py's AST scanner only inspects test files whose
filename contains the substring "router" -- test_routing_plugins.py
doesn't match (routing != router), so it silently skipped this file
and flagged _run_routing_plugins/_filter_by_routing_plugin_candidates
as untested despite the direct unit tests added for them.
* fix(router): fail closed when plugins are configured but the resolved
routing path can't run them
Router.completion() (and other sync entry points) resolves deployments
via the synchronous get_available_deployment(), which never runs
async_pre_routing_hook and therefore never runs the routing-plugin
pipeline. async_get_available_deployment() itself falls back to that
same synchronous method for routing strategies without an async-native
selector (e.g. legacy "usage-based-routing" v1). Both paths would let a
policy plugin (e.g. a deny-all rule) be silently bypassed.
Raise instead of silently proceeding when self.routing_plugins is
configured and the sync path is reached, since applying the pipeline to
every selector path is a larger change out of scope for this PR.
Per review: https://github.com/BerriAI/litellm/pull/32972/changes/BASE..bdfb583c2c6f8df10004fb249e11629d41ce71fa#r3565373303
* feat(router): random-pick multi-model complexity tiers
Tier pools already make sense without adaptive; stop pinning lists to
index 0 and shuffle within the classified tier instead.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(ci): format complexity router config
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(ci): use PEP 585 types for tier pools
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
Selecting the LLM classifier without picking a model only surfaced a
toast on submit; the classifier model select now gets the same red
outline and helper text as the tier and embedding selects once a submit
attempt has failed.
Clicking Add Auto Router with the name empty returned early with only a
toast, so blank tier selects never got their inline error state. The
empty-name branch now sets showValidationErrors and triggers antd
validation on the name field, so every unfilled mandatory field is
flagged at once. Adds a regression test for the tab component.
The Add Auto Router complexity tab let chat models fill the embedding-model
slot (and vice versa) since neither dropdown filtered on ModelGroup.mode, and
submit only required at least one of the four tiers instead of all four. Adds
getMissingTiersError alongside the existing getSemanticConfigError, and
highlights unfilled tier/embedding selects inline once a submit attempt fails.
* refactor(ui): convert endpoint usage charts to shadcn/recharts
Adds a LineChart wrapper to the shared charts kit, mirroring the
BarChart/AreaChart composition with connectNulls and curveType props,
and converts EndpointUsageBarChart and EndpointUsageLineChart from
tremor to the shared wrappers. Both endpoint chart tests now assert on
real recharts SVG output instead of tremor mocks.
* refactor(ui): drop unused endpointData prop from EndpointUsageLineChart
* fix(ui): point endpoint chart test type imports at the UsagePage types alias after colocation move
* refactor(ui): colocate the usage view, keeping the shared usage components
Split for the usage (UsagePage) segment. Most of the folder is the usage page's
own view, but four pieces are reused elsewhere and stay in @/components/UsagePage:
TopKeyView (old-usage), KeyModelUsageView and value_formatters (activity_metrics),
and the shared types (activity_metrics, chartUtils). The other 21 files move into
usage/_components, preserving the folder structure.
The external consumers import only the retained files, so they are untouched. The
moved files' imports of the retained files become @/components/UsagePage paths,
other escaping relative imports are absolutized, and lint suppressions are re-keyed
for moved files only. No behavior change.
* refactor(ui): colocate the mcp-servers view, keeping the shared mcp_tools surface
* feat(ui): adopt openapi-react-query and convert useCustomers to $api
Add openapi-react-query and expose $api = createQueryClient(fetchClient)
alongside fetchClient. Rewrite useCustomers as
$api.useQuery("get", "/customer/list", {}, { enabled, select }), which
derives the query key from method + path (dropping the hand-written
createQueryKeys entry and the manual key) and forwards the request signal
for cancellation. The response type still flows from schema.d.ts as
CustomerResponse[]. Tests assert the path, the admin/token enabled gate,
and the empty-body select fallback.
* test(ui): read the last render's options in useCustomers helper
The lastCallOptions helper was named for the last call but read
mock.calls[0]. Harmless while each test renders once, but it would
silently assert against first-render options if a test ever re-renders.
Read the final call instead.
XecGuard's async_logging_hook wrote a bare dict to
standard_logging_object["guardrail_information"] while the typed
contract is Optional[List[StandardLoggingGuardrailInformation]].
Readers that iterated the field walked dict keys, raised on
info.get, or silently dropped the entry from guardrail usage
tracking and spend-log writes
Construct the typed entry and append it to the existing list or
create a new one, matching the shared helper pattern. Record the
configured guardrail name instead of a hardcoded "xecguard" and
pass the GuardrailEventHooks enum for guardrail_mode
Split for the usage (UsagePage) segment. Most of the folder is the usage page's
own view, but four pieces are reused elsewhere and stay in @/components/UsagePage:
TopKeyView (old-usage), KeyModelUsageView and value_formatters (activity_metrics),
and the shared types (activity_metrics, chartUtils). The other 21 files move into
usage/_components, preserving the folder structure.
The external consumers import only the retained files, so they are untouched. The
moved files' imports of the retained files become @/components/UsagePage paths,
other escaping relative imports are absolutized, and lint suppressions are re-keyed
for moved files only. No behavior change.
* feat(proxy): add expires filter to GET /key/list
Add an opt-in expires query param to GET /key/list so callers can fetch
only expired or only active keys without paginating every page and
filtering client-side. 'expired' matches keys whose expires is in the
past (NULL expires excluded); 'active' matches keys that never expire or
expire in the future. Omitting the param preserves existing behavior for
every caller. An unrecognized value returns HTTP 400 rather than silently
returning all keys.
The filter is pushed to the database via the existing Prisma where
builder so callers avoid pulling the full key table into application
memory.
Resolves LIT-3387
* refactor(proxy): declare VALID_EXPIRES_FILTER_VALUES before its first use
max_tokens=1 makes reasoning models (o1/o3/...) return a 400 "max_tokens
reached" because reasoning tokens count against the cap, so a reachable
reasoning tier showed a false failure in Test Connection. Live-verified: o3
400s with the cap and succeeds without it.
Extract the request shape into a pure buildModelGroupTestRequest and cover it
with a test asserting the chat body carries no max_tokens (or
max_completion_tokens), so this regression is caught in unit tests instead of
only against a live reasoning model.
* feat(guardrails): add pre_mcp_call support to Content Filter
* test(guardrails): cover canonical MCP key gate under pre_mcp_call mode
* fix(guardrails): scan MCP arguments per value and gate mixed-mode scans by call type
* fix(guardrails): cap MCP argument scan depth and register the walker with the recursion detector
* test(guardrails): update LIT-4226 UI settings tests for content filter pre_mcp_call support
* fix(guardrails): use builtin generics in MCP scan annotations to satisfy strict-rule budget
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* refactor(ui): convert entity usage and usage page charts to shadcn/recharts
Swap the tremor BarChart/DonutChart render sites in EntityUsage,
SpendByProvider, TopKeyView, TopModelView, KeyModelUsageView and
UsagePageView to the shared shadcn/recharts wrappers. Convert the two
sole-chart Daily Spend cards and the KeyModelUsageView card to the
shadcn Card primitives.
Close the donut parity gap with strictly additive optional DonutChart
props: showLabel/label render a center total (tremor showed
valueFormatter(sum) by default) and startAngle/endAngle forward to the
Pie so both provider donuts keep tremor's clockwise-from-12 layout.
Defaults preserve the previous wrapper behavior.
DailyData and two site-local row types move from interface to type
alias so they satisfy the wrappers' Record<string, unknown> constraint;
interfaces lack implicit index signatures.
Tests now assert on real recharts output: bar/sector counts, cyan
fills, axis labels, donut center totals, and the TopKeyView bar-click
drill-down into the key info modal. The dead tremor chart mocks in
UsagePageView.test.tsx are removed and lint metrics/suppressions are
regenerated for the dropped tremor imports.
* fix(ui): compute donut center label only when shown and assert Model Usage renders as a card title
Live testing showed the first cut was broken: /health/test_connection merges
{...configParams, ...requestParams}, so passing the public model_group name as
the request model overrode the resolved provider model and every tier failed
with "LLM Provider NOT provided". The frontend only has the public group name,
not the underlying litellm_params, so it cannot build the request that endpoint
needs.
Switch to testing each model group the way production actually routes it: send a
minimal request to /v1/chat/completions (or /v1/embeddings for the embedding
model) by public group name through the shared apiClient. The router resolves
the group, credentials, and provider itself, so a green row means the tier is
genuinely reachable. Verified live: voyage embedding returns 200, a tier with a
bad key returns the real provider auth error.
Also address Greptile feedback: rows now update progressively as each probe
settles instead of all at once, and TIER_ORDER is derived through a
`satisfies Record<keyof ComplexityTiers, null>` guard so adding a tier without
listing it is a compile error.
* fix(ui): show sidebar copy confirmation only on a successful write
The sidebar account menu's copy button switched to the checkmark
synchronously, before the clipboard write settled, so it confirmed a
copy that never happened when navigator.clipboard was undefined on
non-secure origins or when writeText rejected. The handler now guards
navigator.clipboard, awaits the write, and flips to the checkmark only
on success
Also updates the header accent emoji in the same menu
* refactor(ui): extract a shared CopyButton for the sidebar account menu
The copy-icon-to-checkmark pattern was hand-rolled in several places,
including the sidebar account menu whose private copy button held the
false-confirmation bug. Extract a single canonical CopyButton into
components/shared, built on the Button primitive with a guarded and
awaited clipboard write so the checkmark appears only on a real
success, and have SidebarAccountMenu consume it
The success and failure-mode coverage now lives in the shared
component's own test; the sidebar test keeps one case asserting the
email row is wired to it
The consolidated auto-router tab dropped the Test Connection button because
the shared prepareModelAddRequest helper returns an empty array for an auto
router (it has no model_mappings), so the caller crashed destructuring
result[0].litellmParamsObj. That is the crash in #31590 and the open PR
#31794. #31794 only silenced the crash by pointing the test at
auto_router/complexity_router, which is not a provider model, so the
/health/test_connection health check (a real litellm.ahealth_check
completion) would still error.
Bring the button back and make it meaningful: an auto router dispatches to
saved model groups, so Test Connection now probes those directly. It builds
a deduped target list from the configured tiers (tiers sharing a model group
collapse to one probe) plus the embedding model when semantic keyword
matching is on, then runs a live /health/test_connection against each and
shows per-target pass/fail. This never touches prepareModelAddRequest, so the
original destructure crash cannot recur.
Scope is the recommended complexity router only; the to-be-deprecated
semantic router is untouched. No backend changes.
Supersedes #31794. Resolves#31590.
