Three CI gates failed on the initial push; all three were real:
1. lint (strict-rule budget): BLE001 +5, PERF401 +1 over base. The defensive
'except Exception' catches are intentional (an unplaceable name or a
malformed deployment must never fail a request or brick a router), so they
now carry '# noqa: BLE001' with justifications, matching the convention
already used in router.py and common_request_processing.py. The candidate
-spelling loop becomes a generator passed to list.extend (PERF401).
Both rules are now back at base parity.
2. code-quality (router_code_coverage): the checker only scans test files whose
filename contains 'router', so tests/.../test_canonical_model_resolution.py
was invisible to it and both new Router methods read as untested. Renamed to
test_router_canonical_model_resolution.py and added direct coverage for
_get_canonical_model_index (memoization + fenced build failure). Checker now
reports untested_perc: 0.0.
3. proxy-infra (test_route_non_a2a_model_raises_error_if_not_in_router): a real
regression. The test drives route_request with a Mock() router, so
resolve_canonical_model_name returned a truthy Mock and the hook treated it
as a resolved target instead of raising. Both hooks (routing and auth) now
require an actual non-empty str before acting on a resolution -- correct
hardening independent of the test: a stub or partially-initialised router
must never be read as a grant.
Full affected suite: 517 passed. basedpyright: new module 0 errors.
Co-Authored-By: Claude <noreply@anthropic.com>
basedpyright: new module at 0 errors; modified files back to baseline
(+1 structural 'Return type is Any' on the new route return, matching the
file's existing pattern on every route return). ruff clean.
Co-Authored-By: Claude <noreply@anthropic.com>
Implement Feature A: when a request names a model using one spelling (e.g.
dated 'claude-haiku-4-5-20251001') that the gateway serves under a different
spelling (e.g. 'anthropic/claude-haiku-4-5'), route the request to the serving
deployment instead of failing with a 403/400. Resolution only fires after all
existing routes (exact name, alias, wildcard, default_deployment, team routes)
have declined, so a request that succeeds today is never re-pointed (I1).
Identity is enforced strictly: same provider (never cross-provider like Bedrock
to Vertex), same model per cost-map attestation (never family/version hops like
claude-sonnet-4-5 to claude-sonnet-5) (I2).
Auth is AND-on-target: the caller must be allowed to call the resolved target
group; the requested spelling's presence in an allowlist alone grants nothing,
preventing privilege escalation via stale spellings (I3).
Observability: requested name preserved in request metadata, resolution logged
at INFO per (requested, target) pair, Prometheus counter for cardinality.
Changes:
- litellm/router_utils/canonical_model_resolution.py: new module with
canonicalize, build index, lookup functions.
- litellm/router.py: Router.resolve_canonical_model_name(), index caching +
invalidation, config flag model_name_resolution.
- litellm/types/router.py: RouterGeneralSettings.model_name_resolution field.
- litellm/proxy/route_llm_request.py: resolve hook before 400, re-auth on
target, metadata stamping.
- litellm/proxy/auth/auth_checks.py: canonical lookup in _can_object_call_model
with AND-on-target semantics.
- tests/test_litellm/router_utils/test_canonical_model_resolution.py: 23
property tests covering I1–I4, cross-provider block, ambiguity fail-closed,
auth semantics.
Defaults to 'canonical' (on); users can set model_name_resolution: strict in
router_settings to opt out entirely.
Fixes the Claude Code case: dated Haiku requests now resolve across harness
versions without client-side env vars.
Follow-up: once operator data shows a single provider deployment of all
Anthropic models, Feature B (cross-deployment resolution when only 1 provider
exists) becomes safe to implement under similar guards.
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(proxy): requeue spend logs when the DB write fails with a transport error
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): hardcode the spend log queue cap and drop the stale re-export
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): keep the spend log requeue within the type discipline budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): apply the spend log queue cap to producer appends too
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): lower the spend log queue cap to 1k and make it env configurable
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): bound the spend log queue by bytes instead of row count
A row cap cannot bound memory: a row carries the whole prompt under store_prompts_in_spend_logs, so a cap that rides out an outage of counter-only rows is an OOM once prompts are stored. Every enqueue and dequeue now goes through one pair that tracks what the queue costs and drops the oldest rows past a 64 MB budget.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): make the spend log queue byte budget env configurable
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): use a string default for the spend log queue byte budget env read
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): make the spend log queue byte total a public attribute
The queue it accounts for is already public, and a private name only bought reportPrivateUsage errors at every call site.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: shivam <shivam@berri.ai>
Dropping it falls back to the public cost map's tier table, whose rates outrank the
zeros written beside them, so a PTU deployment on a tiered model keeps billing its
traffic per token. Stored empty, the tiers no longer apply and the zeros win
tiered_pricing is a list, so the 0.0 the flat-rate zeroing stores does not
even validate. Supplying tiers alongside PTU config gets the same 400 as a
flat rate; tiers already stored are dropped from both blobs
parse_prompt_tokens_details and parse_completion_tokens_details are imported
by four modules, so the leading underscore made every import a
reportPrivateUsage violation
The previous assertion read data-chips, which is derived from the anchor prop
being truthy, so it stayed true even when the ref never reached the DOM and the
popup was still anchored to the inner input.
Stub distinct widths on the chips container and the input, then read the width
the positioner resolved. Reverting the anchor wiring now reports the input's
width instead of the field's, which is the actual bug.
Base UI positions a combobox popup against the Combobox.Input by default. In
chips mode the visible field is the ComboboxChips wrapper and the input is a
smaller box nested inside it, so every chips-combobox in the dashboard opened
its popup 11px right of the field and 17px past its right edge.
shadcn ships the wiring for this and their combobox-multiple example uses it:
useComboboxAnchor on the chips container, passed to ComboboxContent as anchor.
The anchor prop also drives data-chips, which cancels the extra min-width an
ordinary combobox wants. Every chips site in the dashboard omitted it.
The anchor is attached through Base UI's render prop rather than a plain ref,
because React 18 drops refs on function components and ComboboxChips is one.
Adds MultiSelect's first test, covering the anchor wiring plus selection,
chip rendering and custom values.
Images nested inside an Anthropic `tool_result` block were dropped when the
request was adapted for an OpenAI-compatible provider, because the OpenAI tool
message shape only carried text. Hoist those images out of the tool result and
into a following user message so the model can still see them, and widen the
tool message content type to accept image parts.
The tab strip carried overflow-x-auto directly on the TabsList. CSS forces
overflow-y from visible to auto once overflow-x is not visible, and the line
variant's active-tab underline is an absolutely positioned ::after that hangs
5px below its trigger, so the strip picked up a pixel of vertical scroll on top
of the horizontal scroll it actually wants.
The scroll container now lives on a wrapper whose bottom padding leaves room for
the underline, offset by a matching negative margin so the row keeps its exact
geometry.
* fix(mcp): drop caller host and configured upstream headers from logged metadata
The synthetic request that carries MCP client headers into
add_litellm_data_to_request forwarded the caller's Host header, and
Request.url is built from it, so a caller chose the proxy_server_request
url and the metadata endpoint that every logging callback records.
_upstream_credential_headers also only knew the configured client side
auth header and the x-mcp- prefix family, so a header name declared in
mcp_servers.<name>.extra_headers reached logging metadata in cleartext.
Those names are admin chosen, so no prefix rule can recognize them; read
them off the server registry instead. The header is still forwarded
upstream, which is what extra_headers is for. authorization is left out
because clean_headers already strips it and claiming it here would move
authenticated_with_header on the oauth passthrough config.
The Responses bridge tests stub the server manager, so their fakes gain
the registry accessor the sanitizer now reads.
* fix(mcp): drop caller host from the sanitized header mapping too
The synthetic request stopped forwarding host, but the parallel sanitizer
did not, so a forged hostname still reached the guardrail payload and the
list_tools spend row. Drop it there as well.
Exempt the configured identity headers from the upstream credential set.
get_user_from_headers resolves end user attribution off the same request
this module reconstructs, and it only fills end_user_id when auth left it
unset, so claiming user_header_name or a user_header_mappings name would
lose attribution on the MCP paths that authenticate upstream.
Drop the isinstance guard on extra_headers entries: the field is typed
list[str], so the check is dead and basedpyright scores it.
* fix(mcp): accept a bare user_header_mappings entry when exempting identity headers
get_internal_user_header_from_mapping and get_customer_user_header_from_mapping
both normalize a single mapping to a one element list, and config_settings.md
documents the key as a dict. Iterating the bare form yields its keys instead,
so the exemption silently matched nothing and an identity header also named in
an MCP server's extra_headers was dropped after all.
* feat(search): add Nimble as a search provider
Adds `NimbleSearchConfig` so `search_provider: nimble` works across the SDK,
the proxy /v1/search endpoint, the Search Tools dashboard, and spend tracking.
Nimble's /v2/search already uses the Perplexity unified spec's parameter names,
so the request transform is close to a pass-through. `search_domain_filter`
splits into include_domains/exclude_domains on the spec's `-` prefix, `country`
is upper-cased to the ISO form Nimble documents, and everything else is
forwarded so focus, search_depth, time_range and the rest stay reachable. On the
response side, snippet prefers `content` and falls back to `description`, and a
malformed body raises an attributed error rather than reporting an empty search.
Also tightens `BaseSearchConfig.get_supported_perplexity_optional_params` to
return `frozenset[str]` instead of a bare mutable `set`, which every caller
already treats as read-only.
* fix(search): surface Nimble error bodies instead of empty results
Greptile flagged that a null or absent `results` degraded to a successful empty
search. A search with no hits comes back as `"results": []`, verified against the
live API, so the field is now required and anything else raises the attributed
schema error the other malformed bodies already take.
Also unwraps Nimble's second error envelope. Collection failures return
`{"success", "task_id", "message"}` rather than the `{"detail"}` shape validation
errors use, and only the latter was being read.
Drops comments that restated the adjacent code.
* docs(search): drop the Nimble param list from the transform docstring
It restated the vendor's API reference, which the module docstring already links,
and would go stale the moment Nimble adds a focus mode.
Shadow eval only answered "should this key adopt this auto-router". Once a key
is on the router it is invisible to the feature, because the sampling gate skips
any request the shadowed router already served, so post-adoption quality
regressions go unmeasured.
Reverse mode inverts the arms: sample the traffic the router did serve and
duplicate it against a fixed baseline_model, judged by the same blind pairwise
judge. Same job table, same attempt rows, same aggregates.
real_* stays the arm the caller was served and shadow_* the duplicated one, so
in reverse real_model is the router's pick and shadow_model is the baseline. The
active-job slot becomes one per (key, direction) so both directions can run at
once, and tier attribution in reverse reads the control request's routing
decision rather than the shadow call's write-back.
