Accept read-only Mapping[str, Any] instead of mutable dict[str, Any] in
_effective_turn_off_message_logging's parameter to satisfy LIT001
(mutable collections in type annotations). Convert to dict for the
function that expects Dict.
The classifier and semantic-embedding sub-calls now capture proxy_server_request,
but neither forwarded the caller's turn_off_message_logging opt-out. A caller who
disabled message logging still had their prompt stored in the clear in these
internal sub-calls' spend-log rows, since should_redact_message_logging reads the
flag per-call and this internal call never inherited it.
The classifier read its metadata only from litellm_metadata, which the proxy
populates just for LITELLM_METADATA_ROUTES (/v1/messages, /v1/responses, ...);
/v1/chat/completions puts it under metadata, so the classifier call arrived
unattributed and _should_track_cost_callback dropped it, leaving no spend-log
row at all for the captured request body to show up in.
Also log response_format in the wire shape litellm actually sends
(type_to_response_format_param) instead of the bare pydantic JSON schema
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(cli): read base_url from persistent config file
Adds a lite config command group (set/get/unset) backed by
~/.litellm/config.json so users no longer need to export
LITELLM_PROXY_URL in every shell session. Resolution order is
--base-url flag, then LITELLM_PROXY_URL, then the config file,
then the localhost default. A config-file base_url counts as an
explicit server choice for lite auth print-token, matching the
env var semantics it replaces.
* fix(cli): harden config persistence after review feedback
Rejects base_url values containing a query string or fragment,
including bare trailing ? or # which parse as empty but still
corrupt every joined request URL. Writes config.json and token.json
atomically through a shared write_private_json helper (0600 at
creation, fsync, os.replace) so an interrupted save can no longer
truncate the file or leave it world-readable. Warns on stderr when
an existing config file is invalid instead of silently ignoring it,
including invalid UTF-8. Resolves the eager --version flag through
the same env, config file, default chain as every other command,
and reads the config file once per invocation so base_url and
base_url_explicit always come from the same snapshot.
* fix(cli): resolve --version after option parsing
The eager --version callback ran before --base-url and --api-key were
parsed, so it could not see an explicitly named server. Combined with
the env fallback added for config-file support, that sent the resolved
API key to whichever server the config file pointed at even when the
user named a different one on the command line. Making the flag a
normal option and handling it in the group callback gives the version
request the same flag, env, config, default precedence as every other
command, and lets the stored-token lookup stay origin-checked.
An interactive oauth2 MCP server created with explicit endpoint URLs and no issuer served
400 "authorization url is not configured" from /authorize about a minute after creation,
with the admin's endpoints intact in the row the whole time (#34985). Discovery wrote its
trust-on-first-use issuer into the same column an admin writes, so the next registry build
read the gateway's own output back as an admin pin, anchored the server to RFC 8414
section 3.3, and discarded the stored endpoint columns; one transient metadata fetch
failure then had nothing to serve, and the reload fast path pinned the broken entry until
an unrelated config write
The core of the fix is a deletion. The gateway no longer writes discovery results anywhere:
the OAuth columns and credentials.scopes carry admin intent alone, and everything discovery
learns lives on the in-memory registry entry, as the existing carry-forward already
assumes. With no gateway write there is no value whose provenance a later build can
misread, so the accidental anchoring cannot be expressed
Deleting the write cannot fix a row a released version already stamped, which still reads
as pinned, so a one-time startup heal clears those stamps. The signal is necessarily a
heuristic: updated_by records only the most recent writer and no audit trail says which
field it touched. A row is therefore healed only on the full signature of the defect, which
is discovery as the last writer plus an issuer plus at least one configured endpoint column
that anchoring is actively discarding; rows with an issuer but no configured endpoints are
left alone, since for them both paths resolve from the same upstream document. Every heal
logs the cleared value so an admin who pinned deliberately can re-pin, and the heal records
its own actor, which makes it idempotent
The reload fast path exempts servers missing an endpoint their flow needs, so failed
discovery retries on the normal reload cadence rather than waiting for a config write. Flow
requirements are read through effective_oauth2_flow, the column-first shape-fallback judge
every flow decision uses, so a legacy null-flow M2M row is classified exactly as the
request path classifies it instead of re-discovering forever; a dcr_bridge server with no
configured client needs its registration endpoint for the relay arm, and an entra_obo
server needs a scope, both of which discovery can supply. Retries back off per server,
doubling from one reload cadence to a fifteen-minute cap, so a permanently unresolvable
server cannot re-run the RFC 9728 to 8414 chain and re-log its warning every cycle forever
Deployments with store_model_in_db unset or false loaded MCP servers exactly once at
startup, leaving that retry with no driver, so they now refresh the registry on the same
reload interval. That job deliberately calls a reload-only entry point rather than the
startup composite, keeping the one-time oauth2_flow backfill and issuer heal out of a
recurring path
Losing the persisted trust-on-first-use issuer also means the issuer column no longer
changes underneath the OAuth token identity, so user tokens are purged only when an admin
actually edits the server
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
`test_content_schema_uses_discriminator` fetches Google's live Interactions
OpenAPI document and required an OpenAPI `discriminator` on the `Content` union.
Google has since dropped that keyword and now pins `type` with a `const` on each
variant instead, so the assertion fails on the current spec and the `misc` shard
is red on every open PR against staging
The information the transformation actually needs did not change: a content part
is still routed by reading its `type`, and each variant still declares exactly
one distinct value for it. So the test now asserts that property directly, and
accepts either spelling, a `discriminator` on the union or a `const` (or
single-value `enum`) on each member
It stays a real check rather than a weakened one. Against the live spec it fails
if TextContent loses its type property, if `text` is renamed, if two variants
claim the same type value, if `Content` stops being a union of named variants,
or if a discriminator appears on some property other than `type`
* feat(ui): link organization teams to their team detail pages
On the organization info page the teams shown for an org were plain
badges, so walking to a team meant copying its id and finding it by
hand on the teams page
Team badges now link to /teams?team=<team_id>, which opens that team's
detail page directly since #35112. Adds a shared BadgeLink (a badge
rendered as a real anchor with modifier-aware client-side navigation,
so cmd-click opens a new tab) and a teamDetailHref builder for reuse by
future entity links
* fix(ui): format BadgeLink, split its modifier-click chain, and size it up
prettier wanted the Badge props wrapped, and local/no-long-condition-chain
flagged the four-way modifier-click guard; the guard is now two named
conditions. Linked badges also render slightly larger (text-sm, roomier
padding) than plain badges so clickable entries stand out
* feat(ui): size org model badges to match the linked team badges
BadgeLink's href is now optional; without one it renders the same
enlarged plain badge (no pointer, no hover), so the org page's model
badges share the component and the size while staying non-clickable
* feat(dashscope): add qwen3.7-plus and qwen3.7-max to the model cost map
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: limit backup cost map diff to the new dashscope entries
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(cost_calculator): adjust tier-only alias assertion for mapped qwen3.7-plus
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(dashscope): drop redundant cost map pinning tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(cost_calculator): point tier-only alias check at an unmapped model
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: shivam <shivam@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Drops tests/e2e/guardrails/test_presidio_guardrail_e2e.py and the
PresidioParamsBody it was the only caller of.
Both cases were red on most stage runs between 07-25 and 07-29: pre_call
failed 6 of 11 runs, post_call 6 of 11, with post_call reporting the raw
address reaching the caller while apply_to_output was set.
The cause was propagation, not masking. GuardrailsClient.register() posts
/guardrails and returns immediately with no readiness wait, unlike
ProxyClient._await_model_servable or GuardrailsClient._await_team, and the
data plane only picks a new guardrail up on its next periodic DB sync. Calls
issued before that sync pass the raw value through. #34833 has since made
both cases poll to the deadline, and on the current build each masks on the
first attempt, so the suite is expected to be green now; it is being removed
because it spends real provider money on every retry and because a pod
replaced mid-poll still reproduces the old failure.
The three guardrail.presidio.* rows stay in coverage_registry/guardrail.yaml
and go uncovered on purpose, so Presidio reads as a tier-P0 gap in Grafana
rather than dropping out of the denominator.
Three defects kept LiteLLM's OTel metrics from reaching an OTLP backend.
OTEL_EXPORTER_OTLP_HEADERS is W3C Baggage encoded per the OTLP spec, so its
values are percent-encoded. litellm split the string on "," and "=" and passed
the raw value straight to the exporter, so a vendor that documents
"Authorization=Basic%20<token>" got a literal "%20" on the wire and the backend
rejected the credential. Grafana Cloud documents exactly that shape, which made
its OTLP gateway unreachable. Header parsing now delegates to the OTel SDK's own
W3C Baggage parser in liberal mode, so percent-encoded values decode and values
that were never encoded keep working. It moves from model/utils.py to
plumbing/providers.py because model/ is deliberately free of opentelemetry
imports; providers.parse_headers was already the entry point every caller used.
The OTLP metric exporters then overrode histogram temporality to delta.
Prometheus and Mimir, which back Grafana Cloud's OTLP gateway, reject delta
histograms outright: the gateway answers 400 "invalid temporality and type
combination" and drops the entire batch, so every GenAI metric was silently lost
while traces kept flowing. Backends that prefer delta still accept cumulative, so
the SDK default is the compatible choice in both directions, and the enterprise
billing exporter already relies on it.
Three GenAI instruments also carried names no convention or backend defines, so
nothing downstream could chart them. Time to first token and time per output
token take their semconv names, gen_ai.server.time_to_first_token and
gen_ai.server.time_per_output_token; the gen_ai.client.response.* spellings
litellm used are not conventions at all. Cost has no semconv instrument, so it
takes gen_ai.usage.cost, the name backends already query for spend. All three are
listed verbatim in Grafana Cloud's AI Observability integration reference, so its
prebuilt panels find them. Both engines now read the names from the shared Metric
constants rather than repeating string literals, so v1 and v2 cannot drift.
The renames are breaking for anyone charting the former names; the docs and the
release changelog carry the migration note.
router.py imports Mapping from collections.abc and again from typing,
which ruff flags as a redefinition and fails the lint CI job on every
open PR. Keep the collections.abc import
* feat(ui): deep link organization detail page via ?org= query param
The organizations page kept the selected organization in React state, so
an org detail page had no URL: it could not be shared, bookmarked, or
opened from another page, and the browser back button dropped you out of
the page instead of closing the detail view
Adds useOrgDetailRouting reading ?org= (same pattern as the api-keys,
models, logs, and teams deep links) and derives the open organization in
OrganizationsPanel from the URL
* fix(ui): reset org edit mode on plain row selection and type test mocks
Greptile P1: with the selected org now URL-derived, browser Back leaves
the detail view without running onClose, so a stale editOrg=true made
the next plain row click open on the Settings tab. Reset the flag on
row selection, matching the teams page
Greptile P2: type the panel test's captured table and detail-view props
from the real components instead of any
The teams page kept the selected team in React state, so a team detail
page had no URL: it could not be shared, bookmarked, or opened from
another page, and the browser back button dropped you out of the page
instead of closing the detail view
Adds useTeamDetailRouting reading ?team= (same pattern as the api-keys,
models, and logs deep links) and derives the open team in Teams.tsx from
the URL. TeamInfo now also derives team-admin rights from the fetched
team data, so team admins arriving via a deep link are not stuck with a
read-only view
* feat(ui): shareable log links via log_id query param on the logs page
Clicking a log row now writes ?log_id=<request_id> to the URL, closing the
drawer removes it, and loading the logs page with ?log_id= opens the drawer
for that log. When the log is not in the loaded page, it is fetched by
request_id (the backend already drops the date window for id lookups), so
links keep working for logs of any age. Drawer open state derives from the
URL, mirroring the models page ?model= pattern.
* fix(ui): close the log drawer on browser back after opening via session id
Session opens now write ?session_id= to the URL instead of holding local
state, so back removes both params and the drawer closes (Greptile P1).
Session views become shareable links as a side effect. In-drawer log
switching now replaces the history entry instead of pushing, so back
always closes the drawer in one step rather than replaying every viewed
log.
* fix(proxy): scope /spend/logs/session/ui to the requesting user's visible logs
Non-admin callers now only receive session rows they could already see on
/spend/logs/ui: their own logs plus logs of teams where they hold the
spend-logs permission. Previously any authenticated user could read any
session's log metadata by id, which shareable ?session_id= links made
trivial to trigger. Admin views are unchanged. Also, clicking a log row
now clears a lingering ?session_id= from the URL so the drawer shows the
clicked log instead of a stale session (Greptile P1).
* feat(ui): chart failed requests as their own series on the cache dashboard
Spend logs for failed requests are stored with an empty call_type, so the
Cache Hits vs API Requests chart lumped them into an Unknown bar that read
as normal LLM API traffic. The activity query now also returns a per-group
failed_rows count (status = 'failure') and the dashboard charts it as a
third stacked series, so failures are visibly separate from successful
requests and cache hits. The chart data transform moves into a pure
summarizeCacheActivity helper with unit tests; header stats keep their
existing semantics (cache hit ratio still counts failures in the
denominator).
* refactor(ui): move cache dashboard aggregation server-side with a typed response
The /global/activity/cache_hits endpoint previously returned raw per
(key, call_type, model) spend-log aggregates typed as LiteLLM_SpendLogs
(wrong), and the dashboard reduced them in the browser: grouping by
call_type, relabeling empty call_type as Unknown, and computing the stat
card totals. All of that now happens server-side. The SQL groups per
call_type and splits cache hits vs successful vs failed requests, a new
cache_activity module validates rows into Pydantic models and computes
totals plus the key-alias/model filter options, and the endpoint declares
a real response_model so schema.d.ts types it correctly. The dashboard
consumes it through a typed $api react-query hook (filters ride the
query key and are applied in SQL instead of the browser), the hand-rolled
summarizeCacheActivity transform and the adminGlobalCacheActivity fetch
helper are deleted, and the refresh button now actually refetches.
The endpoint is UI-internal (hidden from the public swagger), so the
response reshape is not a public API break.
The card variant used viewport breakpoints (md:grid-cols-2 lg:grid-cols-3)
but every card usage sits in a one-third-width grid cell, so on desktop the
narrow card still rendered three internal columns of roughly 100px each and
the text spilled out of its boxes. Switch to Tailwind container queries so
the internal column count follows the card's own width
* fix(scim): stop provisioning nested group ids as internal users
POST/PUT/PATCH /scim/v2/Groups treated every member.value as a user id, so
with the default scim_upsert_user=true an unknown id was auto-created as an
internal user. Entra sends nested groups as members carrying "type": "Group",
which meant every nested group produced a phantom internal user whose id and
email were the group GUID, and those users counted toward licensed seats.
Group members are now classified before they are used: members typed "Group"
are skipped without a database hit, an id that names an existing team is
skipped too (Okta sends untyped ids through filtered paths, so the type alone
is not enough), and only ids that resolve to a user, or that resolve to
nothing at all, keep today's behavior. The user lookup runs before the team
lookup so a user whose id collides with a team id keeps syncing.
The type was previously dropped at parse time on POST/PUT because SCIMMember
had no such field, and on PATCH because the raw member dicts were reduced to
bare ids; both paths now share one resolver and one parser that preserves it.
Member removals no longer upsert: a remove of an id we do not know is an
idempotent no-op rather than a reason to create a user and immediately drop
it, and strict mode (scim_upsert_user=false) no longer rejects it. Removal of
an id that is on the roster but has no user row still cleans up membership.
Responses now state members are of type "User" instead of emitting a null,
and the advertised Group schema documents the members.type sub-attribute.
* fix(scim): harden group member classification after adversarial review
Removals now bypass classification and drop exactly the ids they name,
restoring cleanup of roster entries the old bug left behind. The
team-id fallback only applies to untyped members, so an explicit User
type always provisions even when the id collides with a team. Member
types are normalized before matching; a type other than User or Group
only skips when the id is not an existing user. Non-string type values
are tolerated as absent on every verb instead of failing validation.
Admitted member ids are deduped order-preserving, which also closes a
pre-existing duplicate-row hazard on group creation.
* fix(scim): only treat scim-managed teams as nested groups
A PR reviewer flagged that an untyped SCIM member whose id collides
with an admin-created team was silently skipped, suppressing that
user's provisioning. SCIM group writes (POST, PUT, and every PATCH)
now stamp the team with scim_managed metadata, and the typeless
team-id skip only applies to teams carrying that marker or the
scim_data blob older PUTs already wrote. Admin-created teams stay
unmarked, so a colliding untyped member provisions the user in
permissive mode and returns the standard unknown-user 400 in strict
mode. Teams SCIM touched before this change adopt the marker on their
next group write.
Add regression tests for the db-fetch paths whose converted construction
lines were uncovered: the auth_checks getters (default end user budget, end
user, team membership, access group, team by alias, org by alias, object
permission, managed vector stores, project), get_all_team_memberships and
list_available_teams in team_endpoints, and the proxy admin user info
helper. Each test feeds a mocked prisma row through the real function and
asserts the validated model's fields, so a bad model_validate conversion on
any of these paths now fails a test instead of only dropping coverage.
Convert pydantic table-model construction from Cls(**row.model_dump())
kwargs-unpacking to Cls.model_validate(...) across the management endpoint
hotspot files (team, key, internal user, scim, model management, spend
tracking, auth checks, proxy_server). Unpacking an untyped dict reports one
Any-typed argument per matched model field, so each converted site clears
10-35 diagnostics while running the exact same pydantic validation.
Conversions were limited to models verified to use pydantic's default
__init__; UserAPIKeyAuth and LiteLLM_VerificationTokenView keep their custom
kwargs-rewriting __init__ and are untouched. Two locally-verified helper
params move from Any to object.
Whole-tree basedpyright, measured against the branch point in the same
environment: reportAny 24,431 -> 22,741 (-1,690), reportArgumentType
2,189 -> 2,136 (-53), reportUnknownArgumentType 34,370 -> 34,067 (-303),
reportExplicitAny 7,285 -> 7,283 (-2); total 154,882 -> 152,834 (-2,048)
with no rule increasing anywhere and no per-file increases. No casts, no
suppressions, no behavior changes. Budgets ratcheted: basedpyright -2,048
across 4 rules, ruff ANN401 -2.
* fix(mcp): resolve call_tool by registry without requiring tool map
Multi-worker reloads put MCP servers in the registry from the DB but do
not re-run tools/list on every process. Gating call_tool on
tool_name_to_mcp_server_name_mapping made cold workers 500 with Tool not
found after another worker had already listed the tool. Treat a registry
match on server id/name/alias as enough; upstream rejects unknown tools
* test(e2e): poll MCP register, tools/list, and tools/call across multi-worker lag
Stage multi-worker gateways only load MCP servers and tool maps on the
process that handled the request. Poll until the server is listed, the
tool appears on tools/list, and tools/call is not a cold-worker 500 so
key-access and Datadog MCP e2e stop racing the LB
* Revert "fix(mcp): resolve call_tool by registry without requiring tool map"
This reverts commit 8b56e51e39.
* test(e2e): tighten MCP multi-worker lag classifier
Only retry tools/call on gateway shapes Tool <name> not found and
server_not_found, not any 500 that mentions tool/server not found, so
upstream failures are not retried until the poll deadline
* test(e2e): drop unit file for MCP lag classifier
The live await_call_tool polls already cover multi-worker lag; a separate
string-match unit module is not worth keeping
An auto-router deployment's litellm_params.model (auto_router/...) is the
discriminator the router loads it by, but the model management endpoints
accepted any client-supplied value verbatim; a doubled or stripped prefix
made router init fail on the next load and ignore_invalid_deployments
silently dropped the deployment. Validate writes that supply
litellm_params.model at all three endpoints against the merged params and
reject incoherent values with an actionable 400. Classification is
extracted to router_utils/auto_router_model_naming.py so the Router
predicates and the validation share one source
