_load_persisted_master_key documents leniency on any unreadable prior
state but only caught parse errors; a permissions failure or non-UTF-8
bytes in config.yaml crashed configure instead of skipping the
carry-forward. Catch OSError and UnicodeDecodeError too and pin the
undecodable-file case with a regression test.
lite autoroute up minted a fresh master key and picked a fresh OS-ephemeral
port on every run, so any client configured against one session (an
already-open Claude Code session, a hand-configured script) broke on the
next run. The port is now a stable default (5483, overridable with --port)
that refuses loudly when busy or when 4000 is requested, since proxy_cli
silently rebinds a busy 4000 to a random port. The master key is minted
once, persisted in the generated config.yaml, reused by every later up,
and carried forward when configure regenerates the config.
The ID-JAG landing brought to_server_spec to the C901 boundary and the M2M
branch pushed it one over; the oauth2 sub-mode dispatch now lives in its own
_oauth2_spec helper, mirroring the file's per-mode spec builders
_resolve_v2_auth dropped the resolved client_credentials auth when extra_headers
already carried Authorization (MCPJWTSigner, static_headers), so the upstream got
the injected header instead of the minted token and the one-shot 401 refetch was
lost. M2M now joins token_exchange and authorization_code in the authoritative
set; the conflicting header is dropped
The auth object is the httpx client's auth for the whole MCP session; after a
401 recovery it kept sending the rejected token first, burning a 401 round trip
and the single retry on every subsequent call
An expires_in of zero or below computes a ttl of 0; the entry could never be
served but still occupied a slot in the bounded backend, where it could evict
a live token. The mint still serves the current request and the next get
re-fetches under the per-server lock
Greptile P2s: the per-server lock dict now evicts its oldest entry past
max_locks so ephemeral server ids (REST tools preview) cannot grow it
unbounded, and the min-cache floor is capped at the token's actual lifetime so
an expires_in below the skew is never served past expiry
The graft test pinned the pre-migration contract (M2M defers to v1). Replaced
with two tests pinning the new one: a complete-config M2M server resolves via
the v2 arm into ClientCredentialsBearerAuth, and an incomplete-config server
fails closed with a 500 misconfigured naming the missing grant fields
Replaces the not_implemented stub with a live arm: ClientCredentialsTokenSource
mints and caches the M2M token (rotation-aware identity key, expires_in-driven
TTL, audience and token_endpoint_auth_method support) and
ClientCredentialsBearerAuth retries an upstream 401 exactly once with a freshly
minted token. to_server_spec owns oauth2_flow=client_credentials servers and
fails closed on incomplete grant config instead of connecting unauthenticated
Removes the scheduled workflow that cut litellm_oss_daily_YYYY_MM_DD
branches and the guardrails workflow that only ran on them. The secret
scan and ruff checks that workflow duplicated already run on PRs to
litellm_internal_staging via test-linting.yml, so no coverage is lost.
Retargets contributor-facing messaging in CONTRIBUTING.md, CLAUDE.md,
and the guard-main-branch error output at litellm_internal_staging.
Clients exporting large spend-log ranges were forced into 100-row pages,
which meant a bounded COUNT plus an increasingly deep OFFSET scan per
request. Larger pages reduce both the request count and the cumulative
OFFSET cost for the same result set.
The handler already excludes the heavy JSON columns (messages, response,
proxy_server_request) from the paginated SELECT and bounds the COUNT via
SPEND_LOGS_PAGINATION_COUNT_CAP, so per-row cost does not grow with page
size. 1000 matches the ceiling already used by the user and user-agent
analytics list endpoints.
* fix(fireworks_ai): set Content-Type application/json in validate_environment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(fireworks_ai): delegate chat validate_environment to OpenAIGPTConfig
Instead of re-adding the JSON Content-Type default inside FireworksAIMixin,
FireworksAIConfig now delegates header construction to OpenAIGPTConfig and only
layers the Fireworks-specific x-session-affinity header on top, so the
Content-Type default can no longer drift away from the OpenAI base and reintroduce
the 415.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(fireworks_ai): cover missing api key error path in chat validate_environment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
tests/e2e/conftest.py's pytest_sessionfinish truncated LiteLLM_SpendLogs
against whatever DATABASE_URL resolved to, gated only by "an e2e test body
ran". Pointed at a shared or staging DB, a routine local run wiped real
spend data. It also reached the truncate helper through a sys.path.insert
into quota_management/spend_tracking/spend_e2e_client.py, a cross-suite
import-by-path hack it then unwound in a finally.
The cleanup now routes through a new run_spend_log_cleanup in a top-level
tests/e2e/e2e_db.py, which fires the destructive truncate only when the
operator set E2E_RESET_SPEND_LOGS=1 and an e2e test actually ran. Any other
value (unset, 0, true, empty) leaves the DB untouched, so presence of the
variable alone or a test run alone never arms the truncate. The decision
plus the injectable truncate callable live in that pure helper, and conftest
is a thin adapter that supplies os.environ.get(...), the session stash, and
reset_spend_logs.
reset_spend_logs itself moved from spend_e2e_client.py into e2e_db.py
(implementation unchanged), sitting next to e2e_config and lifecycle so both
conftest and any suite import it by name; the sys.path munging is gone.
Nothing else imported reset_spend_logs, so spend_e2e_client.py drops the
definition, its __all__ entry, and the now-unused os import.
create_model_info_response cast cost-map max_input_tokens / max_output_tokens
with unguarded int(). The surrounding try/except covers only the get_model_info
lookup, so a deployment whose model_info carries a non-numeric limit (e.g.
"128,000" or an empty string) raised inside the per-model listing loop and
failed the entire GET /v1/models and /models response with a 500, taking healthy
deployments down with it. A deployment's model_info is registered into
litellm.model_cost verbatim, so the malformed value reaches the cost map and not
just the router index.
Router.get_configured_token_limits already coerced this safely for the
deployment path; the cost-map path was missed, so the two together still
regressed. Both now share coerce_token_limit in litellm_core_utils, which
returns None for a malformed value so the listing omits that one limit instead
of failing, matching the graceful degradation the endpoint had before the
cost-map switch.
The SERVER_ROOT_PATH fix for the per-server pass-through challenge belongs with its sibling
in exceptions.py (both fabricate a per-server resource_metadata URL and both omit the root
segment), and both are pre-existing paths unrelated to the aggregate discovery this PR adds.
Reverting the server.py change keeps this PR to the aggregate front door and avoids leaving
the two per-server challenge builders inconsistent; the per-server root-path fix lands as its
own change covering both sites.
tests/test_litellm/proxy/test_custom_proxy.py sets SERVER_ROOT_PATH at import time (its app
mounts under a custom path) and never restores it, so in a shared shard the value leaks into the
process. The discovery routes and the 401 challenges now read SERVER_ROOT_PATH to path-insert it
where they previously ignored it, so a leaked value rewrites every resource_metadata URL and the
exact-URL assertions in the delegate, pass-through, and aggregate challenge tests fail depending
on shard order
An autouse fixture clears SERVER_ROOT_PATH for the MCP discovery tests so they deterministically
exercise the default root-mounted deployment; the tests that assert a sub-path deployment set the
value explicitly within their own body. No assertion changed; the leak was invisible before only
because the code ignored the variable
Two RFC 9728 / 8414 discovery fixes on the aggregate front door, both raised by Bugbot on this PR
The aggregate authorization-server document at /.well-known/oauth-authorization-server/mcp used to
defer to a per-server row literally named "mcp", serving issuer {base} while the aggregate
protected-resource document advertises {base}/mcp as its authorization server. A spec client
following that chain fails the RFC 8414 issuer check and cannot sign in. The single segment /mcp is
now reserved for the aggregate so the issuer stays {base}/mcp and matches the protected-resource
document; a server named "mcp" keeps its standard two-segment discovery at
/.well-known/oauth-authorization-server/mcp/mcp
The 401 challenges built the resource_metadata URL as {base}/.well-known/oauth-protected-resource/mcp
with no SERVER_ROOT_PATH segment, but the routes are registered with the path-inserted root segment,
so a proxy mounted under a sub-path pointed DCR clients at a URL that 404s. Both the aggregate
challenge and the pre-existing per-server pass-through challenge now derive the path from one
well_known_root_suffix helper that the route registrations also use, so the advertised URL cannot
drift from the served route
The flag guarded no breaking change: the aggregate discovery lives at new /mcp-suffixed
routes, the challenge only fires at aggregate scope, and the authorize/token/register/admission
arms self-gate on the llm_dcrc_/llm_session_ prefixes. Bare-origin and per-server discovery are
left exactly as they were, and a server literally named mcp keeps its own discovery via
disambiguation, so turning it on for everyone changes nothing about existing flows.
* feat(rust): expose anthropic messages route
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(rust): use provider model for messages upstream
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* feat(rust): stream Anthropic Messages SSE on POST /v1/messages
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* test(rust): prove alias is substituted with provider model on /v1/messages
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(rust): make anthropic messages provider constant available without server feature
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* test(e2e): assert the long budget window keeps blocking after the short window resets
The multi-window budget tests proved the tight window blocks and self-heals
but never asserted the other direction: a long (1d) window whose cap the
accumulated spend already crossed must keep refusing calls even inside a
fresh short window. Adds one test per file (key and team) that drives spend
to a block, waits for the short window's reset_at to strictly advance (the
reset job zeroes that window's counter in the same pass), then polls until
the refusal is attributed to the 1d window ("over 1d budget"), failing
immediately if any call succeeds or a non-budget error leaks. Harness gains
per-window reset_at readback: BudgetWindowState in models.py and
key_window_reset_at / team_window_reset_at on BudgetClient.
* refactor(e2e): hoist shared budget-suite helpers into budget_client
drive_to_block and as_datetime existed as five and four per-file copies in
the budgets suite; both move to budget_client with each file keeping a thin
delegating wrapper so call sites and per-file pacing stay unchanged. The
three /team/info readers in budget_client now share a private _team_info.
Also guard the long-window reset_at snapshots with explicit non-None asserts
so the midnight-roll diagnostic cannot misreport when the window is missing
from the info response (greptile P2s).
* docs(e2e): tighten the multi-window module docstrings
* refactor(e2e): type window reset_at as datetime and expose plain window readers
BudgetWindowState.reset_at becomes a pydantic-parsed datetime, so the
multi-window tests compare real datetimes instead of hand-parsing strings.
The duration-keyed accessors are replaced by two plain readers,
key_budget_windows and team_budget_windows, with the pure window_reset_at
lookup exported; the client no longer encodes one test's access pattern.
* test(e2e): name the tiny short-window cap and comment the wait loops
* test(e2e): surface the 429 budget-block assert in the multi-window tests
drive_to_block now returns the blocking response so a test body can assert
on its shape; the two long-window tests assert status 429 explicitly, which
also pins the multi-window enforcement path's HTTP mapping (the enforcement
suite only covers the single-budget path). Other callers ignore the return
and are unchanged.
* refactor(e2e): scope this PR to the multi-window test, drop the cross-suite hoist
The helper hoist rewrote four unrelated budget test files (reset, reset_advances,
team_member_reset, user_across_keys) to pull drive_to_block and as_datetime out
of budget_client, which is refactor churn beyond this PR's multi-window scope.
This restores those four to their pre-PR state and gives the two multi-window
tests their own inline drive-to-block loop again, so the PR touches only the
multi-window feature: its two tests plus the budget_client window readers and the
reset_at datetime typing they actually use. The suite-wide helper dedup can land
on its own PR
* docs(e2e): number the long-window key test steps inline
* docs(e2e): number the long-window team test steps inline
* feat(spend): track prompt compression saved tokens in daily spend aggregates
Native compression interception now records tokens_before/after/saved into the
request litellm_metadata so savings land in the SpendLog metadata JSON under a
typed compression_savings key. A single normalizer
(extract_compression_saved_tokens) sums that key with Headroom guardrail
tokens_saved; the two writers are disjoint and run at different stages, so
summing never double-counts. The spend-log redactor now preserves purely
numeric compression stats inside guardrail_response so Headroom savings
survive the store_prompts_in_spend_logs=false default. compression_saved_tokens
is threaded through BaseDailySpendTransaction, queue aggregation, the daily
upsert blocks, a new BigInt column on all six daily spend tables, and the
daily activity read path (SpendMetrics, DailySpendMetadata, raw-SQL rollups)
* fix(spend): normalize legacy guardrail shapes and float token stats in compression savings reader
* feat(spend): aggregate compression and prompt caching dollar savings in daily rollups
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(spend): update daily spend aggregation fixtures for savings columns
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): add Cost Optimization dashboard page
New left-nav Cost Optimization page under Observability that surfaces money saved by prompt compression and prompt caching. It reads the daily activity rollup (userDailyActivityCall / get_daily_activity) and never scans SpendLogs, so it stays fast at 1M+ rows.
Renders a Total saved card, per-driver Compression and Prompt caching cards, a savings-over-time area chart, and a savings-by-driver donut, all aggregated in memory from the per-day metrics.compression_savings_spend and metrics.prompt_caching_savings_spend fields.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Move the destructive daily-tag Redis drain back inside the try so a Redis
read failure still releases the pod lock via the finally block, and use a
covariant Mapping for the restore signature. Add regression tests for the
daily-tag requeue-on-failure/no-requeue-on-success paths and the RedisError
swallow branch in restore_transactions_to_redis.
