* fix(cache): make in-memory and disk increments atomic
* refactor(cache): narrow in-memory increment lock scope
* fix(cache): address follow-up review on increment tests/types
* fix(cache): refresh atomic increment coverage
* test(cache): widen increment race window with non-zero _SlowInt seed
The zero seed was falsy, so InMemoryCache.increment_cache's `get_cache(...) or 0`
and DiskCache.get_cache's truthiness guard both discarded the _SlowInt before
__add__ could run, leaving the sleep-based window-widening inert. Seed a non-zero
value and return _SlowInt from __add__ so the sleep fires on every read-modify-write
in both backends, making the concurrency regression deterministic.
* test(cache): cover InMemoryCache.async_increment delegation
Add a focused async test asserting async_increment accumulates through the
locked sync path, exercising the previously uncovered delegation line.
---------
Co-authored-by: Emerson Gomes <emerson.gomes@thalesgroup.com>
It is a non-binary latency SLO threshold rather than a deterministic pass/fail
behavior a single e2e test can assert, so it does not fit the coverage registry's
one-test-per-cell contract. The registry README already flagged the perf cells for
a support-check or prune, and throughput SLO under load is covered structurally by
the Locust load suite. Removing it keeps the denominator to behaviors an e2e test
can deterministically prove.
Caller-supplied bedrock_tags land as AWS resource tags under the proxy's
AWS identity, letting an authenticated caller forge ownership or
cost-allocation labels. Add bedrock_tags to _BANNED_REQUEST_BODY_PARAMS
so per-request tags need general_settings.allow_client_side_credentials
or configurable_clientside_auth_params on the deployment, matching the
aws_bedrock_project_id precedent. Deployment-level bedrock_tags in
litellm_params are unaffected.
Also stop an explicit empty bedrock_tags list in litellm_params from
falling through to optional_params
The model_info / get_model_info_with_id endpoint tests drove refactored
endpoints with bare, unspec'd MagicMock routers and models. Because the
mocks were unspec'd, any attribute or method the (refactored) endpoints
newly read auto-materialized a child MagicMock, and whether that child
was reached depended on process-global state (premium_user, and the real
get_available_models_for_user chain reading litellm globals) that sibling
tests in the same xdist worker mutate. When reached, the MagicMock either
unpacked to empty (a, b = mock.method() -> 'not enough values to unpack
(expected 2, got 0)') or leaked into RouterModelInfo(**model_info) and
failed Pydantic str validation. Pass in isolation, fail under xdist.
The original TestModelInfoEndpoint failure (#33807 CI) was the same class
surfaced by merge skew: #33721 added a get_configured_token_limits unpack
to create_model_info_response, and CI's merge commit ran that against the
un-updated bare-mock test before the #33742 band-aid landed.
Fix (test-only, no product change):
- TestModelInfoEndpoint: mock the real seam (get_available_models_for_user),
configure the router methods the endpoint actually calls, return a real
Deployment, and drop the dead proxy_server.get_key_models/get_team_models/
get_complete_model_list patches the refactor had stranded.
- TestGetModelInfoWithIdBlocked: spec the model mock so unset enterprise
columns read as None instead of child MagicMocks.
- test_ProxyConfig_get_model_info_with_id_missing_model_id_raises: pin
premium_user so the asserted AttributeError no longer flips with the
ambient license global.
_load_persisted_master_key documents leniency on any unreadable prior
state but only caught parse errors; a permissions failure or non-UTF-8
bytes in config.yaml crashed configure instead of skipping the
carry-forward. Catch OSError and UnicodeDecodeError too and pin the
undecodable-file case with a regression test.
lite autoroute up minted a fresh master key and picked a fresh OS-ephemeral
port on every run, so any client configured against one session (an
already-open Claude Code session, a hand-configured script) broke on the
next run. The port is now a stable default (5483, overridable with --port)
that refuses loudly when busy or when 4000 is requested, since proxy_cli
silently rebinds a busy 4000 to a random port. The master key is minted
once, persisted in the generated config.yaml, reused by every later up,
and carried forward when configure regenerates the config.
The ID-JAG landing brought to_server_spec to the C901 boundary and the M2M
branch pushed it one over; the oauth2 sub-mode dispatch now lives in its own
_oauth2_spec helper, mirroring the file's per-mode spec builders
_resolve_v2_auth dropped the resolved client_credentials auth when extra_headers
already carried Authorization (MCPJWTSigner, static_headers), so the upstream got
the injected header instead of the minted token and the one-shot 401 refetch was
lost. M2M now joins token_exchange and authorization_code in the authoritative
set; the conflicting header is dropped
The auth object is the httpx client's auth for the whole MCP session; after a
401 recovery it kept sending the rejected token first, burning a 401 round trip
and the single retry on every subsequent call
An expires_in of zero or below computes a ttl of 0; the entry could never be
served but still occupied a slot in the bounded backend, where it could evict
a live token. The mint still serves the current request and the next get
re-fetches under the per-server lock
Greptile P2s: the per-server lock dict now evicts its oldest entry past
max_locks so ephemeral server ids (REST tools preview) cannot grow it
unbounded, and the min-cache floor is capped at the token's actual lifetime so
an expires_in below the skew is never served past expiry
The graft test pinned the pre-migration contract (M2M defers to v1). Replaced
with two tests pinning the new one: a complete-config M2M server resolves via
the v2 arm into ClientCredentialsBearerAuth, and an incomplete-config server
fails closed with a 500 misconfigured naming the missing grant fields
Replaces the not_implemented stub with a live arm: ClientCredentialsTokenSource
mints and caches the M2M token (rotation-aware identity key, expires_in-driven
TTL, audience and token_endpoint_auth_method support) and
ClientCredentialsBearerAuth retries an upstream 401 exactly once with a freshly
minted token. to_server_spec owns oauth2_flow=client_credentials servers and
fails closed on incomplete grant config instead of connecting unauthenticated
Removes the scheduled workflow that cut litellm_oss_daily_YYYY_MM_DD
branches and the guardrails workflow that only ran on them. The secret
scan and ruff checks that workflow duplicated already run on PRs to
litellm_internal_staging via test-linting.yml, so no coverage is lost.
Retargets contributor-facing messaging in CONTRIBUTING.md, CLAUDE.md,
and the guard-main-branch error output at litellm_internal_staging.
Clients exporting large spend-log ranges were forced into 100-row pages,
which meant a bounded COUNT plus an increasingly deep OFFSET scan per
request. Larger pages reduce both the request count and the cumulative
OFFSET cost for the same result set.
The handler already excludes the heavy JSON columns (messages, response,
proxy_server_request) from the paginated SELECT and bounds the COUNT via
SPEND_LOGS_PAGINATION_COUNT_CAP, so per-row cost does not grow with page
size. 1000 matches the ceiling already used by the user and user-agent
analytics list endpoints.
* fix(fireworks_ai): set Content-Type application/json in validate_environment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(fireworks_ai): delegate chat validate_environment to OpenAIGPTConfig
Instead of re-adding the JSON Content-Type default inside FireworksAIMixin,
FireworksAIConfig now delegates header construction to OpenAIGPTConfig and only
layers the Fireworks-specific x-session-affinity header on top, so the
Content-Type default can no longer drift away from the OpenAI base and reintroduce
the 415.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(fireworks_ai): cover missing api key error path in chat validate_environment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
tests/e2e/conftest.py's pytest_sessionfinish truncated LiteLLM_SpendLogs
against whatever DATABASE_URL resolved to, gated only by "an e2e test body
ran". Pointed at a shared or staging DB, a routine local run wiped real
spend data. It also reached the truncate helper through a sys.path.insert
into quota_management/spend_tracking/spend_e2e_client.py, a cross-suite
import-by-path hack it then unwound in a finally.
The cleanup now routes through a new run_spend_log_cleanup in a top-level
tests/e2e/e2e_db.py, which fires the destructive truncate only when the
operator set E2E_RESET_SPEND_LOGS=1 and an e2e test actually ran. Any other
value (unset, 0, true, empty) leaves the DB untouched, so presence of the
variable alone or a test run alone never arms the truncate. The decision
plus the injectable truncate callable live in that pure helper, and conftest
is a thin adapter that supplies os.environ.get(...), the session stash, and
reset_spend_logs.
reset_spend_logs itself moved from spend_e2e_client.py into e2e_db.py
(implementation unchanged), sitting next to e2e_config and lifecycle so both
conftest and any suite import it by name; the sys.path munging is gone.
Nothing else imported reset_spend_logs, so spend_e2e_client.py drops the
definition, its __all__ entry, and the now-unused os import.
