Greptile flagged the unused ``from unittest.mock import patch``
left over from before the ``configure_proxy`` fixture refactor (the
fixture uses ``monkeypatch``, no ``patch`` calls remain). Also pruned
the now-stale "premium gate" paragraph from the module docstring
since that gate was removed in fbcfd59b1a.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Greptile flagged the ``premium_user is not True`` check as a hard
backwards-incompatible break for OSS users currently running
``enable_oauth2_proxy_auth=True``. They were right: unlike the
api_base case (where the docs already required admin opt-in), this
path was documented as available to OSS users. Adding the gate would
have closed a documented feature, not fixed a vuln.
Reframed the change:
* The **identity-only allowlist** (``ALLOWED_OAUTH2_PROXY_FIELDS`` =
``{user_id, user_email, team_id, team_alias, org_id, models}``) is
the actual security fix — it closes the privesc by rejecting any
mapping to a non-identity field at request time. This is unchanged.
* The **premium gate** was parity-with-siblings (a product decision,
not a security one). Removed. BerriAI can re-add it on their own
schedule with a proper deprecation cycle if they want enterprise-
only gating.
Tests: removed ``test_rejects_when_not_premium``; everything else
(allowlist enforcement, identity passthrough, attack-shape
regression) still passes — 14 tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Greptile flagged that the denylist was incomplete: ``user_max_budget``,
``user_tpm_limit``, ``user_rpm_limit``, and ``user_spend`` were not on
it. Inspection of the auth model showed dozens more privileged fields
across the ``LiteLLM_VerificationTokenView`` hierarchy (team / org /
end-user / region budget / spend / limit fields, plus
``allowed_model_region``, ``rpm_limit_per_model``, etc.) — a denylist
of "privileged fields" is unmaintainable here.
Inverted the model. ``ALLOWED_OAUTH2_PROXY_FIELDS`` is now an
identity-only allowlist: ``user_id``, ``user_email``, ``team_id``,
``team_alias``, ``org_id``, ``models``. Any mapping to a non-identity
field is rejected at request time. Default-secure: a future field
added to ``UserAPIKeyAuth`` is automatically blocked from
header-trust.
Use case for OAuth2-proxy auth is identity assertion from a trusted
upstream. Anything beyond that (privileges, budgets, rate limits) is
policy and should be authenticated with a signature, not a header —
operators who need this should switch to JWT auth.
Tests:
- ``test_refuses_to_map_non_identity_fields`` parametrized over 22
fields including all four ``user_*`` Greptile flagged, plus
team/org/end-user budget/limit fields, plus a fabricated field name
to confirm "anything not on the allowlist" is the rule.
- ``test_allowlist_is_identity_only`` locks in the allowlist's intent
so future additions of budget / role / permission entries are caught
in review.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two cleanups from the /simplify review pass:
* The header-mapping loop had a special-case ``if key == "max_budget":
auth_data[key] = float(value)`` branch. Since ``max_budget`` is now
in ``PRIVILEGED_OAUTH2_PROXY_FIELDS``, the denylist check rejects
the configuration before the loop runs — the float-conversion
branch is unreachable. Removed.
* Four tests independently called
``monkeypatch.setattr(proxy_server, "premium_user", ...)`` and
``monkeypatch.setattr(proxy_server, "general_settings", ...)`` with
almost-identical bodies. Replaced with a ``configure_proxy`` fixture
that yields a single callable —
``configure_proxy(premium=False)`` /
``configure_proxy(mappings={...})`` — so each test's setup is one
line. The previously-unused ``premium_proxy_settings`` fixture is
removed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
``handle_oauth2_proxy_request`` reads HTTP request headers per the
admin-set ``oauth2_config_mappings`` and constructs a
``UserAPIKeyAuth`` from the values. Two failure modes:
1. **Premium parity.** Sibling auth paths
(``enable_oauth2_auth``, ``enable_jwt_auth``) require
``premium_user``; this path did not, so any open-source deployment
could turn the feature on without realising it requires a hardened
reverse-proxy topology. Added the ``premium_user`` gate.
2. **Privileged-field denylist.** Without a denylist, an admin who
maps the wrong header to ``user_role`` (or whose reverse proxy
leaks the header from upstream user input) lets any caller send
``X-User-Role: proxy_admin`` and gain full admin access — Pydantic
coerces the string into the ``LitellmUserRoles.PROXY_ADMIN`` enum.
Mapping any field in ``PRIVILEGED_OAUTH2_PROXY_FIELDS``
(``user_role``, ``api_key``, ``token``, ``permissions``,
``allowed_routes``, budget/limit fields, ``metadata``) raises at
request time so the misconfiguration surfaces loudly rather than
as a silent privesc.
Operators who genuinely need a trusted upstream to assert one of
these privileged fields should switch to JWT auth (signature-validated)
rather than header-trust.
Tests:
- ``test_returns_auth_for_simple_user_id_mapping``: legitimate
identity-only mapping still works.
- ``test_rejects_when_not_premium``: open-source deployments get a
clear enterprise-feature error.
- ``test_refuses_to_map_privileged_fields``: parametrized over every
entry in the denylist — each is rejected at request time.
- ``test_user_role_header_forgery_attack_is_blocked``: end-to-end
shape of the GHSA-5c3m-qffq-4r9m attack; rejected before auth
object construction.
- ``test_safe_fields_still_pass_through``: documented usage
(``user_id``, ``user_email``, ``team_id``, ``models``) is
unaffected.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The test was flaking on unrelated asyncio ERROR records (e.g. "Unclosed
client session" from background tasks in other tests). Restrict the
assertion to records emitted by LiteLLM loggers so the test only fails
on errors actually produced by the code under test.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* feat: add AIHubMix provider to providers.json
* fix: add aihubmix to provider_endpoints_support.json for CI check
---------
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
`prerelease: false` was hardcoded, so dispatching create-release with
`1.84.0rc1`, `1.84.0.dev42`, or legacy `v1.83.13-nightly` would publish
them as stable releases on the GitHub Releases page. Derive the flag
from the tag instead.
The detector matches `rc`, `.dev`, `nightly`, `alpha`, `beta`. PEP 440
post-releases (`1.84.0.post1`) and legacy `-stable[.patch.N]` are
stable maintenance releases per PEP 440, so they intentionally do not
match.
The tag validator required a leading `v`, so dispatching create-release
with `1.84.0` (or `1.84.0rc1`, `1.84.0.dev42`, `1.84.0.post1`) failed
even though those are the new naming convention. Make the leading `v`
optional in both create-release.yml and create-release-branch.yml so
both legacy (`v1.83.10-stable`, `v1.83.14.rc.1`, `v1.82.3.dev.9`,
`v1.82.3-stable.patch.4`, `v1.83.13-nightly`) and new PEP 440 forms are
accepted during the transition. Refresh the input descriptions to show
the new examples.
* Unify cost calc in success_handler dict and typed branches
* Trim verbose comments and docstrings
---------
Co-authored-by: Michael Riad Zaky <michaelr@Mac.localdomain>
Co-authored-by: Michael Riad Zaky <michaelr@Michaels-MacBook-Air.local>
Claude 3.5 Sonnet v2 reached EOL on Bedrock 2026-03-01, returning the same
404 EOL error as 3.7 Sonnet. Sonnet 4.5 supports both InvokeModel and
Converse APIs on Bedrock, so use the same model for both routes.
AWS Bedrock has reached end-of-life for `claude-3-7-sonnet-20250219-v1:0`,
returning 404s with "This model version has reached the end of its life."
Update test references to `claude-sonnet-4-5-20250929-v1:0` (same capability
surface: thinking, tools, prompt caching, PDF input, vision, computer use).
The bedrock/invoke pass-through tests stay on Sonnet 3.5 since Sonnet 4.5
is converse-only on Bedrock.
Companion to the prior commit. process_items only converted empty
`items: {}` to `{"type": "object"}`. But anyOf branches like
`{"type": "array"}` (no items field at all) were untouched, so after
convert_anyof_null_to_nullable stripped the null branch and added
nullable, the array branch was sent to Vertex as
`{"type": "array", "nullable": true}` — which Vertex rejects with
INVALID_ARGUMENT (`any_of[0].items: missing field`).
Make process_items synthesize `items: {"type": "object"}` for any
`type == "array"` schema where items is missing or empty.
Also:
- Convert test_gemini_tool_calling_working_demo to a hermetic mock
test asserting items is present on the array branch in the sent
body. Was previously a real-network call to Vertex and was the
test the user reported still failing in CI.
- Add unit test test_build_vertex_schema_array_branch_missing_items_in_anyof
covering the missing-items shape directly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cache provider config lookups for Vertex Anthropic messages so repeated requests reuse the same config object and preserve credential cache state. Add a regression test to catch any future loss of config reuse.
Made-with: Cursor
* fix(caching): preserve prompt_tokens_details through embedding cache round-trip
The embedding caching layer was dropping prompt_tokens_details (including
image_count) because CachedEmbedding had no field for usage metadata and
the cache retrieval code reconstructed Usage without it. This caused
inconsistent responses where the first call returned image_count but
cached responses did not, breaking cost tracking for multimodal embeddings.
Add prompt_tokens_details to CachedEmbedding, persist per-item details
during cache storage, aggregate them on retrieval, and merge them in
combine_usage() for partial cache hits.
* style: apply Black formatting to caching files
* fix(caching): address Greptile review — cyclic import, guarded construction, nested dict merge
Move PromptTokensDetailsWrapper to inline import to resolve CodeQL cyclic
import warning. Guard PromptTokensDetailsWrapper construction with
try/except to handle unexpected cached keys. Add recursive dict merging
in _merge_prompt_tokens_details for nested fields like
cache_creation_token_details.
convert_anyof_null_to_nullable was stripping the items field from array
branches inside anyOf when a sibling null branch was present, leaving
{"type": "array"} without items. Vertex requires items whenever
type == "array" (even inside anyOf) and rejects the call with
INVALID_ARGUMENT.
Leave the (possibly empty) items in place so the downstream process_items
step can convert {} to {"type": "object"}, which is what Vertex wants.
Also:
- Update test_build_vertex_schema expected output, which was codifying
the broken shape.
- Convert test_gemini_tool_calling_not_working to a hermetic mock test
that asserts the request body sent to Vertex includes items inside
the callbacks anyOf array branch. The previous form made a real
network call and was flaky in CI.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
vi.clearAllMocks does not reset mockImplementation, so the error-notification
test was inadvertently relying on a deleteField stub set up in earlier tests
and would time out when run in isolation.
Previously, useStoreRequestInSpendLogs and useDeleteProxyConfigField
did not refresh the proxyConfig cache on success, so the Logging
Settings form continued to render the pre-save values until React
Query refetched on its own. Wire both hooks to invalidate
proxyConfigKeys on success so any active observer (currently the
Logging Settings page) repulls fresh data.
Export proxyConfigKeys for cross-hook reuse.
Switch the spend-logs save flow from mutateAsync + try/catch to
mutate + callbacks. Errors now surface through a single onError path
(no more double toast on failure), and the delete-then-update sequencing
runs through onSettled instead of awaited promises. handleFormSubmit is
no longer async.
Tighten the corresponding test to assert exactly one error toast fires.
Adds a CLI flag (`--timeout_worker_healthcheck`, env `TIMEOUT_WORKER_HEALTHCHECK`)
that forwards to uvicorn's `timeout_worker_healthcheck` Config kwarg (added in
uvicorn 0.37.0). Lets operators raise the supervisor's worker-ping timeout above
the default 5s when triaging workers being killed and respawned under load.
The helper introspects `uvicorn.Config.__init__` and only sets the kwarg if
supported, otherwise prints a warning - so the existing uvicorn>=0.32.1,<1.0.0
floor pin is unaffected. Gunicorn and Hypercorn paths are unchanged (the uvicorn
supervisor isn't running there); the value is also not passed to the helper at
all on those paths so the "uvicorn too old" warning never fires spuriously.
