Compute supports_reasoning once per non-wildcard health-check resolution path and update the stale default-max-tokens test docstring.
Made-with: Cursor
Apply reasoning-first precedence for background health-check max tokens, parse reasoning env as optional, and raise non-wildcard fallback max_tokens from 1 to 5 for better reliability.
Made-with: Cursor
Audit-B #2. _check_key_admin_access was gated on max_budget/spend
changes only, which meant a non-admin caller could blanket-rewrite
any OTHER field on any key (key_alias, models, tpm_limit, rpm_limit,
metadata, tags, allowed_routes, guardrails, blocked, duration,
permissions, auto_rotate, access_group_ids, object_permission, …)
as long as they avoided budget/spend. Example attack:
POST /key/update {
key: sk-victim-in-org-B,
models: [],
blocked: true,
organization_id: org-A,
}
The caller is org-admin of org-A, which satisfies the route gate;
the handler then wipes models and blocks the victim's key.
Policy after this fix:
- PROXY_ADMIN: always allowed.
- Key OWNER (matching user_id): allowed for non-budget fields;
budget/spend changes still require team/org admin.
- Everyone else: must pass _check_key_admin_access (PROXY_ADMIN /
key-owner / team-admin / org-admin of the key).
Regression test confirms a non-owner INTERNAL_USER cannot rewrite
key_alias/blocked on someone else's key; existing test covers the
owner-can-update-alias case; existing test covers
internal-user-cannot-modify-max-budget.
Continuation of Veria E3NpkuAd / Audit-B hardening. All three are the
same anti-pattern PR #25904 already addressed for _user_is_org_admin
and /user/delete: route-level gate trusts a caller-supplied scope
field, handler operates on a different scope.
1. /user/update no longer silently creates a user when the target
email doesn't exist. Pre-fix, an org admin could supply a fresh
email + caller-chosen budget/models/metadata; the INSERT path
created the user with no org attachment, bypassing /user/new's
org/team authorization. Now require PROXY_ADMIN for the create
branch; return 404 otherwise. Also fixes /user/bulk_update because
it dispatches through the same _update_single_user_helper.
2. /team/bulk_member_add with all_users=true restricted to PROXY_ADMIN.
The flag pulls every user in the database into the target team,
ignoring org scope — any team admin could use it to capture every
user across every org into their team.
3. /team/update now verifies destination-org admin rights. When the
request carries an organization_id that differs from the team's
current org, an org admin of the caller's current org could
previously relocate the team into any other org (draining their
resources, or capturing a team they once administered). Require
PROXY_ADMIN or org-admin of the DESTINATION org for the relocation.
Regression tests for #1 and #3; #2 covered by the existing bulk_add
suite after the gate addition.
Greptile P2. The admin-inject gate only removed tags from data['metadata']
and data['litellm_metadata']; and the
policy engine read directly, so a caller without
allow_client_tags could still drive tag-based policy decisions by moving
tags to the body root. Also strip the root key in the same branch.
Greptile P1 on the /user/delete fix. Per CLAUDE.md 'No N+1 queries',
move the find_many inside the per-user loop to a single batched
fetch with {'user_id': {'in': data.user_ids}} before the loop, then
distribute to a per-user set in memory.
Veria admin-queue finding E3NpkuAd, Audit-B #1. The route-level gate
accepts this call when the caller is PROXY_ADMIN or ORG_ADMIN of any
org named in request_data["organization_id"]/["organizations"]. The
handler processes data.user_ids without cross-checking whether those
users belong to the caller's administered orgs, so an org-admin of
org-A could delete users in org-B via:
{"user_ids": ["victim_in_org_B"], "organization_id": "org-A"}
Add per-target authorization: org-admins may only delete users whose
entire org membership is within their admin scope; targets with any
org outside scope (or no org at all) require PROXY_ADMIN.
Regression test confirms an ORG_ADMIN call fails with 403 and no
cascade delete_many runs.
Post-merge audit found 6 adjacent variants of the VERIA-28 class. All
fixed here with regression tests:
1. Strip widened from 3 named keys to the full user_api_key_* prefix.
The proxy writes a dozen user_api_key_* fields (user_id, alias,
spend, team_id, request_route, end_user_id, …) into
data[_metadata_variable_name]; the 3-key strip left the rest
exploitable for identity/spend forgery in audit logs and guardrails.
2. proxy_server_request['body'] snapshot moved to AFTER the strip.
Was captured at line ~990 before the strip ran, so
standard_logging_object, lago, and spend_tracking readers saw the
attacker-forged payload even though the live data dict was clean.
3. get_tags_from_request_body (auth-time) now coerces JSON-string
metadata via safe_json_loads. Previously crashed with
AttributeError on string metadata (DoS; potential RBAC bypass if
a caller swallowed the exception).
4. get_end_user_id_from_request_body coerces JSON-string
metadata/litellm_metadata. Previously isinstance(dict) guard
caused end-user budget attribution to be silently skipped when
the caller sent metadata as a JSON string.
5. Four hand-rolled 'if data.get("metadata") is None: data["metadata"] = {}'
blocks in proxy_server.py (7160, 7341, 7590, 11375) now guard on
isinstance(dict). They crashed with TypeError when metadata was a
JSON string (DoS).
6. _get_admin_metadata defensively guards with isinstance(dict);
previously AttributeError'd on any leaked string metadata.
Also hoists the inline safe_json_loads import in _guardrail_modification_check
to module level per CLAUDE.md style.
Close three variant bypasses adjacent to VERIA-28 found during post-fix
variant audit:
1. _guardrail_modification_check had the same isinstance(dict) bypass
Veria-AI just flagged on the pre-call strip. A caller sending
`{"metadata": "{…}"}` as a JSON-encoded string (multipart/form-data
or extra_body) skipped the guard, got parsed to dict downstream, and
reached guardrail logic with bypass flags intact. Coerce strings via
safe_json_loads before evaluating.
2. The allow_client_tags strip only covered body metadata.tags and
litellm_metadata.tags — caller-supplied tags arriving via the
x-litellm-tags header or root-level data["tags"] bypassed it. Gate
add_request_tag_to_metadata's result on the same flag.
3. requester_metadata was deepcopied BEFORE the strip, so attacker
injections (user_api_key_metadata shadows, disallowed tags,
_pipeline_managed_guardrails) persisted in the snapshot. The PANW
guardrail (and any future consumer) trusting requester_metadata
would see forged values. Move the deepcopy to after the strip.
Regression tests added for each.
Per VERIA-28's secondary recommendation. The existing check only gated
metadata.guardrails. User-supplied values for disable_global_guardrails
(plural and the original singular typo variant) and opted_out_global_guardrails
are already silently ignored by _get_admin_metadata at read time, but the
silent-ignore makes diagnosis confusing and relies on one specific read
site catching them.
Reject at auth time with a 403 when any of:
- guardrails list (existing)
- disable_global_guardrails (new)
- disable_global_guardrail (new — historical singular-key variant)
- opted_out_global_guardrails (new)
are present in metadata, litellm_metadata, or at the request root, and the
caller's team lacks can_modify_guardrails. Defense in depth: the strip at
the pre-call layer still runs; this check fails loudly one layer earlier
so operators see an explicit 403 rather than a silent-ignore.
Veria AI caught a bypass: metadata can arrive as a JSON string via
multipart/form-data or extra_body, and the existing strip block ran
before the string-to-dict parse. The isinstance(_user_meta, dict)
guard returned False on the string, the strip was skipped, and then
the parse turned the string into a dict — leaving user_api_key_metadata
/ user_api_key_team_metadata / _pipeline_managed_guardrails / tags
intact in the parsed dict.
Move the strip to run AFTER the parse and BEFORE the merge of
litellm_metadata into data[_metadata_variable_name], closing the bypass
for both raw-dict and string-encoded payloads.
Regression test: test_add_litellm_data_to_request_strips_string_encoded_admin_injection.
- Move protected-headers set to module level as a frozenset
- Add x-api-key, x-goog-api-key to protected set (provider credential headers)
- Block x-amz- prefix to cover AWS SigV4 signing headers
- Normalize forwarded header names to lowercase on write
- Log at debug level when a protected header is skipped
- Add unit test covering protected-header drop and non-protected forwarding
Two pre-existing tests codified the pre-fix behavior where any caller-
supplied metadata.tags would flow through to spend logs and routing:
- test_add_key_or_team_level_spend_logs_metadata_to_request exercised
the request/key/team tag merge. Set allow_client_tags=True on the key
metadata so the merge path is still tested under the new regime.
- test_create_file_with_nested_litellm_metadata asserted that
litellm_metadata[tags] form-data propagated to the handler. Drop the
tag field; the test still proves nested form-parser correctness via
spend_logs_metadata and environment.
VERIA-28 (High) follow-up: tag-based routing and tag budget enforcement
read metadata.tags directly from the request, letting an attacker reach
restricted tag-routed deployments or misattribute spend to a victim
team's tag.
Strip metadata.tags (and litellm_metadata.tags) at the pre-call boundary
unless the caller's key or team metadata opts in with
allow_client_tags=True. Default-deny: existing clients that need to pass
routing tags must have the flag set explicitly on their key or team.
Preserves the tag-routing feature for admins who trust their callers;
closes the injection path for everyone else.
Expand the pre-call metadata strip to also remove user_api_key_metadata
and user_api_key_team_metadata. The proxy writes these fields into
data[_metadata_variable_name] with admin-authoritative values, but only
into that one metadata key; the caller's value in the OTHER metadata
key (metadata vs litellm_metadata) would otherwise persist and be
picked up by _get_admin_metadata, letting a caller supply their own
'admin' config to disable guardrails, opt out of global policies, etc.
VERIA-28 (High): Security Policy and Guardrail Bypass via Unsanitized
Request Metadata.
Add regression test at the proxy boundary verifying the strip, and
extend the guardrail test to cover the post-strip admin-config path.
Validate org admin role against all requested organizations instead
of returning on first match. Scope team list queries to the caller's
permitted organizations when filtering by user_id.
* feat(proxy): add NO_OPENAPI env var to disable /openapi.json endpoint (#25696)
* feat(proxy): add NO_OPENAPI env var to disable /openapi.json endpoint - Fixes#25538
* test(proxy): add tests for _get_openapi_url
---------
Co-authored-by: Progressive-engg <lov.kumari55@gmail.com>
* feat(prometheus): add api_provider label to spend metric (#25693)
* feat(prometheus): add api_provider label to spend metric
Add `api_provider` to `litellm_spend_metric` labels so users can
build Grafana dashboards that break down spend by cloud provider
(e.g. bedrock, anthropic, openai, azure, vertex_ai).
The `api_provider` label already exists in UserAPIKeyLabelValues and
is populated from `standard_logging_payload["custom_llm_provider"]`,
but was not included in the spend metric's label list.
* add api_provider to requests metric + add test
Address review feedback:
- Add api_provider to litellm_requests_metric too (same call-site as
spend metric, keeps label sets in sync)
- Add test_api_provider_in_spend_and_requests_metrics following the
existing pattern in test_prometheus_labels.py
* fix: ensure `litellm_metadata` is attached to `pre_call` guardrail to align with `post_call` guardrail (#25641)
* fix: ensure `litellm_metadata` is attached to pre_call to align with post_call
* refactor: remove unused BaseTranslation._ensure_litellm_metadata
* refactor: module level imports for ensure_litellm_metadata and CodeQL
* fix: update based off of Codex comment
* revert: undo usage of `_guardrail_litellm_metadata`
* feat: add pricing entry for openrouter/google/gemini-3.1-flash-lite-preview (#25610)
* fix(bedrock): skip synthetic tool injection for json_object with no schema (#25740)
When response_format={"type": "json_object"} is sent without a JSON
schema, _create_json_tool_call_for_response_format builds a tool with an
empty schema (properties: {}). The model follows the empty schema and
returns {} instead of the actual JSON the caller asked for.
This patch:
- Skips synthetic json_tool_call injection when no schema is provided.
The model already returns JSON when the prompt asks for it.
- Fixes finish_reason: after _filter_json_mode_tools strips all
synthetic tool calls, finish_reason stays "tool_calls" instead of
"stop". Callers (like the OpenAI SDK) misinterpret this as a pending
tool invocation.
json_schema requests with an explicit schema are unchanged.
Co-authored-by: Claude <noreply@anthropic.com>
* fix(utils): allowed_openai_params must not forward unset params as None
`_apply_openai_param_overrides` iterated `allowed_openai_params` and
unconditionally wrote `optional_params[param] = non_default_params.pop(param, None)`
for each entry. If the caller listed a param name but did not actually
send that param in the request, the pop returned `None` and `None` was
still written to `optional_params`. The openai SDK then rejected it as
a top-level kwarg:
AsyncCompletions.create() got an unexpected keyword argument 'enable_thinking'
Reproducer (from #25697):
allowed_openai_params = ["chat_template_kwargs", "enable_thinking"]
body = {"chat_template_kwargs": {"enable_thinking": False}}
Here `enable_thinking` is only present nested inside
`chat_template_kwargs`, so the helper should forward
`chat_template_kwargs` and leave `enable_thinking` alone. Instead it
wrote `optional_params["enable_thinking"] = None`.
Fix: only forward a param if it was actually present in
`non_default_params`. Behavior is unchanged for the happy path (param
sent → still forwarded), and the explicit `None` leakage is gone.
Adds a regression test exercising the helper in isolation so the test
does not depend on any provider-specific `map_openai_params` plumbing.
Fixes#25697
---------
Co-authored-by: lovek629 <59618812+lovek629@users.noreply.github.com>
Co-authored-by: Progressive-engg <lov.kumari55@gmail.com>
Co-authored-by: Ori Kotek <ori.k@codium.ai>
Co-authored-by: Alexander Grattan <51346343+agrattan0820@users.noreply.github.com>
Co-authored-by: Mohana Siddhartha Chivukula <103447836+iamsiddhu3007@users.noreply.github.com>
Co-authored-by: Amiram Mizne <amiramm@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
Noma v1 resolved application_id from user_api_key_alias when no explicit
value was set (PR #16832). Noma v2 (PR #21400) was rewritten from scratch
and this fallback was not ported, causing all requests from shared LiteLLM
instances to appear as a single generic "litellm" application in the Noma
dashboard — breaking per-user traceability.
Fix: after checking dynamic_params and self.application_id, fall back to
user_api_key_alias from litellm_metadata or metadata. This matches the
pattern used by PromptSecurityGuardrail._resolve_key_alias_from_request_data()
and restores the v1 behavior where each API key gets its own application
entry in the Noma dashboard.
Fixes#25794
Co-authored-by: Brendan Smith-Elion <brendan.smith-elion@arcadia.io>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 30, 8) (push) Waiting to run
* fix: make PodLockManager.release_lock atomic compare-and-delete
Re-lands #21226 (reverted in #21469).
release_lock() previously did GET + compare + DEL in separate calls,
leaving a window where another pod could reacquire the lock between
the GET and DEL, causing a stale owner to delete a live lock.
Fix: use a Redis Lua script for atomic compare-and-delete. Script
registration is cached per PodLockManager instance. Falls back to
the old GET+DEL path for cache backends that don't expose
async_register_script.
Original revert was due to e2e tests running in CI without Redis.
Those tests now carry @pytest.mark.skip(reason="Requires Redis connection.")
so this re-land is safe.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: add Lua fallback on execution error + test coverage gaps
Address Greptile review feedback on #24466:
1. Wrap Lua script execution in try/except — if Redis clears loaded
scripts (restart) or scripting is disabled, fall back to GET+DEL
rather than letting the exception propagate and leave the lock held
until TTL. Reset cached script handle so the next call re-registers.
2. Add test_release_lock_lua_path_emits_released_event — verifies
_emit_released_lock_event is called when Lua path returns 1.
3. Add test_release_lock_falls_back_to_get_del_when_lua_execution_fails
— verifies the fallback path is taken and script handle is reset.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Current fix includes
- Updates test case
- Optimized query with docstring. The change leverages deduplication and sorting logic from SQL
- Added a bench script to differentiate peak memory usage before and after
* fix#25506
* address greptile review feedback
* [Test] UI - Models: Add E2E tests for Add Model flow
Add E2E tests covering:
- Test connection with bad credentials shows failure modal
- Adding a specific model and verifying it appears in All Models table
- Adding a wildcard route and verifying it appears in All Models table
- Verifying model dropdown shows provider-specific models (existing test updated)
Added data-testid attributes to UI components to support stable test selectors.
Tests verified passing 3/3 consecutive runs with zero flakiness.
* address greptile review feedback (greploop iteration 1)
Add cleanup helper to delete models created during tests, preventing
stale data accumulation across repeated test runs.
* fix CI: replace data-testid selectors with text/role-based selectors
The data-testid attributes added to React components are not present
in the CI-built UI output. Switch to using getByRole and getByText
selectors which work with the rendered DOM regardless of build cache.
* remove unnecessary cleanup helper
The database is freshly seeded on every test run via seed.sql,
so per-test cleanup is not needed.
---------
Co-authored-by: Yuneng Jiang <yuneng@berri.ai>
Co-authored-by: Krrish Dholakia <krrish+github@berri.ai>
