TogetherAIConfig.get_supported_openai_params called get_model_info(),
whose first line calls litellm.get_supported_openai_params() — which for
together_ai routes straight back into this method. The recursion only
terminated when Python's recursion limit was hit or when
_get_model_info_helper raised "not mapped" at the deepest level. Either
way the try/except caught it, so the bug stayed silent — but the cycle
ran ~332 deep every time, emitting hundreds of DEBUG log lines per
call. Surfaced as "infinite loop" in CI when the success_handler thread
emitted that log spam against an already-closed stderr during test
teardown.
Replace the get_model_info() call with supports_function_calling(),
which uses _get_model_info_helper directly and does not call
get_supported_openai_params. Measured drop from 332 to 2
_get_model_info_helper calls per first uncached lookup.
Also swap the test model from Qwen/Qwen3.5-9B (not in model_cost map)
back to a mapped serverless model, Qwen/Qwen2.5-7B-Instruct-Turbo. The
mapping gap is what made the recursion's tail end raise up into the
success handler during teardown in the first place.
Greptile P2. The admin-inject gate only removed tags from data['metadata']
and data['litellm_metadata']; and the
policy engine read directly, so a caller without
allow_client_tags could still drive tag-based policy decisions by moving
tags to the body root. Also strip the root key in the same branch.
Greptile P1 on the /user/delete fix. Per CLAUDE.md 'No N+1 queries',
move the find_many inside the per-user loop to a single batched
fetch with {'user_id': {'in': data.user_ids}} before the loop, then
distribute to a per-user set in memory.
Veria admin-queue finding E3NpkuAd, Audit-B #1. The route-level gate
accepts this call when the caller is PROXY_ADMIN or ORG_ADMIN of any
org named in request_data["organization_id"]/["organizations"]. The
handler processes data.user_ids without cross-checking whether those
users belong to the caller's administered orgs, so an org-admin of
org-A could delete users in org-B via:
{"user_ids": ["victim_in_org_B"], "organization_id": "org-A"}
Add per-target authorization: org-admins may only delete users whose
entire org membership is within their admin scope; targets with any
org outside scope (or no org at all) require PROXY_ADMIN.
Regression test confirms an ORG_ADMIN call fails with 403 and no
cascade delete_many runs.
Post-merge audit found 6 adjacent variants of the VERIA-28 class. All
fixed here with regression tests:
1. Strip widened from 3 named keys to the full user_api_key_* prefix.
The proxy writes a dozen user_api_key_* fields (user_id, alias,
spend, team_id, request_route, end_user_id, …) into
data[_metadata_variable_name]; the 3-key strip left the rest
exploitable for identity/spend forgery in audit logs and guardrails.
2. proxy_server_request['body'] snapshot moved to AFTER the strip.
Was captured at line ~990 before the strip ran, so
standard_logging_object, lago, and spend_tracking readers saw the
attacker-forged payload even though the live data dict was clean.
3. get_tags_from_request_body (auth-time) now coerces JSON-string
metadata via safe_json_loads. Previously crashed with
AttributeError on string metadata (DoS; potential RBAC bypass if
a caller swallowed the exception).
4. get_end_user_id_from_request_body coerces JSON-string
metadata/litellm_metadata. Previously isinstance(dict) guard
caused end-user budget attribution to be silently skipped when
the caller sent metadata as a JSON string.
5. Four hand-rolled 'if data.get("metadata") is None: data["metadata"] = {}'
blocks in proxy_server.py (7160, 7341, 7590, 11375) now guard on
isinstance(dict). They crashed with TypeError when metadata was a
JSON string (DoS).
6. _get_admin_metadata defensively guards with isinstance(dict);
previously AttributeError'd on any leaked string metadata.
Also hoists the inline safe_json_loads import in _guardrail_modification_check
to module level per CLAUDE.md style.
Three tests inherited by TestBedrockMoonshotInvoke from BaseLLMChatTest
make live AWS Bedrock completion calls: test_developer_role_translation,
test_message_with_name, and test_completion_cost. These have been
crashing llm_translation_testing CI workers (reported as "failed on
setup with worker 'gwN' crashed").
Replace each with a mocked override that intercepts the outgoing
request via HTTPHandler.post / AsyncHTTPHandler.post patching:
- test_developer_role_translation asserts the outgoing body maps the
developer role to system (LiteLLM's translation for non-OpenAI
providers).
- test_message_with_name asserts the outgoing body preserves the user
message.
- test_completion_cost returns a canned moonshot-shaped response body
with usage and asserts response_cost > 0 against the local model
cost map.
Follows the existing HTTPHandler + patch.object(client, "post") pattern
used in test_bedrock_gpt_oss.py and test_bedrock_completion.py. No
network traffic; the three tests now complete in ~0.3s.
Close three variant bypasses adjacent to VERIA-28 found during post-fix
variant audit:
1. _guardrail_modification_check had the same isinstance(dict) bypass
Veria-AI just flagged on the pre-call strip. A caller sending
`{"metadata": "{…}"}` as a JSON-encoded string (multipart/form-data
or extra_body) skipped the guard, got parsed to dict downstream, and
reached guardrail logic with bypass flags intact. Coerce strings via
safe_json_loads before evaluating.
2. The allow_client_tags strip only covered body metadata.tags and
litellm_metadata.tags — caller-supplied tags arriving via the
x-litellm-tags header or root-level data["tags"] bypassed it. Gate
add_request_tag_to_metadata's result on the same flag.
3. requester_metadata was deepcopied BEFORE the strip, so attacker
injections (user_api_key_metadata shadows, disallowed tags,
_pipeline_managed_guardrails) persisted in the snapshot. The PANW
guardrail (and any future consumer) trusting requester_metadata
would see forged values. Move the deepcopy to after the strip.
Regression tests added for each.
Per VERIA-28's secondary recommendation. The existing check only gated
metadata.guardrails. User-supplied values for disable_global_guardrails
(plural and the original singular typo variant) and opted_out_global_guardrails
are already silently ignored by _get_admin_metadata at read time, but the
silent-ignore makes diagnosis confusing and relies on one specific read
site catching them.
Reject at auth time with a 403 when any of:
- guardrails list (existing)
- disable_global_guardrails (new)
- disable_global_guardrail (new — historical singular-key variant)
- opted_out_global_guardrails (new)
are present in metadata, litellm_metadata, or at the request root, and the
caller's team lacks can_modify_guardrails. Defense in depth: the strip at
the pre-call layer still runs; this check fails loudly one layer earlier
so operators see an explicit 403 rather than a silent-ignore.
Veria AI caught a bypass: metadata can arrive as a JSON string via
multipart/form-data or extra_body, and the existing strip block ran
before the string-to-dict parse. The isinstance(_user_meta, dict)
guard returned False on the string, the strip was skipped, and then
the parse turned the string into a dict — leaving user_api_key_metadata
/ user_api_key_team_metadata / _pipeline_managed_guardrails / tags
intact in the parsed dict.
Move the strip to run AFTER the parse and BEFORE the merge of
litellm_metadata into data[_metadata_variable_name], closing the bypass
for both raw-dict and string-encoded payloads.
Regression test: test_add_litellm_data_to_request_strips_string_encoded_admin_injection.
test_add_litellm_data_to_request_duplicate_tags tests the request/key
tag merge when tags overlap. The merge requires caller-supplied tags to
flow through — set allow_client_tags=True on the key so the merge path
stays testable under the new default-deny regime.
Silent strip is the worst debug UX: admin's client sends routing tags,
they disappear, admin can't figure out why. Emit a warning naming the
metadata key the tags came from and telling the admin exactly which
flag to set if this is intentional.
- Move protected-headers set to module level as a frozenset
- Add x-api-key, x-goog-api-key to protected set (provider credential headers)
- Block x-amz- prefix to cover AWS SigV4 signing headers
- Normalize forwarded header names to lowercase on write
- Log at debug level when a protected header is skipped
- Add unit test covering protected-header drop and non-protected forwarding
Two pre-existing tests codified the pre-fix behavior where any caller-
supplied metadata.tags would flow through to spend logs and routing:
- test_add_key_or_team_level_spend_logs_metadata_to_request exercised
the request/key/team tag merge. Set allow_client_tags=True on the key
metadata so the merge path is still tested under the new regime.
- test_create_file_with_nested_litellm_metadata asserted that
litellm_metadata[tags] form-data propagated to the handler. Drop the
tag field; the test still proves nested form-parser correctness via
spend_logs_metadata and environment.
VERIA-28 (High) follow-up: tag-based routing and tag budget enforcement
read metadata.tags directly from the request, letting an attacker reach
restricted tag-routed deployments or misattribute spend to a victim
team's tag.
Strip metadata.tags (and litellm_metadata.tags) at the pre-call boundary
unless the caller's key or team metadata opts in with
allow_client_tags=True. Default-deny: existing clients that need to pass
routing tags must have the flag set explicitly on their key or team.
Preserves the tag-routing feature for admins who trust their callers;
closes the injection path for everyone else.
* Add announcement bar for Trivy compromise resolution notice
Add a Docusaurus announcement bar to the top of the docs site informing
users that the Trivy supply-chain compromise has been mitigated and
resolved. The banner:
- States all affected packages have been deleted and releases are safe
- Links to the Security Townhall blog post for details
- Links to the CI/CD v2 blog post for improvements made
- Uses a green background with closeable dismiss button
Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com>
* Use :::note admonition instead of announcement bar
Replace the Docusaurus announcementBar with a :::note admonition on the
docs index page. The note appears below the hero image with the title
'Security Update' and links to the Security Townhall and CI/CD v2 blog
posts.
Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com>
* Update security notice wording to 'contained'
Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com>
* Move note above hero image and add to root page
- Move the security notice above the product screenshot on /docs
- Add the same notice to the root page (src/pages/index.md)
Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com>
* Update security notice wording
Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Krrish Dholakia <krrish-berri-2@users.noreply.github.com>
Replace the hand-maintained _BLOCKED_NETWORKS CIDR list with a
default-deny check based on ipaddress.is_global (RFC 6890 semantics,
implemented by Python's stdlib). Also reject multicast explicitly —
is_global returns True for public multicast allocations, which are
not legitimate HTTP targets.
Only globally-routable cloud-fabric IPs need explicit exceptions; the
canonical list contains one entry today: Azure Wire Server
(168.63.129.16), an in-fabric service reachable from any Azure VM.
Coverage delta picked up automatically via is_global:
- Alibaba Cloud metadata (100.100.100.200, CGNAT)
- Legacy Oracle metadata (192.0.0.192, IETF Protocol Assignments)
- IPv4 documentation ranges (192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24)
- IPv4 reserved/future-use (240.0.0.0/4) and broadcast
- IPv6 documentation (2001:db8::/32)
Also fix two issues Greptile flagged:
- HTTP relative-redirect hops lost the original hostname because
_extract_redirect_url joined the Location against the rewritten
(IP-based) URL. Join against the pre-rewrite URL so the next hop's
Host header keeps the original hostname.
- Two unit tests performed real socket.getaddrinfo('localhost')
calls. Monkeypatch them.
Add coverage tests for every cloud-metadata IP from the canonical
SSRF dictionary (AWS/GCP/Azure/Alibaba/Oracle/DO/OpenStack) plus the
new multicast/reserved/documentation/broadcast ranges, and a
regression test for redirect-hostname preservation.
Expand the pre-call metadata strip to also remove user_api_key_metadata
and user_api_key_team_metadata. The proxy writes these fields into
data[_metadata_variable_name] with admin-authoritative values, but only
into that one metadata key; the caller's value in the OTHER metadata
key (metadata vs litellm_metadata) would otherwise persist and be
picked up by _get_admin_metadata, letting a caller supply their own
'admin' config to disable guardrails, opt out of global policies, etc.
VERIA-28 (High): Security Policy and Guardrail Bypass via Unsanitized
Request Metadata.
Add regression test at the proxy boundary verifying the strip, and
extend the guardrail test to cover the post-strip admin-config path.
Drop test_bedrock_invoke_messages_injects_thinking_for_clear_thinking_context_management.
Its assertion 'interleaved-thinking-2025-05-14' in betas cannot hold because
anthropic_beta_headers_config.json maps that header to null for the bedrock
provider, so filter_and_transform_beta_headers drops it from the auto-added
beta set before anthropic_beta is written to the request.
The adjacent test_bedrock_invoke_messages_skips_thinking_injection_when_already_enabled
already covers the inverse behavior for the same model, so no coverage is lost.
Two litellm-level flags wired through litellm_settings YAML:
- user_url_validation (bool, default True): master switch. When False,
safe_get/async_safe_get bypass validation and call client.get
directly.
- user_url_allowed_hosts (List[str], default []): per-host allowlist.
Entries are 'host' (matches any port) or 'host:port' (port-specific).
Matched hosts skip the blocked-networks check but still resolve DNS
and still rewrite HTTP to the validated IP, preserving rebinding
protection within the permitted name.
Also fix an existing Host header bug: IPv6 literals (e.g. 2001:db8::1)
were emitted unbracketed, producing ambiguous values like
'2001:db8::1:8080' per RFC 7230 5.4. Bracket them consistently in
_format_host_header.
Greptile P2: _get_admin_metadata used 'litellm_metadata or metadata',
meaning a caller sending a non-empty litellm_metadata would shadow
admin config the proxy had injected into data['metadata']. Admin
exemptions would be silently ignored.
Check both keys and prefer whichever contains admin fields. Add
regression test covering the shadowing scenario.
Greptile P1: six tests in test_url_utils.py performed real DNS
lookups to example.com, violating the tests/test_litellm/ mock-only
rule and risking offline CI failures. Add mock_dns_public and
mock_dns_failure fixtures that monkeypatch socket.getaddrinfo on
the url_utils module.
Greptile P2: move 'import httpx' from inside _extract_redirect_url
to module-level imports per CLAUDE.md style guide.
SDK core modules (image_handling, token_counter) should not import
from litellm.proxy. Move url_utils.py to litellm_core_utils/ so
bare SDK installs without proxy dependencies still work.
Pass follow_redirects through in HTTPHandler.get() — previously the
parameter was accepted but never forwarded to the underlying httpx
client, making sync redirect protection ineffective.
Include port in Host header when non-default (e.g. example.com:8080).
Fix redirect loop to read Location header directly instead of
response.next_request (which is None when follow_redirects=False).
_is_blocked_ip now returns True (blocked) for unparseable addresses
instead of False (allowed). HTTPS URLs are rewritten to validated IPs
when ssl_verify is disabled, closing the DNS rebinding window that
exists without TLS certificate binding.
Check URL scheme before calling safe_get in token counter to avoid
unnecessary DNS resolution on base64-encoded image data.
Add 14 unit tests for validate_url covering blocked networks, scheme
validation, URL rewriting, and DNS failure handling.
Add safe_get() and async_safe_get() helpers that validate each
redirect hop before following. For HTTPS, rely on TLS certificate
binding instead of URL rewriting. Simplify call sites to use the
new helpers.
Add validate_url() utility that resolves DNS once, validates all IPs
against private network ranges, and rewrites the URL to connect to the
validated IP directly. Prevents DNS rebinding by pinning to the resolved
IP. Disable follow_redirects to prevent redirect-based SSRF bypasses.
Applied to all user-supplied URL entry points:
- Image URL fetching in chat completions
- Token counter image dimension fetching
- RAG file ingestion
- MCP OpenAPI spec loading
Include user_api_key_team_metadata alongside user_api_key_metadata in
_get_admin_metadata() so team-level guardrail settings are respected.
Key-level settings take precedence over team-level.
Remove turn_off_message_logging from _supported_callback_params so it
cannot be set via request metadata. Admin controls logging globally
or via key/team configuration.
Update tests to verify user-injected guardrail flags are ignored while
admin-configured flags are respected.
Extract _get_admin_metadata() in CustomGuardrail to deduplicate metadata
lookup. Hoist tag resolution above the deployment loop in budget limiter.
Update stale comment in tag routing.
Read guardrail control flags (disable_global_guardrails, opted_out_global_guardrails)
from admin-configured key metadata instead of the request body. This ensures
callers cannot override admin security policies.
Fix tag-based routing to enforce strict tag checks regardless of whether the
request includes tags. Fix budget limiter to use the same dynamic metadata
key resolution as the tag router for consistent tag extraction.
Validate org admin role against all requested organizations instead
of returning on first match. Scope team list queries to the caller's
permitted organizations when filtering by user_id.
The antd mocks in RouterSettingsForm.test.tsx and index.test.tsx
replaced the entire antd module with only Select, so the Switch and
Button used by nested components failed to render. Use importOriginal
to preserve the rest of antd and override only Select.
Also fix the TagFilteringToggle click assertion — antd's Switch fires
onChange with (checked, event), so toHaveBeenCalledWith(true) was
always going to miss. Assert the checked arg directly instead of
coupling to antd's call signature.
