Commit graph

8516 commits

Author SHA1 Message Date
shivam
b6d5728134
add support for pages param 2026-04-16 19:07:02 -07:00
Yuneng Jiang
00bac08e01
[Test] Mock remaining live Bedrock Moonshot tests
Extends the prior moonshot mocking to cover every inherited
BaseLLMChatTest test that still made a live AWS Bedrock call. Adds
request-body assertions for each override.

New overrides:

- test_content_list_handling: verifies the outgoing body round-trips
  user content in list-of-text form; asserts response.choices[0].
  message.content parses back from the canned response.
- test_pydantic_model_input: verifies a pydantic Message input does
  not raise and produces a parseable response.
- test_response_format_type_text_with_tool_calls_no_tool_choice:
  verifies tools are forwarded and response_format + drop_params do
  not break the call.
- test_streaming: verifies stream=True routes to the
  invoke-with-response-stream endpoint. Bedrock invoke streaming is
  intercepted at the make_sync_call import site rather than via the
  caller-supplied client, because CustomStreamWrapper.fetch_sync_stream
  invokes the stored make_call partial with
  client=litellm.module_level_client, overriding any client passed by
  the caller.

Extracts a shared _make_moonshot_response helper and a
_invoke_with_mocked_post harness so all the sync mocks share one
canned response body.

After this change TestBedrockMoonshotInvoke runs 23 passed, 29
skipped, 0 live-callers, all in under 1s locally.
2026-04-16 17:43:43 -07:00
user
c7c3df2b02
fix(proxy): extend /key/update admin check to non-budget fields
Audit-B #2. _check_key_admin_access was gated on max_budget/spend
changes only, which meant a non-admin caller could blanket-rewrite
any OTHER field on any key (key_alias, models, tpm_limit, rpm_limit,
metadata, tags, allowed_routes, guardrails, blocked, duration,
permissions, auto_rotate, access_group_ids, object_permission, …)
as long as they avoided budget/spend. Example attack:

  POST /key/update {
    key: sk-victim-in-org-B,
    models: [],
    blocked: true,
    organization_id: org-A,
  }

The caller is org-admin of org-A, which satisfies the route gate;
the handler then wipes models and blocks the victim's key.

Policy after this fix:
- PROXY_ADMIN: always allowed.
- Key OWNER (matching user_id): allowed for non-budget fields;
  budget/spend changes still require team/org admin.
- Everyone else: must pass _check_key_admin_access (PROXY_ADMIN /
  key-owner / team-admin / org-admin of the key).

Regression test confirms a non-owner INTERNAL_USER cannot rewrite
key_alias/blocked on someone else's key; existing test covers the
owner-can-update-alias case; existing test covers
internal-user-cannot-modify-max-budget.
2026-04-17 00:41:00 +00:00
user
662d05531d
fix(proxy): close three more org-boundary escape paths
Continuation of Veria E3NpkuAd / Audit-B hardening. All three are the
same anti-pattern PR #25904 already addressed for _user_is_org_admin
and /user/delete: route-level gate trusts a caller-supplied scope
field, handler operates on a different scope.

1. /user/update no longer silently creates a user when the target
   email doesn't exist. Pre-fix, an org admin could supply a fresh
   email + caller-chosen budget/models/metadata; the INSERT path
   created the user with no org attachment, bypassing /user/new's
   org/team authorization. Now require PROXY_ADMIN for the create
   branch; return 404 otherwise. Also fixes /user/bulk_update because
   it dispatches through the same _update_single_user_helper.

2. /team/bulk_member_add with all_users=true restricted to PROXY_ADMIN.
   The flag pulls every user in the database into the target team,
   ignoring org scope — any team admin could use it to capture every
   user across every org into their team.

3. /team/update now verifies destination-org admin rights. When the
   request carries an organization_id that differs from the team's
   current org, an org admin of the caller's current org could
   previously relocate the team into any other org (draining their
   resources, or capturing a team they once administered). Require
   PROXY_ADMIN or org-admin of the DESTINATION org for the relocation.

Regression tests for #1 and #3; #2 covered by the existing bulk_add
suite after the gate addition.
2026-04-17 00:36:28 +00:00
Yuneng Jiang
95e1babf67
[Fix] TogetherAIConfig.get_supported_openai_params recursion
TogetherAIConfig.get_supported_openai_params called get_model_info(),
whose first line calls litellm.get_supported_openai_params() — which for
together_ai routes straight back into this method. The recursion only
terminated when Python's recursion limit was hit or when
_get_model_info_helper raised "not mapped" at the deepest level. Either
way the try/except caught it, so the bug stayed silent — but the cycle
ran ~332 deep every time, emitting hundreds of DEBUG log lines per
call. Surfaced as "infinite loop" in CI when the success_handler thread
emitted that log spam against an already-closed stderr during test
teardown.

Replace the get_model_info() call with supports_function_calling(),
which uses _get_model_info_helper directly and does not call
get_supported_openai_params. Measured drop from 332 to 2
_get_model_info_helper calls per first uncached lookup.

Also swap the test model from Qwen/Qwen3.5-9B (not in model_cost map)
back to a mapped serverless model, Qwen/Qwen2.5-7B-Instruct-Turbo. The
mapping gap is what made the recursion's tail end raise up into the
success handler during teardown in the first place.
2026-04-16 17:20:58 -07:00
user
132063289f
fix(proxy): strip root-level data['tags'] alongside metadata tags
Greptile P2. The admin-inject gate only removed tags from data['metadata']
and data['litellm_metadata'];  and the
policy engine read  directly, so a caller without
allow_client_tags could still drive tag-based policy decisions by moving
tags to the body root. Also strip the root key in the same branch.
2026-04-17 00:20:52 +00:00
user
8c0668f105
perf: batch target membership lookup in delete_user to avoid N+1
Greptile P1 on the /user/delete fix. Per CLAUDE.md 'No N+1 queries',
move the find_many inside the per-user loop to a single batched
fetch with {'user_id': {'in': data.user_ids}} before the loop, then
distribute to a per-user set in memory.
2026-04-17 00:17:57 +00:00
user
467166fdd7
fix(proxy): enforce per-target org authorization on /user/delete
Veria admin-queue finding E3NpkuAd, Audit-B #1. The route-level gate
accepts this call when the caller is PROXY_ADMIN or ORG_ADMIN of any
org named in request_data["organization_id"]/["organizations"]. The
handler processes data.user_ids without cross-checking whether those
users belong to the caller's administered orgs, so an org-admin of
org-A could delete users in org-B via:
  {"user_ids": ["victim_in_org_B"], "organization_id": "org-A"}

Add per-target authorization: org-admins may only delete users whose
entire org membership is within their admin scope; targets with any
org outside scope (or no org at all) require PROXY_ADMIN.

Regression test confirms an ORG_ADMIN call fails with 403 and no
cascade delete_many runs.
2026-04-17 00:11:00 +00:00
user
b4e98d190a
fix(proxy): close 6 more metadata/tag variant bypasses
Post-merge audit found 6 adjacent variants of the VERIA-28 class. All
fixed here with regression tests:

1. Strip widened from 3 named keys to the full user_api_key_* prefix.
   The proxy writes a dozen user_api_key_* fields (user_id, alias,
   spend, team_id, request_route, end_user_id, …) into
   data[_metadata_variable_name]; the 3-key strip left the rest
   exploitable for identity/spend forgery in audit logs and guardrails.

2. proxy_server_request['body'] snapshot moved to AFTER the strip.
   Was captured at line ~990 before the strip ran, so
   standard_logging_object, lago, and spend_tracking readers saw the
   attacker-forged payload even though the live data dict was clean.

3. get_tags_from_request_body (auth-time) now coerces JSON-string
   metadata via safe_json_loads. Previously crashed with
   AttributeError on string metadata (DoS; potential RBAC bypass if
   a caller swallowed the exception).

4. get_end_user_id_from_request_body coerces JSON-string
   metadata/litellm_metadata. Previously isinstance(dict) guard
   caused end-user budget attribution to be silently skipped when
   the caller sent metadata as a JSON string.

5. Four hand-rolled 'if data.get("metadata") is None: data["metadata"] = {}'
   blocks in proxy_server.py (7160, 7341, 7590, 11375) now guard on
   isinstance(dict). They crashed with TypeError when metadata was a
   JSON string (DoS).

6. _get_admin_metadata defensively guards with isinstance(dict);
   previously AttributeError'd on any leaked string metadata.

Also hoists the inline safe_json_loads import in _guardrail_modification_check
to module level per CLAUDE.md style.
2026-04-17 00:08:40 +00:00
Yuneng Jiang
e1da27dc0a
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/amazing-almeida
# Conflicts:
#	tests/test_litellm/llms/bedrock/messages/invoke_transformations/test_anthropic_claude3_transformation.py
2026-04-16 16:54:36 -07:00
Yuneng Jiang
35a186a133
[Test] Mock live Bedrock Moonshot tests in llm_translation
Three tests inherited by TestBedrockMoonshotInvoke from BaseLLMChatTest
make live AWS Bedrock completion calls: test_developer_role_translation,
test_message_with_name, and test_completion_cost. These have been
crashing llm_translation_testing CI workers (reported as "failed on
setup with worker 'gwN' crashed").

Replace each with a mocked override that intercepts the outgoing
request via HTTPHandler.post / AsyncHTTPHandler.post patching:

- test_developer_role_translation asserts the outgoing body maps the
  developer role to system (LiteLLM's translation for non-OpenAI
  providers).
- test_message_with_name asserts the outgoing body preserves the user
  message.
- test_completion_cost returns a canned moonshot-shaped response body
  with usage and asserts response_cost > 0 against the local model
  cost map.

Follows the existing HTTPHandler + patch.object(client, "post") pattern
used in test_bedrock_gpt_oss.py and test_bedrock_completion.py. No
network traffic; the three tests now complete in ~0.3s.
2026-04-16 16:51:49 -07:00
user
76aa97f77b
fix(proxy): close three variant metadata/tag injection paths
Close three variant bypasses adjacent to VERIA-28 found during post-fix
variant audit:

1. _guardrail_modification_check had the same isinstance(dict) bypass
   Veria-AI just flagged on the pre-call strip. A caller sending
   `{"metadata": "{…}"}` as a JSON-encoded string (multipart/form-data
   or extra_body) skipped the guard, got parsed to dict downstream, and
   reached guardrail logic with bypass flags intact. Coerce strings via
   safe_json_loads before evaluating.

2. The allow_client_tags strip only covered body metadata.tags and
   litellm_metadata.tags — caller-supplied tags arriving via the
   x-litellm-tags header or root-level data["tags"] bypassed it. Gate
   add_request_tag_to_metadata's result on the same flag.

3. requester_metadata was deepcopied BEFORE the strip, so attacker
   injections (user_api_key_metadata shadows, disallowed tags,
   _pipeline_managed_guardrails) persisted in the snapshot. The PANW
   guardrail (and any future consumer) trusting requester_metadata
   would see forged values. Move the deepcopy to after the strip.

Regression tests added for each.
2026-04-16 23:35:26 +00:00
user
dc8e03b91f
fix(proxy): expand _guardrail_modification_check to cover all bypass keys
Per VERIA-28's secondary recommendation. The existing check only gated
metadata.guardrails. User-supplied values for disable_global_guardrails
(plural and the original singular typo variant) and opted_out_global_guardrails
are already silently ignored by _get_admin_metadata at read time, but the
silent-ignore makes diagnosis confusing and relies on one specific read
site catching them.

Reject at auth time with a 403 when any of:
- guardrails list (existing)
- disable_global_guardrails (new)
- disable_global_guardrail (new — historical singular-key variant)
- opted_out_global_guardrails (new)

are present in metadata, litellm_metadata, or at the request root, and the
caller's team lacks can_modify_guardrails. Defense in depth: the strip at
the pre-call layer still runs; this check fails loudly one layer earlier
so operators see an explicit 403 rather than a silent-ignore.
2026-04-16 23:14:05 +00:00
Yuneng Jiang
f8f356fac3
test: add parametrized tests for api_key value handling in credential check 2026-04-16 16:08:47 -07:00
user
db19f24d69
fix(proxy): move metadata strip after JSON-string parse
Veria AI caught a bypass: metadata can arrive as a JSON string via
multipart/form-data or extra_body, and the existing strip block ran
before the string-to-dict parse. The isinstance(_user_meta, dict)
guard returned False on the string, the strip was skipped, and then
the parse turned the string into a dict — leaving user_api_key_metadata
/ user_api_key_team_metadata / _pipeline_managed_guardrails / tags
intact in the parsed dict.

Move the strip to run AFTER the parse and BEFORE the merge of
litellm_metadata into data[_metadata_variable_name], closing the bypass
for both raw-dict and string-encoded payloads.

Regression test: test_add_litellm_data_to_request_strips_string_encoded_admin_injection.
2026-04-16 23:00:52 +00:00
user
9622864ef2
test: set allow_client_tags on duplicate-tag merge test
test_add_litellm_data_to_request_duplicate_tags tests the request/key
tag merge when tags overlap. The merge requires caller-supplied tags to
flow through — set allow_client_tags=True on the key so the merge path
stays testable under the new default-deny regime.
2026-04-16 22:50:54 +00:00
Yuneng Jiang
5df7c21c9a
fix: extend x-pass- header protection to cover additional credential headers and add tests
- Move protected-headers set to module level as a frozenset
- Add x-api-key, x-goog-api-key to protected set (provider credential headers)
- Block x-amz- prefix to cover AWS SigV4 signing headers
- Normalize forwarded header names to lowercase on write
- Log at debug level when a protected header is skipped
- Add unit test covering protected-header drop and non-protected forwarding
2026-04-16 15:41:34 -07:00
user
8526628a8f
test: update tag-merge tests for default-deny client tag policy
Two pre-existing tests codified the pre-fix behavior where any caller-
supplied metadata.tags would flow through to spend logs and routing:

- test_add_key_or_team_level_spend_logs_metadata_to_request exercised
  the request/key/team tag merge. Set allow_client_tags=True on the key
  metadata so the merge path is still tested under the new regime.

- test_create_file_with_nested_litellm_metadata asserted that
  litellm_metadata[tags] form-data propagated to the handler. Drop the
  tag field; the test still proves nested form-parser correctness via
  spend_logs_metadata and environment.
2026-04-16 22:40:43 +00:00
user
0e62addd94
fix(proxy): gate caller-supplied routing/budget tags behind allow_client_tags
VERIA-28 (High) follow-up: tag-based routing and tag budget enforcement
read metadata.tags directly from the request, letting an attacker reach
restricted tag-routed deployments or misattribute spend to a victim
team's tag.

Strip metadata.tags (and litellm_metadata.tags) at the pre-call boundary
unless the caller's key or team metadata opts in with
allow_client_tags=True. Default-deny: existing clients that need to pass
routing tags must have the flag set explicitly on their key or team.

Preserves the tag-routing feature for admins who trust their callers;
closes the injection path for everyone else.
2026-04-16 22:31:00 +00:00
user
0602564b66
fix: switch blocklist to RFC 6890 via ipaddress.is_global, block multicast and Azure Wire Server
Replace the hand-maintained _BLOCKED_NETWORKS CIDR list with a
default-deny check based on ipaddress.is_global (RFC 6890 semantics,
implemented by Python's stdlib). Also reject multicast explicitly —
is_global returns True for public multicast allocations, which are
not legitimate HTTP targets.

Only globally-routable cloud-fabric IPs need explicit exceptions; the
canonical list contains one entry today: Azure Wire Server
(168.63.129.16), an in-fabric service reachable from any Azure VM.

Coverage delta picked up automatically via is_global:
- Alibaba Cloud metadata (100.100.100.200, CGNAT)
- Legacy Oracle metadata (192.0.0.192, IETF Protocol Assignments)
- IPv4 documentation ranges (192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24)
- IPv4 reserved/future-use (240.0.0.0/4) and broadcast
- IPv6 documentation (2001:db8::/32)

Also fix two issues Greptile flagged:
- HTTP relative-redirect hops lost the original hostname because
  _extract_redirect_url joined the Location against the rewritten
  (IP-based) URL. Join against the pre-rewrite URL so the next hop's
  Host header keeps the original hostname.
- Two unit tests performed real socket.getaddrinfo('localhost')
  calls. Monkeypatch them.

Add coverage tests for every cloud-metadata IP from the canonical
SSRF dictionary (AWS/GCP/Azure/Alibaba/Oracle/DO/OpenStack) plus the
new multicast/reserved/documentation/broadcast ranges, and a
regression test for redirect-hostname preservation.
2026-04-16 22:03:47 +00:00
user
d0601692b8
fix(proxy): strip user_api_key_metadata injection slots from user input
Expand the pre-call metadata strip to also remove user_api_key_metadata
and user_api_key_team_metadata. The proxy writes these fields into
data[_metadata_variable_name] with admin-authoritative values, but only
into that one metadata key; the caller's value in the OTHER metadata
key (metadata vs litellm_metadata) would otherwise persist and be
picked up by _get_admin_metadata, letting a caller supply their own
'admin' config to disable guardrails, opt out of global policies, etc.

VERIA-28 (High): Security Policy and Guardrail Bypass via Unsanitized
Request Metadata.

Add regression test at the proxy boundary verifying the strip, and
extend the guardrail test to cover the post-strip admin-config path.
2026-04-16 21:48:36 +00:00
Yuneng Jiang
72ff989c12
[Test] Remove dead Bedrock clear_thinking interleaved-thinking-beta assertion
Drop test_bedrock_invoke_messages_injects_thinking_for_clear_thinking_context_management.
Its assertion 'interleaved-thinking-2025-05-14' in betas cannot hold because
anthropic_beta_headers_config.json maps that header to null for the bedrock
provider, so filter_and_transform_beta_headers drops it from the auto-added
beta set before anthropic_beta is written to the request.

The adjacent test_bedrock_invoke_messages_skips_thinking_injection_when_already_enabled
already covers the inverse behavior for the same model, so no coverage is lost.
2026-04-16 14:45:38 -07:00
user
1d3dda9342
feat: add admin opt-out for user URL validation
Two litellm-level flags wired through litellm_settings YAML:

- user_url_validation (bool, default True): master switch. When False,
  safe_get/async_safe_get bypass validation and call client.get
  directly.
- user_url_allowed_hosts (List[str], default []): per-host allowlist.
  Entries are 'host' (matches any port) or 'host:port' (port-specific).
  Matched hosts skip the blocked-networks check but still resolve DNS
  and still rewrite HTTP to the validated IP, preserving rebinding
  protection within the permitted name.

Also fix an existing Host header bug: IPv6 literals (e.g. 2001:db8::1)
were emitted unbracketed, producing ambiguous values like
'2001:db8::1:8080' per RFC 7230 5.4. Bracket them consistently in
_format_host_header.
2026-04-16 21:40:19 +00:00
user
22572eafaf
fix: merge admin metadata from both metadata and litellm_metadata
Greptile P2: _get_admin_metadata used 'litellm_metadata or metadata',
meaning a caller sending a non-empty litellm_metadata would shadow
admin config the proxy had injected into data['metadata']. Admin
exemptions would be silently ignored.

Check both keys and prefer whichever contains admin fields. Add
regression test covering the shadowing scenario.
2026-04-16 21:29:13 +00:00
user
1f50c6fa66
test: mock DNS resolution, hoist httpx import to module level
Greptile P1: six tests in test_url_utils.py performed real DNS
lookups to example.com, violating the tests/test_litellm/ mock-only
rule and risking offline CI failures. Add mock_dns_public and
mock_dns_failure fixtures that monkeypatch socket.getaddrinfo on
the url_utils module.

Greptile P2: move 'import httpx' from inside _extract_redirect_url
to module-level imports per CLAUDE.md style guide.
2026-04-16 21:28:13 +00:00
user
30c6556782
test: bypass SSRF validation in image handling tests 2026-04-16 21:08:06 +00:00
user
1ba2be77ae
refactor: move url_utils to litellm_core_utils to avoid proxy dependency
SDK core modules (image_handling, token_counter) should not import
from litellm.proxy. Move url_utils.py to litellm_core_utils/ so
bare SDK installs without proxy dependencies still work.
2026-04-16 21:07:04 +00:00
user
814d03d1ce
fix: fail-closed on unparseable IPs, rewrite HTTPS when SSL verify disabled
_is_blocked_ip now returns True (blocked) for unparseable addresses
instead of False (allowed). HTTPS URLs are rewritten to validated IPs
when ssl_verify is disabled, closing the DNS rebinding window that
exists without TLS certificate binding.
2026-04-16 21:07:04 +00:00
user
62ec396775
test: mock SSRF validation in openapi spec URL test 2026-04-16 21:07:03 +00:00
user
b94aaa72b0
fix: skip DNS resolution for base64 data in token counter, add unit tests
Check URL scheme before calling safe_get in token counter to avoid
unnecessary DNS resolution on base64-encoded image data.

Add 14 unit tests for validate_url covering blocked networks, scheme
validation, URL rewriting, and DNS failure handling.
2026-04-16 21:07:03 +00:00
user
413f89892b
test: update dynamic callback params test for turn_off_message_logging removal
Verify turn_off_message_logging is no longer extracted from request
kwargs since it is now admin-only.
2026-04-16 21:07:00 +00:00
user
34e9be1ba7
fix: merge team metadata in admin helper, remove turn_off_message_logging from dynamic params
Include user_api_key_team_metadata alongside user_api_key_metadata in
_get_admin_metadata() so team-level guardrail settings are respected.
Key-level settings take precedence over team-level.

Remove turn_off_message_logging from _supported_callback_params so it
cannot be set via request metadata. Admin controls logging globally
or via key/team configuration.

Update tests to verify user-injected guardrail flags are ignored while
admin-configured flags are respected.
2026-04-16 21:06:59 +00:00
user
815a2bed1a
test: add regression tests for cross-org admin escalation
Verify that an org admin of org-A cannot operate on org-B, and that
an admin of both orgs can operate on both.
2026-04-16 21:06:56 +00:00
user
91bfbe6efe
fix(proxy): enforce organization boundaries in admin operations
Validate org admin role against all requested organizations instead
of returning on first match. Scope team list queries to the caller's
permitted organizations when filtering by user_id.
2026-04-16 21:06:56 +00:00
ishaan-berri
6fab790a6e
Merge pull request #25867 from BerriAI/litellm_day_0_opus_4.7_support
Litellm day 0 opus 4.7 support
2026-04-16 22:49:51 +05:30
yuneng-jiang
21c0718850
Merge pull request #25871 from BerriAI/litellm_yj_apr15
[Infra] Merge dev branch
2026-04-16 10:11:48 -07:00
ishaan-berri
44c992416c
Merge pull request #25867 from BerriAI/litellm_day_0_opus_4.7_support
Litellm day 0 opus 4.7 support
2026-04-16 09:42:11 -07:00
Yuneng Jiang
b26f858ab0
fix(ci): authorize langgraph-prebuilt in liccheck.ini
langgraph-prebuilt was previously pulled in as a transitive of langgraph
so PyPI license metadata was reported as unknown. Now that it is
explicitly pinned (==1.0.8) to avoid the broken 1.0.9 release, the
license checker flags it. It is published under MIT by the same
langchain-ai/langgraph repository as langgraph itself.
2026-04-16 09:41:51 -07:00
Yuneng Jiang
dafa1bf97c
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_yj_apr15
# Conflicts:
#	litellm/litellm_core_utils/litellm_logging.py
#	uv.lock
2026-04-16 09:17:20 -07:00
Sameer Kankute
b3d5ff5774
Fix tests + add docs 2026-04-16 21:45:31 +05:30
Sameer Kankute
607412defb
feat(bedrock): inject thinking for clear_thinking context_management on Messages API
Bedrock rejects clear_thinking_20251015 unless thinking is enabled or adaptive.
Inject minimal extended thinking and interleaved-thinking beta when Claude Code
sends context_management without thinking. Adds unit tests.

Made-with: Cursor
2026-04-16 21:11:09 +05:30
Sameer Kankute
4b5c86b8a1
Fix code qa 2026-04-16 19:29:08 +05:30
Tim Ren
dd4a41951f
fix(utils): allowed_openai_params must not forward unset params as None (#25777)
* feat(proxy): add NO_OPENAPI env var to disable /openapi.json endpoint (#25696)

* feat(proxy): add NO_OPENAPI env var to disable /openapi.json endpoint - Fixes #25538

* test(proxy): add tests for _get_openapi_url

---------

Co-authored-by: Progressive-engg <lov.kumari55@gmail.com>

* feat(prometheus): add api_provider label to spend metric (#25693)

* feat(prometheus): add api_provider label to spend metric

Add `api_provider` to `litellm_spend_metric` labels so users can
build Grafana dashboards that break down spend by cloud provider
(e.g. bedrock, anthropic, openai, azure, vertex_ai).

The `api_provider` label already exists in UserAPIKeyLabelValues and
is populated from `standard_logging_payload["custom_llm_provider"]`,
but was not included in the spend metric's label list.

* add api_provider to requests metric + add test

Address review feedback:
- Add api_provider to litellm_requests_metric too (same call-site as
  spend metric, keeps label sets in sync)
- Add test_api_provider_in_spend_and_requests_metrics following the
  existing pattern in test_prometheus_labels.py

* fix: ensure `litellm_metadata` is attached to `pre_call` guardrail to align with `post_call` guardrail (#25641)

* fix: ensure `litellm_metadata` is attached to pre_call to align with post_call

* refactor: remove unused BaseTranslation._ensure_litellm_metadata

* refactor: module level imports for ensure_litellm_metadata and CodeQL

* fix: update based off of Codex comment

* revert: undo usage of `_guardrail_litellm_metadata`

* feat: add pricing entry for openrouter/google/gemini-3.1-flash-lite-preview (#25610)

* fix(bedrock): skip synthetic tool injection for json_object with no schema (#25740)

When response_format={"type": "json_object"} is sent without a JSON
schema, _create_json_tool_call_for_response_format builds a tool with an
empty schema (properties: {}). The model follows the empty schema and
returns {} instead of the actual JSON the caller asked for.

This patch:
- Skips synthetic json_tool_call injection when no schema is provided.
  The model already returns JSON when the prompt asks for it.
- Fixes finish_reason: after _filter_json_mode_tools strips all
  synthetic tool calls, finish_reason stays "tool_calls" instead of
  "stop". Callers (like the OpenAI SDK) misinterpret this as a pending
  tool invocation.

json_schema requests with an explicit schema are unchanged.

Co-authored-by: Claude <noreply@anthropic.com>

* fix(utils): allowed_openai_params must not forward unset params as None

`_apply_openai_param_overrides` iterated `allowed_openai_params` and
unconditionally wrote `optional_params[param] = non_default_params.pop(param, None)`
for each entry. If the caller listed a param name but did not actually
send that param in the request, the pop returned `None` and `None` was
still written to `optional_params`. The openai SDK then rejected it as
a top-level kwarg:

    AsyncCompletions.create() got an unexpected keyword argument 'enable_thinking'

Reproducer (from #25697):

    allowed_openai_params = ["chat_template_kwargs", "enable_thinking"]
    body = {"chat_template_kwargs": {"enable_thinking": False}}

Here `enable_thinking` is only present nested inside
`chat_template_kwargs`, so the helper should forward
`chat_template_kwargs` and leave `enable_thinking` alone. Instead it
wrote `optional_params["enable_thinking"] = None`.

Fix: only forward a param if it was actually present in
`non_default_params`. Behavior is unchanged for the happy path (param
sent → still forwarded), and the explicit `None` leakage is gone.

Adds a regression test exercising the helper in isolation so the test
does not depend on any provider-specific `map_openai_params` plumbing.

Fixes #25697

---------

Co-authored-by: lovek629 <59618812+lovek629@users.noreply.github.com>
Co-authored-by: Progressive-engg <lov.kumari55@gmail.com>
Co-authored-by: Ori Kotek <ori.k@codium.ai>
Co-authored-by: Alexander Grattan <51346343+agrattan0820@users.noreply.github.com>
Co-authored-by: Mohana Siddhartha Chivukula <103447836+iamsiddhu3007@users.noreply.github.com>
Co-authored-by: Amiram Mizne <amiramm@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-04-16 19:04:26 +05:30
Brendan Smith-Elion
265a960472
fix(noma-v2): fall back to key_alias for application_id in Noma dashboard (#25795)
Noma v1 resolved application_id from user_api_key_alias when no explicit
value was set (PR #16832). Noma v2 (PR #21400) was rewritten from scratch
and this fallback was not ported, causing all requests from shared LiteLLM
instances to appear as a single generic "litellm" application in the Noma
dashboard — breaking per-user traceability.

Fix: after checking dynamic_params and self.application_id, fall back to
user_api_key_alias from litellm_metadata or metadata. This matches the
pattern used by PromptSecurityGuardrail._resolve_key_alias_from_request_data()
and restores the v1 behavior where each API key gets its own application
entry in the Noma dashboard.

Fixes #25794

Co-authored-by: Brendan Smith-Elion <brendan.smith-elion@arcadia.io>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 19:04:24 +05:30
Jared Everett
3cbb36aa13
fix(ollama): propagate done_reason='length' as finish_reason for max_tokens truncation (#25824)
* fix(ollama): propagate done_reason='length' as finish_reason for max_tokens truncation

Ollama returns done_reason='length' when a response is cut off by num_predict
(the max_tokens limit). Previously, non-streaming responses hardcoded
finish_reason='stop', and streaming used chunk.get('done_reason', 'stop')
which also defaulted to 'stop' when done_reason was absent.

This meant callers (e.g. the Anthropic pass-through adapter, which maps
OpenAI 'length' -> Anthropic 'max_tokens') could never detect truncation,
making stop_reason always appear as 'end_turn' even for cut-off responses.

Fix: read done_reason from the response JSON in the non-streaming path and
use `chunk.get('done_reason') or 'stop'` in the streaming path, so Ollama's
actual done_reason passes through to the caller unchanged.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Update test_ollama_chat_transformation.py

* Update litellm/llms/ollama/chat/transformation.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-04-16 19:03:41 +05:30
Darien Kindlund
6b2973b29a
fix(vertex): strip version suffix from model name in count_tokens requests (#25800)
The Vertex AI count-tokens endpoint rejects model names that include
version suffixes (@default, @20251001, etc.) with:
"claude-sonnet-4-6@default is not supported for token counting"

The same model without the suffix ("claude-sonnet-4-6") works correctly.

Strip @suffix from both the model parameter and request_data["model"]
in handle_count_tokens_request before sending to the API.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 19:03:40 +05:30
Sameer Kankute
3c9bf32611
fix(bedrock): prevent negative streaming costs with start-only cache usage
Bedrock /v1/messages streams can report cache tokens only on message_start while message_delta carries only uncached input tokens. Merge cache fields onto the final delta usage and clamp negative text-token remainders in cost calc to keep usage/cost consistent.

Made-with: Cursor
2026-04-16 13:08:31 +05:30
Yuneng Jiang
c8cfc5de21
fix(httpx): set response.request and strip content-encoding in MaskedHTTPStatusError
MaskedHTTPStatusError constructs a new httpx.Response from the original
error. Two bugs surfaced under real HTTP error responses:

1. The new Response was created without request=, so response.request
   raised RuntimeError("The .request property has not been set.") for
   any downstream caller (e.g. exception_mapping_utils) that inspected it.

2. The decoded response bytes were passed together with the original
   Content-Encoding header. On construction httpx tried to decompress
   the already-decoded bytes and raised httpx.DecodingError
   ("Error -3 while decompressing data: incorrect header check").

Set response.request to the masked Request and strip Content-Encoding
(and the now-stale Content-Length) before rebuilding the Response.
URL/message masking is unchanged; the new request carries the already
masked URL.

Also update test_logging_key_masking_gemini: the security commit
25f93bed91 moved Gemini API keys from ?key=... URL params to the
x-goog-api-key header, so api_base no longer contains the key.
2026-04-15 22:03:48 -07:00
Yuneng Jiang
070374d03a
fix(ci): authorize RestrictedPython in liccheck.ini
RestrictedPython (ZPL-2.1, a BSD-style permissive license) was added as
a dependency for the custom_code guardrail sandbox, but the license
checker didn't recognize it. Add to authorized packages list.
2026-04-15 21:20:40 -07:00
yuneng-jiang
be1b802501
Merge pull request #25834 from stuxf/fix/path-traversal-guardrail-yaml
fix(proxy): add shared path utilities, prevent directory traversal
2026-04-15 21:01:48 -07:00