Commit graph

18921 commits

Author SHA1 Message Date
mateo-berri
61fcfd986d fix(anthropic): map the dangerous-tool-use beta for Bedrock Mantle so safeguards never reach it without the beta 2026-09-21 13:24:52 -07:00
mateo-berri
b0651d52ec Merge remote-tracking branch 'origin/main' into litellm_safeguards_bedrock_vertex_messages 2026-09-21 13:15:33 -07:00
kerry-berri
246a6ea54a
Merge pull request #42282 from BerriAI/litellm_fal_price_from_response_dims
fix(fal_ai): price images from the dimensions fal returns
2026-09-21 13:08:49 -07:00
yujonglee
18f77e96b5
Merge pull request #42196 from BerriAI/litellm_cache_static_dispatch
feat(rust): scaffold cache foundation for Python parity
2026-09-21 13:05:41 -07:00
yucheng-berri
42519a7680
Merge pull request #42262 from BerriAI/litellm_bedrock_batch_s3_bucket_owner
* fix(bedrock): send s3BucketOwner on batch input and output data config

Resolve s3_bucket_owner from litellm_params, then optional_params, then
AWS_S3_BUCKET_OWNER and emit it on both S3 data configs so cross-account
batch buckets pass Bedrock ownership validation. Omitted when unset

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(bedrock): build batch output config with explicit returns

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 12:59:08 -07:00
kerry
b53f9ad658 fix(fal_ai): validate returned image dimensions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 19:58:59 +00:00
joshua-berri
3549143bcd
Merge pull request #34919 from BerriAI/litellm_fix_mcp_peek_utf8_boundary
fix(mcp): handle split UTF-8 routing previews
2026-09-21 19:56:50 +00:00
yucheng-berri
884407dad4
Merge pull request #42119 from BerriAI/litellm_default_policy_attachments
* feat(policy_engine): add default fallback policy attachments

A policy attachment with default: true applies only when no non-default
attachment matches the request, so an opt-in guardrail policy replaces the
fallback one instead of running alongside it. Supported in config.yaml,
/policies/attachments, the Admin UI Attachments tab and the resolver
(matched_via is prefixed with default:).

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(policy_engine): ignore inapplicable non-default attachments when selecting defaults

A non-default attachment whose policy is missing or whose condition does not match the request
no longer suppresses default attachments. The impact preview marks default counts as an upper bound

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(policy_engine): accept any sequence of policy names in condition matching

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(policy_engine): resolve policies once and apply fallback semantics in get_matching_policies

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 12:55:42 -07:00
mateo-berri
21b5e2d8a1 fix(anthropic): add the dangerous-tool-use beta when safeguards arrive without it on Bedrock Invoke and Vertex 2026-09-21 12:52:38 -07:00
Mateo Wang
2e35ae1065
Merge pull request #42049 from BerriAI/litellm_mantle_native_anthropic_messages
feat(bedrock_mantle): serve /v1/messages for Claude models on Mantle's native Anthropic Messages API
2026-09-21 12:52:16 -07:00
tin-berri
457b01e96d
Merge pull request #42055 from BerriAI/litellm_prompt_caching_request_table
feat(ui): show prompt caching requests and net savings
2026-09-21 12:45:11 -07:00
Mateo Wang
e8d97d381a
Merge pull request #42080 from BerriAI/litellm_prompt_caching_affinity_lookback
fix(router): keep prompt caching affinity when the breakpoint moves
2026-09-21 12:44:34 -07:00
joshua-berri
211ff96943
Merge pull request #39189 from BerriAI/litellm_mcp_list_pagination_lit5594
fix(mcp): paginate prompt and resource discovery
2026-09-21 19:35:17 +00:00
Mateo Wang
0e7cf5113e
Merge pull request #42069 from BerriAI/litellm_redacted_thinking_prompt_caching_pin
fix(token_counter): count replayed redacted_thinking blocks so prompt_caching keeps pinning
2026-09-21 12:35:14 -07:00
Yujong Lee
efcafa7f12 fix(rust): invalidate changed Redis pool settings 2026-09-21 12:30:17 -07:00
mateo-berri
deca6aea79 test(bedrock): assert safeguard_results survive message_start and check the beta by membership 2026-09-21 12:30:11 -07:00
Mateo Wang
d8267d507d
Merge pull request #41956 from BerriAI/litellm_explicit_cache_injection_points_survive_client_marks
fix: apply configured cache_control_injection_points beside client cache_control marks
2026-09-21 12:28:02 -07:00
Joshua Valluru
a835e75620 fix(mcp): handle split UTF-8 routing previews in place 2026-09-21 12:24:19 -07:00
mateo-berri
0e16050100 fix(anthropic): forward Claude Code safeguards and dangerous-tool-use beta to Bedrock Invoke and Vertex on /v1/messages
Claude Code's server-side auto-mode classifier sends a `safeguards` body field
together with the `dangerous-tool-use-2026-09-03` beta. PR #42152 made the
first-party anthropic route pass them through, but the beta header mapping
left the other two Claude platforms at null, so Bedrock Invoke dropped both
(classifier silently disabled) and Vertex forwarded the body field without
the beta, which the platform rejects with "safeguards: Extra inputs are not
permitted" (a 400 Claude Code hides by retrying without them).

Map the beta for bedrock and vertex_ai in the beta headers config and add
`safeguards` to the Bedrock Invoke request allowlist so the pair reaches
both platforms unchanged. Nothing is injected: a client that sends
`safeguards` without the beta still gets the platform's 400, exactly as
api.anthropic.com answers it.
2026-09-21 12:21:50 -07:00
Mateo Wang
e7e4df9098
Merge pull request #42041 from BerriAI/litellm_azure_ai_gpt5_tools_responses_bridge
fix(azure_ai): bridge gpt-5.4+ function tools with reasoning to the Foundry Responses API
2026-09-21 12:21:02 -07:00
Joshua Valluru
bf5dff8986 chore: sync MCP UTF-8 fix with main 2026-09-21 12:15:57 -07:00
kerry
9b54c4b077 refactor(fal_ai): bill flux dev per 1024x1024 megapixel and drop ImageResponse retyping
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 19:10:33 +00:00
kerry
adc4e6a132 fix(fal_ai): price images from the dimensions fal returns
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 19:10:32 +00:00
Mateo Wang
9a90adad32
Merge pull request #42275 from BerriAI/litellm_claude_platform_messages_beta_passthrough
fix(bedrock): forward anthropic-beta headers verbatim on the Claude platform messages path
2026-09-21 12:01:39 -07:00
Yassin Kortam
5673f67727
Merge pull request #42022 from BerriAI/litellm_redis_durable_spend_log_buffer
fix(proxy): park requeued spend logs in Redis so they survive a pod restart during a DB outage
2026-09-21 14:01:34 -05:00
mateo-berri
2fe5c8990e fix(bedrock_mantle): bill Mantle's un-versioned Claude ids from a Mantle cost row
Mantle serves anthropic.claude-haiku-4-5 without the dated -20251001-v1:0
suffix the Bedrock row carries, so the native route billed it at 0. Add a
bedrock_mantle/anthropic.claude-haiku-4-5 row and let a
bedrock_mantle/<region>/<model> name fall back to the region-free
bedrock_mantle/<model> row before the provider-prefixed lookup. Also
satisfy the mutable-collection gate in the native messages transformation.
2026-09-21 12:00:38 -07:00
mateo-berri
4eed951e6f Merge commit '36b8be7d81b' into litellm_mantle_native_anthropic_messages_b4dc
# Conflicts:
#	tests/test_litellm/llms/anthropic/experimental_pass_through/messages/test_anthropic_experimental_pass_through_messages_handler.py
2026-09-21 12:00:37 -07:00
mateo-berri
5ea4fe620f test(router): give each prompt caching check test a fresh callback registry 2026-09-21 11:55:58 -07:00
moe-berri
a83773cfa5
Merge pull request #41886 from BerriAI/litellm_jev_autorouter_launch_1789767495
feat(auto-router): add JEV classifier alongside LLM classifier
2026-09-21 11:55:44 -07:00
mateo-berri
f549c87091 Merge branch 'main' into litellm_prompt_caching_affinity_lookback 2026-09-21 11:55:40 -07:00
Mateo Wang
7cb884cd31
Merge pull request #42120 from BerriAI/litellm_cadence_58065d4_google_genai_proxy_master_key
test(google): boot the unified Google proxy fixture with a real master key
2026-09-21 11:54:03 -07:00
kerry-berri
12379aa1e3
Merge pull request #42095 from BerriAI/litellm_fal_gpt_image_25_flux_dev_edits
feat(fal_ai): add gpt-image-2.5 flare/sunburst, flux/dev and image edits
2026-09-21 11:45:45 -07:00
mateo-berri
e51ccbc759 fix(bedrock): forward anthropic-beta headers verbatim on the Claude platform messages path 2026-09-21 11:39:26 -07:00
mateo-berri
f567fe230e fix: reserve cap slots for direct marks on /v1/messages when extra_body unmarks them 2026-09-21 11:39:25 -07:00
Mateo Wang
4b2e96a5f5
Merge pull request #42143 from BerriAI/litellm_e2e_changed_keep_pytest_log
ci(e2e): fix the stage-mirror batch reds and keep a redacted pytest log
2026-09-21 11:39:25 -07:00
Joshua Valluru
1499d84f5a fix(mcp): paginate optional discovery lists 2026-09-21 11:37:55 -07:00
Moe Khalil
83ec5d6101 fix(auto-router): skip JEV for encrypted delegated tasks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 18:29:21 +00:00
mateo-berri
79d1af9d3c chore: merge origin/main to pick up the pre-call check test move 2026-09-21 11:28:04 -07:00
Yujong Lee
1b6b704ddd refactor(cache): keep native foundation isolated 2026-09-21 11:15:44 -07:00
mateo-berri
c1ba76154e fix(litellm): pass a flat Responses-style function tool through the chat bridge unchanged 2026-09-21 11:14:59 -07:00
ryan-crabbe-berri
cc1a3157d3
Merge pull request #42121 from BerriAI/litellm_utils_model_info_lookup
feat(proxy): add GET /utils/model_info to look up cost map info for unregistered models
2026-09-21 10:42:28 -07:00
kerry
052d93d6dd test(integration): assert full fal image payloads and use existing catalog rows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 17:42:02 +00:00
kerry
7510697355 test(integration): fal image generation and edit wire contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 17:39:53 +00:00
yucheng
e647255909 fix(policy_engine): resolve policies once and apply fallback semantics in get_matching_policies
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 17:36:03 +00:00
ryan-crabbe-berri
5573265013
Merge pull request #41906 from BerriAI/litellm_team_member_budget_source_reset
feat(team): show whether a member follows the team default budget and allow resetting to it
2026-09-21 10:34:08 -07:00
yucheng
d56479767e fix(bedrock): send s3BucketOwner on batch input and output data config
Resolve s3_bucket_owner from litellm_params, then optional_params, then
AWS_S3_BUCKET_OWNER and emit it on both S3 data configs so cross-account
batch buckets pass Bedrock ownership validation. Omitted when unset

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 17:27:12 +00:00
Moe Khalil
bb46e8b774 chore(auto-router): merge main and preserve JEV configuration
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-21 17:16:18 +00:00
yucheng-berri
51ae7ab1fd
Merge pull request #41588 from BerriAI/litellm_ui_session_personal_key_budget_ceiling
* fix(key_generate): use user's budget for UI session personal keys

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* Update litellm/proxy/management_endpoints/key_management_endpoints.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* fix(user_update): evict cached user on max_budget change so the personal key ceiling refreshes on every worker

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: jesus-berri <jesus@berri.ai>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: yucheng <yucheng@berri.ai>
2026-09-21 10:11:29 -07:00
tin-berri
fe34fe6047
Merge pull request #42252 from BerriAI/litellm_heuristic_v2_success_threshold
feat(auto-router): configure heuristic v2 success threshold
2026-09-21 10:06:09 -07:00
yuneng-jiang
da76ba83ad
Merge pull request #42105 from BerriAI/litellm_flip_v2_migration_resolver_default
feat(proxy): default to the v2 migration resolver
2026-09-21 10:05:59 -07:00