Commit graph

3177 commits

Author SHA1 Message Date
moe-berri
11b60a9634 Merge remote-tracking branch 'origin/main' into litellm_memory_active_tools
# Conflicts:
#	ui/litellm-dashboard/src/lib/http/schema.d.ts
2026-09-15 12:42:24 -07:00
mateo-berri
f490172338 test(anthropic): drop docstrings and wrap a long line in the per-turn-control tests 2026-09-15 11:39:01 -07:00
mateo-berri
ce40b5773d Merge remote-tracking branch 'origin/main' into litellm_per_turn_control_beta 2026-09-15 11:11:52 -07:00
mateo-berri
c0fd8f6012 fix(anthropic): merge a case-variant Anthropic-Beta client header instead of clobbering it 2026-09-15 11:03:10 -07:00
Mateo Wang
3ed6c19b8d
Merge pull request #40915 from BerriAI/litellm_internal_copy_37075
fix(vertex-live): bill Gemini Live sessions end to end (internal copy of #37075)
2026-09-15 02:06:49 -07:00
mateo-berri
d5938ff886 Merge remote-tracking branch 'origin/main' into litellm_bedrock_wif_session_policy_coverage 2026-09-15 01:16:45 -07:00
Mateo Wang
e5cb8b7534
Merge pull request #40984 from BerriAI/litellm_anthropic_guardrail_system_and_tool_use
fix(guardrails): scan the Anthropic top-level system prompt and tool_use arguments
2026-09-15 00:55:42 -07:00
mateo-berri
4f27573424 merge: origin/main into litellm_internal_copy_37075 2026-09-15 00:34:18 -07:00
mateo-berri
92714cac0c fix(guardrails): validate tool_use rewrites before writing text rewrites back
A guardrail that rewrites text and hands back tool_use arguments that are
not a JSON object used to leave the text rewrite applied when the request
was rejected, so failure logging saw a half-rewritten request. Every
rejection now happens before any write to system or messages.
2026-09-15 00:07:45 -07:00
moe-berri
4450bfdf79 chore: merge latest main into gateway memory 2026-09-15 00:02:37 -07:00
yuneng-jiang
7e3ca1421e
Merge pull request #41194 from BerriAI/litellm_stream_tool_contract
test(e2e): verify streamed answers and tool continuation
2026-09-15 00:01:49 -07:00
Mateo Wang
1ce66e98a2
Merge pull request #40989 from BerriAI/litellm_responses_bridge_hoist_additional_tools
fix(responses): hoist Codex additional_tools input items into the chat bridge tools
2026-09-14 23:33:38 -07:00
mateo-berri
2bf44ed354 fix(guardrails): reject tool_use rewrites that are not JSON objects 2026-09-14 23:18:47 -07:00
mateo-berri
e3152c011d fix(responses): classify streamed tool calls on the chat name and strip guardrail edits around the grammar block
The streaming bridge restored the namespace before deciding whether a tool call was a custom tool, so a namespaced function sharing a short name with a nested custom tool streamed back as a custom_tool_call. Classify on the raw chat tool name first, the way the non-streaming path already does.

The guardrail merge only stripped the namespace prefix and grammar suffix from the ends of the edited description, so a guardrail appending text after the grammar block left the block in the member description and the chat conversion appended it a second time. Strip the first occurrence of each instead.
2026-09-14 22:58:25 -07:00
Yuneng Jiang
7a7770db0d
test(e2e): verify streamed answers and tool continuation 2026-09-14 22:46:55 -07:00
mateo-berri
1b594fc935 fix(guardrails): scan empty top-level system text blocks too
The hoisted structured row keeps every text block of the top-level system
prompt, empty ones included, while the scanned texts dropped the empty ones.
Guardrails that count one text per slot then came back with more texts than
the handler could place, so their rewrite was rejected. User text blocks were
already scanned empty or not; the system prompt now matches.
2026-09-14 22:24:03 -07:00
yucheng
fe0fb97fd2 fix(guardrails): record not_run when a skipped role mixes text and images
Only image-only unscoped content stays unrecorded; text or tool content
removed by scoping is recorded as not_run even when an image sits beside it.
Also keeps the type-discipline budget flat by returning the reason from the
helper and annotating the accumulator lists _extract_inputs requires.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 05:20:56 +00:00
mateo-berri
04410967ff Merge remote-tracking branch 'origin/main' into litellm_anthropic_guardrail_system_and_tool_use
main's test_one_text_per_row_over_a_system_prompt_is_rejected_by_name assumed the
top-level system prompt stays out of the scanned texts. This branch scans it, so one
text per structured row now lines up and the rewrite is applied; the test asserts that,
and a multi-block system prompt keeps the length-guard rejection covered.
2026-09-14 22:19:18 -07:00
yucheng
714b113c5f fix(guardrails): leave scoped-out image-only input unrecorded and split the not_run helper
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 05:10:32 +00:00
yucheng
9acdebf563 chore: merge origin/main into litellm_lit6314_guardrail_metadata_transfer
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 05:03:31 +00:00
mateo-berri
19dc66a48c fix(anthropic): add the per-turn-control beta when a message carries output_config 2026-09-14 22:01:47 -07:00
Mateo Wang
c16c172a65
Merge pull request #40939 from BerriAI/litellm_responses_per_message_guardrail_rewrite
fix(guardrails): write per-message guardrail rewrites back onto Responses input items
2026-09-14 21:57:29 -07:00
mateo-berri
31b34f7767 test(guardrails): type the Anthropic write-back test helper 2026-09-14 21:46:02 -07:00
mateo-berri
1111658e16 fix(bedrock): grant the AgentCore for-user invoke action in the web identity session policy
The chat and A2A AgentCore handlers send X-Amzn-Bedrock-AgentCore-Runtime-User-Id
when runtimeUserId is set, and AWS requires bedrock-agentcore:InvokeAgentRuntimeForUser
alongside InvokeAgentRuntime on that call, so the ceiling now carries both. The role
identity policy still decides whether a given role may use it

The invalid-token test now uses a neutral example audience
2026-09-14 21:31:45 -07:00
mateo-berri
ae90f1a458 fix(guardrails): type the request payload handed to the Anthropic write-back 2026-09-14 21:28:47 -07:00
kerry
7761d04450 test(vertex): cover malformed batch usage details
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 02:37:51 +00:00
mateo-berri
8b80971586 Merge remote-tracking branch 'origin/main' into litellm_anthropic_guardrail_system_and_tool_use 2026-09-14 19:27:58 -07:00
mateo-berri
0a0a9509dc test(bedrock): type the session policy test helpers with the policy TypedDict 2026-09-14 19:21:23 -07:00
mateo-berri
8d5d5d5824 fix(bedrock): grant rerank, retrieve, agent, and agentcore actions in the web identity session policy 2026-09-14 19:03:55 -07:00
mateo-berri
16c326537f fix(guardrails): define UnappliableRequestRewrite in the shared guardrail translation utils
The three guardrail translation handlers imported the exception from the proxy policy engine through a function-local import, which CodeQL flagged as a cyclic import. The exception and its helper now live next to the handlers in the shared guardrail translation utils, and the tests import it from there.

The Prompt Security modify-mode helper is also restructured into early-return TypedDict displays so the LIT002 budget stays at its limit
2026-09-14 18:39:54 -07:00
kerry
27a486e4d3 test(cost): cover modality guards and image detection fallbacks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 01:34:17 +00:00
mateo-berri
6574d83eae Merge remote-tracking branch 'origin/main' into HEAD
# Conflicts:
#	litellm/proxy/guardrails/guardrail_hooks/prompt_security/prompt_security.py
2026-09-14 18:19:38 -07:00
kerry
4a8ec7b9d8 fix(cost): bill gemini-embedding-2-preview per token like the GA entries
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 01:18:50 +00:00
kerry
6a18105275 fix(vertex): only bill image rate without modality details when every input is an image
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 01:16:00 +00:00
kerry
ac8e1a355c test(vertex): load local pricing in embedding billing tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 01:00:32 +00:00
kerry
e5845c17ff fix(vertex): bill image inputs at the image rate when usage lacks modality details
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 00:51:49 +00:00
kerry
0c91d9157c refactor(vertex): drop unused resolved_files from embed response parsing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 00:45:08 +00:00
kerry
d4f2119b03 fix(cost): bill gemini-embedding-2 per token and stop double charging audio
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 00:41:46 +00:00
moe-berri
49d0e9da59 Merge remote-tracking branch 'origin/main' into litellm_memory_active_tools 2026-09-14 17:16:55 -07:00
yucheng
5ceec4c21e fix(guardrails): cite message scoping only when an unscoped pass finds content
The not_run reason now says after message scoping only when the same messages carry text or tool calls without the skip flags applied. A request that is empty to begin with, whatever the flags, records no scannable content

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-15 00:03:02 +00:00
yucheng
52da64a45b fix(guardrails): only cite message scoping in the not_run reason when scoping is on
A request whose messages carry no scannable content at all, with no skip flag set, now records the neutral reason no scannable content instead of blaming configuration

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 23:56:32 +00:00
yucheng
0519d86346 fix(guardrails): stop labelling image-only input as a not_run scoping skip
Images without text were never dispatched to guardrails before this change, so that gap is not a message scoping skip and must not get a not_run entry

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 23:49:03 +00:00
yucheng
0d0b96ed06 Revert "refactor(guardrails): rename scoped-out evaluation status from not_run to skipped"
This reverts commit b37ce94075.
2026-09-14 23:46:09 +00:00
yucheng
bd9a87ea76 Revert "fix(guardrails): keep legacy not_run neutral and stop labelling image-only input as skipped"
This reverts commit f78dd921c9.
2026-09-14 23:46:09 +00:00
tin-berri
c626ff098b
Merge pull request #40877 from BerriAI/litellm_lit7658_cache_cost_v0_fresh
feat(proxy): predict prompt-cache costs across deployments
2026-09-14 16:25:41 -07:00
yucheng
f78dd921c9 fix(guardrails): keep legacy not_run neutral and stop labelling image-only input as skipped
Some checks failed
ai-gateway image / ai-gateway release image (push) Has been cancelled
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Usage tracking, compliance and the dashboard now treat both not_run (older spend logs) and skipped as unevaluated through a shared UNEVALUATED_GUARDRAIL_STATUSES set, so old records stop counting as passed. The skipped record is no longer written when the request carried images, since images without text were never dispatched to guardrails before this change and that gap is not a message-scoping skip

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:34:33 +00:00
yucheng
b37ce94075 refactor(guardrails): rename scoped-out evaluation status from not_run to skipped
The per-guardrail status a scoped-out evaluation records is now skipped, matching the
skip_*_in_guardrail settings that cause it. Request-level rollup still maps it to not_run
so the StandardLoggingPayload status contract is unchanged

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 22:15:53 +00:00
yassin
fbc1011d27 fix(bedrock): end the realtime session when the client disconnects instead of waiting for Nova Sonic
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 21:42:42 +00:00
yassin
b4d0f4ad26 refactor(realtime): move session ownership marker keys into constants
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 21:33:48 +00:00
yassin
d2342f06ce fix(bedrock): stamp the realtime success ownership marker when Nova Sonic spend is logged
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-14 21:13:29 +00:00