yucheng-berri 9f115be88d fix(bedrock)!: scan Claude invoke-with-response-stream passthrough output with post_call guardrails (#46524) ... * fix(bedrock): scan Claude invoke-with-response-stream passthrough frames with post_call guardrails (LIT-9618) Invoke streams of Claude models now go through the same frame-by-frame relay as converse-stream. Each chunk frame's Anthropic event is scanned by the /v1/messages stream translation, a block ends the stream with an AWS exception frame, and a held stream gets its masks written back into re-encoded chunk frames. Claude is told by its model id or, behind an application inference profile, by the anthropic_version its request body carries. Streams of other model families stay as they were * test(bedrock): cover invoke stream vendor outage, upstream exception and release on scan (LIT-9618) * fix(bedrock): hold Claude invoke streams for guardrails the relay cannot run and mark each scan as streaming (LIT-9618) A guardrail released on scan or run in parallel left a Claude invoke-with-response-stream unscanned, while converse-stream holds the whole stream and scans it once. Claude invoke streams now take the same buffered path, with the rewrites written back into the chunk frames that carried each text. Each invoke scan also sends stream true and the events relayed so far as responses, as /v1/messages does, so a vendor can tie the scans of one streamed reply together. * refactor(bedrock): mark the request dicts the buffered stream paths hand to post_call hooks (LIT-9618) * fix(bedrock): scan a Claude invoke stream the upstream breaks off as a finished stream (LIT-9618) An invoke stream that ended on an upstream exception, without a message_delta stop_reason, was scanned as still in flight, so its tool input never reached the guardrail and a held stream released it unscanned. The scan and its scan key now close such a stream the way converse-stream treats the end of its upstream. * test(bedrock): pass the converse-stream endpoint where the tool call without input test frames its stream (LIT-9618) * test(bedrock): expect the ValidationException main now returns when a guardrail blocks a Claude invoke stream before its first frame (LIT-9618) * test(bedrock): type the Claude invoke event helpers in the passthrough guardrail tests * test(bedrock): type the remaining parameters of the Claude invoke stream tests * refactor(bedrock): bind the buffered invoke rewrite in its condition --------- Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>		2026-10-17 01:17:19 -07:00
..
a2a	chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)	2026-10-13 15:35:23 -07:00
aiml	chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)	2026-10-13 15:35:23 -07:00
aiohttp_openai	refactor(types): replace Any with proven types in 137 files (#44478)	2026-10-05 11:01:45 -07:00
anthropic	fix(caching): keep prompt cache usage on streamed cache hits (#47073)	2026-10-16 23:28:11 -07:00
apiserpent	test(unit): add package markers to migrated unit test directories	2026-09-20 11:53:14 +00:00
aws_polly	refactor(sdk): separate core AWS and tokenizer dependencies (#44447)	2026-10-08 19:07:57 -07:00
azure	fix(openai): send chat completions through client.post so the SDK's per-message param transform stops holding the event loop (#46967)	2026-10-16 18:06:01 -07:00
azure_ai	fix(azure_ai): count Grok reasoning tokens in completion_tokens (#46969)	2026-10-16 16:54:38 -07:00
base_llm	feat: add Nace Drex decisions provider (#46523)	2026-10-16 18:07:50 -07:00
baseten	test(unit): make every tests/unit directory a package so pytest collection is unique	2026-09-20 11:50:59 +00:00
bedrock	fix(bedrock)!: scan Claude invoke-with-response-stream passthrough output with post_call guardrails (#46524)	2026-10-17 01:17:19 -07:00
bedrock_mantle	fix(bedrock): drop dangerous-tool-use beta without safeguards (#44471)	2026-10-16 21:00:37 +00:00
black_forest_labs	refactor(types): replace Any with proven types in 137 files (#44478)	2026-10-05 11:01:45 -07:00
brave	chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)	2026-10-13 15:35:23 -07:00
bytez	test(unit): make every tests/unit directory a package so pytest collection is unique	2026-09-20 11:50:59 +00:00
cerebras	test(unit): make every tests/unit directory a package so pytest collection is unique	2026-09-20 11:50:59 +00:00
chat	fix(router): read the first stream chunk before returning so a failure before any token runs through the retry and fallback budget (#46493)	2026-10-14 14:19:25 -07:00
chatgpt	fix(responses): keep the provider code and message on top-level stream error events (#46438)	2026-10-14 16:16:46 -07:00
clarifai	refactor(types): replace Any with proven types in 137 files (#44478)	2026-10-05 11:01:45 -07:00
claude_code	refactor(types): replace Any with proven types in 137 files (#44478)	2026-10-05 11:01:45 -07:00
cloudflare	test: move offline logging, secret manager and provider tests from legacy dirs into tests/unit (#45554)	2026-10-09 09:48:10 -07:00
codex	refactor(types): replace Any with proven types in 137 files (#44478)	2026-10-05 11:01:45 -07:00
cohere	chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)	2026-10-13 15:35:23 -07:00
cometapi	chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)	2026-10-13 15:35:23 -07:00
compactifai	test: move tests/test_litellm/llms into tests/unit/llms (#43191)	2026-09-25 12:43:23 -07:00
crusoe	fix(crusoe): update base URL to current endpoint (#33121)	2026-10-15 19:10:58 -07:00
custom_httpx	fix(anthropic): keep signed empty thinking blocks and honor thinking.block_binding (#46342)	2026-10-15 18:49:53 +00:00
dashscope	feat(volcengine): add Doubao-Seedance video generation via the OpenAI /v1/videos surface (#46156)	2026-10-16 13:52:15 +00:00
databricks	feat(router): mid-stream fallback continuation for chat completions (#41127)	2026-10-16 13:51:32 +00:00
dataforseo	chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)	2026-10-13 15:35:23 -07:00
datarobot	test(unit): make every tests/unit directory a package so pytest collection is unique	2026-09-20 11:50:59 +00:00
deepagents	feat: add litellm.agent() to run claude code, codex, opencode and deep agents through the ai gateway (#43885)	2026-10-01 22:27:49 +00:00
deepgram	test: move the unit half of 126 mixed legacy files into tests/unit (#45090)	2026-10-07 14:07:43 -07:00
deepinfra	refactor(llms): expose public names for private provider helpers (#45037)	2026-10-07 08:49:25 -07:00
deepseek	fix(llms): sanitize trailing slashes in provider get_complete_url (#42887)	2026-10-15 02:59:46 -07:00
docker_model_runner	test(unit): make every tests/unit directory a package so pytest collection is unique	2026-09-20 11:50:59 +00:00
duckduckgo	chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)	2026-10-13 15:35:23 -07:00
e2b	refactor(types): replace Any with proven types in 137 files (#44478)	2026-10-05 11:01:45 -07:00
edenai	test: move tests/test_litellm/llms into tests/unit/llms (#43191)	2026-09-25 12:43:23 -07:00
elevenlabs	test: move the unit half of 126 mixed legacy files into tests/unit (#45090)	2026-10-07 14:07:43 -07:00
exa_ai	fix(llms): sanitize trailing slashes in provider get_complete_url (#42887)	2026-10-15 02:59:46 -07:00
fal_ai	chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)	2026-10-13 15:35:23 -07:00
fastcrw	fix(llms): sanitize trailing slashes in provider get_complete_url (#42887)	2026-10-15 02:59:46 -07:00
featherless_ai	refactor(llms): expose public names for private provider helpers (#45037)	2026-10-07 08:49:25 -07:00
firecrawl	fix(llms): sanitize trailing slashes in provider get_complete_url (#42887)	2026-10-15 02:59:46 -07:00
fireworks_ai	fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444)	2026-10-14 21:40:14 -07:00
gdc	test: move tests/test_litellm/llms into tests/unit/llms (#43191)	2026-09-25 12:43:23 -07:00
gemini	fix(realtime)!: keep turn_detection.type when a transcript guardrail disables auto-response (#44939)	2026-10-16 15:24:40 +00:00
gigachat	fix(llms): sanitize trailing slashes in provider get_complete_url (#42887)	2026-10-15 02:59:46 -07:00
github_copilot	chore(github_copilot): sync model catalog with Copilot's served models (#45386)	2026-10-09 15:19:08 -07:00
google_pse	chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)	2026-10-13 15:35:23 -07:00
gradient_ai	fix(llms): sanitize trailing slashes in provider get_complete_url (#42887)	2026-10-15 02:59:46 -07:00
groq	fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444)	2026-10-14 21:40:14 -07:00
heroku	fix(llms): sanitize trailing slashes in provider get_complete_url (#42887)	2026-10-15 02:59:46 -07:00
hosted_vllm	test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759)	2026-10-11 07:58:53 +00:00
huggingface	test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759)	2026-10-11 07:58:53 +00:00
hyperbolic	refactor(llms): expose public names for private provider helpers (#45037)	2026-10-07 08:49:25 -07:00
inception	refactor(llms): expose public names for private provider helpers (#45037)	2026-10-07 08:49:25 -07:00
infinity	chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)	2026-10-13 15:35:23 -07:00
jina_ai	fix(llms): sanitize trailing slashes in provider get_complete_url (#42887)	2026-10-15 02:59:46 -07:00
lambda_ai	test: move the unit half of 126 mixed legacy files into tests/unit (#45090)	2026-10-07 14:07:43 -07:00
langflow	fix(ci): stop stale CI reds, keep unit tests off the host env, retry CyberArk policy conflicts (#43294)	2026-09-26 09:25:13 -07:00
langgraph	chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)	2026-10-13 15:35:23 -07:00
laya	feat: add Bespoke Nimble gateway and OSS classifier support (#44246)	2026-10-02 19:37:08 -07:00
lemonade	chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)	2026-10-13 15:35:23 -07:00
linkup	fix(llms): sanitize trailing slashes in provider get_complete_url (#42887)	2026-10-15 02:59:46 -07:00
litellm_proxy	fix(openai): send chat completions through client.post so the SDK's per-message param transform stops holding the event loop (#46967)	2026-10-16 18:06:01 -07:00
llamafile	refactor(llms): expose public names for private provider helpers (#45037)	2026-10-07 08:49:25 -07:00
lm_studio	refactor(llms): expose public names for private provider helpers (#45037)	2026-10-07 08:49:25 -07:00
manus	test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758)	2026-10-11 00:17:21 -07:00
meta	fix(realtime)!: keep turn_detection.type when a transcript guardrail disables auto-response (#44939)	2026-10-16 15:24:40 +00:00
meta_llama	refactor(llms): expose public names for private provider helpers (#45037)	2026-10-07 08:49:25 -07:00
microsoft_365_copilot	feat(microsoft_365_copilot): add Microsoft 365 Copilot chat provider with OAuth token exchange (#45158)	2026-10-09 15:19:07 -07:00
milvus	test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940)	2026-10-11 06:24:57 -07:00
minimax	feat(minimax): add MiniMax-H3 and H3-Max video generation via /v1/videos (#43705)	2026-10-15 05:48:36 -07:00
mistral	fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444)	2026-10-14 21:40:14 -07:00
modelscope	fix(llms): sanitize trailing slashes in provider get_complete_url (#42887)	2026-10-15 02:59:46 -07:00
mongodb	test(unit): make every tests/unit directory a package so pytest collection is unique	2026-09-20 11:50:59 +00:00
moonshot	fix(llms): sanitize trailing slashes in provider get_complete_url (#42887)	2026-10-15 02:59:46 -07:00
nadir	test: move tests/test_litellm/llms into tests/unit/llms (#43191)	2026-09-25 12:43:23 -07:00
nebius	test: move tests/test_litellm/llms into tests/unit/llms (#43191)	2026-09-25 12:43:23 -07:00
neosantara	refactor(llms): expose public names for private provider helpers (#45037)	2026-10-07 08:49:25 -07:00
nimble	test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807)	2026-10-07 05:02:28 +00:00
nlp_cloud	chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)	2026-10-13 15:35:23 -07:00
novita	test(unit): make every tests/unit directory a package so pytest collection is unique	2026-09-20 11:50:59 +00:00
nscale	test(unit): make every tests/unit directory a package so pytest collection is unique	2026-09-20 11:50:59 +00:00
nvidia_nim	fix(openai): send chat completions through client.post so the SDK's per-message param transform stops holding the event loop (#46967)	2026-10-16 18:06:01 -07:00
nvidia_riva	test(unit): make every tests/unit directory a package so pytest collection is unique	2026-09-20 11:50:59 +00:00
oci	fix(oci): drop non-function tools under drop_params and gate reasoning_effort on declared levels (#40732)	2026-10-15 22:20:02 +00:00
ocr	test(unit): make every tests/unit directory a package so pytest collection is unique	2026-09-20 11:50:59 +00:00
ollama	fix(llms): sanitize trailing slashes in provider get_complete_url (#42887)	2026-10-15 02:59:46 -07:00
oobabooga	test(unit): make every tests/unit directory a package so pytest collection is unique	2026-09-20 11:50:59 +00:00
openai	fix(openai): send chat completions through client.post so the SDK's per-message param transform stops holding the event loop (#46967)	2026-10-16 18:06:01 -07:00
openai_like	fix(openai_like)!: surface provider response headers on non-streaming chat completions (#46928)	2026-10-16 17:25:50 -07:00
opencode	feat: add litellm.agent() to run claude code, codex, opencode and deep agents through the ai gateway (#43885)	2026-10-01 22:27:49 +00:00
openrouter	chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)	2026-10-13 15:35:23 -07:00
opensandbox	chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)	2026-10-13 15:35:23 -07:00
ovhcloud	chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)	2026-10-13 15:35:23 -07:00
parallel_ai	test: add __init__.py to phase 12 unit test directories	2026-09-20 12:36:15 +00:00
parasail	refactor(llms): expose public names for private provider helpers (#45037)	2026-10-07 08:49:25 -07:00
pass_through	fix(bedrock)!: scan Claude invoke-with-response-stream passthrough output with post_call guardrails (#46524)	2026-10-17 01:17:19 -07:00
perplexity	fix(openai): send chat completions through client.post so the SDK's per-message param transform stops holding the event loop (#46967)	2026-10-16 18:06:01 -07:00
pg_vector	fix(security): remove the publicly known master key from the repo (#44718)	2026-10-06 10:55:24 -07:00
publicai	test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759)	2026-10-11 07:58:53 +00:00
ragflow	test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940)	2026-10-11 06:24:57 -07:00
recraft	refactor(types): replace Any with proven types in 137 files (#44478)	2026-10-05 11:01:45 -07:00
reducto	test: move tests/test_litellm/llms into tests/unit/llms (#43191)	2026-09-25 12:43:23 -07:00
replicate	chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)	2026-10-13 15:35:23 -07:00
runwayml	test: add __init__.py to phase 12 unit test directories	2026-09-20 12:36:15 +00:00
s3_vectors	test: add __init__.py to intermediate phase 12 unit test directories	2026-09-20 12:37:21 +00:00
sagemaker	feat(sagemaker)!: inference components and OpenAI-compatible embeddings (#44985)	2026-10-12 22:25:52 +00:00
sail	test(e2e): align completion, SAIL, and spend-log fixtures with supported contracts (#43902)	2026-09-30 14:21:22 -07:00
sambanova	chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)	2026-10-13 15:35:23 -07:00
sap	fix(sap): correctly handle cache_control (#39122)	2026-10-06 08:54:06 -07:00
scaledown	feat(scaledown): add native text provider support (#44168)	2026-10-10 14:01:17 -07:00
scaleway	test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759)	2026-10-11 07:58:53 +00:00
searchapi	chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)	2026-10-13 15:35:23 -07:00
searxng	chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)	2026-10-13 15:35:23 -07:00
serper	chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)	2026-10-13 15:35:23 -07:00
snowflake	fix(bedrock)!: keep reasoning, thinking blocks and the json_mode finish_reason on fake streams (#46863)	2026-10-16 17:27:57 +00:00
soniox	feat(soniox): decode proxy form fields for context, translation and flags (#40605)	2026-10-16 06:37:34 +00:00
stability	refactor(types): replace Any with proven types in 137 files (#44478)	2026-10-05 11:01:45 -07:00
strands_decider	feat(strands_decider): call a Strands Decider on an AgentCore runtime ARN (#45678)	2026-10-09 22:51:19 -07:00
tavily	fix(llms): sanitize trailing slashes in provider get_complete_url (#42887)	2026-10-15 02:59:46 -07:00
tencent	refactor(anthropic): rename experimental_pass_through to pass_through (#43329)	2026-09-26 13:00:50 -07:00
tinyfish	test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807)	2026-10-07 05:02:28 +00:00
together_ai	fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns (#46444)	2026-10-14 21:40:14 -07:00
tool_loop	feat(harness): add Harness.TOOL_LOOP, a minimal in-process tool-calling loop (#44391)	2026-10-03 17:33:01 +00:00
topaz	fix(llms): sanitize trailing slashes in provider get_complete_url (#42887)	2026-10-15 02:59:46 -07:00
triton	test: move offline logging, secret manager and provider tests from legacy dirs into tests/unit (#45554)	2026-10-09 09:48:10 -07:00
v0	test: move the unit half of 126 mixed legacy files into tests/unit (#45090)	2026-10-07 14:07:43 -07:00
valkey	test: migrate legacy provider tests to tests/unit (wave 2, phase 13)	2026-09-20 14:18:49 +00:00
vercel_ai_gateway	refactor(llms): expose public names for private provider helpers (#45037)	2026-10-07 08:49:25 -07:00
vertex_ai	fix(count_tokens): apply context_management edits in the Gemini /v1/messages/count_tokens count (#46843)	2026-10-16 17:53:06 +00:00
vllm	fix(vllm): write spend log rows for /vllm passthrough calls to router models (#45733)	2026-10-14 12:40:48 -07:00
volcengine	fix(openai): send chat completions through client.post so the SDK's per-message param transform stops holding the event loop (#46967)	2026-10-16 18:06:01 -07:00
voyage	fix(llms): sanitize trailing slashes in provider get_complete_url (#42887)	2026-10-15 02:59:46 -07:00
wandb	test: move tests/test_litellm/llms into tests/unit/llms (#43191)	2026-09-25 12:43:23 -07:00
watsonx	chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388)	2026-10-13 15:35:23 -07:00
xai	fix: preserve explicit provider for async image, audio, and realtime routes (#36346)	2026-10-15 05:21:43 -07:00
xinference	test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807)	2026-10-07 05:02:28 +00:00
you_com	test: migrate phase 14 wave 2 provider tests to tests/unit	2026-09-20 11:01:40 +00:00
zai	test: add __init__.py to every tests/unit directory this migration touches	2026-09-20 11:52:09 +00:00
__init__.py	test(unit): add package markers to migrated unit test directories	2026-09-20 11:53:14 +00:00
test_cache_control_and_reasoning.py	test: move tests/test_litellm/llms into tests/unit/llms (#43191)	2026-09-25 12:43:23 -07:00
test_custom_llm.py	test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807)	2026-10-07 05:02:28 +00:00
test_file_content_block.py	refactor(llms): expose public names for private provider helpers (#45037)	2026-10-07 08:49:25 -07:00
test_file_search_responses.py	refactor: expose core private helpers under public names (#44871)	2026-10-07 00:05:27 +00:00
test_lifecycle_fix.py	refactor(llms): expose public names for private provider helpers (#45037)	2026-10-07 08:49:25 -07:00
test_oss_decision.py	feat(decisions): add Laya and Bespoke providers to the Add Model form (#45260)	2026-10-14 08:11:11 -07:00
test_polling_url_origin_match.py	test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit (#43199)	2026-09-25 17:10:13 -07:00
test_predibase_transformation.py	fix(prompt-templates): treat initial_prompt_value and final_prompt_value as optional (#39762)	2026-10-16 10:30:59 +00:00