litellm/tests/test_litellm/llms
tin-berri e50a42051c
fix(websearch): restore snippet text in native web_search_tool_result blocks (LIT-5315) (#36228)
* fix(websearch): restore snippet text in native web_search_tool_result blocks (LIT-5315)

The build_web_search_tool_result_block method copied url/title/page_age but
hardcoded encrypted_content to empty string, never reading SearchResult.snippet.
This left every native block content-free, forcing clients to web_fetch each
result to recover evidence—the reported symptom.

The Anthropic spec carries page text only in encrypted_content (an opaque
server-issued blob we cannot mint), so snippet is emitted as an additive key
alongside the spec fields. encrypted_content stays empty rather than holding
plaintext, which would assert encryption semantics that don't hold.

The anthropic SDK's BaseModel sets extra='allow', so the additive snippet key
survives SDK parsing. litellm has no typed model for web_search_result at all,
so nothing drops it internally. Turn-2 replay behavior is unaffected: the
empty encrypted_content already exists today.

Tests:
- Updated test_shape_with_results to assert snippet present
- Added test_snippet_carried_for_every_result to cover multi-result ordering
- Added test_missing_snippet_degrades_to_empty_string for edge case
- Mutation check: reverting source-only yields 3 test failures, restored to 117 passed

Fixes: LIT-5315
Co-Authored-By: Claude <noreply@anthropic.com>

* fix(websearch): make synthesized web_search blocks replayable by native clients

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(websearch): flatten a resultless replayed search block so Bedrock accepts the next turn

The flatten added for LIT-5315 bails when the replayed web_search_tool_result
carries an empty content list, but that is exactly what the interceptor emits
when a search legitimately returns nothing and when a search raises. The block
survived into the outbound body, Bedrock rejected the tag, and the conversation
died on the following turn just as it did before the flatten existed.

An empty content list has no encrypted_content to respect and no evidence to
preserve, so it flattens safely, and its paired server_tool_use goes with it.
The rendered text now says so explicitly rather than emitting a bare header.

Adds the multi-turn replay coverage that existed nowhere: the outbound Bedrock
invoke body is asserted free of both block types, parametrized over the
results-present and resultless cases, and built from the interceptor's own
builder so the fixture cannot drift from what it emits.

Resolves LIT-5320

* test(websearch): pin flatten idempotency for the agentic-loop re-entry

The agentic loop re-enters the same /v1/messages entry point for its follow-up
call and hands it the original client history, so the flatten runs again over
already-flattened messages once per iteration. Bedrock always takes that path,
since its config reports web search as natively handled and the short-circuit
is skipped.

A pass that appended the rendered text instead of replacing the block would
duplicate the evidence on every iteration and re-ship the unsupported tag, and
no existing single-pass test sees it. Mutation checked: keeping the original
block alongside the rendered text fails this test on its own.

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Yassin Kortam <yassin@berri.ai>
2026-08-07 17:28:55 -07:00
..
a2a/chat fix(a2a): populate response usage in a2a chat transformation (#31980) 2026-07-03 09:28:36 +05:30
aiml/image_generation feat(aiml): add openai/gpt-image-2 image model (#31323) 2026-06-25 16:41:43 -07:00
amazon_nova/chat style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
anthropic fix(websearch): restore snippet text in native web_search_tool_result blocks (LIT-5315) (#36228) 2026-08-07 17:28:55 -07:00
apiserpent fix(search): block server credential leak to caller-supplied api_base (#30682) 2026-06-23 13:01:21 -07:00
azure Revert "revert: "fix(caching): close evicted LLM clients so their connections are reclaimed (#35492)"" 2026-08-04 18:23:06 -07:00
azure_ai chore(tests): replace a customer name and domain with neutral placeholders 2026-07-21 15:09:15 -07:00
base_llm fix(azure_storage): honor AZURE_STORAGE_ENDPOINT_SUFFIX for sovereign clouds (#35806) 2026-08-04 16:06:41 -07:00
baseten/chat style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
bedrock fix(websearch): restore snippet text in native web_search_tool_result blocks (LIT-5315) (#36228) 2026-08-07 17:28:55 -07:00
bedrock_mantle test(pricing): cover gpt-5.6 cache-cost plumbing and bedrock_mantle responses billing 2026-07-30 21:57:20 -07:00
black_forest_labs Litellm OSS Staging 010626 (#29422) 2026-06-01 21:42:51 -07:00
bytez/chat fix: preserve safe provider model path segments 2026-04-29 22:09:54 -07:00
chat chore(oss): litellm oss staging 120626 (#30292) 2026-06-12 09:49:25 -07:00
chatgpt Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
cloudflare fix(cloudflare): route native Workers AI provider through OpenAI-compatible endpoint (#31053) 2026-06-23 12:09:11 -07:00
cohere chore: litellm oss staging (#31185) 2026-06-26 09:17:44 -07:00
cometapi/chat style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
compactifai style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
crusoe fix(crusoe): remove trailing slashes from API base URLs and fix list indentation 2026-05-01 17:27:52 +05:30
custom_httpx fix(http): stop pooled clients persisting cookies on the aiohttp jar too (#36149) 2026-08-07 11:05:59 -07:00
dashscope Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
databricks Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
datarobot style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
deepgram style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
deepinfra style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
deepseek chore: litellm oss staging (#31185) 2026-06-26 09:17:44 -07:00
docker_model_runner style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
elevenlabs test: avoid provider test module name collision 2026-04-29 22:36:57 -07:00
fal_ai/image_generation feat(fal_ai): add Nano Banana / Gemini 2.5 Flash Image generation support (#29798) 2026-06-06 11:16:44 -07:00
fastcrw/search chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
featherless_ai/chat style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
fireworks_ai fix(fireworks_ai): correct Kimi K2.5/K2.6/K2.7 max output token limits 2026-07-30 01:43:59 +00:00
gdc/chat feat(gdc): implement Google Distributed Cloud (GDC) Gemini provider (#31895) 2026-07-01 17:31:07 -07:00
gemini fix(realtime): stop second Gemini Live setup, retry hung handshake, close guardrail bypass (#31519) 2026-06-28 08:52:20 +05:30
github_copilot Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
groq fix(groq): translate web_search_options to the browser_search tool (#34971) 2026-08-04 09:08:11 -07:00
heroku style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
hosted_vllm fix(bedrock): normalize Messages system role and adaptive-thinking for Claude Invoke (#31364) 2026-06-27 11:35:36 -07:00
huggingface fix(tests): remove importlib.reload of http_handler that breaks client injection in later tests (#34336) 2026-07-23 01:11:35 +00:00
inception Litellm oss staging 030626 (#29578) 2026-06-03 11:01:51 -07:00
jina_ai/embedding fix(jina_ai): resolve the documented JINA_API_KEY as a fallback (#35992) 2026-08-05 14:13:42 -07:00
langflow feat(agents): add LangFlow agent provider with A2A session bridging (#28963) 2026-06-02 14:45:56 -07:00
lemonade Litellm OSS Staging (#29161) 2026-06-01 21:22:35 -07:00
litellm_proxy chore: simplify ownership tracking — drop thin stores, in-memory fallback, hand-rolled cache 2026-05-05 00:23:32 +00:00
llamafile/chat style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
lm_studio style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
manus fix: encode upstream URL path identifiers 2026-04-29 22:02:39 -07:00
meta_llama fix: add missing OpenAI chat completion params to OPENAI_CHAT_COMPLETION_PARAMS (#21360) 2026-02-16 20:31:21 -08:00
minimax style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
mistral feat(mistral): add mistral/mistral-ocr-2512 (OCR 3) to cost map (#31463) 2026-06-26 10:29:07 -07:00
modelscope chore(oss): litellm oss staging 150626 (#30463) 2026-06-16 12:06:41 -07:00
moonshot chore: litellm oss staging (#31185) 2026-06-26 09:17:44 -07:00
nebius test: fixes 2025-05-31 12:42:56 -07:00
neosantara Litellm oss staging 050626 (#29774) 2026-06-05 13:51:51 -07:00
novita/chat Litellm fix GitHub action testing (#11163) 2025-05-26 14:41:42 -07:00
nscale/chat Litellm fix GitHub action testing (#11163) 2025-05-26 14:41:42 -07:00
nvidia_riva feat(audio_transcription): add NVIDIA Riva STT provider (#27185) 2026-05-05 17:17:51 -07:00
oci feat: litellm oss 110626 (#30202) 2026-06-11 22:30:26 -07:00
ocr Add OCR guardrail_translation handler and support (#22145) 2026-02-28 17:39:36 -08:00
ollama Litellm OSS Staging (#29161) 2026-06-01 21:22:35 -07:00
oobabooga/chat chore(proxy): clean up request parameter validation and provider destination handling (#34189) 2026-07-22 00:57:58 +00:00
openai fix(guardrails): scan function-role results and dedupe returned tools 2026-08-06 00:52:16 -07:00
openai_like Revert "chore(ci): sync litellm_internal_staging into daily OSS branch (#33337)" (#33339) 2026-07-14 19:32:25 -07:00
openrouter chore: litellm oss staging160626 (#30527) 2026-06-16 18:23:13 -07:00
ovhcloud fix(ovhcloud): remove dead transform_response override 2026-04-28 23:09:17 +05:30
parallel_ai fix(search): block server credential leak to caller-supplied api_base (#30682) 2026-06-23 13:01:21 -07:00
parasail Litellm oss staging 080626 (#29932) 2026-06-08 13:49:52 -07:00
pass_through test: remove tests that mutation analysis proved assert nothing 2026-07-23 23:52:09 -07:00
perplexity chore: litellm oss staging (#30968) 2026-06-23 07:31:44 -07:00
pg_vector/vector_stores fix: encode upstream URL path identifiers 2026-04-29 22:02:39 -07:00
publicai style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
ragflow/chat fix: encode upstream URL path identifiers 2026-04-29 22:02:39 -07:00
recraft style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
reducto Litellm oss staging 04 21 2026 2 (#26569) 2026-05-20 21:25:19 -07:00
runwayml fix: encode upstream URL path identifiers 2026-04-29 22:02:39 -07:00
s3_vectors Fix extra body error 2026-04-29 08:34:31 +05:30
sagemaker test(sagemaker): assert make_sync_call maps non-200 to SagemakerError 2026-07-23 04:31:11 +00:00
sambanova Feat/sambanova embeddings (#13308) 2025-08-12 17:15:26 -07:00
sap style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
scaleway Feature/add audio support for scaleway (#26110) 2026-04-20 14:49:41 -07:00
snowflake chore(oss): litellm oss staging 120626 (#30292) 2026-06-12 09:49:25 -07:00
soniox chore: litellm oss staging (#30745) 2026-06-18 13:55:35 -07:00
stability style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
tencent feat(tencent): add Tencent TokenHub as a provider (#31903) 2026-07-02 18:31:59 -07:00
tinyfish feat(tinyfish): make search provider permissive, attribute errors (#31997) 2026-07-03 10:17:11 -07:00
vercel_ai_gateway style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
vertex_ai Merge pull request #35004 from mgeorgaklis/fix/gemini-thought-signature-duplication 2026-07-31 11:51:18 -07:00
volcengine chore(lint): remove litellm/types from the ruff lint exclusion 2026-08-05 01:10:15 -07:00
voyage chore(oss): litellm oss staging 120626 (#30292) 2026-06-12 09:49:25 -07:00
wandb (feat): Add W&B Inference to LiteLLM 2025-09-11 00:07:30 +05:30
watsonx fix(watsonx): wrap string embedding input in array for WatsonX API (#30897) 2026-06-20 15:50:44 -07:00
xai Litellm oss 090626 (#30021) 2026-06-10 10:34:07 -07:00
you_com Litellm oss staging 050626 (#29774) 2026-06-05 13:51:51 -07:00
zai style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_cache_control_and_reasoning.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_file_content_block.py Litellm oss staging (#28161) 2026-05-18 16:27:44 -07:00
test_file_search_responses.py fix(tests): use canonical litellm_enterprise import path (#27699) 2026-05-12 12:32:57 -07:00
test_lifecycle_fix.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_oom_fixes.py style: run black formatter on files from main merge 2026-04-17 13:02:59 -07:00
test_polling_url_origin_match.py chore(security): close two unaddressed SSRF cases 2026-05-01 18:43:47 +00:00
test_predibase_transformation.py refactor(predibase): migrate transform_request and transform_response… (#25249) 2026-04-27 08:58:22 +05:30