scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-29 06:33:28 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 267
litellm/tests/unit/llms
History
devin-ai-integration[bot] c581c85559
fix(vertex_ai): unwrap tool-emulated json_schema envelopes and send output_config.format for Claude (#45871) 
* fix(vertex_ai): unwrap tool-emulated json_schema envelopes and send output_config.format for Claude

Vertex AI Claude deployments that emulate response_format through a forced
tool sometimes nest the schema object under an extra key such as
`parameters` or `input`, and that envelope reached message.content with a
200. The converter now unwraps exactly one nested object that matches the
schema, leaving a payload that is itself schema-shaped alone.

Native structured-output rows sent the deprecated top-level `output_format`,
which Vertex rejects with a 400. The Vertex request now carries it as
`output_config.format`, with an explicit caller value winning.

* fix(anthropic): unwrap tool-emulated json_schema envelopes on streams and tool-name wrappers

* test(anthropic): expect the json tool description through a shared constant

* fix(anthropic): drop the json tool description that makes sonnet-5 omit required keys

* fix(anthropic): keep schema-accepted answers whole and decide held streams at the second key

An answer the json_schema accepts is returned as is before any envelope rule runs, the tool-name
wrapper is unwrapped only when the schema rejects it, a malformed schema turns the unwrap off, a
streamed answer whose first key is a schema property named name or tool_name waits for its second
key before streaming, the held fragments go through a StringIO, and a prefix that stays undecided
past 4 KB is held to the end

* test(anthropic): keep the stream family cells to providers whose stream iterator gets the schema

* fix(anthropic): unwrap json tool envelopes by structure instead of schema validation

The unwrap now reads only the schema's top-level property names. A lone
undeclared key holding an object, alone or beside a name equal to the json
tool's own name, is the envelope, and the unwrap repeats while the inner
object shares no key with the schema. jsonschema and referencing are no
longer imported on the request path. The stream holds only an answer that
opens with the tool's name or with an undeclared key holding an object, and a
schema without top-level properties streams as it arrives.

* fix(anthropic): walk json tool envelopes iteratively

* fix(anthropic): keep json tool arguments too deep to decode instead of raising

* fix(anthropic): unwrap json tool envelopes beside undeclared string labels

* fix(anthropic): keep json answers whole on schemas that allow extra keys and stream held text as sent

A schema that allows undeclared keys and requires none now keeps an undeclared wrapper key on both the stream and non-stream paths. When no envelope is found, the held stream text is now sent exactly as the model wrote it. On Vertex /v1/messages, output_format now wins over output_config.format when both are set, which matches the Anthropic adapter.

* test(anthropic): cover label envelopes, dynamic-key schemas, as-written held streams and output_format precedence

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-14 00:12:19 +00:00
..
a2a chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
aiml chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
aiohttp_openai refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
anthropic fix(vertex_ai): unwrap tool-emulated json_schema envelopes and send output_config.format for Claude (#45871) 2026-10-14 00:12:19 +00:00
apiserpent
aws_polly refactor(sdk): separate core AWS and tokenizer dependencies (#44447) 2026-10-08 19:07:57 -07:00
azure test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
azure_ai chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
base_llm feat(decisions): support Clef-omni audio and video (#45811) 2026-10-12 20:29:49 +00:00
baseten
bedrock fix(vertex_ai): unwrap tool-emulated json_schema envelopes and send output_config.format for Claude (#45871) 2026-10-14 00:12:19 +00:00
bedrock_mantle fix(bedrock_mantle): send max_completion_tokens for gpt-5.6 and gpt-6 chat completions (#43982) 2026-10-11 18:28:48 +00:00
black_forest_labs refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
brave chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
bytez
cerebras
chat refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
chatgpt refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
clarifai refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
claude_code refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
cloudflare test: move offline logging, secret manager and provider tests from legacy dirs into tests/unit (#45554) 2026-10-09 09:48:10 -07:00
codex refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
cohere chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
cometapi chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
compactifai
crusoe refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
custom_httpx test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
dashscope refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
databricks fix(databricks): accept service_tier on requests and carry the served tier on responses (#46111) 2026-10-12 18:22:10 -07:00
dataforseo chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
datarobot
deepagents feat: add litellm.agent() to run claude code, codex, opencode and deep agents through the ai gateway (#43885) 2026-10-01 22:27:49 +00:00
deepgram test: move the unit half of 126 mixed legacy files into tests/unit (#45090) 2026-10-07 14:07:43 -07:00
deepinfra refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
deepseek test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
docker_model_runner
duckduckgo chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
e2b refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
edenai
elevenlabs test: move the unit half of 126 mixed legacy files into tests/unit (#45090) 2026-10-07 14:07:43 -07:00
exa_ai chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
fal_ai chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
fastcrw chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
featherless_ai refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
firecrawl chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
fireworks_ai chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
gdc
gemini chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
gigachat test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
github_copilot chore(github_copilot): sync model catalog with Copilot's served models (#45386) 2026-10-09 15:19:08 -07:00
google_pse chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
gradient_ai refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
groq test: move the unit half of 126 mixed legacy files into tests/unit (#45090) 2026-10-07 14:07:43 -07:00
heroku
hosted_vllm test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
huggingface test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
hyperbolic refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
inception refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
infinity chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
jina_ai test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
lambda_ai test: move the unit half of 126 mixed legacy files into tests/unit (#45090) 2026-10-07 14:07:43 -07:00
langflow
langgraph chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
laya feat: add Bespoke Nimble gateway and OSS classifier support (#44246) 2026-10-02 19:37:08 -07:00
lemonade chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
linkup chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
litellm_proxy test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
llamafile refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
lm_studio refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
manus test: migrate SDK and proxy-booting legacy CircleCI tests to their tiers (#45758) 2026-10-11 00:17:21 -07:00
meta
meta_llama refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
microsoft_365_copilot feat(microsoft_365_copilot): add Microsoft 365 Copilot chat provider with OAuth token exchange (#45158) 2026-10-09 15:19:07 -07:00
milvus test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
minimax test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
mistral chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
modelscope refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
mongodb
moonshot
nadir
nebius
neosantara refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
nimble test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
nlp_cloud chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
novita
nscale
nvidia_nim test: move offline logging, secret manager and provider tests from legacy dirs into tests/unit (#45554) 2026-10-09 09:48:10 -07:00
nvidia_riva
oci refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
ocr
ollama chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
oobabooga
openai fix(openai): carry cache_control to prompt_cache_breakpoint on chat completions to gpt-5.6 (#46101) 2026-10-13 15:45:49 -07:00
openai_like feat(providers): add CoralBricks as a native provider (#44700) 2026-10-12 21:56:59 +00:00
opencode feat: add litellm.agent() to run claude code, codex, opencode and deep agents through the ai gateway (#43885) 2026-10-01 22:27:49 +00:00
openrouter chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
opensandbox chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
ovhcloud chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
parallel_ai
parasail refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
pass_through feat(guardrails): per-mode stream_scope with bedrock stream and pass-through fixes (#43801) 2026-10-08 13:46:53 -07:00
perplexity chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
pg_vector fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
publicai test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
ragflow test: migrate legacy unit dirs and unrun folders into tests/unit, tests/integration and tests/e2e (#45940) 2026-10-11 06:24:57 -07:00
recraft refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
reducto
replicate chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
runwayml
s3_vectors
sagemaker feat(sagemaker)!: inference components and OpenAI-compatible embeddings (#44985) 2026-10-12 22:25:52 +00:00
sail test(e2e): align completion, SAIL, and spend-log fixtures with supported contracts (#43902) 2026-09-30 14:21:22 -07:00
sambanova chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
sap fix(sap): correctly handle cache_control (#39122) 2026-10-06 08:54:06 -07:00
scaledown feat(scaledown): add native text provider support (#44168) 2026-10-10 14:01:17 -07:00
scaleway test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
searchapi chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
searxng chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
serper chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
snowflake refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
soniox refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
stability refactor(types): replace Any with proven types in 137 files (#44478) 2026-10-05 11:01:45 -07:00
strands_decider feat(strands_decider): call a Strands Decider on an AgentCore runtime ARN (#45678) 2026-10-09 22:51:19 -07:00
tavily chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
tencent
tinyfish test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
together_ai test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00
tool_loop feat(harness): add Harness.TOOL_LOOP, a minimal in-process tool-calling loop (#44391) 2026-10-03 17:33:01 +00:00
triton test: move offline logging, secret manager and provider tests from legacy dirs into tests/unit (#45554) 2026-10-09 09:48:10 -07:00
v0 test: move the unit half of 126 mixed legacy files into tests/unit (#45090) 2026-10-07 14:07:43 -07:00
valkey
vercel_ai_gateway refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
vertex_ai fix(vertex_ai): unwrap tool-emulated json_schema envelopes and send output_config.format for Claude (#45871) 2026-10-14 00:12:19 +00:00
volcengine test: migrate tests/llm_translation into unit, integration and e2e tiers (#45759) 2026-10-11 07:58:53 +00:00
voyage chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
wandb
watsonx chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
xai chore(typing): clear 670 basedpyright Any errors by validating provider payloads at the boundary (#45388) 2026-10-13 15:35:23 -07:00
xinference test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
you_com
zai
__init__.py
test_cache_control_and_reasoning.py
test_custom_llm.py test: move whole-unit legacy test files into tests/unit and delete dead skips (#44807) 2026-10-07 05:02:28 +00:00
test_file_content_block.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
test_file_search_responses.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
test_lifecycle_fix.py refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
test_oss_decision.py feat: add Bespoke Nimble gateway and OSS classifier support (#44246) 2026-10-02 19:37:08 -07:00
test_polling_url_origin_match.py
test_predibase_transformation.py test: migrate local_testing SDK and provider tests out of build_and_test (#45760) 2026-10-10 23:14:25 +00:00