litellm/litellm
Mateo Wang be6d082bdc
fix(anthropic)!: send /v1/messages PDF documents to OpenAI-shaped providers as file parts (#47166)
* fix(anthropic): send /v1/messages PDF documents to OpenAI-shaped providers as file parts

* refactor(anthropic): hold the chat completions document change for a later commit

* fix(anthropic): send /v1/messages PDFs as file parts and keep file parts on azure_ai

* fix(snowflake): turn file parts into document blocks for Cortex Claude

* fix(prompt_security): sanitize chat file parts that carry file_data

* fix: answer 400 or drop PDFs for models that cannot read them

Models whose model_info or cost map entry sets supports_pdf_input to
false, and Ollama, now get the PDF dropped under drop_params and a 400
from litellm without it. GigaChat uploads file parts, SAP keeps sending
them as image_url data URLs, azure_ai passes dict content through, and
annotations other than url_citation move to provider_specific_fields
instead of failing the response.

* fix(router): scope supports_pdf_input to its deployment and keep provider annotation keys

Mark the azure_ai chat models in the cost map that cannot read PDFs, so drop_params drops the PDF and
litellm answers its own 400 without it. Keep url_citation annotations whole, move only typed provider
annotations to provider_specific_fields, and split them the same way when a stream is rebuilt.

* fix(vertex_ai): send PDF file parts to Vertex partner models as image_url

Vertex AI's OpenAI-compatible partner endpoint rejects the chat `file` part, so PDFs from /v1/messages
reach those models as the image_url data URL they took before. Mark native DeepSeek chat models as
unable to read PDFs, and treat a file part as a PDF by its format or filename when it has no data URL.

* fix(cost-map): leave native DeepSeek PDF support unset

/v1/messages reaches native DeepSeek through its own Anthropic endpoint, so the marks never applied there
and would only change chat completions, which this PR does not otherwise touch.

* fix(main): drop a user turn that held only a PDF under drop_params

Dropping the PDF from a PDF-only user turn left an empty content list, which
Foundry and OpenAI-shaped providers reject with a 400, so drop_params still
failed for that request. The emptied user turn is now removed.

* fix: read OCI PDFs, gate deployment base models, and stop empty drops

OCI generic chat sends file parts as image data URLs again, so OCI Gemini
keeps reading PDFs from /v1/messages and chat file parts.

The PDF gate now reads model_info.base_model, drops PDFs under drop_params
for azure_ai deployments that never declared PDF support (the old flattening
behavior), and answers 400 when dropping the PDF leaves no user message.

The cost map marks the Vertex partner text models that a live probe showed
answering without reading the PDF.

* test(integration): cover the PDF input gate, messages PDF file parts and provider annotations

Wire cells for the supports_pdf_input check on chat, chat streaming, /v1/messages and the Responses bridge, its model_info, base_model, cost map and Ollama sources, drop_params at deployment and request level, azure_ai file parts, sibling deployment isolation, Anthropic SDK PDFs reaching azure as titled file parts, Gemini and Bedrock document controls, Prompt Security sanitizing a /v1/messages PDF, and OpenRouter file annotations live, cached and streamed

* test(integration): cover Snowflake Claude and SAP in the messages PDF wire cells

* fix(azure_ai): flatten non-PDF file parts to text as before and cover the GigaChat PDF upload

* fix(main): PDF check reads the deployment's top-level base_model

* fix(main): keep roles alternating after drop_params removes a PDF-only turn

When ensure_alternating_roles is set, re-run role normalization on the messages left after the PDF check drops a turn, so two assistant turns never end up next to each other

* test: type the PDF and annotation tests the PR added

* test: type the GigaChat, SAP and Snowflake PDF tests
2026-10-29 08:06:12 -07:00
..
_v2 feat(cache): select Rust caching through explicit cache objects (#43601) 2026-09-29 00:01:44 +00:00
a2a_protocol fix(streaming): keep merged think tags balanced and split SSE lines only on CR/LF (#48755) 2026-10-28 17:00:29 -07:00
accounting feat(accounting): add accounting contracts, PricingSnapshot and ledger schema (LIT-10193) (#48862) 2026-10-28 14:39:25 -07:00
anthropic_interface fix(anthropic): make opening usage estimates opt-in (#48633) 2026-10-27 12:57:21 -07:00
assistants refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
batch_completion
batches fix(batches): apply the Vertex regional endpoint uplift to batch cost (#47466) 2026-10-20 01:51:51 +00:00
caching refactor(types): replace Any with proven types in 14 files (#48928) 2026-10-29 07:26:07 -07:00
chat_completions refactor(rust-bridge): detect Python-only features in the catalog instead of Rust declines (#46714) 2026-10-20 15:06:25 -07:00
completion_extras fix(responses): drop a malformed prompt_cache_breakpoint under drop_params on the direct /v1/responses path (#45330) 2026-10-29 03:37:54 -07:00
compression refactor(llms): expose public names for private provider helpers (#45037) 2026-10-07 08:49:25 -07:00
containers feat(openai)!: honor per-deployment workload identity on non-chat routes (#39966) 2026-10-16 22:25:15 +00:00
decisions fix(decisions): honor base_url alias for custom backends (#48505) 2026-10-26 18:46:43 -07:00
embeddings refactor(rust-bridge): decide native routing in the catalog, drop the bypass predicates (#46750) 2026-10-20 15:06:24 -07:00
endpoints/speech/speech_to_completion_bridge feat(vertex-ai)!: support gemini 3.1 flash tts (#31915) 2026-10-21 04:18:42 +00:00
evals
experimental_mcp_client refactor(mcp): drop the experimental label from the MCP server and client paths (#48513) 2026-10-27 17:47:50 +00:00
extraction feat(extraction): add /v1/extraction with Cloudflare Clairic support (#47473) 2026-10-20 15:51:45 -07:00
files refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
fine_tuning fix(managed_files): send provider file ids when creating a fine-tuning job from managed files (#46552) 2026-10-19 14:40:15 -07:00
google_genai fix(google_genai): read one-object and null spellings of repeated fields in countTokens and generateContent (#48771) 2026-10-28 21:46:06 -07:00
harness refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
images feat(providers): add Kyma API as an OpenAI-compatible provider (#48756) 2026-10-29 05:19:21 -07:00
integrations refactor(types): replace Any with proven types in 14 files (#48928) 2026-10-29 07:26:07 -07:00
interactions feat(providers): add Kyma API as an OpenAI-compatible provider (#48756) 2026-10-29 05:19:21 -07:00
litellm_core_utils fix(anthropic)!: send /v1/messages PDF documents to OpenAI-shaped providers as file parts (#47166) 2026-10-29 08:06:12 -07:00
llms fix(anthropic)!: send /v1/messages PDF documents to OpenAI-shaped providers as file parts (#47166) 2026-10-29 08:06:12 -07:00
mcp_client refactor(mcp): drop the experimental label from the MCP server and client paths (#48513) 2026-10-27 17:47:50 +00:00
messages fix(anthropic): drop the non-spec custom object from /v1/messages tool definitions (#48893) 2026-10-29 03:36:20 -07:00
models refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
ocr refactor(rust-bridge): decide native routing in the catalog, drop the bypass predicates (#46750) 2026-10-20 15:06:24 -07:00
passthrough fix(bedrock)!: apply the deployment guardrailConfig on /bedrock pass-through routes (#47429) 2026-10-20 14:48:19 -07:00
proxy fix(anthropic)!: send /v1/messages PDF documents to OpenAI-shaped providers as file parts (#47166) 2026-10-29 08:06:12 -07:00
proxy_auth refactor(types): replace Any with real types across 29 more backend files 2026-09-08 10:14:07 +00:00
rag refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
realtime_api feat(soniox): add realtime speech-to-text on /v1/realtime (#44822) 2026-10-29 07:58:38 -07:00
repositories feat(daily-activity): per-guardrail spend breakdown in user, team and organization daily activity (#48702) 2026-10-28 15:35:08 -07:00
rerank_api refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
responses fix(cost): bill a deployment's per-second rates on native Anthropic /v1/messages (#42417) 2026-10-29 05:29:03 -07:00
router_strategy refactor(router): fence the router surface and document the router-core boundary (#48839) 2026-10-29 00:01:52 +00:00
router_utils fix(openai): warn when the GPT-5 reasoning effort gate drops a level under drop_params (#47761) 2026-10-29 04:46:29 -07:00
rust_bridge build(model-catalog): ship the cost map inside the native extension (#48907) 2026-10-29 07:55:02 -07:00
sandbox refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
search fix(search): price Tavily advanced and Linkup deep searches at their own cost map entries (#47411) 2026-10-19 23:32:31 -07:00
secret_managers refactor(types): replace Any with proven types in 28 files (#48903) 2026-10-28 16:12:10 -07:00
skills refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
systemone fix(decisions): log every request field the caller sent on /v1/systemone and /v1/decisions (#47183) 2026-10-19 14:19:14 -07:00
telemetry refactor: clean up fresh tech debt in per-user Copilot auth, Ternary FOCUS export and telemetry (#45769) 2026-10-15 04:29:27 +00:00
text_completions fix(completions): omit stream usage unless requested (#47953) (#48052) 2026-10-23 19:36:20 +00:00
tracing refactor(rust-bridge): require the native extension (#48852) 2026-10-28 13:16:56 -07:00
types fix(anthropic)!: send /v1/messages PDF documents to OpenAI-shaped providers as file parts (#47166) 2026-10-29 08:06:12 -07:00
vector_store_files fix(secret_managers)!: resolve oidc/file and oidc/env refs in admin-configured api_key per request (#47341) 2026-10-20 01:43:59 +00:00
vector_stores fix(proxy): enforce team and key Allowed Vector Stores when no vector store registry is loaded (#48915) 2026-10-28 23:25:14 -07:00
videos feat(dashscope): add Wan 3.0, Wan 2.7 and HappyHorse video generation via /v1/videos (#43706) 2026-10-14 10:10:07 -07:00
__init__.py feat(providers): add Kyma API as an OpenAI-compatible provider (#48756) 2026-10-29 05:19:21 -07:00
_internal_context.py fix(logging): mark mirrored silent_model calls with is_silent_experiment in spend logs (#48888) 2026-10-29 05:12:50 -07:00
_lazy_imports.py refactor(typing): clear 1181 basedpyright Any errors across 431 files (#47789) 2026-10-24 11:31:09 -07:00
_lazy_imports_registry.py feat(decisions): support Ollama decision models (#48871) 2026-10-28 12:35:26 -07:00
_logging.py feat(proxy)!: add suppress_startup_output to turn off startup prints (#47708) 2026-10-29 03:47:04 -07:00
_process_defaults.py perf(cli): start lite without initializing the litellm SDK (#46803) 2026-10-17 04:53:32 +00:00
_redis.py refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
_redis_credential_provider.py refactor: expose core private helpers under public names (#44871) 2026-10-07 00:05:27 +00:00
_service_logger.py fix(caching): report Redis circuit-breaker metrics through the Prometheus services logger (#46484) 2026-10-14 14:29:54 -07:00
_uuid.py fix: support Python 3.15 (#48359) 2026-10-26 16:00:11 +00:00
_version.py feat(sdk): build core from an independent packaging manifest (#44340) 2026-10-08 18:40:14 -07:00
anthropic_beta_gated_fields.py fix(anthropic)!: drop beta-gated body fields under drop_params (#48793) 2026-10-29 06:25:31 -07:00
anthropic_beta_headers_config.json fix(anthropic)!: drop beta-gated body fields under drop_params (#48793) 2026-10-29 06:25:31 -07:00
anthropic_beta_headers_manager.py fix(anthropic)!: drop beta-gated body fields under drop_params (#48793) 2026-10-29 06:25:31 -07:00
blog_posts.json
budget_manager.py refactor(typing): clear 1143 basedpyright Any errors across 273 files (#46574) 2026-10-15 21:08:47 +00:00
constants.py fix(llms): move later developer messages for models declared system-first (#48897) 2026-10-29 04:12:13 -07:00
cost.json
cost_calculator.py fix(cost): bill a deployment's per-second rates on native Anthropic /v1/messages (#42417) 2026-10-29 05:29:03 -07:00
exceptions.py fix(exceptions)!: honor status code when mapping OpenAI invalid_request_error exceptions (#32537) 2026-10-29 08:05:47 -07:00
main.py fix(anthropic)!: send /v1/messages PDF documents to OpenAI-shaped providers as file parts (#47166) 2026-10-29 08:06:12 -07:00
model_prices_and_context_window_backup.json fix(anthropic)!: send /v1/messages PDF documents to OpenAI-shaped providers as file parts (#47166) 2026-10-29 08:06:12 -07:00
policy_templates_backup.json fix(packaging): keep wheel paths under Windows MAX_PATH for Store Python (#43903) 2026-09-30 22:20:36 +00:00
provider_endpoints_support_backup.json feat(soniox): add realtime speech-to-text on /v1/realtime (#44822) 2026-10-29 07:58:38 -07:00
py.typed
router.py fix(logging): mark mirrored silent_model calls with is_silent_experiment in spend logs (#48888) 2026-10-29 05:12:50 -07:00
scheduler.py fix(scheduler): remove a request's queue entry once it stops waiting (#43061) 2026-10-07 17:34:38 -07:00
setup_wizard.py fix(model_prices): correct Claude Sonnet 5.5 capabilities and provider keys (#43587) 2026-09-28 19:12:31 +00:00
timeout.py
tool_loop.py fix(responses): replay the original Responses item order, phase, and boundaries on the next chat turn (#48912) 2026-10-29 00:56:02 -07:00
utils.py feat(soniox): add realtime speech-to-text on /v1/realtime (#44822) 2026-10-29 07:58:38 -07:00