* refactor(typing): clear basedpyright Any errors (wave 1)
* refactor(typing): clear basedpyright Any errors (wave 2)
* refactor(typing): clear basedpyright Any errors (wave 3)
* refactor(typing): clear basedpyright Any errors (wave 4)
* refactor(typing): clear basedpyright Any errors (wave 5)
* refactor(typing): clear basedpyright Any errors (wave 6)
* fix(typing): keep the cached-plan fallback's direct query_first calls so span naming resolves
* fix(logging): keep callbacks running when litellm_params is a params model
should_run_callback and the sync success and failure handlers read litellm_params through any object with a get method again, as they did before this branch, so a logging hook that returns a GenericLiteLLMParams no longer stops sync logging for that request
Adds behavior tests for the changed lines codecov listed as not run
* fix: read the regional uplift for every data residency again and reuse one dict TypeAdapter
The typed lookup only read the eu and us multipliers, so a deployment that set regional_processing_uplift_multiplier_fedramp stopped applying it. /model/update and the model write paths now share one module-level TypeAdapter instead of building one per call.
* fix: bill regional uplift multipliers float() accepts, bytes included
* fix: raise the in operator's own error from one shared json_contains, and name the bad uplift type
* fix(bedrock, gemini, deepseek, databricks, gigachat, anthropic): answer 400 for a hosted MCP tool a route cannot run, drop it under drop_params
A `type: "mcp"` tool sent to a route that rebuilds tools for its provider
(Bedrock Converse and Nova, Bedrock Invoke Claude, Vertex AI Claude, Vertex AI
and Google AI Studio Gemini, DeepSeek, Databricks Claude, GigaChat) was dropped
silently and the request still answered 200. These routes now answer
UnsupportedParamsError (400) naming the model, route and server, and pointing
to the LiteLLM MCP gateway. Under drop_params the tool is dropped with a
warning, other tools are kept, and no empty tools field is sent.
Anthropic, Azure AI Claude and Claude Platform on AWS keep mapping the tool to
the MCP connector. Native Responses providers and raw OpenAI-compatible
forwarders are unchanged.
* fix(mcp): check hosted MCP tools once per request and keep base tool_choice handling
Each route now runs one request-level step before its param mapping. When the drop leaves no tools,
the request sends no tools, tool_choice or parallel_tool_calls. Converse and GigaChat go back to their
base tool_choice and empty-tools handling.
* refactor(deepseek): hold the sent tools in a tuple
* fix(mcp): keep tool_choice for legacy functions when every hosted MCP tool drops
* fix(mcp): answer 400 for MCP servers a route cannot run on Messages, Ollama and the Responses playground
/v1/messages requests with mcp_servers on a chat-adapter or Responses-bridge
route now answer 400 naming each server, or drop it with a warning under
drop_params, instead of being forwarded or silently ignored. Ollama generate
no longer pastes a hosted MCP tool (headers included) into the prompt, and
ollama_chat filters hosted MCP tools the same way. The Admin UI Responses
playground now sends litellm_proxy/mcp references like the other endpoints.
* fix(ollama): leave JSON mode off when drop_params drops every hosted MCP tool
Ollama generate set format json for any request with tools, even after the
hosted MCP check dropped all of them, so a plain answer came back as JSON or
as a call to a tool the model never had. The check now runs before the JSON
mode switch, which stays on only while a tool is left for the prompt.
* fix(messages): honor additional_drop_params for mcp_servers and read the beta map only for hosted MCP tools
* fix(mcp_security): accept gateway server names and toolsets the gateway resolves
The guardrail compared litellm_proxy/mcp/<name> refs against registry keys, which are server ids, so a saved server referenced by name, alias, or toolset name was blocked although the gateway routes it. It now resolves each ref the way the gateway does and still blocks unknown names.
* fix(tool_fields): keep MCP drop warnings on one log line
Server labels, model names, and routes come from the caller, so a line break in one could forge extra log lines. The drop warnings now strip CR and LF from each value.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(anthropic): send /v1/messages PDF documents to OpenAI-shaped providers as file parts
* refactor(anthropic): hold the chat completions document change for a later commit
* fix(anthropic): send /v1/messages PDFs as file parts and keep file parts on azure_ai
* fix(snowflake): turn file parts into document blocks for Cortex Claude
* fix(prompt_security): sanitize chat file parts that carry file_data
* fix: answer 400 or drop PDFs for models that cannot read them
Models whose model_info or cost map entry sets supports_pdf_input to
false, and Ollama, now get the PDF dropped under drop_params and a 400
from litellm without it. GigaChat uploads file parts, SAP keeps sending
them as image_url data URLs, azure_ai passes dict content through, and
annotations other than url_citation move to provider_specific_fields
instead of failing the response.
* fix(router): scope supports_pdf_input to its deployment and keep provider annotation keys
Mark the azure_ai chat models in the cost map that cannot read PDFs, so drop_params drops the PDF and
litellm answers its own 400 without it. Keep url_citation annotations whole, move only typed provider
annotations to provider_specific_fields, and split them the same way when a stream is rebuilt.
* fix(vertex_ai): send PDF file parts to Vertex partner models as image_url
Vertex AI's OpenAI-compatible partner endpoint rejects the chat `file` part, so PDFs from /v1/messages
reach those models as the image_url data URL they took before. Mark native DeepSeek chat models as
unable to read PDFs, and treat a file part as a PDF by its format or filename when it has no data URL.
* fix(cost-map): leave native DeepSeek PDF support unset
/v1/messages reaches native DeepSeek through its own Anthropic endpoint, so the marks never applied there
and would only change chat completions, which this PR does not otherwise touch.
* fix(main): drop a user turn that held only a PDF under drop_params
Dropping the PDF from a PDF-only user turn left an empty content list, which
Foundry and OpenAI-shaped providers reject with a 400, so drop_params still
failed for that request. The emptied user turn is now removed.
* fix: read OCI PDFs, gate deployment base models, and stop empty drops
OCI generic chat sends file parts as image data URLs again, so OCI Gemini
keeps reading PDFs from /v1/messages and chat file parts.
The PDF gate now reads model_info.base_model, drops PDFs under drop_params
for azure_ai deployments that never declared PDF support (the old flattening
behavior), and answers 400 when dropping the PDF leaves no user message.
The cost map marks the Vertex partner text models that a live probe showed
answering without reading the PDF.
* test(integration): cover the PDF input gate, messages PDF file parts and provider annotations
Wire cells for the supports_pdf_input check on chat, chat streaming, /v1/messages and the Responses bridge, its model_info, base_model, cost map and Ollama sources, drop_params at deployment and request level, azure_ai file parts, sibling deployment isolation, Anthropic SDK PDFs reaching azure as titled file parts, Gemini and Bedrock document controls, Prompt Security sanitizing a /v1/messages PDF, and OpenRouter file annotations live, cached and streamed
* test(integration): cover Snowflake Claude and SAP in the messages PDF wire cells
* fix(azure_ai): flatten non-PDF file parts to text as before and cover the GigaChat PDF upload
* fix(main): PDF check reads the deployment's top-level base_model
* fix(main): keep roles alternating after drop_params removes a PDF-only turn
When ensure_alternating_roles is set, re-run role normalization on the messages left after the PDF check drops a turn, so two assistant turns never end up next to each other
* test: type the PDF and annotation tests the PR added
* test: type the GigaChat, SAP and Snowflake PDF tests
* fix(llms): sanitize trailing slashes in provider get_complete_url
* test(llms): move trailing-slash url tests into each provider's mapped unit file
* fix(llms): strip trailing slashes in Perplexity search and Ollama embedding URLs and run the Topaz tests in CI
* fix(llms): strip trailing slashes in search, DashScope, VolcEngine, GigaChat and Gradient AI URL builders
* fix(fastcrw): strip a trailing slash from api_base and expect one slash in the GigaChat tests
* fix(gigachat): strip a trailing slash from the pass-through api_base
* test(integration): cover trailing-slash api_base across provider URL builders
Wire cells for chat, streaming, messages, responses, embeddings, search and pass-through with and without a trailing slash, an owned-proxy module for the flag-gated providers, env bases and a worker kill, two chaos bursts, and a Topaz SDK cell
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test: migrate selected llm translation contracts
* test: migrate anthropic completion translation tests
* test: migrate bedrock and cohere translation tests
* test: tighten bedrock translation migration
* test: migrate azure and openai model translation tests
* test: migrate openai and small-provider translation tests
* test: migrate gemini and vertex translation tests
* test: migrate optional params and rerank translation tests
* test: migrate huggingface, json and openrouter translation tests
* test: fix openrouter integration expectations
* test: migrate databricks, xai, cometapi and fireworks translation tests
* test: migrate ovhcloud, prompt and small provider translation tests
* test: split reasoning effort grid into unit and e2e
* test: migrate router and realtime guardrail translation tests
* test: migrate google interactions translation tests
* test: migrate a2a, evals, skills and hosted vllm translation tests
* test: remove remaining llm translation suite
* test: tighten migrated llm translation unit tests and land missing rows
Restores rows the first pass dropped or weakened (bedrock agentcore and openwebui thinking stream, anthropic prefix, citations and tool_choice, azure agents handler, huggingface bare api_base, optional params num_retries, alternating roles, rerank response type), replaces truthiness and call-count checks with whole-value assertions, and moves tests to the mirror file of the code they cover
* test: move cohere, databricks, fireworks and gemini rows to mirror files over mocked HTTP
Restores three TestCohereV2Transform methods the first pass nested out of collection, replaces direct transform calls with respx-backed completion calls, and drops the databricks and fireworks files that sat outside their mirror paths
* test: close lost-intent gaps found in review of the llm translation migration
Points the hosted-mock guard at the triton test's new integration path, restores exact rerank result assertions for infinity, drops the reasoning e2e fallback that passed for any multi-line answer, and adds the mixtral invoke system prompt variants, eval deletion, and async pdf url download that the legacy suite exercised
* test: assert provider acceptance in the reasoning effort e2e
The live half of the split grid proves each route accepts reasoning_effort. claude-fable-5-1 on the direct route answers without surfacing reasoning content, so a reasoning-presence check either fails on head or needs a fallback that passes for any answer. Wire shape stays covered by the unit grid
* test: cover async and stream-end paths the legacy translation suite exercised
Adds async variants for bedrock converse file and thinking content, bedrock embedding timeout and region resolution, interaction stream termination and success logging, xai reasoning token folding, and bytes chunks with a prefix before data:. Also scopes the vertex safety settings global in test_vertex_safety_settings with monkeypatch so it stops leaking into gemini request bodies on the same worker
* test: type the new llm translation e2e calls and mark moved fixture dirs as packages
* test: name the reasoning effort e2e models as module constants
* test: assert reasoning content, register the new gemini coverage ids and give the guardrail config e2e a unique name
* test: block the prompt the shared guardrail denies and drop the reasoning presence check
* test: drop the duplicate litellm import left by the main sync
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* test: delete unconditionally skipped legacy tests
* test: move whole-unit legacy test files into tests/unit
* test: keep moved legacy tests free of import-time global state
* test: keep the module-level invocation scan pointed at tests/local_testing
* ci: drop the agent_testing CircleCI job emptied by the move
* test: fix moved-test isolation and router coverage
* test: add Tinyfish search package marker
* test: isolate moved tests from logger state leaks
* test: isolate Helicone logging fixture state
* test: isolate Vertex pass-through credentials between moved tests
* test: cancel S3 periodic flush tasks started by moved tests
* ci: restore CircleCI assistant test selection after move
* test: prevent Datadog datetime import shadowing
* ci: drop the litellm_assistants_api_testing CircleCI job emptied by the move
* test: deduplicate imports in rebased unit tests
* test: remove duplicate passthrough router patch import
* test: remove moved legacy source files after rebase
* test: align moved tests with rebased main
* test: carry main's legacy-file edits into moved destinations
* test: make the moved cost map fallback tests assert the fetch and the backup
The four fallback cases only checked the result was non-empty, so they still
passed with integrity validation disabled. They now inject a mock client, assert
one fetch happened, and assert the result is exactly the local backup with the
fallback reason recorded.
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* refactor(types): prove runtime types at harness, search, rag and client boundaries
Replace Any with adapter-validated types in the litellm.agent() harness, the
search provider transformations, RAG ingestion and query, the vector store
pre-call hook and registry, the galileo and opik logging integrations and the
proxy client CLI. Each boundary gets unit tests for well-formed and malformed
payloads.
* chore(typing): prove types at more provider boundaries and restore search transformations
Second pass over a2a, embedding, rerank, image, audio and small provider
modules. Search transformations go back to their previous form because
validating their response bodies would change the proxy status for
malformed upstream bodies from 400 to 500.
* refactor(types): prove types at logging, files, rerank, image, audio and management boundaries
Replace Any with validated or annotation-only types in 37 more files: logging
integrations, token counters, provider files/rerank/image generation/audio
transcription transformations, pass-through logging handlers and management
endpoints. No proxy HTTP status or error type changes.
* refactor(types): prove types at repository, spend, files and router boundaries
Replace Any with repository table accessors, validated mappings and
annotation-only types in 33 more files: Prisma repositories, the enterprise
batch and responses cost checkers, budget reservation, files endpoints,
management endpoints, the policy registry, the adaptive and complexity routers
and the secret managers. No proxy HTTP status or error type changes.
* test(types): run the aiohttp transformation test in-process and cover repository row conversion
The aiohttp chat transformation test no longer starts a server. It feeds the
transformation a response whose json() returns the body under test.
The proxy unit shards now exercise stored model rows whose params are JSON
strings and the object permission create and update paths.
Move the wave 1 phase 7 batch (fireworks_ai, gemini, gigachat, github_copilot; 20 files) from tests/test_litellm to tests/unit after judging every test function under a behaviour mutation. Seven wiring or mock-echo tests that stayed green are deleted. The fireworks cost calculator tests get a local model_cost save/restore fixture since the tests/unit tree has no shared conftest for it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
