mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-12 23:01:41 +00:00
6d828e5759
* feat(messages): passthrough /v1/messages to native endpoints via supported_endpoints
The unified /v1/messages proxy endpoint always translated inbound Anthropic
requests down to /v1/chat/completions (or the Responses API for openai) when the
deployment's provider lacked a native Anthropic-messages config, dropping
Anthropic-only features like cache_control and thinking. Some customers run
OpenAI-compatible servers (self-hosted vLLM, DeepSeek's Anthropic endpoint, etc.)
that also natively expose /v1/messages and want the raw Anthropic payload
forwarded untranslated, while keeping provider openai so /v1/chat/completions to
the same deployment stays native.
Opt in per deployment via model_info.supported_endpoints containing
/v1/messages. When present, the gate routes to a generic, provider-agnostic
OpenAILikeAnthropicMessagesConfig that POSTs the Anthropic payload to
{api_base}/v1/messages with Bearer auth, instead of translating. Default
behavior is unchanged. Generalizes and supersedes the hosted_vllm-only,
env-var-toggled PR #28745.
* fix(messages): preserve standard-cased caller headers in native passthrough
The OpenAI-like Anthropic passthrough config only checked for lowercase header
names before injecting Bearer auth, anthropic-version, and content-type
defaults. A caller sending standard-cased Authorization, Anthropic-Version, or
Content-Type was treated as missing those headers, so LiteLLM added duplicate
lowercase variants and overwrote the caller's credential/version at the HTTP
layer. Header presence is now checked case-insensitively and the merge no longer
mutates the caller dict.
Also moves the feature docs out of the main repo (docs live in litellm-docs).
* fix(openai_like/messages): delegate to parent transform and inject anthropic-beta headers
The passthrough config bypassed the parent transform and skipped header beta injection. Both gaps cause native /v1/messages features (context management, advisor tool, fast mode, structured outputs, reasoning_effort, advisor stripping) to silently degrade on opted-in deployments. Reuse the parent's pipeline and call _update_headers_with_anthropic_beta after merging defaults
* fix: normalize anthropic-beta header key case before beta injection
* style: collapse anthropic-beta header normalization to single line
ruff format --check requires the comprehension on one line (it fits within
the 120 char limit); fixes the lint job failure on the bugbot autofix commit
* fix(messages): forward anthropic-beta to native passthrough upstream
The shared anthropic_messages HTTP handler ran update_headers_with_filtered_beta
with the deployment's custom_llm_provider after validate. For the native
/v1/messages passthrough that provider is openai, which has no beta-header
mapping, so every anthropic-beta value (caller-supplied or feature-derived for
speed/context_management/etc.) was stripped to empty before the upstream
request, breaking beta passthrough to the Anthropic-compatible endpoint.
Beta filtering only makes sense on cross-provider translation paths where the
upstream cannot understand Anthropic betas. Gate it on a new
should_filter_anthropic_beta_headers() that defaults to True (bedrock, vertex_ai,
native anthropic unchanged) and is overridden to False by
OpenAILikeAnthropicMessagesConfig, whose upstream is a native Anthropic endpoint,
so betas pass through verbatim.
* chore: remove accidentally committed local QA logs and config
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
|
||
|---|---|---|
| .. | ||
| aiml/image_generation | ||
| amazon_nova/chat | ||
| anthropic | ||
| apiserpent | ||
| azure | ||
| azure_ai | ||
| base_llm | ||
| baseten/chat | ||
| bedrock | ||
| bedrock_mantle | ||
| black_forest_labs | ||
| bytez/chat | ||
| chat | ||
| chatgpt | ||
| cloudflare | ||
| cohere | ||
| cometapi/chat | ||
| compactifai | ||
| crusoe | ||
| custom_httpx | ||
| dashscope | ||
| databricks | ||
| datarobot | ||
| deepgram | ||
| deepinfra | ||
| deepseek | ||
| docker_model_runner | ||
| elevenlabs | ||
| fal_ai/image_generation | ||
| fastcrw/search | ||
| featherless_ai/chat | ||
| fireworks_ai | ||
| gemini | ||
| github_copilot | ||
| heroku | ||
| hosted_vllm | ||
| huggingface | ||
| inception | ||
| jina_ai/embedding | ||
| langflow | ||
| lemonade | ||
| litellm_proxy | ||
| llamafile/chat | ||
| lm_studio | ||
| manus | ||
| meta_llama | ||
| minimax | ||
| mistral | ||
| modelscope | ||
| moonshot | ||
| nebius | ||
| neosantara | ||
| novita/chat | ||
| nscale/chat | ||
| nvidia_riva | ||
| oci | ||
| ocr | ||
| ollama | ||
| openai | ||
| openai_like | ||
| openrouter | ||
| ovhcloud | ||
| parallel_ai | ||
| parasail | ||
| pass_through | ||
| perplexity | ||
| pg_vector/vector_stores | ||
| publicai | ||
| ragflow/chat | ||
| recraft | ||
| reducto | ||
| runwayml | ||
| s3_vectors | ||
| sagemaker | ||
| sambanova | ||
| sap | ||
| scaleway | ||
| snowflake | ||
| soniox | ||
| stability | ||
| tinyfish | ||
| vercel_ai_gateway | ||
| vertex_ai | ||
| volcengine | ||
| voyage | ||
| wandb | ||
| watsonx | ||
| xai | ||
| you_com | ||
| zai | ||
| test_cache_control_and_reasoning.py | ||
| test_file_content_block.py | ||
| test_file_search_responses.py | ||
| test_lifecycle_fix.py | ||
| test_oom_fixes.py | ||
| test_polling_url_origin_match.py | ||
| test_predibase_transformation.py | ||