mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-29 06:33:28 +00:00
693dae2a86
vertex_ai/ deployments of Claude models now serve /v1/chat/completions, streamed and not, instead of being refused. The request is the Anthropic chat translation built for the vertex_ai host, as Python's VertexAIAnthropicConfig builds it on top of AnthropicConfig: capabilities read the vertex_ai/ cost map entry first and native structured output reads only that entry, service_tier and speed are refused or dropped with drop_params, and the Anthropic API betas are not added. The body drops model, carries anthropic_version vertex-2023-10-16, folds output_format into output_config and keeps effort only for models that take it, and names its context management and caller betas in anthropic_beta and the anthropic-beta header, filtered to what Vertex accepts, with the web search beta alone when there are none. The call goes to the publisher rawPredict URL for the project and location, regional or global, or a custom api_base or VERTEXAI_API_BASE, with a Google access token from vertex_credentials, the environment or ADC. Streams call streamRawPredict with stream true, alt=sse on a custom base, decode the Anthropic events as the anthropic/ path does and hand usage to the hooks through ReportedUsage. Every remote image is downloaded first, since Vertex takes none by URL, and the response names the requested model. |
||
|---|---|---|
| .. | ||
| azure-chat.svg | ||
| bedrock_messages-flow.svg | ||
| budgets.svg | ||
| chat-streaming.svg | ||
| gemini-chat.svg | ||
| hooks-pipeline.svg | ||
| loadtest-rig.svg | ||
| model-capacity.svg | ||
| openai-compat-chat.svg | ||
| openai_chat-flow.svg | ||
| ops-drain.svg | ||
| ops-metrics.svg | ||
| prometheus.svg | ||
| rate-limits.svg | ||
| request-hooks.svg | ||
| routing-cooldown.svg | ||
| routing-request-flow.svg | ||
| s3-log.svg | ||
| shared_state-redis.svg | ||
| spend-writer.svg | ||
| vertex-claude-chat.svg | ||
| virtual_keys-auth-flow.svg | ||
