Merge remote-tracking branch 'origin' into litellm_new_user_collision_handling

2026-10-08 03:08:45 +00:00 · 2025-12-06 19:58:35 -08:00 · 2025-12-06 19:58:35 -08:00 · 30eaa1c099
commit 30eaa1c099
parent a2bf6e169c a2688c7017
345 changed files with 7384 additions and 1060 deletions
--- a/README.md
+++ b/README.md
@ -274,8 +274,6 @@ echo 'LITELLM_MASTER_KEY="sk-1234"' > .env
 # password generator to get a random hash for litellm salt key
 echo 'LITELLM_SALT_KEY="sk-1234"' >> .env

-source .env
-
 # Start
 docker compose up
 ```
--- a/ci_cd/security_scans.sh
+++ b/ci_cd/security_scans.sh
@ -76,6 +76,8 @@ run_grype_scans() {
        "GHSA-4xh5-x5gv-qwph"
        "CVE-2025-8291" # no fix available as of Oct 11, 2025
        "GHSA-5j98-mcp5-4vw2"
+        "CVE-2025-13836" # Python 3.13 HTTP response reading OOM/DoS - no fix available in base image
+        "CVE-2025-12084" # Python 3.13 xml.dom.minidom quadratic algorithm - no fix available in base image
    )

    # Build JSON array of allowlisted CVE IDs for jq
--- a/cookbook/misc/RELEASE_NOTES_GENERATION_INSTRUCTIONS.md
+++ b/cookbook/misc/RELEASE_NOTES_GENERATION_INSTRUCTIONS.md
@ -404,6 +404,93 @@ This release has a known issue...
  - **New Providers** - Provider name, supported endpoints, description
  - **New LLM API Endpoints** (optional) - Endpoint, method, description, documentation link
 - Only include major new provider integrations, not minor provider updates
+- **IMPORTANT**: When adding new providers, also update `provider_endpoints_support.json` in the repository root (see Section 13)
+
+### 12. Section Header Counts
+
+**Always include counts in section headers for:**
+- **New Providers** - Add count in parentheses: `### New Providers (X new providers)`
+- **New LLM API Endpoints** - Add count in parentheses: `### New LLM API Endpoints (X new endpoints)`
+- **New Model Support** - Add count in parentheses: `#### New Model Support (X new models)`
+
+**Format:**
+```markdown
+### New Providers (4 new providers)
+
+| Provider | Supported LiteLLM Endpoints | Description |
+| -------- | --------------------------- | ----------- |
+...
+
+### New LLM API Endpoints (2 new endpoints)
+
+| Endpoint | Method | Description | Documentation |
+| -------- | ------ | ----------- | ------------- |
+...
+
+#### New Model Support (32 new models)
+
+| Provider | Model | Context Window | Input ($/1M tokens) | Output ($/1M tokens) | Features |
+| -------- | ----- | -------------- | ------------------- | -------------------- | -------- |
+...
+```
+
+**Counting Rules:**
+- Count each row in the table (excluding the header row)
+- For models, count each model entry in the pricing table
+- For providers, count each new provider added
+- For endpoints, count each new API endpoint added
+
+### 13. Update provider_endpoints_support.json
+
+**When adding new providers or endpoints, you MUST also update `provider_endpoints_support.json` in the repository root.**
+
+This file tracks which endpoints are supported by each LiteLLM provider and is used to generate documentation.
+
+**Required Steps:**
+1. For each new provider added to the release notes, add a corresponding entry to `provider_endpoints_support.json`
+2. For each new endpoint type added, update the schema comment and add the endpoint to relevant providers
+
+**Provider Entry Format:**
+```json
+"provider_slug": {
+  "display_name": "Provider Name (`provider_slug`)",
+  "url": "https://docs.litellm.ai/docs/providers/provider_slug",
+  "endpoints": {
+    "chat_completions": true,
+    "messages": true,
+    "responses": true,
+    "embeddings": false,
+    "image_generations": false,
+    "audio_transcriptions": false,
+    "audio_speech": false,
+    "moderations": false,
+    "batches": false,
+    "rerank": false,
+    "a2a": true
+  }
+}
+```
+
+**Available Endpoint Types:**
+- `chat_completions` - `/chat/completions` endpoint
+- `messages` - `/messages` endpoint (Anthropic format)
+- `responses` - `/responses` endpoint (OpenAI/Anthropic unified)
+- `embeddings` - `/embeddings` endpoint
+- `image_generations` - `/image/generations` endpoint
+- `audio_transcriptions` - `/audio/transcriptions` endpoint
+- `audio_speech` - `/audio/speech` endpoint
+- `moderations` - `/moderations` endpoint
+- `batches` - `/batches` endpoint
+- `rerank` - `/rerank` endpoint
+- `ocr` - `/ocr` endpoint
+- `search` - `/search` endpoint
+- `vector_stores` - `/vector_stores` endpoint
+- `a2a` - `/a2a/{agent}/message/send` endpoint (A2A Protocol)
+
+**Checklist:**
+- [ ] All new providers from release notes are added to `provider_endpoints_support.json`
+- [ ] Endpoint support flags accurately reflect provider capabilities
+- [ ] Documentation URL points to correct provider docs page

 ## Example Command Workflow

--- a/deploy/charts/litellm-helm/Chart.yaml
+++ b/deploy/charts/litellm-helm/Chart.yaml
@ -18,7 +18,7 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.4.9
+version: 0.4.10

 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to
--- a/deploy/charts/litellm-helm/templates/deployment.yaml
+++ b/deploy/charts/litellm-helm/templates/deployment.yaml
@ -6,6 +6,9 @@ metadata:
  name: {{ include "litellm.fullname" . }}
  labels:
    {{- include "litellm.labels" . | nindent 4 }}
+    {{- if .Values.deploymentLabels }}
+    {{- toYaml .Values.deploymentLabels | nindent 4 }}
+    {{- end }}
 spec:
  {{- if not .Values.autoscaling.enabled }}
  replicas: {{ .Values.replicaCount }}
@ -126,6 +129,12 @@ spec:
            - configMapRef:
                name: {{ . }}
          {{- end }}
+          {{- if .Values.command }}
+          command: {{ toYaml .Values.command | nindent 12 }}
+          {{- end }}
+          {{- if .Values.args }}
+          args: {{ toYaml .Values.args | nindent 12 }}
+          {{- else }}
          args:
            - --config
            - /etc/litellm/config.yaml
@ -133,6 +142,7 @@ spec:
            - --num_workers
            - {{ .Values.numWorkers | quote }}
            {{- end }}
+          {{- end }}
          ports:
            - name: http
              containerPort: {{ .Values.service.port }}
--- a/deploy/charts/litellm-helm/tests/deployment_command_args_labels_tests.yaml
+++ b/deploy/charts/litellm-helm/tests/deployment_command_args_labels_tests.yaml
@ -0,0 +1,68 @@
+suite: test deployment command, args, and deploymentLabels
+templates:
+  - deployment.yaml
+  - configmap-litellm.yaml
+tests:
+  - it: should override args when custom args specified
+    template: deployment.yaml
+    set:
+      args:
+        - --custom-arg1
+        - value1
+        - --custom-arg2
+    asserts:
+      - equal:
+          path: spec.template.spec.containers[0].args
+          value:
+            - --custom-arg1
+            - value1
+            - --custom-arg2
+  - it: should set custom command when specified
+    template: deployment.yaml
+    set:
+      command:
+        - /bin/sh
+        - -c
+    asserts:
+      - equal:
+          path: spec.template.spec.containers[0].command
+          value:
+            - /bin/sh
+            - -c
+  - it: should set custom command and args together
+    template: deployment.yaml
+    set:
+      command:
+        - python
+        - -u
+      args:
+        - my_script.py
+        - --verbose
+    asserts:
+      - equal:
+          path: spec.template.spec.containers[0].command
+          value:
+            - python
+            - -u
+      - equal:
+          path: spec.template.spec.containers[0].args
+          value:
+            - my_script.py
+            - --verbose
+  - it: should add deploymentLabels to deployment metadata
+    template: deployment.yaml
+    set:
+      deploymentLabels:
+        environment: production
+        team: platform
+        version: v1.2.3
+    asserts:
+      - equal:
+          path: metadata.labels.environment
+          value: production
+      - equal:
+          path: metadata.labels.team
+          value: platform
+      - equal:
+          path: metadata.labels.version
+          value: v1.2.3
--- a/deploy/charts/litellm-helm/values.yaml
+++ b/deploy/charts/litellm-helm/values.yaml
@ -30,6 +30,7 @@ serviceAccount:

 # annotations for litellm deployment
 deploymentAnnotations: {}
+deploymentLabels: {}
 # annotations for litellm pods
 podAnnotations: {}
 podLabels: {}
@ -253,6 +254,11 @@ envVars: {}
 # Additional environment variables to be added to the deployment as a list of k8s env vars
 extraEnvVars: {}

+# if you want to override the container command, you can do so here
+command: {}
+# if you want to override the container args, you can do so here
+args: {}
+
 # - name: EXTRA_ENV_VAR
 #   value: EXTRA_ENV_VAR_VALUE
 # Pod Disruption Budget
--- a/docker/Dockerfile.non_root
+++ b/docker/Dockerfile.non_root
@ -10,18 +10,20 @@ WORKDIR /app

 # Install build dependencies including Node.js for UI build
 USER root
-RUN apk add --no-cache \
-    python3 \
-    py3-pip \
-    clang \
-    llvm \
-    lld \
-    gcc \
-    linux-headers \
-    build-base \
-    bash \
-    nodejs \
-    npm \
+RUN for i in 1 2 3; do \
+  apk add --no-cache \
+  python3 \
+  py3-pip \
+  clang \
+  llvm \
+  lld \
+  gcc \
+  linux-headers \
+  build-base \
+  bash \
+  nodejs \
+  npm && break || sleep 5; \
+  done \
  && pip install --no-cache-dir --upgrade pip build

 # Copy project files
@ -37,7 +39,7 @@ RUN npm install -g npm@latest && npm cache clean --force

 RUN cd /app/ui/litellm-dashboard && \
  if [ -f "/app/enterprise/enterprise_ui/enterprise_colors.json" ]; then \
-    cp /app/enterprise/enterprise_ui/enterprise_colors.json ./ui_colors.json; \
+  cp /app/enterprise/enterprise_ui/enterprise_colors.json ./ui_colors.json; \
  fi

 RUN cd /app/ui/litellm-dashboard && rm -f package-lock.json
@ -50,11 +52,11 @@ RUN cp -r /app/ui/litellm-dashboard/out/* /tmp/litellm_ui/

 RUN cd /tmp/litellm_ui && \
  for html_file in *.html; do \
-    if [ "$html_file" != "index.html" ] && [ -f "$html_file" ]; then \
-      folder_name="${html_file%.html}" && \
-      mkdir -p "$folder_name" && \
-      mv "$html_file" "$folder_name/index.html"; \
-    fi; \
+  if [ "$html_file" != "index.html" ] && [ -f "$html_file" ]; then \
+  folder_name="${html_file%.html}" && \
+  mkdir -p "$folder_name" && \
+  mv "$html_file" "$folder_name/index.html"; \
+  fi; \
  done

 RUN cd /app/ui/litellm-dashboard && rm -rf ./out
@ -72,8 +74,12 @@ WORKDIR /app

 # Install runtime dependencies
 USER root
-RUN apk upgrade --no-cache && \
-  apk add --no-cache python3 py3-pip bash openssl tzdata nodejs npm supervisor
+RUN for i in 1 2 3; do \
+  apk upgrade --no-cache && break || sleep 5; \
+  done \
+  && for i in 1 2 3; do \
+  apk add --no-cache python3 py3-pip bash openssl tzdata nodejs npm supervisor && break || sleep 5; \
+  done

 # Copy only necessary artifacts from builder stage for runtime
 COPY . .
@ -91,7 +97,7 @@ RUN pip install *.whl /wheels/* --no-index --find-links=/wheels/ \

 # Remove test files and keys from dependencies
 RUN find /usr/lib -type f -path "*/tornado/test/*" -delete && \
-    find /usr/lib -type d -path "*/tornado/test" -delete
+  find /usr/lib -type d -path "*/tornado/test" -delete

 # Install semantic_router and aurelio-sdk using script
 RUN chmod +x docker/install_auto_router.sh && ./docker/install_auto_router.sh
--- a/docker/build_from_pip/Dockerfile.build_from_pip
+++ b/docker/build_from_pip/Dockerfile.build_from_pip
@ -1,17 +1,16 @@
-FROM cgr.dev/chainguard/python:latest-dev
+FROM python:3.13-alpine

-USER root
 WORKDIR /app

 ENV HOME=/home/litellm
 ENV PATH="${HOME}/venv/bin:$PATH"

 # Install runtime dependencies
-# Note: The base image has Python 3.14, but python3-dev installs Python 3.13 which conflicts.
-# The -dev variant should include Python headers, but if compilation fails, we may need
-# to install python-3.14-dev specifically (if available in the repo)
+# Note: Using Python 3.13 for compatibility with ddtrace and other packages
+# rust and cargo are required for building ddtrace from source
+# musl-dev and libffi-dev are needed for some Python packages on Alpine
 RUN apk update && \
-    apk add --no-cache gcc openssl openssl-dev
+    apk add --no-cache gcc musl-dev libffi-dev openssl openssl-dev rust cargo

 RUN python -m venv ${HOME}/venv
 RUN ${HOME}/venv/bin/pip install --no-cache-dir --upgrade pip
--- a/docs/my-website/docs/projects/GraphRAG.md
+++ b/docs/my-website/docs/projects/GraphRAG.md
@ -0,0 +1,8 @@
+
+# Microsoft GraphRAG
+
+GraphRAG is a data pipeline and transformation suite that extracts meaningful, structured data from unstructured text using the power of LLMs. It uses a graph-based approach to RAG (Retrieval-Augmented Generation) that leverages knowledge graphs to improve reasoning over private datasets.
+
+- [Github](https://github.com/microsoft/graphrag)
+- [Docs](https://microsoft.github.io/graphrag/)
+- [Paper](https://arxiv.org/pdf/2404.16130)
--- a/docs/my-website/docs/providers/amazon_nova.md
+++ b/docs/my-website/docs/providers/amazon_nova.md
@ -0,0 +1,291 @@
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Amazon Nova
+
+| Property | Details |
+|-------|-------|
+| Description | Amazon Nova is a family of foundation models built by Amazon that deliver frontier intelligence and industry-leading price performance. |
+| Provider Route on LiteLLM | `amazon_nova/` |
+| Provider Doc | [Amazon Nova ↗](https://docs.aws.amazon.com/nova/latest/userguide/what-is-nova.html) |
+| Supported OpenAI Endpoints | `/chat/completions`, `v1/responses` |
+| Other Supported Endpoints | `v1/messages`, `/generateContent` | 
+
+## Authentication
+
+Amazon Nova uses API key authentication. You can obtain your API key from the [Amazon Nova developer console ↗](https://nova.amazon.com/dev/documentation).
+
+```bash
+export AMAZON_NOVA_API_KEY="your-api-key"
+```
+
+## Usage
+
+<Tabs>
+<TabItem value="sdk" label="SDK">
+
+```python
+import os
+from litellm import completion
+
+# Set your API key
+os.environ["AMAZON_NOVA_API_KEY"] = "your-api-key"
+
+response = completion(
+    model="amazon_nova/nova-micro-v1",
+    messages=[
+        {"role": "system", "content": "You are a helpful assistant"},
+        {"role": "user", "content": "Hello, how are you?"}
+    ]
+)
+
+print(response)
+```
+
+</TabItem>
+<TabItem value="proxy" label="PROXY">
+
+### 1. Setup config.yaml
+
+```yaml
+model_list:
+  - model_name: amazon-nova-micro
+    litellm_params:
+      model: amazon_nova/nova-micro-v1
+      api_key: os.environ/AMAZON_NOVA_API_KEY
+```
+### 2. Start the proxy
+```bash
+litellm --config /path/to/config.yaml
+```
+
+### 3. Test it
+
+```bash
+curl --location 'http://0.0.0.0:4000/chat/completions' \
+--header 'Content-Type: application/json' \
+--data '{
+    "model": "amazon-nova-micro",
+    "messages": [
+        {
+            "role": "user",
+            "content": "Hello, how are you?"
+        }
+    ]
+}'
+```
+
+</TabItem>
+</Tabs>
+
+## Supported Models
+
+| Model Name | Usage | Context Window |
+|------------|-------|----------------|
+| Nova Micro | `completion(model="amazon_nova/nova-micro-v1", messages=messages)` | 128K tokens |
+| Nova Lite | `completion(model="amazon_nova/nova-lite-v1", messages=messages)` | 300K tokens |
+| Nova Pro | `completion(model="amazon_nova/nova-pro-v1", messages=messages)` | 300K tokens |
+| Nova Premier | `completion(model="amazon_nova/nova-premier-v1", messages=messages)` | 1M tokens |
+
+## Usage - Streaming
+
+<Tabs>
+<TabItem value="sdk" label="SDK">
+
+```python
+import os
+from litellm import completion
+
+os.environ["AMAZON_NOVA_API_KEY"] = "your-api-key"
+
+response = completion(
+    model="amazon_nova/nova-micro-v1",
+    messages=[
+        {"role": "system", "content": "You are a helpful assistant"},
+        {"role": "user", "content": "Tell me about machine learning"}
+    ],
+    stream=True
+)
+
+for chunk in response:
+    print(chunk.choices[0].delta.content or "", end="")
+```
+
+</TabItem>
+<TabItem value="proxy" label="PROXY">
+
+```bash
+curl --location 'http://0.0.0.0:4000/chat/completions' \
+--header 'Content-Type: application/json' \
+--data '{
+    "model": "amazon-nova-micro",
+    "messages": [
+        {
+            "role": "user",
+            "content": "Tell me about machine learning"
+        }
+    ],
+    "stream": true
+}'
+```
+
+</TabItem>
+</Tabs>
+
+## Usage - Function Calling / Tool Usage
+
+<Tabs>
+<TabItem value="sdk" label="SDK">
+
+```python
+import os
+from litellm import completion
+
+os.environ["AMAZON_NOVA_API_KEY"] = "your-api-key"
+
+tools = [
+    {
+        "type": "function",
+        "function": {
+            "name": "getCurrentWeather",
+            "description": "Get the current weather in a given city",
+            "parameters": {
+                "type": "object",
+                "properties": {
+                    "location": {
+                        "type": "string",
+                        "description": "City and country e.g. San Francisco, CA"
+                    }
+                },
+                "required": ["location"]
+            }
+        }
+    }
+]
+
+response = completion(
+    model="amazon_nova/nova-micro-v1",
+    messages=[
+        {"role": "user", "content": "What's the weather like in San Francisco?"}
+    ],
+    tools=tools
+)
+
+print(response)
+```
+
+</TabItem>
+<TabItem value="proxy" label="PROXY">
+
+```bash
+curl --location 'http://0.0.0.0:4000/chat/completions' \
+--header 'Content-Type: application/json' \
+--data '{
+    "model": "amazon-nova-micro",
+    "messages": [
+        {
+            "role": "user",
+            "content": "What'\''s the weather like in San Francisco?"
+        }
+    ],
+    "tools": [
+        {
+            "type": "function",
+            "function": {
+                "name": "getCurrentWeather",
+                "description": "Get the current weather in a given city",
+                "parameters": {
+                    "type": "object",
+                    "properties": {
+                        "location": {
+                            "type": "string",
+                            "description": "City and country e.g. San Francisco, CA"
+                        }
+                    },
+                    "required": ["location"]
+                }
+            }
+        }
+    ]
+}'
+```
+
+</TabItem>
+</Tabs>
+
+## Set temperature, top_p, etc.
+
+<Tabs>
+<TabItem value="sdk" label="SDK">
+
+```python
+import os
+from litellm import completion
+
+os.environ["AMAZON_NOVA_API_KEY"] = "your-api-key"
+
+response = completion(
+    model="amazon_nova/nova-pro-v1",
+    messages=[
+        {"role": "user", "content": "Write a creative story"}
+    ],
+    temperature=0.8,
+    max_tokens=500,
+    top_p=0.9
+)
+
+print(response)
+```
+
+</TabItem>
+<TabItem value="proxy" label="PROXY">
+
+**Set on yaml**
+
+```yaml
+model_list:
+  - model_name: amazon-nova-pro
+    litellm_params:
+      model: amazon_nova/nova-pro-v1
+      temperature: 0.8
+      max_tokens: 500
+      top_p: 0.9
+```
+**Set on request**
+```bash
+curl --location 'http://0.0.0.0:4000/chat/completions' \
+--header 'Content-Type: application/json' \
+--data '{
+    "model": "amazon-nova-pro",
+    "messages": [
+        {
+            "role": "user",
+            "content": "Write a creative story"
+        }
+    ],
+    "temperature": 0.8,
+    "max_tokens": 500,
+    "top_p": 0.9
+}'
+```
+
+</TabItem>
+</Tabs>
+
+## Model Comparison
+
+| Model | Best For | Speed | Cost | Context |
+|-------|----------|-------|------|---------|
+| **Nova Micro** | Simple tasks, high throughput | Fastest | Lowest | 128K |
+| **Nova Lite** | Balanced performance | Fast | Low | 300K |
+| **Nova Pro** | Complex reasoning | Medium | Medium | 300K |
+| **Nova Premier** | Most advanced tasks | Slower | Higher | 1M |
+
+## Error Handling
+
+Common error codes and their meanings:
+
+- `401 Unauthorized`: Invalid API key
+- `429 Too Many Requests`: Rate limit exceeded
+- `400 Bad Request`: Invalid request format
+- `500 Internal Server Error`: Service temporarily unavailable
--- a/docs/my-website/docs/providers/gemini.md
+++ b/docs/my-website/docs/providers/gemini.md
@ -2006,3 +2006,34 @@ curl -L -X POST 'http://localhost:4000/v1/chat/completions' \
 </TabItem>
 </Tabs>

+### Image Generation Pricing
+
+Gemini image generation models (like `gemini-3-pro-image-preview`) return `image_tokens` in the response usage. These tokens are priced differently from text tokens:
+
+| Token Type | Price per 1M tokens | Price per token |
+|------------|---------------------|-----------------|
+| Text output | $12 | $0.000012 |
+| Image output | $120 | $0.00012 |
+
+The number of image tokens depends on the output resolution:
+
+| Resolution | Tokens per image | Cost per image |
+|------------|------------------|----------------|
+| 1K-2K (1024x1024 to 2048x2048) | 1,120 | $0.134 |
+| 4K (4096x4096) | 2,000 | $0.24 |
+
+LiteLLM automatically calculates costs using `output_cost_per_image_token` from the model pricing configuration.
+
+**Example response usage:**
+```json
+{
+    "completion_tokens_details": {
+        "reasoning_tokens": 225,
+        "text_tokens": 0,
+        "image_tokens": 1120
+    }
+}
+```
+
+For more details, see [Google's Gemini pricing documentation](https://ai.google.dev/gemini-api/docs/pricing).
+
--- a/docs/my-website/docs/providers/openai.md
+++ b/docs/my-website/docs/providers/openai.md
@ -191,6 +191,7 @@ os.environ["OPENAI_BASE_URL"] = "https://your_host/v1"     # OPTIONAL
 | gpt-5.1 | `response = completion(model="gpt-5.1", messages=messages)` |
 | gpt-5.1-codex | `response = completion(model="gpt-5.1-codex", messages=messages)` |
 | gpt-5.1-codex-mini | `response = completion(model="gpt-5.1-codex-mini", messages=messages)` |
+| gpt-5.1-codex-max | `response = completion(model="gpt-5.1-codex-max", messages=messages)` |
 | gpt-4.1 | `response = completion(model="gpt-4.1", messages=messages)` |
 | gpt-4.1-mini | `response = completion(model="gpt-4.1-mini", messages=messages)` |
 | gpt-4.1-nano | `response = completion(model="gpt-4.1-nano", messages=messages)` |
@ -427,7 +428,7 @@ Expected Response:

 ### Advanced: Using `reasoning_effort` with `summary` field

-By default, `reasoning_effort` accepts a string value (`"none"`, `"minimal"`, `"low"`, `"medium"`, `"high"`) and only sets the effort level without including a reasoning summary.
+By default, `reasoning_effort` accepts a string value (`"none"`, `"minimal"`, `"low"`, `"medium"`, `"high"`, `"xhigh"`—`"xhigh"` is only supported on `gpt-5.1-codex-max`) and only sets the effort level without including a reasoning summary.

 To opt-in to the `summary` feature, you can pass `reasoning_effort` as a dictionary. **Note:** The `summary` field requires your OpenAI organization to have verification status. Using `summary` without verification will result in a 400 error from OpenAI.

@ -494,10 +495,12 @@ curl -X POST 'http://0.0.0.0:4000/chat/completions' \
 | `gpt-5-codex` | `adaptive` | `low`, `medium`, `high` (no `minimal`) |
 | `gpt-5.1-codex` | `adaptive` | `low`, `medium`, `high` (no `minimal`) |
 | `gpt-5.1-codex-mini` | `adaptive` | `low`, `medium`, `high` (no `minimal`) |
+| `gpt-5.1-codex-max` | `adaptive` | `low`, `medium`, `high`, `xhigh` (no `minimal`) |
 | `gpt-5-pro` | `high` | `high` only |

 **Note:**
 - GPT-5.1 introduced a new `reasoning_effort="none"` setting for faster, lower-latency responses. This replaces the `"minimal"` setting from GPT-5.
+- `gpt-5.1-codex-max` is the only model that supports `reasoning_effort="xhigh"`. All other models will reject this value.
 - `gpt-5-pro` only accepts `reasoning_effort="high"`. Other values will return an error.
 - When `reasoning_effort` is not set (None), OpenAI defaults to the value shown in the "Default" column.

@ -509,7 +512,7 @@ The `verbosity` parameter controls the length and detail of responses from GPT-5

 **Supported models:** `gpt-5`, `gpt-5.1`, `gpt-5-mini`, `gpt-5-nano`, `gpt-5-pro`

-**Note:** GPT-5-Codex models (`gpt-5-codex`, `gpt-5.1-codex`, `gpt-5.1-codex-mini`) do **not** support the `verbosity` parameter.
+**Note:** GPT-5-Codex models (`gpt-5-codex`, `gpt-5.1-codex`, `gpt-5.1-codex-mini`, `gpt-5.1-codex-max`) do **not** support the `verbosity` parameter.

 **Use cases:**
 - **`"low"`**: Best for concise answers or simple code generation (e.g., SQL queries)
@ -988,4 +991,4 @@ response = completion(

 LiteLLM supports OpenAI's video generation models including Sora.

-For detailed documentation on video generation, see [OpenAI Video Generation →](./openai/video_generation.md)
+For detailed documentation on video generation, see [OpenAI Video Generation →](./openai/video_generation.md)
--- a/docs/my-website/docs/proxy/customer_usage.md
+++ b/docs/my-website/docs/proxy/customer_usage.md
@ -0,0 +1,110 @@
+import Image from '@theme/IdealImage';
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+# Customer Usage
+
+Track and visualize end-user spend directly in the dashboard. Monitor customer-level usage analytics, spend logs, and activity metrics to understand how your customers are using your LLM services.
+
+This feature is **available in v1.80.8-stable and above**.
+
+## Overview
+
+Customer Usage enables you to track spend and usage for individual customers (end users) by passing an ID in your API requests. This allows you to:
+
+- Track spend per customer automatically
+- View customer-level usage analytics in the Admin UI
+- Filter spend logs and activity metrics by customer ID
+- Set budgets and rate limits per customer
+- Monitor customer usage patterns and trends
+
+<Image img={require('../../img/customer_usage.png')} />
+
+## How to Track Spend
+
+Track customer spend by including a `user` field in your API requests. The customer ID will be automatically tracked and associated with all spend from that request.
+
+### Example using cURL
+
+Make a `/chat/completions` call with the `user` field containing your customer ID:
+
+```bash showLineNumbers title="Track spend with customer ID"
+curl -X POST 'http://0.0.0.0:4000/chat/completions' \
+  --header 'Content-Type: application/json' \
+  --header 'Authorization: Bearer sk-1234' \ # 👈 YOUR PROXY KEY
+  --data '{
+    "model": "gpt-3.5-turbo",
+    "user": "customer-123", # 👈 CUSTOMER ID
+    "messages": [
+      {
+        "role": "user",
+        "content": "What is the capital of France?"
+      }
+    ]
+  }'
+```
+
+The customer ID (`customer-123`) will be automatically upserted into the database with the new spend. If the customer ID already exists, spend will be incremented.
+
+### Example using OpenWebUI
+
+See the [Open WebUI tutorial](../tutorials/openweb_ui.md) for detailed instructions on connecting Open WebUI to LiteLLM and tracking customer usage.
+
+## How to View Spend
+
+### View Spend in Admin UI
+
+Navigate to the Customer Usage tab in the Admin UI to view customer-level spend analytics:
+
+#### 1. Access Customer Usage
+
+Go to the Usage page in the Admin UI (`PROXY_BASE_URL/ui/?login=success&page=new_usage`) and click on the **Customer Usage** tab.
+
+<Image img={require('../../img/customer_usage_ui_navigation.png')} />
+
+#### 2. View Customer Analytics
+
+The Customer Usage dashboard provides:
+
+- **Total spend per customer**: View aggregated spend across all customers
+- **Daily spend trends**: See how customer spend changes over time
+- **Model usage breakdown**: Understand which models each customer uses
+- **Activity metrics**: Track requests, tokens, and success rates per customer
+
+<Image img={require('../../img/customer_usage_analytics.png')} />
+
+#### 3. Filter by Customer
+
+Use the customer filter dropdown to view spend for specific customers:
+
+- Select one or more customer IDs from the dropdown
+- View filtered analytics, spend logs, and activity metrics
+- Compare spend across different customers
+
+<Image img={require('../../img/customer_usage_filter.png')} />
+
+## Use Cases
+
+### Customer Billing
+
+Track spend per customer to accurately bill your end users:
+
+- Monitor individual customer usage
+- Generate invoices based on actual spend
+- Set spending limits per customer
+
+### Usage Analytics
+
+Understand how different customers use your service:
+
+- Identify high-value customers
+- Analyze usage patterns
+- Optimize resource allocation
+
+---
+
+## Related Features
+
+- [Customers / End-User Budgets](./customers.md) - Set budgets and rate limits for customers
+- [Cost Tracking](./cost_tracking.md) - Comprehensive cost tracking and analytics
+- [Billing](./billing.md) - Bill customers based on their usage
--- a/docs/my-website/docs/proxy/deploy.md
+++ b/docs/my-website/docs/proxy/deploy.md
@ -26,8 +26,6 @@ echo 'LITELLM_MASTER_KEY="sk-1234"' > .env
 # password generator to get a random hash for litellm salt key
 echo 'LITELLM_SALT_KEY="sk-1234"' >> .env

-source .env
-
 # Start
 docker compose up
 ```
@ -1072,4 +1070,4 @@ A: We explored MySQL but that was hard to maintain and led to bugs for customers

 **Q: If there is Postgres downtime, how does LiteLLM react? Does it fail-open or is there API downtime?**

-A: You can gracefully handle DB unavailability if it's on your VPC. See our production guide for more details: [Gracefully Handle DB Unavailability](https://docs.litellm.ai/docs/proxy/prod#6-if-running-litellm-on-vpc-gracefully-handle-db-unavailability)
+A: You can gracefully handle DB unavailability if it's on your VPC. See our production guide for more details: [Gracefully Handle DB Unavailability](https://docs.litellm.ai/docs/proxy/prod#6-if-running-litellm-on-vpc-gracefully-handle-db-unavailability)
--- a/docs/my-website/docs/proxy/docker_quick_start.md
+++ b/docs/my-website/docs/proxy/docker_quick_start.md
@ -52,8 +52,6 @@ echo 'LITELLM_MASTER_KEY="sk-1234"' > .env
 # password generator to get a random hash for litellm salt key
 echo 'LITELLM_SALT_KEY="sk-1234"' >> .env

-source .env
-
 # Start
 docker compose up
 ```
--- a/docs/my-website/docs/proxy/dynamic_rate_limit.md
+++ b/docs/my-website/docs/proxy/dynamic_rate_limit.md
@ -175,7 +175,37 @@ general_settings:
 litellm --config /path/to/config.yaml
 ```

-#### 2. Create Keys with Priority Levels
+### Set priority on either a team or a key
+
+Priority can be set at either the **team level** or **key level**. Team-level priority takes precedence over key-level priority.
+
+**Option A: Set Priority on Team (Recommended)**
+
+All keys within a team will inherit the team's priority. This is useful when you want all keys for a specific environment or project to have the same priority.
+
+```bash
+curl -X POST 'http://0.0.0.0:4000/team/new' \
+-H 'Authorization: Bearer sk-1234' \
+-H 'Content-Type: application/json' \
+-d '{
+  "team_alias": "production-team",
+  "metadata": {"priority": "prod"}
+}'
+```
+
+Create a key for this team:
+```bash
+curl -X POST 'http://0.0.0.0:4000/key/generate' \
+-H 'Authorization: Bearer sk-1234' \
+-H 'Content-Type: application/json' \
+-d '{
+  "team_id": "team-id-from-previous-response"
+}'
+```
+
+**Option B: Set Priority on Individual Keys**
+
+Set priority directly on the key. This is useful when you need fine-grained control per key.

 **Production Key:**
 ```bash
@ -205,7 +235,7 @@ curl -X POST 'http://0.0.0.0:4000/key/generate' \
 -d '{}'
 ```

-**Expected Response for both:**
+**Expected Response:**
 ```json
 {
  "key": "sk-...",
@ -214,6 +244,11 @@ curl -X POST 'http://0.0.0.0:4000/key/generate' \
 }
 ```

+**Priority Resolution Order:**
+1. If key belongs to a team with `metadata.priority` set → use team priority
+2. Else if key has `metadata.priority` set → use key priority  
+3. Else → use `default_priority` from config
+
 #### 3. Test Priority Allocation

 **Test Production Key (should get 9 RPM):**
--- a/docs/my-website/docs/response_api.md
+++ b/docs/my-website/docs/response_api.md
@ -81,6 +81,85 @@ for event in stream:
            f.write(image_bytes)
 ```

+#### Image Generation (Non-streaming)
+
+Image generation is supported for models that generate images. Generated images are returned in the `output` array with `type: "image_generation_call"`.
+
+**Gemini (Google AI Studio):**
+```python showLineNumbers title="Gemini Image Generation"
+import litellm
+import base64
+
+# Gemini image generation models don't require tools parameter
+response = litellm.responses(
+    model="gemini/gemini-2.5-flash-image",
+    input="Generate a cute cat playing with yarn"
+)
+
+# Access generated images from output
+for item in response.output:
+    if item.type == "image_generation_call":
+        # item.result contains pure base64 (no data: prefix)
+        image_bytes = base64.b64decode(item.result)
+
+        # Save the image
+        with open(f"generated_{item.id}.png", "wb") as f:
+            f.write(image_bytes)
+
+print(f"Image saved: generated_{response.output[0].id}.png")
+```
+
+**OpenAI:**
+```python showLineNumbers title="OpenAI Image Generation"
+import litellm
+import base64
+
+# OpenAI models require tools parameter for image generation
+response = litellm.responses(
+    model="openai/gpt-4o",
+    input="Generate a futuristic city at sunset",
+    tools=[{"type": "image_generation"}]
+)
+
+# Access generated images from output
+for item in response.output:
+    if item.type == "image_generation_call":
+        image_bytes = base64.b64decode(item.result)
+        with open(f"generated_{item.id}.png", "wb") as f:
+            f.write(image_bytes)
+```
+
+**Response Format:**
+
+When image generation is successful, the response contains:
+
+```json
+{
+  "id": "resp_abc123",
+  "status": "completed",
+  "output": [
+    {
+      "type": "image_generation_call",
+      "id": "resp_abc123_img_0",
+      "status": "completed",
+      "result": "iVBORw0KGgo..."  // Pure base64 string (no data: prefix)
+    }
+  ]
+}
+```
+
+**Supported Models:**
+
+| Provider | Models | Requires `tools` Parameter |
+|----------|--------|---------------------------|
+| Google AI Studio | `gemini/gemini-2.5-flash-image` | ❌ No |
+| Vertex AI | `vertex_ai/gemini-2.5-flash-image-preview` | ❌ No |
+| OpenAI | `gpt-4o`, `gpt-4o-mini`, `gpt-4.1`, `gpt-4.1-mini`, `gpt-4.1-nano`, `o3` | ✅ Yes |
+| AWS Bedrock | Stability AI, Amazon Nova Canvas models | Model-specific |
+| Fal AI | Various image generation models | Check model docs |
+
+**Note:** The `result` field contains pure base64-encoded image data without the `data:image/png;base64,` prefix. You must decode it with `base64.b64decode()` before saving.
+
 #### GET a Response
 ```python showLineNumbers title="Get Response by ID"
 import litellm
--- a/docs/my-website/img/customer_usage.png
+++ b/docs/my-website/img/customer_usage.png
--- a/docs/my-website/img/customer_usage_analytics.png
+++ b/docs/my-website/img/customer_usage_analytics.png
--- a/docs/my-website/img/customer_usage_filter.png
+++ b/docs/my-website/img/customer_usage_filter.png
--- a/docs/my-website/img/customer_usage_ui_navigation.png
+++ b/docs/my-website/img/customer_usage_ui_navigation.png
--- a/docs/my-website/package-lock.json
+++ b/docs/my-website/package-lock.json
@ -14619,9 +14619,9 @@
      }
    },
    "node_modules/mdast-util-to-hast": {
-      "version": "13.2.0",
-      "resolved": "https://registry.npmjs.org/mdast-util-to-hast/-/mdast-util-to-hast-13.2.0.tgz",
-      "integrity": "sha512-QGYKEuUsYT9ykKBCMOEDLsU5JRObWQusAolFMeko/tYPufNkRffBAQjIE+99jbA87xv6FgmjLtwjh9wBWajwAA==",
+      "version": "13.2.1",
+      "resolved": "https://registry.npmjs.org/mdast-util-to-hast/-/mdast-util-to-hast-13.2.1.tgz",
+      "integrity": "sha512-cctsq2wp5vTsLIcaymblUriiTcZd0CwWtCbLvrOzYCDZoWyMNV8sZ7krj09FSnsiJi3WVsHLM4k6Dq/yaPyCXA==",
      "license": "MIT",
      "dependencies": {
        "@types/hast": "^3.0.0",
--- a/docs/my-website/package.json
+++ b/docs/my-website/package.json
@ -61,6 +61,7 @@
    "mermaid": ">=11.10.0",
    "gray-matter": "4.0.3",
    "glob": ">=11.1.0",
-    "node-forge": ">=1.3.2"
+    "node-forge": ">=1.3.2",
+    "mdast-util-to-hast": ">=13.2.1"
  }
-}
+}
--- a/docs/my-website/release_notes/v1.80.8-stable/index.md
+++ b/docs/my-website/release_notes/v1.80.8-stable/index.md
@ -0,0 +1,607 @@
+---
+title: "[Preview] v1.80.8.rc.1 - Introducing A2A Agent Gateway"
+slug: "v1-80-8"
+date: 2025-12-06T10:00:00
+authors:
+  - name: Krrish Dholakia
+    title: CEO, LiteLLM
+    url: https://www.linkedin.com/in/krish-d/
+    image_url: https://pbs.twimg.com/profile_images/1298587542745358340/DZv3Oj-h_400x400.jpg
+  - name: Ishaan Jaff
+    title: CTO, LiteLLM
+    url: https://www.linkedin.com/in/reffajnaahsi/
+    image_url: https://pbs.twimg.com/profile_images/1613813310264340481/lz54oEiB_400x400.jpg
+hide_table_of_contents: false
+---
+
+import Image from '@theme/IdealImage';
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+
+## Deploy this version
+
+<Tabs>
+<TabItem value="docker" label="Docker">
+
+``` showLineNumbers title="docker run litellm"
+docker run \
+-e STORE_MODEL_IN_DB=True \
+-p 4000:4000 \
+ghcr.io/berriai/litellm:v1.80.8.rc.1
+```
+
+</TabItem>
+
+<TabItem value="pip" label="Pip">
+
+``` showLineNumbers title="pip install litellm"
+pip install litellm==1.80.8
+```
+
+</TabItem>
+</Tabs>
+
+---
+
+## Key Highlights
+
+- **Agent Gateway (A2A)** - [Invoke agents through the AI Gateway with request/response logging and access controls](../../docs/a2a)
+- **Guardrails API v2** - [Generic Guardrail API with streaming support, structured messages, and tool call checks](../../docs/adding_provider/generic_guardrail_api)
+- **Customer (End User) Usage UI** - [Track and visualize end-user spend directly in the dashboard](../../docs/proxy/customer_usage)
+- **vLLM Batch + Files API** - [Support for batch and files API with vLLM deployments](../../docs/batches)
+- **Dynamic Rate Limiting on Teams** - [Enable dynamic rate limits and priority reservation on team-level](../../docs/proxy/team_budgets)
+- **Google Cloud Chirp3 HD** - [New text-to-speech provider with Chirp3 HD voices](../../docs/text_to_speech)
+
+---
+
+### Agent Gateway (A2A)
+
+<Image 
+  img={require('../../img/a2a_gateway.png')}
+  style={{width: '100%', display: 'block', margin: '2rem auto'}}
+/>
+
+<br/>
+
+This release introduces **A2A Agent Gateway** for LiteLLM, allowing you to invoke and manage A2A agents with the same controls you have for LLM APIs.
+
+As a **LiteLLM Gateway Admin**, you can now do the following:
+    - **Request/Response Logging** - Every agent invocation is logged to the Logs page with full request and response tracking.
+    - **Access Control** - Control which Team/Key can access which agents.
+
+As a developer, you can continue using the A2A SDK, all you need to do is point you `A2AClient` to the LiteLLM proxy URL and your API key.
+
+**Works with the A2A SDK:**
+
+```python
+from a2a.client import A2AClient
+
+client = A2AClient(
+    base_url="http://localhost:4000",  # Your LiteLLM proxy
+    api_key="sk-1234"                   # LiteLLM API key
+)
+
+response = client.send_message(
+    agent_id="my-agent",
+    message="What's the status of my order?"
+)
+```
+
+Get started with Agent Gateway here: [Agent Gateway Documentation](../../docs/a2a)
+
+---
+
+### Customer (End User) Usage UI
+
+<Image
+img={require('../../img/customer_usage.png')}
+style={{width: '100%', display: 'block', margin: '2rem auto'}}
+/>
+
+Users can now filter usage statistics by customers, providing the same granular filtering capabilities available for teams and organizations.
+
+**Details:**
+
+- Filter usage analytics, spend logs, and activity metrics by customer ID
+- View customer-level breakdowns alongside existing team and user-level filters
+- Consistent filtering experience across all usage and analytics views
+
+---
+
+## New Providers and Endpoints
+
+### New Providers (5 new providers)
+
+| Provider | Supported LiteLLM Endpoints | Description |
+| -------- | ------------------- | ----------- |
+| **[Z.AI (Zhipu AI)](../../docs/providers/zai)** | `/v1/chat/completions`, `/v1/responses`, `/v1/messages` | Built-in support for Zhipu AI GLM models |
+| **[RAGFlow](../../docs/providers/ragflow)** | `/v1/chat/completions`, `/v1/responses`, `/v1/messages`, `/v1/vector_stores` | RAG-based chat completions with vector store support |
+| **[PublicAI](../../docs/providers/publicai)** | `/v1/chat/completions`, `/v1/responses`, `/v1/messages` | OpenAI-compatible provider via JSON config |
+| **[Google Cloud Chirp3 HD](../../docs/text_to_speech)** | `/v1/audio/speech`, `/v1/audio/speech/stream` | Text-to-speech with Google Cloud Chirp3 HD voices |
+
+### New LLM API Endpoints (2 new endpoints)
+
+| Endpoint | Method | Description | Documentation |
+| -------- | ------ | ----------- | ------------- |
+| `/v1/agents/invoke` | POST | Invoke A2A agents through the AI Gateway | [Agent Gateway](../../docs/a2a) |
+| `/cursor/chat/completions` | POST | Cursor BYOK endpoint - accepts Responses API input, returns Chat Completions output | [Cursor Integration](../../docs/tutorials/cursor_integration) |
+
+---
+
+## New Models / Updated Models
+
+#### New Model Support (33 new models)
+
+| Provider | Model | Context Window | Input ($/1M tokens) | Output ($/1M tokens) | Features |
+| -------- | ----- | -------------- | ------------------- | -------------------- | -------- |
+| OpenAI | `gpt-5.1-codex-max` | 400K | $1.25 | $10.00 | Reasoning, vision, PDF input, responses API |
+| Azure | `azure/gpt-5.1-codex-max` | 400K | $1.25 | $10.00 | Reasoning, vision, PDF input, responses API |
+| Anthropic | `claude-opus-4-5` | 200K | $5.00 | $25.00 | Computer use, reasoning, vision |
+| Bedrock | `global.anthropic.claude-opus-4-5-20251101-v1:0` | 200K | $5.00 | $25.00 | Computer use, reasoning, vision |
+| Bedrock | `amazon.nova-2-lite-v1:0` | 1M | $0.30 | $2.50 | Reasoning, vision, video, PDF input |
+| Bedrock | `amazon.titan-image-generator-v2:0` | - | - | $0.008/image | Image generation |
+| Fireworks | `fireworks_ai/deepseek-v3p2` | 164K | $1.20 | $1.20 | Function calling, response schema |
+| Fireworks | `fireworks_ai/kimi-k2-instruct-0905` | 262K | $0.60 | $2.50 | Function calling, response schema |
+| DeepSeek | `deepseek/deepseek-v3.2` | 164K | $0.28 | $0.40 | Reasoning, function calling |
+| Mistral | `mistral/mistral-large-3` | 256K | $0.50 | $1.50 | Function calling, vision |
+| Azure AI | `azure_ai/mistral-large-3` | 256K | $0.50 | $1.50 | Function calling, vision |
+| Moonshot | `moonshot/kimi-k2-0905-preview` | 262K | $0.60 | $2.50 | Function calling, web search |
+| Moonshot | `moonshot/kimi-k2-turbo-preview` | 262K | $1.15 | $8.00 | Function calling, web search |
+| Moonshot | `moonshot/kimi-k2-thinking-turbo` | 262K | $1.15 | $8.00 | Function calling, web search |
+| OpenRouter | `openrouter/deepseek/deepseek-v3.2` | 164K | $0.28 | $0.40 | Reasoning, function calling |
+| Databricks | `databricks/databricks-claude-haiku-4-5` | 200K | $1.00 | $5.00 | Reasoning, function calling |
+| Databricks | `databricks/databricks-claude-opus-4` | 200K | $15.00 | $75.00 | Reasoning, function calling |
+| Databricks | `databricks/databricks-claude-opus-4-1` | 200K | $15.00 | $75.00 | Reasoning, function calling |
+| Databricks | `databricks/databricks-claude-opus-4-5` | 200K | $5.00 | $25.00 | Reasoning, function calling |
+| Databricks | `databricks/databricks-claude-sonnet-4` | 200K | $3.00 | $15.00 | Reasoning, function calling |
+| Databricks | `databricks/databricks-claude-sonnet-4-1` | 200K | $3.00 | $15.00 | Reasoning, function calling |
+| Databricks | `databricks/databricks-gemini-2-5-flash` | 1M | $0.30 | $2.50 | Function calling |
+| Databricks | `databricks/databricks-gemini-2-5-pro` | 1M | $1.25 | $10.00 | Function calling |
+| Databricks | `databricks/databricks-gpt-5` | 400K | $1.25 | $10.00 | Function calling |
+| Databricks | `databricks/databricks-gpt-5-1` | 400K | $1.25 | $10.00 | Function calling |
+| Databricks | `databricks/databricks-gpt-5-mini` | 400K | $0.25 | $2.00 | Function calling |
+| Databricks | `databricks/databricks-gpt-5-nano` | 400K | $0.05 | $0.40 | Function calling |
+| Vertex AI | `vertex_ai/chirp` | - | $30.00/1M chars | - | Text-to-speech (Chirp3 HD) |
+| Z.AI | `zai/glm-4.6` | 200K | $0.60 | $2.20 | Function calling |
+| Z.AI | `zai/glm-4.5` | 128K | $0.60 | $2.20 | Function calling |
+| Z.AI | `zai/glm-4.5v` | 128K | $0.60 | $1.80 | Function calling, vision |
+| Z.AI | `zai/glm-4.5-flash` | 128K | Free | Free | Function calling |
+| Vertex AI | `vertex_ai/bge-large-en-v1.5` | - | - | - | BGE Embeddings |
+
+#### Features
+
+- **[OpenAI](../../docs/providers/openai)**
+    - Add `gpt-5.1-codex-max` model pricing and configuration - [PR #17541](https://github.com/BerriAI/litellm/pull/17541)
+    - Add xhigh reasoning effort for gpt-5.1-codex-max - [PR #17585](https://github.com/BerriAI/litellm/pull/17585)
+    - Add clear error message for empty LLM endpoint responses - [PR #17445](https://github.com/BerriAI/litellm/pull/17445)
+
+- **[Azure OpenAI](../../docs/providers/azure/azure)**
+    - Allow reasoning_effort='none' for Azure gpt-5.1 models - [PR #17311](https://github.com/BerriAI/litellm/pull/17311)
+
+- **[Anthropic](../../docs/providers/anthropic)**
+    - Add `claude-opus-4-5` alias to pricing data - [PR #17313](https://github.com/BerriAI/litellm/pull/17313)
+    - Parse `<budget:thinking>` blocks for opus 4.5 - [PR #17534](https://github.com/BerriAI/litellm/pull/17534)
+    - Update new Anthropic features as reviewed - [PR #17142](https://github.com/BerriAI/litellm/pull/17142)
+    - Skip empty text blocks in Anthropic system messages - [PR #17442](https://github.com/BerriAI/litellm/pull/17442)
+
+- **[Bedrock](../../docs/providers/bedrock)**
+    - Add Nova embedding support - [PR #17253](https://github.com/BerriAI/litellm/pull/17253)
+    - Add support for Bedrock Qwen 2 imported model - [PR #17461](https://github.com/BerriAI/litellm/pull/17461)
+    - Bedrock OpenAI model support - [PR #17368](https://github.com/BerriAI/litellm/pull/17368)
+    - Add support for file content download for Bedrock batches - [PR #17470](https://github.com/BerriAI/litellm/pull/17470)
+    - Make streaming chunk size configurable in Bedrock API - [PR #17357](https://github.com/BerriAI/litellm/pull/17357)
+    - Add experimental latest-user filtering for Bedrock - [PR #17282](https://github.com/BerriAI/litellm/pull/17282)
+    - Handle Cohere v4 embed response dictionary format - [PR #17220](https://github.com/BerriAI/litellm/pull/17220)
+    - Remove not compatible beta header from Bedrock - [PR #17301](https://github.com/BerriAI/litellm/pull/17301)
+    - Add model price and details for Global Opus 4.5 Bedrock endpoint - [PR #17380](https://github.com/BerriAI/litellm/pull/17380)
+
+- **[Gemini (Google AI Studio + Vertex AI)](../../docs/providers/gemini)**
+    - Add better handling in image generation for Gemini models - [PR #17292](https://github.com/BerriAI/litellm/pull/17292)
+    - Fix reasoning_content showing duplicate content in streaming responses - [PR #17266](https://github.com/BerriAI/litellm/pull/17266)
+    - Handle partial JSON chunks after first valid chunk - [PR #17496](https://github.com/BerriAI/litellm/pull/17496)
+    - Fix Gemini 3 last chunk thinking block - [PR #17403](https://github.com/BerriAI/litellm/pull/17403)
+    - Fix Gemini image_tokens treated as text tokens in cost calculation - [PR #17554](https://github.com/BerriAI/litellm/pull/17554)
+    - Make sure that media resolution is only for Gemini 3 model - [PR #17137](https://github.com/BerriAI/litellm/pull/17137)
+
+- **[Vertex AI](../../docs/providers/vertex)**
+    - Add Google Cloud Chirp3 HD support on /speech - [PR #17391](https://github.com/BerriAI/litellm/pull/17391)
+    - Add BGE Embeddings support - [PR #17362](https://github.com/BerriAI/litellm/pull/17362)
+    - Handle global location for Vertex AI image generation endpoint - [PR #17255](https://github.com/BerriAI/litellm/pull/17255)
+    - Add Google Private API Endpoint to Vertex AI fields - [PR #17382](https://github.com/BerriAI/litellm/pull/17382)
+
+- **[Z.AI (Zhipu AI)](../../docs/providers/zai)**
+    - Add Z.AI as built-in provider - [PR #17307](https://github.com/BerriAI/litellm/pull/17307)
+
+- **[GitHub Copilot](../../docs/providers/github_copilot)**
+    - Add Embedding API support - [PR #17278](https://github.com/BerriAI/litellm/pull/17278)
+    - Preserve encrypted_content in reasoning items for multi-turn conversations - [PR #17130](https://github.com/BerriAI/litellm/pull/17130)
+
+- **[Databricks](../../docs/providers/databricks)**
+    - Update Databricks model pricing and add new models - [PR #17277](https://github.com/BerriAI/litellm/pull/17277)
+
+- **[OVHcloud](../../docs/providers/ovhcloud)**
+    - Add support of audio transcription for OVHcloud - [PR #17305](https://github.com/BerriAI/litellm/pull/17305)
+
+- **[Mistral](../../docs/providers/mistral)**
+    - Add Mistral Large 3 model support - [PR #17547](https://github.com/BerriAI/litellm/pull/17547)
+
+- **[Moonshot](../../docs/providers/moonshot)**
+    - Fix missing Moonshot turbo models and fix incorrect pricing - [PR #17432](https://github.com/BerriAI/litellm/pull/17432)
+
+- **[Together AI](../../docs/providers/togetherai)**
+    - Add context window exception mapping for Together AI - [PR #17284](https://github.com/BerriAI/litellm/pull/17284)
+
+- **[WatsonX](../../docs/providers/watsonx/index)**
+    - Allow passing zen_api_key dynamically - [PR #16655](https://github.com/BerriAI/litellm/pull/16655)
+    - Fix Watsonx Audio Transcription API - [PR #17326](https://github.com/BerriAI/litellm/pull/17326)
+    - Fix audio transcriptions, don't force content type in request headers - [PR #17546](https://github.com/BerriAI/litellm/pull/17546)
+
+- **[Fireworks AI](../../docs/providers/fireworks_ai)**
+    - Add new model `fireworks_ai/kimi-k2-instruct-0905` - [PR #17328](https://github.com/BerriAI/litellm/pull/17328)
+    - Add `fireworks/deepseek-v3p2` - [PR #17395](https://github.com/BerriAI/litellm/pull/17395)
+
+- **[DeepSeek](../../docs/providers/deepseek)**
+    - Support Deepseek 3.2 with Reasoning - [PR #17384](https://github.com/BerriAI/litellm/pull/17384)
+
+- **[Nova Lite 2](../../docs/providers/bedrock)**
+    - Add Nova Lite 2 reasoning support with reasoningConfig - [PR #17371](https://github.com/BerriAI/litellm/pull/17371)
+
+- **[Ollama](../../docs/providers/ollama)**
+    - Fix auth not working with ollama.com - [PR #17191](https://github.com/BerriAI/litellm/pull/17191)
+
+- **[Groq](../../docs/providers/groq)**
+    - Fix supports_response_schema before using json_tool_call workaround - [PR #17438](https://github.com/BerriAI/litellm/pull/17438)
+
+- **[vLLM](../../docs/providers/vllm)**
+    - Fix empty response + vLLM streaming - [PR #17516](https://github.com/BerriAI/litellm/pull/17516)
+
+- **[Azure AI](../../docs/providers/azure_ai)**
+    - Migrate Anthropic provider to Azure AI - [PR #17202](https://github.com/BerriAI/litellm/pull/17202)
+    - Fix GA path for Azure OpenAI realtime models - [PR #17260](https://github.com/BerriAI/litellm/pull/17260)
+
+- **[Bedrock TwelveLabs](../../docs/providers/bedrock#twelvelabs-pegasus---video-understanding)**
+    - Add support for TwelveLabs Pegasus video understanding - [PR #17193](https://github.com/BerriAI/litellm/pull/17193)
+
+### Bug Fixes
+
+- **[Bedrock](../../docs/providers/bedrock)**
+    - Fix extra_headers in messages API bedrock invoke - [PR #17271](https://github.com/BerriAI/litellm/pull/17271)
+    - Fix Bedrock models in model map - [PR #17419](https://github.com/BerriAI/litellm/pull/17419)
+    - Make Bedrock converse messages respect modify_params as expected - [PR #17427](https://github.com/BerriAI/litellm/pull/17427)
+    - Fix Anthropic beta headers for Bedrock imported Qwen models - [PR #17467](https://github.com/BerriAI/litellm/pull/17467)
+    - Preserve usage from JSON response for OpenAI provider in Bedrock - [PR #17589](https://github.com/BerriAI/litellm/pull/17589)
+
+- **[SambaNova](../../docs/providers/sambanova)**
+    - Fix acompletion throws error with SambaNova models - [PR #17217](https://github.com/BerriAI/litellm/pull/17217)
+
+- **General**
+    - Fix AttributeError when metadata is null in request body - [PR #17306](https://github.com/BerriAI/litellm/pull/17306)
+    - Fix 500 error for malformed request - [PR #17291](https://github.com/BerriAI/litellm/pull/17291)
+    - Respect custom LLM provider in header - [PR #17290](https://github.com/BerriAI/litellm/pull/17290)
+    - Replace deprecated .dict() with .model_dump() in streaming_handler - [PR #17359](https://github.com/BerriAI/litellm/pull/17359)
+
+---
+
+## LLM API Endpoints
+
+#### Features
+
+- **[Responses API](../../docs/response_api)**
+    - Add cost tracking for responses API - [PR #17258](https://github.com/BerriAI/litellm/pull/17258)
+    - Map output_tokens_details of responses API to completion_tokens_details - [PR #17458](https://github.com/BerriAI/litellm/pull/17458)
+    - Add image generation support for Responses API - [PR #16586](https://github.com/BerriAI/litellm/pull/16586)
+
+- **[Batch API](../../docs/batches)**
+    - Add vLLM batch+files API support - [PR #15823](https://github.com/BerriAI/litellm/pull/15823)
+    - Fix optional parameter default value - [PR #17434](https://github.com/BerriAI/litellm/pull/17434)
+    - Add status parameter as optional for FileObject - [PR #17431](https://github.com/BerriAI/litellm/pull/17431)
+
+- **[Video Generation API](../../docs/videos)**
+    - Add passthrough cost tracking for Veo - [PR #17296](https://github.com/BerriAI/litellm/pull/17296)
+
+- **[OCR API](../../docs/ocr)**
+    - Add missing OCR and aOCR to CallTypes enum - [PR #17435](https://github.com/BerriAI/litellm/pull/17435)
+
+- **General**
+    - Support routing to only websearch supported deployments - [PR #17500](https://github.com/BerriAI/litellm/pull/17500)
+
+#### Bugs
+
+- **General**
+    - Fix streaming error validation - [PR #17242](https://github.com/BerriAI/litellm/pull/17242)
+    - Add length validation for empty tool_calls in delta - [PR #17523](https://github.com/BerriAI/litellm/pull/17523)
+
+---
+
+## Management Endpoints / UI
+
+#### Features
+
+- **New Login Page**
+    - New Login Page UI - [PR #17443](https://github.com/BerriAI/litellm/pull/17443)
+    - Refactor /login route - [PR #17379](https://github.com/BerriAI/litellm/pull/17379)
+    - Add auto_redirect_to_sso to UI Config - [PR #17399](https://github.com/BerriAI/litellm/pull/17399)
+    - Add Auto Redirect to SSO to New Login Page - [PR #17451](https://github.com/BerriAI/litellm/pull/17451)
+
+- **Customer (End User) Usage**
+    - Customer (end user) Usage feature - [PR #17498](https://github.com/BerriAI/litellm/pull/17498)
+    - Customer Usage UI - [PR #17506](https://github.com/BerriAI/litellm/pull/17506)
+    - Add Info Banner for Customer Usage - [PR #17598](https://github.com/BerriAI/litellm/pull/17598)
+
+- **Virtual Keys**
+    - Standardize API Key vs Virtual Key in UI - [PR #17325](https://github.com/BerriAI/litellm/pull/17325)
+    - Add User Alias Column to Internal User Table - [PR #17321](https://github.com/BerriAI/litellm/pull/17321)
+    - Delete Credential Enhancements - [PR #17317](https://github.com/BerriAI/litellm/pull/17317)
+
+- **Models + Endpoints**
+    - Show all credential values on Edit Credential Modal - [PR #17397](https://github.com/BerriAI/litellm/pull/17397)
+    - Change Edit Team Models Shown to Match Create Team - [PR #17394](https://github.com/BerriAI/litellm/pull/17394)
+    - Support Images in Compare UI - [PR #17562](https://github.com/BerriAI/litellm/pull/17562)
+
+- **Callbacks**
+    - Show all callbacks on UI - [PR #16335](https://github.com/BerriAI/litellm/pull/16335)
+    - Credentials to use React Query - [PR #17465](https://github.com/BerriAI/litellm/pull/17465)
+
+- **Management Routes**
+    - Allow admin viewer to access global tag usage - [PR #17501](https://github.com/BerriAI/litellm/pull/17501)
+    - Allow wildcard routes for nonproxy admin (SCIM) - [PR #17178](https://github.com/BerriAI/litellm/pull/17178)
+    - Return 404 when a user is not found on /user/info - [PR #16850](https://github.com/BerriAI/litellm/pull/16850)
+
+- **OCI Configuration**
+    - Enable Oracle Cloud Infrastructure configuration via UI - [PR #17159](https://github.com/BerriAI/litellm/pull/17159)
+
+#### Bugs
+
+- **UI Fixes**
+    - Fix Request and Response Panel JSONViewer - [PR #17233](https://github.com/BerriAI/litellm/pull/17233)
+    - Adding Button Loading States to Edit Settings - [PR #17236](https://github.com/BerriAI/litellm/pull/17236)
+    - Fix Various Text, button state, and test changes - [PR #17237](https://github.com/BerriAI/litellm/pull/17237)
+    - Fix Fallbacks Immediately Deleting before API resolves - [PR #17238](https://github.com/BerriAI/litellm/pull/17238)
+    - Remove Feature Flags - [PR #17240](https://github.com/BerriAI/litellm/pull/17240)
+    - Fix metadata tags and model name display in UI for Azure passthrough - [PR #17258](https://github.com/BerriAI/litellm/pull/17258)
+    - Change labeling around Vertex Fields - [PR #17383](https://github.com/BerriAI/litellm/pull/17383)
+    - Remove second scrollbar when sidebar is expanded + tooltip z index - [PR #17436](https://github.com/BerriAI/litellm/pull/17436)
+    - Fix Select in Edit Membership Modal - [PR #17524](https://github.com/BerriAI/litellm/pull/17524)
+    - Change useAuthorized Hook to redirect to new Login Page - [PR #17553](https://github.com/BerriAI/litellm/pull/17553)
+
+- **SSO**
+    - Fix the generic SSO provider - [PR #17227](https://github.com/BerriAI/litellm/pull/17227)
+    - Clear SSO integration for all users - [PR #17287](https://github.com/BerriAI/litellm/pull/17287)
+    - Fix SSO users not added to Entra synced team - [PR #17331](https://github.com/BerriAI/litellm/pull/17331)
+
+- **Auth / JWT**
+    - JWT Auth - Allow using regular OIDC flow with user info endpoints - [PR #17324](https://github.com/BerriAI/litellm/pull/17324)
+    - Fix litellm user auth not passing issue - [PR #17342](https://github.com/BerriAI/litellm/pull/17342)
+    - Add other routes in JWT auth - [PR #17345](https://github.com/BerriAI/litellm/pull/17345)
+    - Fix new org team validate against org - [PR #17333](https://github.com/BerriAI/litellm/pull/17333)
+    - Fix litellm_enterprise ensure imported routes exist - [PR #17337](https://github.com/BerriAI/litellm/pull/17337)
+    - Use organization.members instead of deprecated organization field - [PR #17557](https://github.com/BerriAI/litellm/pull/17557)
+
+- **Organizations/Teams**
+    - Fix organization max budget not enforced - [PR #17334](https://github.com/BerriAI/litellm/pull/17334)
+    - Fix budget update to allow null max_budget - [PR #17545](https://github.com/BerriAI/litellm/pull/17545)
+
+---
+
+## AI Integrations (2 new integrations)
+
+### Logging (1 new integration)
+
+#### New Integration
+
+- **[Weave](../../docs/proxy/logging)**
+    - Basic Weave OTEL integration - [PR #17439](https://github.com/BerriAI/litellm/pull/17439)
+
+#### Improvements & Fixes
+
+- **[DataDog](../../docs/proxy/logging#datadog)**
+    - Fix Datadog callback regression when ddtrace is installed - [PR #17393](https://github.com/BerriAI/litellm/pull/17393)
+
+- **[Arize Phoenix](../../docs/observability/arize_integration)**
+    - Fix clean arize-phoenix traces - [PR #16611](https://github.com/BerriAI/litellm/pull/16611)
+
+- **[MLflow](../../docs/proxy/logging#mlflow)**
+    - Fix MLflow streaming spans for Anthropic passthrough - [PR #17288](https://github.com/BerriAI/litellm/pull/17288)
+
+- **[Langfuse](../../docs/proxy/logging#langfuse)**
+    - Fix Langfuse logger test mock setup - [PR #17591](https://github.com/BerriAI/litellm/pull/17591)
+
+- **General**
+    - Improve PII anonymization handling in logging callbacks - [PR #17207](https://github.com/BerriAI/litellm/pull/17207)
+
+### Guardrails (1 new integration)
+
+#### New Integration
+
+- **[Generic Guardrail API](../../docs/adding_provider/generic_guardrail_api)**
+    - Generic Guardrail API - allows guardrail providers to add INSTANT support for LiteLLM w/out PR to repo - [PR #17175](https://github.com/BerriAI/litellm/pull/17175)
+    - Guardrails API V2 - user api key metadata, session id, specify input type (request/response), image support - [PR #17338](https://github.com/BerriAI/litellm/pull/17338)
+    - Guardrails API - add streaming support - [PR #17400](https://github.com/BerriAI/litellm/pull/17400)
+    - Guardrails API - support tool call checks on OpenAI `/chat/completions`, OpenAI `/responses`, Anthropic `/v1/messages` - [PR #17459](https://github.com/BerriAI/litellm/pull/17459)
+    - Guardrails API - new `structured_messages` param - [PR #17518](https://github.com/BerriAI/litellm/pull/17518)
+    - Correctly map a v1/messages call to the anthropic unified guardrail - [PR #17424](https://github.com/BerriAI/litellm/pull/17424)
+    - Support during_call event type for unified guardrails - [PR #17514](https://github.com/BerriAI/litellm/pull/17514)
+
+#### Improvements & Fixes
+
+- **[Noma Guardrail](../../docs/proxy/guardrails/noma_security)**
+    - Refactor Noma guardrail to use shared Responses transformation and include system instructions - [PR #17315](https://github.com/BerriAI/litellm/pull/17315)
+
+- **[Presidio](../../docs/proxy/guardrails/pii_masking_v2)**
+    - Handle empty content and error dict responses in guardrails - [PR #17489](https://github.com/BerriAI/litellm/pull/17489)
+    - Fix Presidio guardrail test TypeError and license base64 decoding error - [PR #17538](https://github.com/BerriAI/litellm/pull/17538)
+
+- **[Tool Permissions](../../docs/proxy/guardrails/tool_permission)**
+    - Add regex-based tool_name/tool_type matching for tool-permission - [PR #17164](https://github.com/BerriAI/litellm/pull/17164)
+    - Add images for tool permission guardrail documentation - [PR #17322](https://github.com/BerriAI/litellm/pull/17322)
+
+- **[AIM Guardrails](../../docs/proxy/guardrails/aim_security)**
+    - Fix AIM guardrail tests - [PR #17499](https://github.com/BerriAI/litellm/pull/17499)
+
+- **[Bedrock Guardrails](../../docs/proxy/guardrails/bedrock)**
+    - Fix Bedrock Guardrail indent and import - [PR #17378](https://github.com/BerriAI/litellm/pull/17378)
+
+- **General Guardrails**
+    - Mask all matching keywords in content filter - [PR #17521](https://github.com/BerriAI/litellm/pull/17521)
+    - Ensure guardrail metadata is preserved in request_data - [PR #17593](https://github.com/BerriAI/litellm/pull/17593)
+    - Fix apply_guardrail method and improve test isolation - [PR #17555](https://github.com/BerriAI/litellm/pull/17555)
+
+### Secret Managers
+
+- **[CyberArk](../../docs/secret_managers/cyberark)**
+    - Allow setting SSL verify to false - [PR #17433](https://github.com/BerriAI/litellm/pull/17433)
+
+- **General**
+    - Make email and secret manager operations independent in key management hooks - [PR #17551](https://github.com/BerriAI/litellm/pull/17551)
+
+---
+
+## Spend Tracking, Budgets and Rate Limiting
+
+- **Rate Limiting**
+    - Parallel Request Limiter with /messages - [PR #17426](https://github.com/BerriAI/litellm/pull/17426)
+    - Allow using dynamic rate limit/priority reservation on teams - [PR #17061](https://github.com/BerriAI/litellm/pull/17061)
+    - Dynamic Rate Limiter - Fix token count increases/decreases by 1 instead of actual count + Redis TTL - [PR #17558](https://github.com/BerriAI/litellm/pull/17558)
+
+- **Spend Logs**
+    - Deprecate `spend/logs` & add `spend/logs/v2` - [PR #17167](https://github.com/BerriAI/litellm/pull/17167)
+    - Optimize SpendLogs queries to use timestamp filtering for index usage - [PR #17504](https://github.com/BerriAI/litellm/pull/17504)
+
+- **Enforce User Param**
+    - Enforce support of enforce_user_param to OpenAI post endpoints - [PR #17407](https://github.com/BerriAI/litellm/pull/17407)
+
+---
+
+## MCP Gateway
+
+- **MCP Configuration**
+    - Remove URL format validation for MCP server endpoints - [PR #17270](https://github.com/BerriAI/litellm/pull/17270)
+    - Add stack trace to MCP error message - [PR #17269](https://github.com/BerriAI/litellm/pull/17269)
+
+- **MCP Tool Results**
+    - Preserve tool metadata in CallToolResult - [PR #17561](https://github.com/BerriAI/litellm/pull/17561)
+
+---
+
+## Agent Gateway (A2A)
+
+- **Agent Invocation**
+    - Allow invoking agents through AI Gateway - [PR #17440](https://github.com/BerriAI/litellm/pull/17440)
+    - Allow tracking request/response in "Logs" Page - [PR #17449](https://github.com/BerriAI/litellm/pull/17449)
+
+- **Agent Access Control**
+    - Enforce Allowed agents by key, team + add agent access groups on backend - [PR #17502](https://github.com/BerriAI/litellm/pull/17502)
+
+- **Agent Gateway UI**
+    - Allow testing agents on UI - [PR #17455](https://github.com/BerriAI/litellm/pull/17455)
+    - Set allowed agents by key, team - [PR #17511](https://github.com/BerriAI/litellm/pull/17511)
+
+---
+
+## Performance / Loadbalancing / Reliability improvements
+
+- **Audio/Speech Performance**
+    - Fix `/audio/speech` performance by using `shared_sessions` - [PR #16739](https://github.com/BerriAI/litellm/pull/16739)
+
+- **Memory Optimization**
+    - Prevent memory leak in aiohttp connection pooling - [PR #17388](https://github.com/BerriAI/litellm/pull/17388)
+    - Lazy-load utils to reduce memory + import time - [PR #17171](https://github.com/BerriAI/litellm/pull/17171)
+
+- **Database**
+    - Update default database connection number - [PR #17353](https://github.com/BerriAI/litellm/pull/17353)
+    - Update default proxy_batch_write_at number - [PR #17355](https://github.com/BerriAI/litellm/pull/17355)
+    - Add background health checks to db - [PR #17528](https://github.com/BerriAI/litellm/pull/17528)
+
+- **Proxy Caching**
+    - Fix proxy caching between requests in aiohttp transport - [PR #17122](https://github.com/BerriAI/litellm/pull/17122)
+
+- **Session Management**
+    - Fix session consistency, move Lasso API version away from source code - [PR #17316](https://github.com/BerriAI/litellm/pull/17316)
+    - Conditionally pass enable_cleanup_closed to aiohttp TCPConnector - [PR #17367](https://github.com/BerriAI/litellm/pull/17367)
+
+- **Vector Store**
+    - Fix vector store configuration synchronization failure - [PR #17525](https://github.com/BerriAI/litellm/pull/17525)
+
+---
+
+## Documentation Updates
+
+- **Provider Documentation**
+    - Add Azure AI Foundry documentation for Claude models - [PR #17104](https://github.com/BerriAI/litellm/pull/17104)
+    - Document responses and embedding API for GitHub Copilot - [PR #17456](https://github.com/BerriAI/litellm/pull/17456)
+    - Add gpt-5.1-codex-max to OpenAI provider documentation - [PR #17602](https://github.com/BerriAI/litellm/pull/17602)
+    - Update Instructions For Phoenix Integration - [PR #17373](https://github.com/BerriAI/litellm/pull/17373)
+
+- **Guides**
+    - Add guide on how to debug gateway error vs provider error - [PR #17387](https://github.com/BerriAI/litellm/pull/17387)
+    - Agent Gateway documentation - [PR #17454](https://github.com/BerriAI/litellm/pull/17454)
+    - A2A Permission management documentation - [PR #17515](https://github.com/BerriAI/litellm/pull/17515)
+    - Update docs to link agent hub - [PR #17462](https://github.com/BerriAI/litellm/pull/17462)
+
+- **Projects**
+    - Add Google ADK and Harbor to projects - [PR #17352](https://github.com/BerriAI/litellm/pull/17352)
+    - Add Microsoft Agent Lightning to projects - [PR #17422](https://github.com/BerriAI/litellm/pull/17422)
+
+- **Cleanup**
+    - Cleanup: Remove orphan docs pages and Docusaurus template files - [PR #17356](https://github.com/BerriAI/litellm/pull/17356)
+    - Remove `source .env` from docs - [PR #17466](https://github.com/BerriAI/litellm/pull/17466)
+
+---
+
+## Infrastructure / CI/CD
+
+- **Helm Chart**
+    - Add ingress-only labels - [PR #17348](https://github.com/BerriAI/litellm/pull/17348)
+
+- **Docker**
+    - Add retry logic to apk package installation in Dockerfile.non_root - [PR #17596](https://github.com/BerriAI/litellm/pull/17596)
+    - Chainguard fixes - [PR #17406](https://github.com/BerriAI/litellm/pull/17406)
+
+- **OpenAPI Schema**
+    - Refactor add_schema_to_components to move definitions to components/schemas - [PR #17389](https://github.com/BerriAI/litellm/pull/17389)
+
+- **Security**
+    - Fix security vulnerability: update mdast-util-to-hast to 13.2.1 - [PR #17601](https://github.com/BerriAI/litellm/pull/17601)
+    - Bump jws from 3.2.2 to 3.2.3 - [PR #17494](https://github.com/BerriAI/litellm/pull/17494)
+
+---
+
+## New Contributors
+
+* @weichiet made their first contribution in [PR #17242](https://github.com/BerriAI/litellm/pull/17242)
+* @AndyForest made their first contribution in [PR #17220](https://github.com/BerriAI/litellm/pull/17220)
+* @omkar806 made their first contribution in [PR #17217](https://github.com/BerriAI/litellm/pull/17217)
+* @v0rtex20k made their first contribution in [PR #17178](https://github.com/BerriAI/litellm/pull/17178)
+* @hxomer made their first contribution in [PR #17207](https://github.com/BerriAI/litellm/pull/17207)
+* @orgersh92 made their first contribution in [PR #17316](https://github.com/BerriAI/litellm/pull/17316)
+* @dannykopping made their first contribution in [PR #17313](https://github.com/BerriAI/litellm/pull/17313)
+* @rioiart made their first contribution in [PR #17333](https://github.com/BerriAI/litellm/pull/17333)
+* @codgician made their first contribution in [PR #17278](https://github.com/BerriAI/litellm/pull/17278)
+* @epistoteles made their first contribution in [PR #17277](https://github.com/BerriAI/litellm/pull/17277)
+* @kothamah made their first contribution in [PR #17368](https://github.com/BerriAI/litellm/pull/17368)
+* @flozonn made their first contribution in [PR #17371](https://github.com/BerriAI/litellm/pull/17371)
+* @richardmcsong made their first contribution in [PR #17389](https://github.com/BerriAI/litellm/pull/17389)
+* @matt-greathouse made their first contribution in [PR #17384](https://github.com/BerriAI/litellm/pull/17384)
+* @mossbanay made their first contribution in [PR #17380](https://github.com/BerriAI/litellm/pull/17380)
+* @mhielpos-asapp made their first contribution in [PR #17376](https://github.com/BerriAI/litellm/pull/17376)
+* @Joilence made their first contribution in [PR #17367](https://github.com/BerriAI/litellm/pull/17367)
+* @deepaktammali made their first contribution in [PR #17357](https://github.com/BerriAI/litellm/pull/17357)
+* @axiomofjoy made their first contribution in [PR #16611](https://github.com/BerriAI/litellm/pull/16611)
+* @DevajMody made their first contribution in [PR #17445](https://github.com/BerriAI/litellm/pull/17445)
+* @andrewtruong made their first contribution in [PR #17439](https://github.com/BerriAI/litellm/pull/17439)
+* @AnasAbdelR made their first contribution in [PR #17490](https://github.com/BerriAI/litellm/pull/17490)
+* @dominicfeliton made their first contribution in [PR #17516](https://github.com/BerriAI/litellm/pull/17516)
+* @kristianmitk made their first contribution in [PR #17504](https://github.com/BerriAI/litellm/pull/17504)
+* @rgshr made their first contribution in [PR #17130](https://github.com/BerriAI/litellm/pull/17130)
+* @dominicfallows made their first contribution in [PR #17489](https://github.com/BerriAI/litellm/pull/17489)
+* @irfansofyana made their first contribution in [PR #17467](https://github.com/BerriAI/litellm/pull/17467)
+* @GusBricker made their first contribution in [PR #17191](https://github.com/BerriAI/litellm/pull/17191)
+* @OlivverX made their first contribution in [PR #17255](https://github.com/BerriAI/litellm/pull/17255)
+* @withsmilo made their first contribution in [PR #17585](https://github.com/BerriAI/litellm/pull/17585)
+
+---
+
+## Full Changelog
+
+**[View complete changelog on GitHub](https://github.com/BerriAI/litellm/compare/v1.80.7-nightly...v1.80.8)**
+
--- a/docs/my-website/sidebars.js
+++ b/docs/my-website/sidebars.js
@ -130,6 +130,16 @@ const sidebars = {
      },
      items: [
        "proxy/docker_quick_start",
+        {
+          type: "link",
+          label: "A2A Agent Gateway",
+          href: "https://docs.litellm.ai/docs/a2a",
+        },
+        {
+          type: "link",
+          label: "MCP Gateway",
+          href: "https://docs.litellm.ai/docs/mcp",
+        },
        {
          "type": "category",
          "label": "Config.yaml",
@ -186,6 +196,7 @@ const sidebars = {
          label: "Architecture",
          items: [
            "proxy/architecture",
+            "proxy/multi_tenant_architecture",
            "proxy/control_plane_and_data_plane",
            "proxy/db_deadlocks",
            "proxy/db_info",
@ -224,6 +235,7 @@ const sidebars = {
            "proxy/team_budgets",
            "proxy/tag_budgets",
            "proxy/customers",
+            "proxy/customer_usage",
            "proxy/dynamic_rate_limit",
            "proxy/rate_limit_tiers",
            "proxy/temporary_budget_increase",
@ -317,6 +329,14 @@ const sidebars = {
        slug: "/supported_endpoints",
      },
      items: [
+        {
+          type: "category",
+          label: "/a2a - A2A Agent Gateway",
+          items: [
+            "a2a",
+            "a2a_agent_permissions",
+          ],
+        },
        "assistants",
        {
          type: "category",
@ -472,6 +492,11 @@ const sidebars = {
          id: "provider_registration/index",
          label: "Integrate as a Model Provider",
        },
+        {
+          type: "doc",
+          id: "contributing/adding_openai_compatible_providers",
+          label: "Add OpenAI-Compatible Provider (JSON)",
+        },
        {
          type: "doc",
          id: "provider_registration/add_model_pricing",
@ -797,6 +822,7 @@ const sidebars = {
          type: "category",
          label: "Adding Providers",
          items: [
+            "contributing/adding_openai_compatible_providers",
            "adding_provider/directory_structure",
            "adding_provider/new_rerank_provider",
          ]
@ -823,13 +849,14 @@ const sidebars = {
              "Learn how to deploy + call models from different providers on LiteLLM",
            slug: "/project",
          },
-          items: [
+          items: [            
            "projects/smolagents",
            "projects/mini-swe-agent",
            "projects/openai-agents",
            "projects/Google ADK",
            "projects/Agent Lightning",
            "projects/Harbor",
+            "projects/GraphRAG",
            "projects/Docq.AI",
            "projects/PDL",
            "projects/OpenInterpreter",
--- a/litellm-proxy-extras/dist/litellm_proxy_extras-0.4.10-py3-none-any.whl
+++ b/litellm-proxy-extras/dist/litellm_proxy_extras-0.4.10-py3-none-any.whl
--- a/litellm-proxy-extras/dist/litellm_proxy_extras-0.4.10.tar.gz
+++ b/litellm-proxy-extras/dist/litellm_proxy_extras-0.4.10.tar.gz
--- a/litellm-proxy-extras/dist/litellm_proxy_extras-0.4.11-py3-none-any.whl
+++ b/litellm-proxy-extras/dist/litellm_proxy_extras-0.4.11-py3-none-any.whl
--- a/litellm-proxy-extras/dist/litellm_proxy_extras-0.4.11.tar.gz
+++ b/litellm-proxy-extras/dist/litellm_proxy_extras-0.4.11.tar.gz
--- a/litellm-proxy-extras/litellm_proxy_extras/migrations/20251204124859_add_end_user_spend_table/migration.sql
+++ b/litellm-proxy-extras/litellm_proxy_extras/migrations/20251204124859_add_end_user_spend_table/migration.sql
@ -0,0 +1,42 @@
+-- CreateTable
+CREATE TABLE "LiteLLM_DailyEndUserSpend" (
+    "id" TEXT NOT NULL,
+    "end_user_id" TEXT,
+    "date" TEXT NOT NULL,
+    "api_key" TEXT NOT NULL,
+    "model" TEXT,
+    "model_group" TEXT,
+    "custom_llm_provider" TEXT,
+    "mcp_namespaced_tool_name" TEXT,
+    "prompt_tokens" BIGINT NOT NULL DEFAULT 0,
+    "completion_tokens" BIGINT NOT NULL DEFAULT 0,
+    "cache_read_input_tokens" BIGINT NOT NULL DEFAULT 0,
+    "cache_creation_input_tokens" BIGINT NOT NULL DEFAULT 0,
+    "spend" DOUBLE PRECISION NOT NULL DEFAULT 0.0,
+    "api_requests" BIGINT NOT NULL DEFAULT 0,
+    "successful_requests" BIGINT NOT NULL DEFAULT 0,
+    "failed_requests" BIGINT NOT NULL DEFAULT 0,
+    "created_at" TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP,
+    "updated_at" TIMESTAMP(3) NOT NULL,
+
+    CONSTRAINT "LiteLLM_DailyEndUserSpend_pkey" PRIMARY KEY ("id")
+);
+
+-- CreateIndex
+CREATE INDEX "LiteLLM_DailyEndUserSpend_date_idx" ON "LiteLLM_DailyEndUserSpend"("date");
+
+-- CreateIndex
+CREATE INDEX "LiteLLM_DailyEndUserSpend_end_user_id_idx" ON "LiteLLM_DailyEndUserSpend"("end_user_id");
+
+-- CreateIndex
+CREATE INDEX "LiteLLM_DailyEndUserSpend_api_key_idx" ON "LiteLLM_DailyEndUserSpend"("api_key");
+
+-- CreateIndex
+CREATE INDEX "LiteLLM_DailyEndUserSpend_model_idx" ON "LiteLLM_DailyEndUserSpend"("model");
+
+-- CreateIndex
+CREATE INDEX "LiteLLM_DailyEndUserSpend_mcp_namespaced_tool_name_idx" ON "LiteLLM_DailyEndUserSpend"("mcp_namespaced_tool_name");
+
+-- CreateIndex
+CREATE UNIQUE INDEX "LiteLLM_DailyEndUserSpend_end_user_id_date_api_key_model_cu_key" ON "LiteLLM_DailyEndUserSpend"("end_user_id", "date", "api_key", "model", "custom_llm_provider", "mcp_namespaced_tool_name");
+
--- a/litellm-proxy-extras/litellm_proxy_extras/schema.prisma
+++ b/litellm-proxy-extras/litellm_proxy_extras/schema.prisma
@ -465,6 +465,34 @@ model LiteLLM_DailyOrganizationSpend {
  @@index([mcp_namespaced_tool_name])
 }

+// Track daily end user (customer) spend metrics per model and key
+model LiteLLM_DailyEndUserSpend {
+  id                  String   @id @default(uuid())
+  end_user_id         String?
+  date                String
+  api_key             String   
+  model               String?
+  model_group         String?  
+  custom_llm_provider String?
+  mcp_namespaced_tool_name String?
+  prompt_tokens       BigInt      @default(0)
+  completion_tokens   BigInt      @default(0)
+  cache_read_input_tokens     BigInt      @default(0)
+  cache_creation_input_tokens BigInt      @default(0)
+  spend               Float    @default(0.0)
+  api_requests        BigInt      @default(0)
+  successful_requests BigInt      @default(0)
+  failed_requests     BigInt      @default(0)
+  created_at          DateTime @default(now())
+  updated_at          DateTime @updatedAt
+  @@unique([end_user_id, date, api_key, model, custom_llm_provider, mcp_namespaced_tool_name])
+  @@index([date])
+  @@index([end_user_id])
+  @@index([api_key])
+  @@index([model])
+  @@index([mcp_namespaced_tool_name])
+}
+
 // Track daily team spend metrics per model and key
 model LiteLLM_DailyTeamSpend {
  id                  String   @id @default(uuid())
--- a/litellm-proxy-extras/pyproject.toml
+++ b/litellm-proxy-extras/pyproject.toml
@ -1,6 +1,6 @@
 [tool.poetry]
 name = "litellm-proxy-extras"
-version = "0.4.9"
+version = "0.4.11"
 description = "Additional files for the LiteLLM Proxy. Reduces the size of the main litellm package."
 authors = ["BerriAI"]
 readme = "README.md"
@ -22,7 +22,7 @@ requires = ["poetry-core"]
 build-backend = "poetry.core.masonry.api"

 [tool.commitizen]
-version = "0.4.9"
+version = "0.4.11"
 version_files = [
    "pyproject.toml:version",
    "../requirements.txt:litellm-proxy-extras==",
--- a/litellm/init.py
+++ b/litellm/init.py
@ -265,6 +265,7 @@ heroku_key: Optional[str] = None
 cometapi_key: Optional[str] = None
 ovhcloud_key: Optional[str] = None
 lemonade_key: Optional[str] = None
+amazon_nova_api_key: Optional[str] = None
 common_cloud_provider_auth_params: dict = {
    "params": ["project", "region_name", "token"],
    "providers": ["vertex_ai", "bedrock", "watsonx", "azure", "vertex_ai_beta"],
@ -520,6 +521,7 @@ perplexity_models: Set = set()
 watsonx_models: Set = set()
 gemini_models: Set = set()
 xai_models: Set = set()
+zai_models: Set = set()
 deepseek_models: Set = set()
 runwayml_models: Set = set()
 azure_ai_models: Set = set()
@ -571,6 +573,7 @@ ovhcloud_models: Set = set()
 ovhcloud_embedding_models: Set = set()
 lemonade_models: Set = set()
 docker_model_runner_models: Set = set()
+amazon_nova_models: Set = set()


 def is_bedrock_pricing_only_model(key: str) -> bool:
@ -711,6 +714,8 @@ def add_known_models():
            text_completion_codestral_models.add(key)
        elif value.get("litellm_provider") == "xai":
            xai_models.add(key)
+        elif value.get("litellm_provider") == "zai":
+            zai_models.add(key)
        elif value.get("litellm_provider") == "fal_ai":
            fal_ai_models.add(key)
        elif value.get("litellm_provider") == "deepseek":
@ -811,6 +816,8 @@ def add_known_models():
            lemonade_models.add(key)
        elif value.get("litellm_provider") == "docker_model_runner":
            docker_model_runner_models.add(key)
+        elif value.get("litellm_provider") == "amazon_nova":
+            amazon_nova_models.add(key)


 add_known_models()
@ -872,6 +879,7 @@ model_list = list(
    | gemini_models
    | text_completion_codestral_models
    | xai_models
+    | zai_models
    | fal_ai_models
    | deepseek_models
    | azure_ai_models
@ -960,6 +968,7 @@ models_by_provider: dict = {
    "aleph_alpha": aleph_alpha_models,
    "text-completion-codestral": text_completion_codestral_models,
    "xai": xai_models,
+    "zai": zai_models,
    "fal_ai": fal_ai_models,
    "deepseek": deepseek_models,
    "runwayml": runwayml_models,
@ -1010,6 +1019,7 @@ models_by_provider: dict = {
    "ovhcloud": ovhcloud_models | ovhcloud_embedding_models,
    "lemonade": lemonade_models,
    "clarifai": clarifai_models,
+    "amazon_nova": amazon_nova_models,
 }

 # mapping for those models which have larger equivalents
@ -1300,6 +1310,7 @@ from .llms.friendliai.chat.transformation import FriendliaiChatConfig
 from .llms.jina_ai.embedding.transformation import JinaAIEmbeddingConfig
 from .llms.xai.chat.transformation import XAIChatConfig
 from .llms.xai.common_utils import XAIModelInfo
+from .llms.zai.chat.transformation import ZAIChatConfig
 from .llms.aiml.chat.transformation import AIMLChatConfig
 from .llms.volcengine.chat.transformation import (
    VolcEngineChatConfig as VolcEngineConfig,
@ -1353,6 +1364,7 @@ from .llms.ovhcloud.embedding.transformation import OVHCloudEmbeddingConfig
 from .llms.cometapi.embed.transformation import CometAPIEmbeddingConfig
 from .llms.lemonade.chat.transformation import LemonadeChatConfig
 from .llms.snowflake.embedding.transformation import SnowflakeEmbeddingConfig
+from .llms.amazon_nova.chat.transformation import AmazonNovaChatConfig
 from .main import *  # type: ignore

 # Skills API
@ -1497,10 +1509,46 @@ def set_global_gitlab_config(config: Dict[str, Any]) -> None:
 # Lazy loading system for heavy modules to reduce initial import time and memory usage

 if TYPE_CHECKING:
+    from litellm.types.utils import ModelInfo as _ModelInfoType
+    
+    # Cost calculator functions
    cost_per_token: Callable[..., Tuple[float, float]]
    completion_cost: Callable[..., float]
    response_cost_calculator: Any
    modify_integration: Any
+    
+    # Utils functions - type stubs for truly lazy loaded functions only
+    # (functions NOT imported via "from .main import *")
+    get_response_string: Callable[..., str]
+    supports_function_calling: Callable[..., bool]
+    supports_web_search: Callable[..., bool]
+    supports_url_context: Callable[..., bool]
+    supports_response_schema: Callable[..., bool]
+    supports_parallel_function_calling: Callable[..., bool]
+    supports_vision: Callable[..., bool]
+    supports_audio_input: Callable[..., bool]
+    supports_audio_output: Callable[..., bool]
+    supports_system_messages: Callable[..., bool]
+    supports_reasoning: Callable[..., bool]
+    acreate: Callable[..., Any]
+    get_max_tokens: Callable[..., int]
+    get_model_info: Callable[..., _ModelInfoType]
+    register_prompt_template: Callable[..., None]
+    validate_environment: Callable[..., dict]
+    check_valid_key: Callable[..., bool]
+    register_model: Callable[..., None]
+    encode: Callable[..., list]
+    decode: Callable[..., str]
+    _calculate_retry_after: Callable[..., float]
+    _should_retry: Callable[..., bool]
+    get_supported_openai_params: Callable[..., Optional[list]]
+    get_api_base: Callable[..., Optional[str]]
+    get_first_chars_messages: Callable[..., str]
+    get_provider_fields: Callable[..., List]
+    get_valid_models: Callable[..., list]
+    
+    # Response types - truly lazy loaded only (not in main.py or elsewhere)
+    ModelResponseListIterator: Type[Any]


 def __getattr__(name: str) -> Any:
--- a/litellm/completion_extras/litellm_responses_transformation/transformation.py
+++ b/litellm/completion_extras/litellm_responses_transformation/transformation.py
@ -632,6 +632,8 @@ class LiteLLMResponsesTransformationHandler(CompletionTransformationBridge):
            return Reasoning(effort="none")  # type: ignore
        elif reasoning_effort == "high":
            return Reasoning(effort="high")
+        elif reasoning_effort == "xhigh":
+            return Reasoning(effort="xhigh")  # type: ignore[typeddict-item]
        elif reasoning_effort == "medium":
            return Reasoning(effort="medium")
        elif reasoning_effort == "low":
--- a/litellm/constants.py
+++ b/litellm/constants.py
@ -149,6 +149,7 @@ REDIS_UPDATE_BUFFER_KEY = "litellm_spend_update_buffer"
 REDIS_DAILY_SPEND_UPDATE_BUFFER_KEY = "litellm_daily_spend_update_buffer"
 REDIS_DAILY_TEAM_SPEND_UPDATE_BUFFER_KEY = "litellm_daily_team_spend_update_buffer"
 REDIS_DAILY_ORG_SPEND_UPDATE_BUFFER_KEY = "litellm_daily_org_spend_update_buffer"
+REDIS_DAILY_END_USER_SPEND_UPDATE_BUFFER_KEY = "litellm_daily_end_user_spend_update_buffer"
 REDIS_DAILY_TAG_SPEND_UPDATE_BUFFER_KEY = "litellm_daily_tag_spend_update_buffer"
 MAX_REDIS_BUFFER_DEQUEUE_COUNT = int(os.getenv("MAX_REDIS_BUFFER_DEQUEUE_COUNT", 100))
 MAX_SIZE_IN_MEMORY_QUEUE = int(os.getenv("MAX_SIZE_IN_MEMORY_QUEUE", 10000))
@ -413,6 +414,7 @@ LITELLM_CHAT_PROVIDERS = [
    "ovhcloud",
    "lemonade",
    "docker_model_runner",
+    "amazon_nova",
 ]

 LITELLM_EMBEDDING_PROVIDERS_SUPPORTING_INPUT_ARRAY_OF_TOKENS = [
@ -538,6 +540,7 @@ openai_compatible_endpoints: List = [
    "https://api.friendli.ai/serverless/v1",
    "api.sambanova.ai/v1",
    "api.x.ai/v1",
+    "ollama.com",
    "api.galadriel.ai/v1",
    "api.llama.com/compat/v1/",
    "api.featherless.ai/v1",
--- a/litellm/cost_calculator.py
+++ b/litellm/cost_calculator.py
@ -860,9 +860,9 @@ def completion_cost(  # noqa: PLR0915
                    or isinstance(completion_response, dict)
                ):  # tts returns a custom class
                    if isinstance(completion_response, dict):
-                        usage_obj: Optional[Union[dict, Usage]] = (
-                            completion_response.get("usage", {})
-                        )
+                        usage_obj: Optional[
+                            Union[dict, Usage]
+                        ] = completion_response.get("usage", {})
                    else:
                        usage_obj = getattr(completion_response, "usage", {})
                    if isinstance(usage_obj, BaseModel) and not _is_known_usage_objects(
@ -1066,13 +1066,14 @@ def completion_cost(  # noqa: PLR0915
                        # If model is like "tavily-search", construct "tavily/search" for cost lookup
                        search_model = f"{custom_llm_provider}/search"

-                    prompt_cost, completion_cost_result = (
-                        search_provider_cost_per_query(
-                            model=search_model,
-                            custom_llm_provider=custom_llm_provider,
-                            number_of_queries=number_of_queries,
-                            optional_params=optional_params,
-                        )
+                    (
+                        prompt_cost,
+                        completion_cost_result,
+                    ) = search_provider_cost_per_query(
+                        model=search_model,
+                        custom_llm_provider=custom_llm_provider,
+                        number_of_queries=number_of_queries,
+                        optional_params=optional_params,
                    )

                    # Return the total cost (prompt_cost + completion_cost, but for search it's just prompt_cost)
@ -1080,11 +1081,13 @@ def completion_cost(  # noqa: PLR0915

                    # Apply discount
                    original_cost = _final_cost
-                    _final_cost, discount_percent, discount_amount = (
-                        _apply_cost_discount(
-                            base_cost=_final_cost,
-                            custom_llm_provider=custom_llm_provider,
-                        )
+                    (
+                        _final_cost,
+                        discount_percent,
+                        discount_amount,
+                    ) = _apply_cost_discount(
+                        base_cost=_final_cost,
+                        custom_llm_provider=custom_llm_provider,
                    )

                    # Store cost breakdown in logging object if available
@ -1329,9 +1332,8 @@ def response_cost_calculator(
            response_cost = 0.0
        else:
            if isinstance(response_object, BaseModel):
-                response_object._hidden_params["optional_params"] = optional_params
-
                if hasattr(response_object, "_hidden_params"):
+                    response_object._hidden_params["optional_params"] = optional_params
                    provider_response_cost = get_response_cost_from_hidden_params(
                        response_object._hidden_params
                    )
--- a/litellm/integrations/mlflow.py
+++ b/litellm/integrations/mlflow.py
@ -129,8 +129,11 @@ class MlflowLogger(CustomLogger):
        self._add_chunk_events(span, response_obj)

        # If this is the final chunk, end the span. The final chunk
-        # has complete_streaming_response that gathers the full response.
-        if final_response := kwargs.get("complete_streaming_response"):
+        # has the assembled streaming response (key differs between sync/async paths).
+        final_response = kwargs.get("complete_streaming_response") or kwargs.get(
+            "async_complete_streaming_response"
+        )
+        if final_response:
            end_time_ns = int(end_time.timestamp() * 1e9)

            self._extract_and_set_chat_attributes(span, kwargs, final_response)
@ -153,7 +156,9 @@ class MlflowLogger(CustomLogger):
                span.add_event(
                    SpanEvent(
                        name="streaming_chunk",
-                        attributes={"delta": json.dumps(choice.delta.model_dump())},
+                        attributes={
+                            "delta": json.dumps(choice.delta.model_dump, default=str)
+                        },
                    )
                )
        except Exception:
--- a/litellm/litellm_core_utils/get_llm_provider_logic.py
+++ b/litellm/litellm_core_utils/get_llm_provider_logic.py
@ -229,6 +229,9 @@ def get_llm_provider(  # noqa: PLR0915
                    elif endpoint == "api.deepseek.com/v1":
                        custom_llm_provider = "deepseek"
                        dynamic_api_key = get_secret_str("DEEPSEEK_API_KEY")
+                    elif endpoint == "ollama.com":
+                        custom_llm_provider = "ollama"
+                        dynamic_api_key = get_secret_str("OLLAMA_API_KEY")
                    elif endpoint == "https://api.friendli.ai/serverless/v1":
                        custom_llm_provider = "friendliai"
                        dynamic_api_key = get_secret_str(
@ -401,6 +404,8 @@ def get_llm_provider(  # noqa: PLR0915
            custom_llm_provider = "lemonade"
        elif model.startswith("clarifai/"):
            custom_llm_provider = "clarifai"
+        elif model.startswith("amazon_nova"):
+            custom_llm_provider = "amazon_nova"
        if not custom_llm_provider:
            if litellm.suppress_debug_info is False:
                print()  # noqa
@ -469,11 +474,13 @@ def _get_openai_compatible_provider_info(  # noqa: PLR0915
    model = model.split("/", 1)[1]

    # Check JSON providers FIRST (before hardcoded ones)
-    from litellm.llms.openai_like.json_loader import JSONProviderRegistry
    from litellm.llms.openai_like.dynamic_config import create_config_class
+    from litellm.llms.openai_like.json_loader import JSONProviderRegistry

    if JSONProviderRegistry.exists(custom_llm_provider):
        provider_config = JSONProviderRegistry.get(custom_llm_provider)
+        if provider_config is None:
+            raise ValueError(f"Provider {custom_llm_provider} not found")
        config_class = create_config_class(provider_config)
        api_base, dynamic_api_key = config_class()._get_openai_compatible_provider_info(
            api_base, api_key
@ -556,6 +563,13 @@ def _get_openai_compatible_provider_info(  # noqa: PLR0915
            or "https://api.studio.nebius.ai/v1"
        )  # type: ignore
        dynamic_api_key = api_key or get_secret_str("NEBIUS_API_KEY")
+    elif custom_llm_provider == "ollama":
+        api_base = (
+            api_base
+            or get_secret("OLLAMA_API_BASE")
+            or "http://localhost:11434"
+        )  # type: ignore
+        dynamic_api_key = api_key or get_secret_str("OLLAMA_API_KEY")
    elif (custom_llm_provider == "ai21_chat") or (
        custom_llm_provider == "ai21" and model in litellm.ai21_chat_models
    ):
@ -675,12 +689,12 @@ def _get_openai_compatible_provider_info(  # noqa: PLR0915
            api_base, api_key
        )
    elif custom_llm_provider == "zai":
-        api_base = (
-            api_base
-            or get_secret_str("ZAI_API_BASE")
-            or "https://api.z.ai/api/paas/v4"
+        (
+            api_base,
+            dynamic_api_key,
+        ) = litellm.ZAIChatConfig()._get_openai_compatible_provider_info(
+            api_base, api_key
        )
-        dynamic_api_key = api_key or get_secret_str("ZAI_API_KEY")
    elif custom_llm_provider == "together_ai":
        api_base = (
            api_base
--- a/litellm/litellm_core_utils/llm_cost_calc/utils.py
+++ b/litellm/litellm_core_utils/llm_cost_calc/utils.py
@ -583,9 +583,11 @@ def generic_cost_per_token(
        reasoning_tokens = completion_tokens_details["reasoning_tokens"]
        image_tokens = completion_tokens_details["image_tokens"]

-    if text_tokens == 0:
+    # Only assume all tokens are text if there's NO breakdown at all
+    # If image_tokens, audio_tokens, or reasoning_tokens exist, respect text_tokens=0
+    has_token_breakdown = image_tokens > 0 or audio_tokens > 0 or reasoning_tokens > 0
+    if text_tokens == 0 and not has_token_breakdown:
        text_tokens = usage.completion_tokens
-    if text_tokens == usage.completion_tokens:
        is_text_tokens_total = True
    ## TEXT COST
    completion_cost = float(text_tokens) * completion_base_cost
--- a/litellm/litellm_core_utils/streaming_handler.py
+++ b/litellm/litellm_core_utils/streaming_handler.py
@ -737,6 +737,7 @@ class CustomStreamWrapper:
            or (
                "tool_calls" in model_response.choices[0].delta
                and model_response.choices[0].delta["tool_calls"] is not None
+                and len(model_response.choices[0].delta["tool_calls"]) > 0
            )
            or (
                "function_call" in model_response.choices[0].delta
--- a/litellm/llms/amazon_nova/chat/transformation.py
+++ b/litellm/llms/amazon_nova/chat/transformation.py
@ -0,0 +1,115 @@
+"""
+Translate from OpenAI's `/v1/chat/completions` to Amazon Nova's `/v1/chat/completions`
+"""
+from typing import Any, List, Optional, Tuple
+
+import httpx
+
+import litellm
+from litellm.litellm_core_utils.litellm_logging import Logging as LiteLLMLoggingObj
+from litellm.secret_managers.main import get_secret_str
+from litellm.types.llms.openai import (
+    AllMessageValues,
+)
+from litellm.types.utils import ModelResponse
+
+from ...openai_like.chat.transformation import OpenAILikeChatConfig
+
+
+class AmazonNovaChatConfig(OpenAILikeChatConfig):
+    max_completion_tokens: Optional[int] = None
+    max_tokens: Optional[int] = None
+    metadata: Optional[int] = None
+    temperature: Optional[int] = None
+    top_p: Optional[int] = None
+    tools: Optional[list] = None
+    reasoning_effort: Optional[list] = None
+
+    def __init__(
+        self,
+        max_completion_tokens: Optional[int] = None,
+        max_tokens: Optional[int] = None,
+        temperature: Optional[int] = None,
+        top_p: Optional[int] = None,
+        tools: Optional[list] = None,
+        reasoning_effort: Optional[list] = None,
+    ) -> None:
+        locals_ = locals().copy()
+        for key, value in locals_.items():
+            if key != "self" and value is not None:
+                setattr(self.__class__, key, value)
+
+    @property
+    def custom_llm_provider(self) -> Optional[str]:
+        return "amazon_nova"
+
+    @classmethod
+    def get_config(cls):
+        return super().get_config()
+
+    def _get_openai_compatible_provider_info(
+        self, api_base: Optional[str], api_key: Optional[str]
+    ) -> Tuple[Optional[str], Optional[str]]:
+        # Amazon Nova is openai compatible, we just need to set this to custom_openai and have the api_base be Nova's endpoint
+        api_base = (
+            api_base
+            or get_secret_str("AMAZON_NOVA_API_BASE")
+            or "https://api.nova.amazon.com/v1"
+        )  # type: ignore
+        
+        # Get API key from multiple sources
+        key = (
+            api_key
+            or litellm.amazon_nova_api_key
+            or get_secret_str("AMAZON_NOVA_API_KEY")
+            or litellm.api_key
+        )
+        return api_base, key
+    
+    def get_supported_openai_params(self, model: str) -> List:
+        return [
+            "top_p",
+            "temperature",
+            "max_tokens",
+            "max_completion_tokens",
+            "metadata",
+            "stop",
+            "stream",
+            "stream_options",
+            "tools",
+            "tool_choice",
+            "reasoning_effort"
+        ]
+
+    def transform_response(
+        self,
+        model: str,
+        raw_response: httpx.Response,
+        model_response: ModelResponse,
+        logging_obj: LiteLLMLoggingObj,
+        request_data: dict,
+        messages: List[AllMessageValues],
+        optional_params: dict,
+        litellm_params: dict,
+        encoding: Any,
+        api_key: Optional[str] = None,
+        json_mode: Optional[bool] = None,
+    ) -> ModelResponse:
+        model_response = super().transform_response(
+            model=model,
+            model_response=model_response,
+            raw_response=raw_response,
+            messages=messages,
+            logging_obj=logging_obj,
+            request_data=request_data,
+            encoding=encoding,
+            optional_params=optional_params,
+            json_mode=json_mode,
+            litellm_params=litellm_params,
+            api_key=api_key,
+        )
+
+        # Storing amazon_nova in the model response for easier cost calculation later
+        setattr(model_response, "model", "amazon-nova/" + model)
+
+        return model_response
--- a/litellm/llms/amazon_nova/cost_calculation.py
+++ b/litellm/llms/amazon_nova/cost_calculation.py
@ -0,0 +1,21 @@
+"""
+Helper util for handling amazon nova cost calculation
+- e.g.: prompt caching
+"""
+
+from typing import TYPE_CHECKING, Tuple
+
+from litellm.litellm_core_utils.llm_cost_calc.utils import generic_cost_per_token
+
+if TYPE_CHECKING:
+    from litellm.types.utils import Usage
+
+
+def cost_per_token(model: str, usage: "Usage") -> Tuple[float, float]:
+    """
+    Calculates the cost per token for a given model, prompt tokens, and completion tokens.
+    Follows the same logic as Anthropic's cost per token calculation.
+    """
+    return generic_cost_per_token(
+        model=model, usage=usage, custom_llm_provider="amazon_nova"
+    )
--- a/litellm/llms/anthropic/common_utils.py
+++ b/litellm/llms/anthropic/common_utils.py
@ -12,7 +12,7 @@ from litellm.litellm_core_utils.prompt_templates.common_utils import (
 )
 from litellm.llms.base_llm.base_utils import BaseLLMModelInfo, BaseTokenCounter
 from litellm.llms.base_llm.chat.transformation import BaseLLMException
-from litellm.types.llms.anthropic import AllAnthropicToolsValues, AnthropicMcpServerTool
+from litellm.types.llms.anthropic import AllAnthropicToolsValues, AnthropicMcpServerTool, ANTHROPIC_HOSTED_TOOLS
 from litellm.types.llms.openai import AllMessageValues
 from litellm.types.utils import TokenCountResponse

@ -72,6 +72,17 @@ class AnthropicModelInfo(BaseLLMModelInfo):
                return tool["type"]
        return None

+    def is_web_search_tool_used(
+        self, tools: Optional[List[AllAnthropicToolsValues]]
+    ) -> bool:
+        """Returns True if web_search tool is used"""
+        if tools is None:
+            return False
+        for tool in tools:
+            if "type" in tool and tool["type"].startswith(ANTHROPIC_HOSTED_TOOLS.WEB_SEARCH.value):
+                return True
+        return False
+
    def is_pdf_used(self, messages: List[AllMessageValues]) -> bool:
        """
        Set to true if media passed into messages.
@ -252,6 +263,7 @@ class AnthropicModelInfo(BaseLLMModelInfo):
        pdf_used: bool = False,
        file_id_used: bool = False,
        mcp_server_used: bool = False,
+        web_search_tool_used: bool = False,
        tool_search_used: bool = False,
        programmatic_tool_calling_used: bool = False,
        input_examples_used: bool = False,
@ -292,9 +304,12 @@ class AnthropicModelInfo(BaseLLMModelInfo):
        if user_anthropic_beta_headers is not None:
            betas.update(user_anthropic_beta_headers)

-        # Don't send any beta headers to Vertex, Vertex has failed requests when they are sent
+        # Don't send any beta headers to Vertex, except web search which is required
        if is_vertex_request is True:
-            pass
+            # Vertex AI requires web search beta header for web search to work
+            if web_search_tool_used:
+                from litellm.types.llms.anthropic import ANTHROPIC_BETA_HEADER_VALUES
+                headers["anthropic-beta"] = ANTHROPIC_BETA_HEADER_VALUES.WEB_SEARCH_2025_03_05.value
        elif len(betas) > 0:
            headers["anthropic-beta"] = ",".join(betas)

@ -325,6 +340,7 @@ class AnthropicModelInfo(BaseLLMModelInfo):
        )
        pdf_used = self.is_pdf_used(messages=messages)
        file_id_used = self.is_file_id_used(messages=messages)
+        web_search_tool_used = self.is_web_search_tool_used(tools=tools)
        tool_search_used = self.is_tool_search_used(tools=tools)
        programmatic_tool_calling_used = self.is_programmatic_tool_calling_used(tools=tools)
        input_examples_used = self.is_input_examples_used(tools=tools)
@ -338,6 +354,7 @@ class AnthropicModelInfo(BaseLLMModelInfo):
            pdf_used=pdf_used,
            api_key=api_key,
            file_id_used=file_id_used,
+            web_search_tool_used=web_search_tool_used,
            is_vertex_request=optional_params.get("is_vertex_request", False),
            user_anthropic_beta_headers=user_anthropic_beta_headers,
            mcp_server_used=mcp_server_used,
--- a/litellm/llms/anthropic/experimental_pass_through/messages/transformation.py
+++ b/litellm/llms/anthropic/experimental_pass_through/messages/transformation.py
@ -103,7 +103,7 @@ class AnthropicMessagesConfig(BaseAnthropicMessagesConfig):
                status_code=400,
            )
        ####### get required params for all anthropic messages requests ######
-        verbose_logger.debug(f"🔍 TRANSFORMATION DEBUG - Messages: {messages}")
+        verbose_logger.debug(f"TRANSFORMATION DEBUG - Messages: {messages}")
        anthropic_messages_request: AnthropicMessagesRequest = AnthropicMessagesRequest(
            messages=messages,
            max_tokens=max_tokens,
--- a/litellm/llms/bedrock/chat/converse_transformation.py
+++ b/litellm/llms/bedrock/chat/converse_transformation.py
@ -960,7 +960,10 @@ class AmazonConverseConfig(BaseConfig):
            bedrock_tools = _bedrock_tools_pt(filtered_tools)

        # Set anthropic_beta in additional_request_params if we have any beta features
-        if anthropic_beta_list:
+        # ONLY apply to Anthropic/Claude models - other models (e.g., Qwen, Llama) don't support this field
+        # and will error with "unknown variant anthropic_beta" if included
+        base_model = BedrockModelInfo.get_base_model(model)
+        if anthropic_beta_list and base_model.startswith("anthropic"):
            # Remove duplicates while preserving order
            unique_betas = []
            seen = set()
--- a/litellm/llms/bedrock/chat/invoke_handler.py
+++ b/litellm/llms/bedrock/chat/invoke_handler.py
@ -674,33 +674,39 @@ class BedrockLLM(BaseAWSLLM):
            )

        ## CALCULATING USAGE - bedrock returns usage in the headers
-        bedrock_input_tokens = response.headers.get(
-            "x-amzn-bedrock-input-token-count", None
-        )
-        bedrock_output_tokens = response.headers.get(
-            "x-amzn-bedrock-output-token-count", None
-        )
-
-        prompt_tokens = int(
-            bedrock_input_tokens or litellm.token_counter(messages=messages)
-        )
-
-        completion_tokens = int(
-            bedrock_output_tokens
-            or litellm.token_counter(
-                text=model_response.choices[0].message.content,  # type: ignore
-                count_response_tokens=True,
+        # Skip if usage was already set (e.g., from JSON response for OpenAI provider)
+        if not hasattr(model_response, "usage") or getattr(model_response, "usage", None) is None:
+            bedrock_input_tokens = response.headers.get(
+                "x-amzn-bedrock-input-token-count", None
+            )
+            bedrock_output_tokens = response.headers.get(
+                "x-amzn-bedrock-output-token-count", None
            )
-        )

-        model_response.created = int(time.time())
-        model_response.model = model
-        usage = Usage(
-            prompt_tokens=prompt_tokens,
-            completion_tokens=completion_tokens,
-            total_tokens=prompt_tokens + completion_tokens,
-        )
-        setattr(model_response, "usage", usage)
+            prompt_tokens = int(
+                bedrock_input_tokens or litellm.token_counter(messages=messages)
+            )
+
+            completion_tokens = int(
+                bedrock_output_tokens
+                or litellm.token_counter(
+                    text=model_response.choices[0].message.content,  # type: ignore
+                    count_response_tokens=True,
+                )
+            )
+
+            model_response.created = int(time.time())
+            model_response.model = model
+            usage = Usage(
+                prompt_tokens=prompt_tokens,
+                completion_tokens=completion_tokens,
+                total_tokens=prompt_tokens + completion_tokens,
+            )
+            setattr(model_response, "usage", usage)
+        else:
+            # Ensure created and model are set even if usage was already set
+            model_response.created = int(time.time())
+            model_response.model = model

        return model_response

--- a/litellm/llms/oci/chat/transformation.py
+++ b/litellm/llms/oci/chat/transformation.py
@ -416,15 +416,34 @@ class OCIChatConfig(BaseConfig):
                "Please install it with: pip install cryptography"
            ) from e

+        # Handle oci_key - it should be a string (PEM content)
+        oci_key_content = None
+        if oci_key:
+            if isinstance(oci_key, str):
+                oci_key_content = oci_key
+                # Fix common issues with PEM content
+                # Replace escaped newlines with actual newlines
+                oci_key_content = oci_key_content.replace("\\n", "\n")
+                # Ensure proper line endings
+                if "\r\n" in oci_key_content:
+                    oci_key_content = oci_key_content.replace("\r\n", "\n")
+            else:
+                raise OCIError(
+                    status_code=400,
+                    message=f"oci_key must be a string containing the PEM private key content. "
+                    f"Got type: {type(oci_key).__name__}",
+                )
+
        private_key = (
-            load_private_key_from_str(oci_key)
-            if oci_key
+            load_private_key_from_str(oci_key_content)
+            if oci_key_content
            else load_private_key_from_file(oci_key_file) if oci_key_file else None
        )

        if private_key is None:
-            raise Exception(
-                "Private key is required for OCI authentication. Please provide either oci_key or oci_key_file."
+            raise OCIError(
+                status_code=400,
+                message="Private key is required for OCI authentication. Please provide either oci_key or oci_key_file.",
            )

        signature = private_key.sign(
--- a/litellm/llms/openai/chat/gpt_5_transformation.py
+++ b/litellm/llms/openai/chat/gpt_5_transformation.py
@ -25,6 +25,12 @@ class OpenAIGPT5Config(OpenAIGPTConfig):
    def is_model_gpt_5_codex_model(cls, model: str) -> bool:
        """Check if the model is specifically a GPT-5 Codex variant."""
        return "gpt-5-codex" in model
+
+    @classmethod
+    def is_model_gpt_5_1_codex_max_model(cls, model: str) -> bool:
+        """Check if the model is the gpt-5.1-codex-max variant."""
+        model_name = model.split("/")[-1]  # handle provider prefixes
+        return model_name == "gpt-5.1-codex-max"
    
    @classmethod
    def is_model_gpt_5_1_model(cls, model: str) -> bool:
@ -66,6 +72,22 @@ class OpenAIGPT5Config(OpenAIGPTConfig):
        model: str,
        drop_params: bool,
    ) -> dict:
+        reasoning_effort = (
+            non_default_params.get("reasoning_effort")
+            or optional_params.get("reasoning_effort")
+        )
+        if reasoning_effort is not None and reasoning_effort == "xhigh":
+            if not self.is_model_gpt_5_1_codex_max_model(model):
+                if litellm.drop_params or drop_params:
+                    non_default_params.pop("reasoning_effort", None)
+                else:
+                    raise litellm.utils.UnsupportedParamsError(
+                        message=(
+                            "reasoning_effort='xhigh' is only supported for gpt-5.1-codex-max."
+                        ),
+                        status_code=400,
+                    )
+
        ################################################################
        # max_tokens is not supported for gpt-5 models on OpenAI API
        # Relevant issue: https://github.com/BerriAI/litellm/issues/13381
@ -79,10 +101,6 @@ class OpenAIGPT5Config(OpenAIGPTConfig):
            temperature_value: Optional[float] = non_default_params.pop("temperature")
            if temperature_value is not None:
                is_gpt_5_1 = self.is_model_gpt_5_1_model(model)
-                reasoning_effort = (
-                    non_default_params.get("reasoning_effort") 
-                    or optional_params.get("reasoning_effort")
-                )
                
                # gpt-5.1 supports any temperature when reasoning_effort="none" (or not specified, as it defaults to "none")
                if is_gpt_5_1 and (reasoning_effort == "none" or reasoning_effort is None):
--- a/litellm/llms/openai/chat/guardrail_translation/handler.py
+++ b/litellm/llms/openai/chat/guardrail_translation/handler.py
@ -164,7 +164,7 @@ class OpenAIChatCompletionsHandler(BaseTranslation):
            for tool_call_idx, tool_call in enumerate(tool_calls):
                if isinstance(tool_call, dict):
                    # Add the full tool call object to the list
-                    tool_calls_to_check.append(ChatCompletionToolParam(**tool_call))
+                    tool_calls_to_check.append(cast(ChatCompletionToolParam, tool_call))
                    tool_call_task_mappings.append((msg_idx, int(tool_call_idx)))

    async def _apply_guardrail_responses_to_input_texts(
@ -380,20 +380,20 @@ class OpenAIChatCompletionsHandler(BaseTranslation):

                if isinstance(content, str):
                    # String content - accumulate for this choice
-                    key = (choice_idx, None)
-                    if key not in combined_texts:
-                        combined_texts[key] = ""
-                    combined_texts[key] += content
+                    str_key: Tuple[int, Optional[int]] = (choice_idx, None)
+                    if str_key not in combined_texts:
+                        combined_texts[str_key] = ""
+                    combined_texts[str_key] += content

                elif isinstance(content, list):
                    # List content - accumulate for each content item
                    for content_idx, content_item in enumerate(content):
                        text_str = content_item.get("text")
                        if text_str:
-                            key = (choice_idx, content_idx)
-                            if key not in combined_texts:
-                                combined_texts[key] = ""
-                            combined_texts[key] += text_str
+                            list_key: Tuple[int, Optional[int]] = (choice_idx, content_idx)
+                            if list_key not in combined_texts:
+                                combined_texts[list_key] = ""
+                            combined_texts[list_key] += text_str

        # Step 2: Create lists for guardrail processing
        texts_to_check: List[str] = []
@ -401,9 +401,9 @@ class OpenAIChatCompletionsHandler(BaseTranslation):
        task_mappings: List[Tuple[int, Optional[int]]] = []
        # Track (choice_index, content_index) for each combined text

-        for (choice_idx, content_idx), combined_text in combined_texts.items():
+        for (map_choice_idx, map_content_idx), combined_text in combined_texts.items():
            texts_to_check.append(combined_text)
-            task_mappings.append((choice_idx, content_idx))
+            task_mappings.append((map_choice_idx, map_content_idx))

        # Step 3: Apply guardrail to all combined texts in batch
        if texts_to_check:
@ -503,7 +503,7 @@ class OpenAIChatCompletionsHandler(BaseTranslation):

        # Determine content source and tool calls based on choice type
        content = None
-        tool_calls = None
+        tool_calls: Optional[List[Any]] = None
        if isinstance(choice, litellm.Choices):
            content = choice.message.content
            tool_calls = choice.message.tool_calls
@ -686,15 +686,15 @@ class OpenAIChatCompletionsHandler(BaseTranslation):

                if isinstance(content, str):
                    # String content
-                    key = (choice_idx_in_response, None)
-                    if key in guardrail_map:
-                        if key not in already_set:
+                    str_key: Tuple[int, Optional[int]] = (choice_idx_in_response, None)
+                    if str_key in guardrail_map:
+                        if str_key not in already_set:
                            # First chunk - set the complete guardrailed text
                            if isinstance(choice, litellm.StreamingChoices):
-                                choice.delta.content = guardrail_map[key]
+                                choice.delta.content = guardrail_map[str_key]
                            elif isinstance(choice, litellm.Choices):
-                                choice.message.content = guardrail_map[key]
-                            already_set[key] = True
+                                choice.message.content = guardrail_map[str_key]
+                            already_set[str_key] = True
                        else:
                            # Subsequent chunks - clear the content
                            if isinstance(choice, litellm.StreamingChoices):
@ -706,12 +706,12 @@ class OpenAIChatCompletionsHandler(BaseTranslation):
                    # List content - handle each content item
                    for content_idx, content_item in enumerate(content):
                        if "text" in content_item:
-                            key = (choice_idx_in_response, content_idx)
-                            if key in guardrail_map:
-                                if key not in already_set:
+                            list_key: Tuple[int, Optional[int]] = (choice_idx_in_response, content_idx)
+                            if list_key in guardrail_map:
+                                if list_key not in already_set:
                                    # First chunk - set the complete guardrailed text
-                                    content_item["text"] = guardrail_map[key]
-                                    already_set[key] = True
+                                    content_item["text"] = guardrail_map[list_key]
+                                    already_set[list_key] = True
                                else:
                                    # Subsequent chunks - clear the text
                                    content_item["text"] = ""
--- a/litellm/llms/openai_like/dynamic_config.py
+++ b/litellm/llms/openai_like/dynamic_config.py
@ -19,11 +19,11 @@ def create_config_class(provider: SimpleProviderConfig):
    """Generate config class dynamically from JSON configuration"""

    # Choose base class
-    base_class = (
+    base_class: type = (
        OpenAIGPTConfig if provider.base_class == "openai_gpt" else OpenAILikeChatConfig
    )

-    class JSONProviderConfig(base_class):
+    class JSONProviderConfig(base_class):  # type: ignore[valid-type,misc]
        @overload
        def _transform_messages(
            self, messages: List[AllMessageValues], model: str, is_async: Literal[True]
@ -87,6 +87,9 @@ def create_config_class(provider: SimpleProviderConfig):
            if not api_base:
                api_base = provider.base_url

+            if api_base is None:
+                raise ValueError(f"api_base is required for provider {provider.slug}")
+
            if not api_base.endswith("/chat/completions"):
                api_base = f"{api_base}/chat/completions"

--- a/litellm/llms/vertex_ai/gemini/transformation.py
+++ b/litellm/llms/vertex_ai/gemini/transformation.py
@ -116,7 +116,7 @@ def _process_gemini_image(
            is not None
        ):
            file_data = FileDataType(file_uri=image_url, mime_type=image_type)
-            part: PartType = {"file_data": file_data}
+            part = {"file_data": file_data}
            
            if media_resolution_enum is not None and model is not None:
                from .vertex_and_google_ai_studio_gemini import VertexGeminiConfig
@ -129,7 +129,7 @@ def _process_gemini_image(
            image = convert_to_anthropic_image_obj(image_url, format=format)
            _blob: BlobType = {"data": image["data"], "mime_type": image["media_type"]}
            
-            part: PartType = {"inline_data": cast(BlobType, _blob)}
+            part = {"inline_data": cast(BlobType, _blob)}
            
            if media_resolution_enum is not None and model is not None:
                from .vertex_and_google_ai_studio_gemini import VertexGeminiConfig
--- a/litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py
+++ b/litellm/llms/vertex_ai/gemini/vertex_and_google_ai_studio_gemini.py
@ -1085,24 +1085,26 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):
    def _extract_thinking_blocks_from_parts(
        self, parts: List[HttpxPartType]
    ) -> List[ChatCompletionThinkingBlock]:
-        """Extract thinking blocks from parts if present"""
+        """Extract thinking blocks from parts if present.
+
+        Per Google's docs (https://ai.google.dev/gemini-api/docs/thinking):
+        - Parts with `thought: true` contain thinking/reasoning content
+        - `thoughtSignature` is a separate token for multi-turn context preservation,
+          it does NOT indicate that the content is thinking (a part can have
+          thoughtSignature without thought: true, e.g., function calls)
+        """
        thinking_blocks: List[ChatCompletionThinkingBlock] = []
        for part in parts:
-            if "thoughtSignature" in part:
-                part_copy = part.copy()
-                part_copy.pop("thoughtSignature")
-
-                text_content = part_copy.get("text")
-                if isinstance(text_content, str) and text_content.strip() == "":
-                    continue
-
-                thinking_blocks.append(
-                    ChatCompletionThinkingBlock(
-                        type="thinking",
-                        thinking=json.dumps(part_copy),
-                        signature=part["thoughtSignature"],
-                    )
-                )
+            if part.get("thought") is True:
+                thinking_text = part.get("text", "")
+                block: ChatCompletionThinkingBlock = {
+                    "type": "thinking",
+                    "thinking": thinking_text,
+                }                
+                signature = part.get("thoughtSignature")
+                if signature is not None:
+                    block["signature"] = signature
+                thinking_blocks.append(block)
        return thinking_blocks

    def _extract_image_response_from_parts(
--- a/litellm/llms/vertex_ai/image_generation/vertex_gemini_transformation.py
+++ b/litellm/llms/vertex_ai/image_generation/vertex_gemini_transformation.py
@ -140,7 +140,11 @@ class VertexAIGeminiImageGenerationConfig(BaseImageGenerationConfig, VertexLLM):
        if not vertex_project or not vertex_location:
            raise ValueError("vertex_project and vertex_location are required for Vertex AI")

-        base_url = f"https://{vertex_location}-aiplatform.googleapis.com"
+        # Handle global location differently (no region prefix in URL)
+        if vertex_location == "global":
+            base_url = "https://aiplatform.googleapis.com"
+        else:
+            base_url = f"https://{vertex_location}-aiplatform.googleapis.com"

        return f"{base_url}/v1/projects/{vertex_project}/locations/{vertex_location}/publishers/google/models/{model_name}:generateContent"

--- a/litellm/llms/vertex_ai/text_to_speech/transformation.py
+++ b/litellm/llms/vertex_ai/text_to_speech/transformation.py
@ -220,7 +220,7 @@ class VertexAITextToSpeechConfig(BaseTextToSpeechConfig, VertexBase):
        Returns:
            Tuple of (mapped_voice_str, mapped_params)
        """
-        mapped_params = {}
+        mapped_params: Dict[str, Any] = {}

        ##########################################################
        # Map voice using helper
--- a/litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/experimental_pass_through/transformation.py
+++ b/litellm/llms/vertex_ai/vertex_ai_partner_models/anthropic/experimental_pass_through/transformation.py
@ -5,6 +5,7 @@ from litellm.llms.anthropic.experimental_pass_through.messages.transformation im
 )
 from litellm.types.llms.vertex_ai import VertexPartnerProvider
 from litellm.types.router import GenericLiteLLMParams
+from litellm.types.llms.anthropic import ANTHROPIC_BETA_HEADER_VALUES, ANTHROPIC_HOSTED_TOOLS

 from ....vertex_llm_base import VertexBase

@ -49,6 +50,15 @@ class VertexAIPartnerModelsAnthropicMessagesConfig(AnthropicMessagesConfig, Vert
            )

        headers["content-type"] = "application/json"
+        
+        # Add web search beta header for Vertex AI only if not already set
+        if "anthropic-beta" not in headers:
+            tools = optional_params.get("tools", [])
+            for tool in tools:
+                if isinstance(tool, dict) and tool.get("type", "").startswith(ANTHROPIC_HOSTED_TOOLS.WEB_SEARCH.value):
+                    headers["anthropic-beta"] = ANTHROPIC_BETA_HEADER_VALUES.WEB_SEARCH_2025_03_05.value
+                    break
+        
        return headers, api_base

    def get_complete_url(
--- a/litellm/llms/vertex_ai/vertex_llm_base.py
+++ b/litellm/llms/vertex_ai/vertex_llm_base.py
@ -90,9 +90,15 @@ class VertexBase:
                    else ""
                )
                if isinstance(environment_id, str) and "aws" in environment_id:
-                    creds = self._credentials_from_identity_pool_with_aws(json_obj)
+                    creds = self._credentials_from_identity_pool_with_aws(
+                        json_obj,
+                        scopes=["https://www.googleapis.com/auth/cloud-platform"],
+                    )
                else:
-                    creds = self._credentials_from_identity_pool(json_obj)
+                    creds = self._credentials_from_identity_pool(
+                        json_obj,
+                        scopes=["https://www.googleapis.com/auth/cloud-platform"],
+                    )
            # Check if the JSON object contains Authorized User configuration (via gcloud auth application-default login)
            elif "type" in json_obj and json_obj["type"] == "authorized_user":
                creds = self._credentials_from_authorized_user(
@ -131,15 +137,21 @@ class VertexBase:
        return creds, project_id

    # Google Auth Helpers -- extracted for mocking purposes in tests
-    def _credentials_from_identity_pool(self, json_obj):
+    def _credentials_from_identity_pool(self, json_obj, scopes):
        from google.auth import identity_pool

-        return identity_pool.Credentials.from_info(json_obj)
+        creds = identity_pool.Credentials.from_info(json_obj)
+        if scopes and hasattr(creds, "requires_scopes") and creds.requires_scopes:
+            creds = creds.with_scopes(scopes)
+        return creds

-    def _credentials_from_identity_pool_with_aws(self, json_obj):
+    def _credentials_from_identity_pool_with_aws(self, json_obj, scopes):
        from google.auth import aws

-        return aws.Credentials.from_info(json_obj)
+        creds = aws.Credentials.from_info(json_obj)
+        if scopes and hasattr(creds, "requires_scopes") and creds.requires_scopes:
+            creds = creds.with_scopes(scopes)
+        return creds

    def _credentials_from_authorized_user(self, json_obj, scopes):
        import google.oauth2.credentials
@ -300,7 +312,7 @@ class VertexBase:
    ) -> Tuple[Optional[str], str]:
        """
        for cloudflare ai gateway - https://github.com/BerriAI/litellm/issues/4317
-        
+
        Handles custom api_base for:
        1. Gemini (Google AI Studio) - constructs /models/{model}:{endpoint}
        2. Vertex AI with standard proxies - constructs {api_base}:{endpoint}
@ -328,7 +340,7 @@ class VertexBase:
                        "Missing gemini_api_key, please set `GEMINI_API_KEY`"
                    )
                if gemini_api_key is not None:
-                    auth_header = {"x-goog-api-key": gemini_api_key}  # type: ignore[assignment] 
+                    auth_header = {"x-goog-api-key": gemini_api_key}  # type: ignore[assignment]
            else:
                # For Vertex AI
                if use_psc_endpoint_format:
@ -396,9 +408,7 @@ class VertexBase:
            )

            ### SET RUNTIME ENDPOINT ###
-            version = (
-                "v1beta1" if should_use_v1beta1_features is True else "v1"
-            )
+            version = "v1beta1" if should_use_v1beta1_features is True else "v1"
            url, endpoint = _get_vertex_url(
                mode=mode,
                model=model,
@ -675,13 +685,13 @@ class VertexBase:
    def safe_get_vertex_ai_project(litellm_params: dict) -> Optional[str]:
        """
        Safely get Vertex AI project without mutating the litellm_params dict.
-        
+
        Unlike get_vertex_ai_project(), this does NOT pop values from the dict,
        making it safe to call multiple times with the same litellm_params.
-        
+
        Args:
            litellm_params: Dictionary containing Vertex AI parameters
-            
+
        Returns:
            Vertex AI project ID or None
        """
@ -696,13 +706,13 @@ class VertexBase:
    def safe_get_vertex_ai_credentials(litellm_params: dict) -> Optional[str]:
        """
        Safely get Vertex AI credentials without mutating the litellm_params dict.
-        
+
        Unlike get_vertex_ai_credentials(), this does NOT pop values from the dict,
        making it safe to call multiple times with the same litellm_params.
-        
+
        Args:
            litellm_params: Dictionary containing Vertex AI parameters
-            
+
        Returns:
            Vertex AI credentials or None
        """
@ -716,13 +726,13 @@ class VertexBase:
    def safe_get_vertex_ai_location(litellm_params: dict) -> Optional[str]:
        """
        Safely get Vertex AI location without mutating the litellm_params dict.
-        
+
        Unlike get_vertex_ai_location(), this does NOT pop values from the dict,
        making it safe to call multiple times with the same litellm_params.
-        
+
        Args:
            litellm_params: Dictionary containing Vertex AI parameters
-            
+
        Returns:
            Vertex AI location/region or None
        """
--- a/litellm/llms/zai/init.py
+++ b/litellm/llms/zai/init.py
--- a/litellm/llms/zai/chat/init.py
+++ b/litellm/llms/zai/chat/init.py
--- a/litellm/llms/zai/chat/transformation.py
+++ b/litellm/llms/zai/chat/transformation.py
@ -0,0 +1,33 @@
+from typing import Optional, Tuple
+
+from litellm.secret_managers.main import get_secret_str
+
+from ...openai.chat.gpt_transformation import OpenAIGPTConfig
+
+ZAI_API_BASE = "https://api.z.ai/api/paas/v4"
+
+
+class ZAIChatConfig(OpenAIGPTConfig):
+    @property
+    def custom_llm_provider(self) -> Optional[str]:
+        return "zai"
+
+    def _get_openai_compatible_provider_info(
+        self, api_base: Optional[str], api_key: Optional[str]
+    ) -> Tuple[Optional[str], Optional[str]]:
+        api_base = api_base or get_secret_str("ZAI_API_BASE") or ZAI_API_BASE
+        dynamic_api_key = api_key or get_secret_str("ZAI_API_KEY")
+        return api_base, dynamic_api_key
+
+    def get_supported_openai_params(self, model: str) -> list:
+        return [
+            "max_tokens",
+            "stream",
+            "stream_options",
+            "temperature",
+            "top_p",
+            "stop",
+            "tools",
+            "tool_choice",
+        ]
+
--- a/litellm/main.py
+++ b/litellm/main.py
@ -52,10 +52,13 @@ from pydantic import BaseModel
 from typing_extensions import overload

 import litellm
+
 # client must be imported from litellm as it's a decorator used at function definition time
 from litellm import client
+
 # Other utils are imported directly to avoid circular imports
 from litellm.utils import exception_type, get_litellm_params, get_optional_params
+
 # Logging is imported lazily when needed to avoid loading litellm_logging at import time
 if TYPE_CHECKING:
    from litellm.litellm_core_utils.litellm_logging import Logging
@ -390,7 +393,7 @@ async def acompletion(
    top_logprobs: Optional[int] = None,
    deployment_id=None,
    reasoning_effort: Optional[
-        Literal["none", "minimal", "low", "medium", "high", "default"]
+        Literal["none", "minimal", "low", "medium", "high", "xhigh", "default"]
    ] = None,
    verbosity: Optional[Literal["low", "medium", "high"]] = None,
    safety_identifier: Optional[str] = None,
@ -1003,7 +1006,7 @@ def completion(  # type: ignore # noqa: PLR0915
    user: Optional[str] = None,
    # openai v1.0+ new params
    reasoning_effort: Optional[
-        Literal["none", "minimal", "low", "medium", "high", "default"]
+        Literal["none", "minimal", "low", "medium", "high", "xhigh", "default"]
    ] = None,
    verbosity: Optional[Literal["low", "medium", "high"]] = None,
    response_format: Optional[Union[dict, Type[BaseModel]]] = None,
@ -2662,6 +2665,35 @@ def completion(  # type: ignore # noqa: PLR0915
            )

            response = model_response
+        elif custom_llm_provider == "amazon_nova":
+            api_key = (
+                api_key
+                or litellm.amazon_nova_api_key
+                or get_secret_str("AMAZON_NOVA_API_KEY")
+                or litellm.api_key
+            )
+            api_base = (
+                api_base
+                or litellm.api_base
+                or get_secret_str("AMAZON_NOVA_API_BASE")
+                or "https://api.nova.amazon.com/v1"
+            )
+            response = openai_like_chat_completion.completion(
+                model=model,
+                messages=messages,
+                api_base=api_base,
+                model_response=model_response,
+                print_verbose=print_verbose,
+                optional_params=optional_params,
+                litellm_params=litellm_params,
+                logger_fn=logger_fn,
+                encoding=encoding,
+                api_key=api_key,
+                logging_obj=logging,
+                timeout=timeout,
+                custom_llm_provider=custom_llm_provider,
+                custom_prompt_dict=custom_prompt_dict,
+            )
        elif custom_llm_provider == "huggingface":
            huggingface_key = (
                api_key
@ -3503,6 +3535,9 @@ def completion(  # type: ignore # noqa: PLR0915
                or get_secret("OLLAMA_API_BASE")
                or "http://localhost:11434"
            )
+            if api_key is not None and "Authorization" not in headers:
+                headers["Authorization"] = f"Bearer {api_key}"
+
            response = base_llm_http_handler.completion(
                model=model,
                stream=stream,
@ -3536,6 +3571,9 @@ def completion(  # type: ignore # noqa: PLR0915
                or os.environ.get("OLLAMA_API_KEY")
                or litellm.api_key
            )
+            if api_key is not None and "Authorization" not in headers:
+                headers["Authorization"] = f"Bearer {api_key}"
+

            response = base_llm_http_handler.completion(
                model=model,
--- a/litellm/model_prices_and_context_window_backup.json
+++ b/litellm/model_prices_and_context_window_backup.json
@ -5164,6 +5164,19 @@
        "supports_function_calling": true,
        "supports_tool_choice": true
    },
+    "azure_ai/mistral-large-3": {
+        "input_cost_per_token": 5e-07,
+        "litellm_provider": "azure_ai",
+        "max_input_tokens": 256000,
+        "max_output_tokens": 8191,
+        "max_tokens": 8191,
+        "mode": "chat",
+        "output_cost_per_token": 1.5e-06,
+        "source": "https://azure.microsoft.com/en-us/blog/introducing-mistral-large-3-in-microsoft-foundry-open-capable-and-ready-for-production-workloads/",
+        "supports_function_calling": true,
+        "supports_tool_choice": true,
+        "supports_vision": true
+    },
    "azure_ai/mistral-medium-2505": {
        "input_cost_per_token": 4e-07,
        "litellm_provider": "azure_ai",
@ -12134,6 +12147,7 @@
        "max_tokens": 65536,
        "mode": "image_generation",
        "output_cost_per_image": 0.134,
+        "output_cost_per_image_token": 1.2e-04,
        "output_cost_per_token": 1.2e-05,
        "output_cost_per_token_batches": 6e-06,
        "source": "https://ai.google.dev/gemini-api/docs/pricing",
@ -13871,6 +13885,7 @@
        "max_tokens": 65536,
        "mode": "image_generation",
        "output_cost_per_image": 0.134,
+        "output_cost_per_image_token": 1.2e-04,
        "output_cost_per_token": 1.2e-05,
        "rpm": 1000,
        "tpm": 4000000,
@ -16940,6 +16955,60 @@
        "supports_response_schema": true,
        "supports_tool_choice": true
    },
+    "amazon-nova/nova-micro-v1": {
+        "input_cost_per_token": 3.5e-08,
+        "litellm_provider": "amazon_nova",
+        "max_input_tokens": 128000,
+        "max_output_tokens": 10000,
+        "max_tokens": 10000,
+        "mode": "chat",
+        "output_cost_per_token": 1.4e-07,
+        "supports_function_calling": true,
+        "supports_prompt_caching": true,
+        "supports_response_schema": true
+    },
+    "amazon-nova/nova-lite-v1": {
+        "input_cost_per_token": 6e-08,
+        "litellm_provider": "amazon_nova",
+        "max_input_tokens": 300000,
+        "max_output_tokens": 10000,
+        "max_tokens": 10000,
+        "mode": "chat",
+        "output_cost_per_token": 2.4e-07,
+        "supports_function_calling": true,
+        "supports_pdf_input": true,
+        "supports_prompt_caching": true,
+        "supports_response_schema": true,
+        "supports_vision": true
+    },
+    "amazon-nova/nova-premier-v1": {
+        "input_cost_per_token": 2.5e-06,
+        "litellm_provider": "amazon_nova",
+        "max_input_tokens": 1000000,
+        "max_output_tokens": 10000,
+        "max_tokens": 10000,
+        "mode": "chat",
+        "output_cost_per_token": 1.25e-05,
+        "supports_function_calling": true,
+        "supports_pdf_input": true,
+        "supports_prompt_caching": false,
+        "supports_response_schema": true,
+        "supports_vision": true
+    },
+    "amazon-nova/nova-pro-v1": {
+        "input_cost_per_token": 8e-07,
+        "litellm_provider": "amazon_nova",
+        "max_input_tokens": 300000,
+        "max_output_tokens": 10000,
+        "max_tokens": 10000,
+        "mode": "chat",
+        "output_cost_per_token": 3.2e-06,
+        "supports_function_calling": true,
+        "supports_pdf_input": true,
+        "supports_prompt_caching": true,
+        "supports_response_schema": true,
+        "supports_vision": true
+    },
    "groq/deepseek-r1-distill-llama-70b": {
        "input_cost_per_token": 7.5e-07,
        "litellm_provider": "groq",
@ -18745,6 +18814,21 @@
        "supports_response_schema": true,
        "supports_tool_choice": true
    },
+    "mistral/mistral-large-3": {
+        "input_cost_per_token": 5e-07,
+        "litellm_provider": "mistral",
+        "max_input_tokens": 256000,
+        "max_output_tokens": 8191,
+        "max_tokens": 8191,
+        "mode": "chat",
+        "output_cost_per_token": 1.5e-06,
+        "source": "https://docs.mistral.ai/models/mistral-large-3-25-12",
+        "supports_assistant_prefill": true,
+        "supports_function_calling": true,
+        "supports_response_schema": true,
+        "supports_tool_choice": true,
+        "supports_vision": true
+    },
    "mistral/mistral-medium": {
        "input_cost_per_token": 2.7e-06,
        "litellm_provider": "mistral",
@ -25774,6 +25858,7 @@
        "max_tokens": 65536,
        "mode": "image_generation",
        "output_cost_per_image": 0.134,
+        "output_cost_per_image_token": 1.2e-04,
        "output_cost_per_token": 1.2e-05,
        "output_cost_per_token_batches": 6e-06,
        "source": "https://docs.cloud.google.com/vertex-ai/generative-ai/docs/models/gemini/3-pro-image"
--- a/litellm/proxy/_experimental/mcp_server/mcp_server_manager.py
+++ b/litellm/proxy/_experimental/mcp_server/mcp_server_manager.py
@ -1272,19 +1272,19 @@ class MCPServerManager:
        prefix = get_server_prefix(server)

        for tool in tools:
-            prefixed_name = add_server_prefix_to_name(tool.name, prefix)
+            tool_copy = tool.model_copy(deep=True)

-            name_to_use = prefixed_name if add_prefix else tool.name
+            original_name = tool_copy.name
+            prefixed_name = add_server_prefix_to_name(original_name, prefix)

-            tool_obj = MCPTool(
-                name=name_to_use,
-                description=tool.description,
-                inputSchema=tool.inputSchema,
-            )
-            prefixed_tools.append(tool_obj)
+            name_to_use = prefixed_name if add_prefix else original_name
+
+            # Preserve all tool fields including metadata/_meta by avoiding mutation
+            tool_copy.name = name_to_use
+            prefixed_tools.append(tool_copy)

            # Update tool to server mapping for resolution (support both forms)
-            self.tool_name_to_mcp_server_name_mapping[tool.name] = prefix
+            self.tool_name_to_mcp_server_name_mapping[original_name] = prefix
            self.tool_name_to_mcp_server_name_mapping[prefixed_name] = prefix

        verbose_logger.info(
@ -1953,9 +1953,9 @@ class MCPServerManager:
                            server_name_from_prefix
                        ):
                            return server
-                    elif normalize_server_name(server.server_name) == normalize_server_name(
-                        server_name_from_prefix
-                    ):
+                    elif normalize_server_name(
+                        server.server_name
+                    ) == normalize_server_name(server_name_from_prefix):
                        return server

        return None
--- a/litellm/proxy/_experimental/mcp_server/server.py
+++ b/litellm/proxy/_experimental/mcp_server/server.py
@ -6,7 +6,7 @@ LiteLLM MCP Server Routes
 import asyncio
 import contextlib
 from datetime import datetime
-from typing import Any, AsyncIterator, Dict, List, Optional, Tuple, Union
+from typing import Any, AsyncIterator, Dict, List, Optional, Tuple, Union, cast

 from fastapi import FastAPI, HTTPException
 from pydantic import AnyUrl, ConfigDict
@ -72,7 +72,13 @@ if MCP_AVAILABLE:
        auth_context_var,
    )
    from mcp.server.streamable_http_manager import StreamableHTTPSessionManager
-    from mcp.types import EmbeddedResource, ImageContent, Prompt, TextContent
+    from mcp.types import (
+        CallToolResult,
+        EmbeddedResource,
+        ImageContent,
+        Prompt,
+        TextContent,
+    )
    from mcp.types import Tool as MCPTool

    from litellm.proxy._experimental.mcp_server.auth.litellm_auth_handler import (
@ -234,7 +240,7 @@ if MCP_AVAILABLE:
    @server.call_tool()
    async def mcp_server_tool_call(
        name: str, arguments: Dict[str, Any] | None
-    ) -> List[Union[TextContent, ImageContent, EmbeddedResource]]:
+    ) -> CallToolResult:
        """
        Call a specific tool with the provided arguments

@ -300,26 +306,37 @@ if MCP_AVAILABLE:
            )
        except BlockedPiiEntityError as e:
            verbose_logger.error(f"BlockedPiiEntityError in MCP tool call: {str(e)}")
-            # Return error as text content for MCP protocol
-            return [
-                TextContent(
-                    text=f"Error: Blocked PII entity detected - {str(e)}", type="text"
-                )
-            ]
+            return CallToolResult(
+                content=[
+                    TextContent(
+                        text=f"Error: Blocked PII entity detected - {str(e)}",
+                        type="text",
+                    )
+                ],
+                isError=True,
+            )
        except GuardrailRaisedException as e:
            verbose_logger.error(f"GuardrailRaisedException in MCP tool call: {str(e)}")
-            # Return error as text content for MCP protocol
-            return [
-                TextContent(text=f"Error: Guardrail violation - {str(e)}", type="text")
-            ]
+            return CallToolResult(
+                content=[
+                    TextContent(
+                        text=f"Error: Guardrail violation - {str(e)}", type="text"
+                    )
+                ],
+                isError=True,
+            )
        except HTTPException as e:
            verbose_logger.error(f"HTTPException in MCP tool call: {str(e)}")
-            # Return error as text content for MCP protocol
-            return [TextContent(text=f"Error: {str(e.detail)}", type="text")]
+            return CallToolResult(
+                content=[TextContent(text=f"Error: {str(e.detail)}", type="text")],
+                isError=True,
+            )
        except Exception as e:
            verbose_logger.exception(f"MCP mcp_server_tool_call - error: {e}")
-            # Return error as text content for MCP protocol
-            return [TextContent(text=f"Error: {str(e)}", type="text")]
+            return CallToolResult(
+                content=[TextContent(text=f"Error: {str(e)}", type="text")],
+                isError=True,
+            )

        return response

@ -1173,7 +1190,7 @@ if MCP_AVAILABLE:
        oauth2_headers: Optional[Dict[str, str]] = None,
        raw_headers: Optional[Dict[str, str]] = None,
        **kwargs: Any,
-    ) -> List[Union[TextContent, ImageContent, EmbeddedResource]]:
+    ) -> CallToolResult:
        """
        Call a specific tool with the provided arguments (handles prefixed tool names)
        """
@ -1237,9 +1254,9 @@ if MCP_AVAILABLE:
            "litellm_logging_obj", None
        )
        if litellm_logging_obj:
-            litellm_logging_obj.model_call_details["mcp_tool_call_metadata"] = (
-                standard_logging_mcp_tool_call
-            )
+            litellm_logging_obj.model_call_details[
+                "mcp_tool_call_metadata"
+            ] = standard_logging_mcp_tool_call
            litellm_logging_obj.model = f"MCP: {name}"
        # Check if tool exists in local registry first (for OpenAPI-based tools)
        # These tools are registered with their prefixed names
@ -1247,7 +1264,8 @@ if MCP_AVAILABLE:
        local_tool = global_mcp_tool_registry.get_tool(name)
        if local_tool:
            verbose_logger.debug(f"Executing local registry tool: {name}")
-            response = await _handle_local_mcp_tool(name, arguments)
+            local_content = await _handle_local_mcp_tool(name, arguments)
+            response = CallToolResult(content=cast(Any, local_content), isError=False)

        # Try managed MCP server tool (pass the full prefixed name)
        # Primary and recommended way to use external MCP servers
@ -1279,7 +1297,12 @@ if MCP_AVAILABLE:
            # Deprecated: Local MCP Server Tool
            #########################################################
            else:
-                response = await _handle_local_mcp_tool(original_tool_name, arguments)
+                local_content = await _handle_local_mcp_tool(
+                    original_tool_name, arguments
+                )
+                response = CallToolResult(
+                    content=cast(Any, local_content), isError=False
+                )

        #########################################################
        # Post MCP Tool Call Hook
@ -1432,7 +1455,7 @@ if MCP_AVAILABLE:
        oauth2_headers: Optional[Dict[str, str]] = None,
        raw_headers: Optional[Dict[str, str]] = None,
        litellm_logging_obj: Optional[Any] = None,
-    ) -> List[Union[TextContent, ImageContent, EmbeddedResource]]:
+    ) -> CallToolResult:
        """Handle tool execution for managed server tools"""
        # Import here to avoid circular import
        from litellm.proxy.proxy_server import proxy_logging_obj
@ -1449,7 +1472,7 @@ if MCP_AVAILABLE:
            proxy_logging_obj=proxy_logging_obj,
        )
        verbose_logger.debug("CALL TOOL RESULT: %s", call_tool_result)
-        return call_tool_result.content  # type: ignore[return-value]
+        return call_tool_result

    async def _handle_local_mcp_tool(
        name: str, arguments: Dict[str, Any]
@ -1741,14 +1764,16 @@ if MCP_AVAILABLE:
        )
        auth_context_var.set(auth_user)

-    def get_auth_context() -> Tuple[
-        Optional[UserAPIKeyAuth],
-        Optional[str],
-        Optional[List[str]],
-        Optional[Dict[str, Dict[str, str]]],
-        Optional[Dict[str, str]],
-        Optional[Dict[str, str]],
-    ]:
+    def get_auth_context() -> (
+        Tuple[
+            Optional[UserAPIKeyAuth],
+            Optional[str],
+            Optional[List[str]],
+            Optional[Dict[str, Dict[str, str]]],
+            Optional[Dict[str, str]],
+            Optional[Dict[str, str]],
+        ]
+    ):
        """
        Get the UserAPIKeyAuth from the auth context variable.

--- a/litellm/proxy/_experimental/out/_next/static/chunks/1200-64d099608f321062.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/1200-64d099608f321062.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/1200-72b27753cc974b47.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/1200-72b27753cc974b47.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/1301-c5ca003a7988f6b1.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/1301-c5ca003a7988f6b1.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/1518-499d06fabfcafb58.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/1518-499d06fabfcafb58.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/1518-af5eabc2040f5a96.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/1518-af5eabc2040f5a96.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/1739-a97d403afe23a96f.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/1739-a97d403afe23a96f.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/1971-92070b200b9aaa46.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/1971-92070b200b9aaa46.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/2004-e6c5ce8d6dc32432.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/2004-e6c5ce8d6dc32432.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/2012-4854cbc146d8a7eb.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/2012-4854cbc146d8a7eb.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/2012-dcbd62e829c6106f.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/2012-dcbd62e829c6106f.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/2202-721dd881f2afe3d2.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/2202-721dd881f2afe3d2.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/2249-985208cb7064c804.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/2249-985208cb7064c804.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/2273-fdf410d28cc9d394.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/2273-fdf410d28cc9d394.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/2409-e94c05c6f11bb939.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/2409-e94c05c6f11bb939.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/3163-e261e5767e016074.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/3163-e261e5767e016074.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/3367-58830187e9e5b9fa.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/3367-58830187e9e5b9fa.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/3705-05649f5df18d8716.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/3705-05649f5df18d8716.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/3801-3f7b66ca5919fd60.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/3801-3f7b66ca5919fd60.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/4292-a3e9c22c4ffc7d9a.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/4292-a3e9c22c4ffc7d9a.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/4679-12d9b222cdd9a545.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/4679-12d9b222cdd9a545.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/4679-bdd70e8457d0a482.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/4679-bdd70e8457d0a482.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/5301-92cec930d7b4b830.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/5301-92cec930d7b4b830.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/5333-e9bc197d1822e3ad.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/5333-e9bc197d1822e3ad.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/5458-3a5d500e8deb5b23.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/5458-3a5d500e8deb5b23.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/5830-0662e9fc2ba82b07.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/5830-0662e9fc2ba82b07.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/5869-426268ba6ad0ce0c.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/5869-426268ba6ad0ce0c.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/611-3b24a9c382a17460.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/611-3b24a9c382a17460.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/630-9c30ebb65854ac3f.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/630-9c30ebb65854ac3f.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/6600-f82a8329e442461d.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/6600-f82a8329e442461d.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/6609-d93906f43161f066.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/6609-d93906f43161f066.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/7155-036c6fcc23f65f77.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/7155-036c6fcc23f65f77.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/7164-c65c1ea80db7bbed.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/7164-c65c1ea80db7bbed.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/7526-997a4faae4b21df5.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/7526-997a4faae4b21df5.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/7641-2d52ba6eada33bb5.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/7641-2d52ba6eada33bb5.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/7641-90e15c72e10330f1.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/7641-90e15c72e10330f1.js
--- a/litellm/proxy/_experimental/out/_next/static/chunks/7692-2d8e89cbe1f5ea87.js
+++ b/litellm/proxy/_experimental/out/_next/static/chunks/7692-2d8e89cbe1f5ea87.js
--- a/Show more
+++ b/Show more