mirror of
https://github.com/BerriAI/litellm.git
synced 2026-08-28 05:25:59 +00:00
Merge branch 'main' into litellm_agents_budget_limits
This commit is contained in:
Harshit Jain
GitHub
commit
86b5e2d873
721 changed files with 15981 additions and 6624 deletions
252
docs/my-website/docs/a2a_agent_headers.md
Normal file
252
docs/my-website/docs/a2a_agent_headers.md
Normal file
|
|
@ -0,0 +1,252 @@
|
|||
import Tabs from '@theme/Tabs';
|
||||
import TabItem from '@theme/TabItem';
|
||||
|
||||
# A2A Agent Authentication Headers
|
||||
|
||||
Forward authentication credentials (Bearer tokens, API keys, etc.) from clients to backend A2A agents.
|
||||
|
||||
## Overview
|
||||
|
||||
When LiteLLM proxies a request to a backend A2A agent, the agent may require its own authentication headers. There are three ways to supply them:
|
||||
|
||||
| Method | Who configures | How it works |
|
||||
|---|---|---|
|
||||
| **Static headers** | Admin (UI / API) | Always sent, regardless of client request |
|
||||
| **Forward client headers** | Admin (UI / API) | Header names to extract from client request and forward |
|
||||
| **Convention-based** | Client (no admin config) | Client sends `x-a2a-{agent_name}-{header}` — automatically routed |
|
||||
|
||||
All three methods can be combined. **Static headers always win** on key conflicts.
|
||||
|
||||
---
|
||||
|
||||
## Method 1 — Static Headers
|
||||
|
||||
Admin-configured headers that are always sent to the backend agent. Use this for server-to-server tokens or internal credentials that clients should never see or override.
|
||||
|
||||
<Tabs>
|
||||
<TabItem value="ui" label="UI">
|
||||
|
||||
1. Go to **Agents** in the LiteLLM dashboard.
|
||||
2. Create or edit an agent.
|
||||
3. Open the **Authentication Headers** panel.
|
||||
4. Under **Static Headers**, click **Add Static Header** and fill in the header name and value.
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="api" label="REST API">
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:4000/v1/agents \
|
||||
-H "Authorization: Bearer sk-admin" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"agent_name": "my-agent",
|
||||
"agent_card_params": { ... },
|
||||
"static_headers": {
|
||||
"Authorization": "Bearer internal-server-token",
|
||||
"X-Internal-Service": "litellm-proxy"
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
To update an existing agent:
|
||||
|
||||
```bash
|
||||
curl -X PATCH http://localhost:4000/v1/agents/{agent_id} \
|
||||
-H "Authorization: Bearer sk-admin" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"static_headers": {
|
||||
"Authorization": "Bearer new-token"
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
**Client call — no special headers needed:**
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:4000/a2a/my-agent \
|
||||
-H "Authorization: Bearer sk-client-key" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"jsonrpc": "2.0", "id": "1", "method": "message/send",
|
||||
"params": { "message": { "role": "user", "parts": [{"kind": "text", "text": "Hello"}], "messageId": "msg-1" } }
|
||||
}'
|
||||
```
|
||||
|
||||
The backend agent receives `Authorization: Bearer internal-server-token` without the client ever knowing the value.
|
||||
|
||||
---
|
||||
|
||||
## Method 2 — Forward Client Headers
|
||||
|
||||
Admin specifies a list of header **names**. When the client sends a request that includes those headers, LiteLLM extracts their values and forwards them to the backend agent. The client controls the values; the admin controls which headers are eligible to be forwarded.
|
||||
|
||||
<Tabs>
|
||||
<TabItem value="ui" label="UI">
|
||||
|
||||
1. Go to **Agents** in the LiteLLM dashboard.
|
||||
2. Create or edit an agent.
|
||||
3. Open the **Authentication Headers** panel.
|
||||
4. Under **Forward Client Headers**, type header names and press **Enter** (e.g. `x-api-key`, `Authorization`).
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="api" label="REST API">
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:4000/v1/agents \
|
||||
-H "Authorization: Bearer sk-admin" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"agent_name": "my-agent",
|
||||
"agent_card_params": { ... },
|
||||
"extra_headers": ["x-api-key", "x-user-token"]
|
||||
}'
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
**Client call — include the forwarded headers:**
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:4000/a2a/my-agent \
|
||||
-H "Authorization: Bearer sk-client-key" \
|
||||
-H "x-api-key: user-secret-value" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{ ... }'
|
||||
```
|
||||
|
||||
The backend agent receives `x-api-key: user-secret-value`.
|
||||
|
||||
:::note
|
||||
Header name matching is **case-insensitive**. If the client sends `X-API-Key` and `extra_headers` lists `x-api-key`, they match.
|
||||
:::
|
||||
|
||||
---
|
||||
|
||||
## Method 3 — Convention-Based Forwarding
|
||||
|
||||
Clients can forward headers to a specific agent without any admin pre-configuration by using the naming convention:
|
||||
|
||||
```
|
||||
x-a2a-{agent_name_or_id}-{header_name}: value
|
||||
```
|
||||
|
||||
LiteLLM parses these headers automatically and routes them to the matching agent only.
|
||||
|
||||
**Examples:**
|
||||
|
||||
| Client header sent | Agent name/ID | Forwarded as |
|
||||
|---|---|---|
|
||||
| `x-a2a-my-agent-authorization: Bearer tok` | `my-agent` | `authorization: Bearer tok` |
|
||||
| `x-a2a-my-agent-x-api-key: secret` | `my-agent` | `x-api-key: secret` |
|
||||
| `x-a2a-abc123-authorization: Bearer tok` | agent ID `abc123` | `authorization: Bearer tok` |
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:4000/a2a/my-agent \
|
||||
-H "Authorization: Bearer sk-client-key" \
|
||||
-H "x-a2a-my-agent-authorization: Bearer agent-specific-token" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{ ... }'
|
||||
```
|
||||
|
||||
The `x-a2a-other-agent-authorization` header sent in the same request is **not** forwarded to `my-agent` — it is silently ignored.
|
||||
|
||||
:::tip Matches both agent name and agent ID
|
||||
Both the human-readable name (e.g. `my-agent`) and the UUID (e.g. `abc123-...`) are valid. Use whichever is convenient for the client.
|
||||
:::
|
||||
|
||||
---
|
||||
|
||||
## Merge Precedence
|
||||
|
||||
When multiple methods supply the same header name, **static headers win**:
|
||||
|
||||
```
|
||||
dynamic (forwarded/convention) → merged ← static (overlays, wins)
|
||||
```
|
||||
|
||||
Example:
|
||||
|
||||
| Source | `Authorization` value |
|
||||
|---|---|
|
||||
| Client sends (via `extra_headers` or convention) | `Bearer client-token` |
|
||||
| Admin-configured `static_headers` | `Bearer server-token` |
|
||||
| **What the backend agent receives** | **`Bearer server-token`** |
|
||||
|
||||
This ensures admin-controlled credentials cannot be overridden by client requests.
|
||||
|
||||
---
|
||||
|
||||
## Combining All Three Methods
|
||||
|
||||
```bash
|
||||
# Register agent with static + forwarded headers
|
||||
curl -X POST http://localhost:4000/v1/agents \
|
||||
-H "Authorization: Bearer sk-admin" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"agent_name": "my-agent",
|
||||
"agent_card_params": { ... },
|
||||
"static_headers": {
|
||||
"X-Internal-Token": "secret123"
|
||||
},
|
||||
"extra_headers": ["x-user-id"]
|
||||
}'
|
||||
|
||||
# Client call using all three mechanisms
|
||||
curl -X POST http://localhost:4000/a2a/my-agent \
|
||||
-H "Authorization: Bearer sk-client-key" \
|
||||
-H "x-user-id: user-42" \
|
||||
-H "x-a2a-my-agent-x-request-id: req-abc" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{ ... }'
|
||||
```
|
||||
|
||||
The backend agent receives:
|
||||
|
||||
```
|
||||
X-Internal-Token: secret123 ← static header (always)
|
||||
x-user-id: user-42 ← forwarded (in extra_headers)
|
||||
x-request-id: req-abc ← convention-based (x-a2a-my-agent-*)
|
||||
X-LiteLLM-Trace-Id: <uuid> ← LiteLLM internal
|
||||
X-LiteLLM-Agent-Id: <agent-id> ← LiteLLM internal
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Header Isolation
|
||||
|
||||
Each agent invocation uses an isolated HTTP connection. Headers configured for agent A are **never** sent to agent B, even if both agents are running and receiving requests simultaneously.
|
||||
|
||||
---
|
||||
|
||||
## API Reference
|
||||
|
||||
### `POST /v1/agents` / `PATCH /v1/agents/{agent_id}`
|
||||
|
||||
| Field | Type | Description |
|
||||
|---|---|---|
|
||||
| `static_headers` | `object` | `{"Header-Name": "value"}` — always forwarded |
|
||||
| `extra_headers` | `string[]` | Header names to extract from client request and forward |
|
||||
|
||||
### Agent Response
|
||||
|
||||
Both fields are returned in `GET /v1/agents` and `GET /v1/agents/{agent_id}`:
|
||||
|
||||
```json
|
||||
{
|
||||
"agent_id": "...",
|
||||
"agent_name": "my-agent",
|
||||
"static_headers": { "X-Internal-Token": "secret123" },
|
||||
"extra_headers": ["x-user-id"],
|
||||
...
|
||||
}
|
||||
```
|
||||
|
||||
:::caution
|
||||
`static_headers` values are stored in the database and returned by the API. Treat them as you would any credential — do not store sensitive long-lived tokens here if your API is publicly accessible. Consider using short-lived tokens or environment-injected secrets instead.
|
||||
:::
|
||||
|
|
@ -704,6 +704,63 @@ asyncio.run(main())
|
|||
|
||||
[Learn more about customer management →](./proxy/customers)
|
||||
|
||||
## Calling the Proxy's /v1/responses Endpoint
|
||||
|
||||
When calling your LiteLLM Proxy's `/v1/responses` endpoint to use MCP tools, **always use `server_url: "litellm_proxy"`** in the tools array. This tells the proxy to use its configured MCP servers.
|
||||
|
||||
:::important Do not use the full proxy URL
|
||||
Using `server_url: "https://your-proxy.com/mcp"` is incorrect when the request is already going to the proxy. The proxy needs the literal value `litellm_proxy` to route to its configured MCP servers.
|
||||
:::
|
||||
|
||||
```bash title="Correct: Using litellm_proxy" showLineNumbers
|
||||
curl --location 'https://your-proxy.com/v1/responses' \
|
||||
--header 'Content-Type: application/json' \
|
||||
--header "Authorization: Bearer $LITELLM_API_KEY" \
|
||||
--data '{
|
||||
"model": "gpt-4",
|
||||
"tools": [
|
||||
{
|
||||
"type": "mcp",
|
||||
"server_label": "litellm",
|
||||
"server_url": "litellm_proxy",
|
||||
"require_approval": "never"
|
||||
}
|
||||
],
|
||||
"input": "Run available tools",
|
||||
"tool_choice": "required"
|
||||
}'
|
||||
```
|
||||
|
||||
### Sending Custom Headers to MCP Servers
|
||||
|
||||
To pass custom headers (e.g., API keys, auth tokens) to specific MCP servers, use either:
|
||||
|
||||
**Option 1: Request headers** – Add `x-mcp-{server_alias}-{header_name}` to your request headers. The proxy forwards these to the matching MCP server.
|
||||
|
||||
```bash
|
||||
# Send Authorization header to the "weather2" MCP server
|
||||
--header 'x-mcp-weather2-authorization: Bearer your-token'
|
||||
|
||||
# Send custom header to the "github" MCP server
|
||||
--header 'x-mcp-github-x-api-key: your-api-key'
|
||||
```
|
||||
|
||||
**Option 2: Headers in tool config** – Include a `headers` object in the tool definition. These are merged with request headers.
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "mcp",
|
||||
"server_label": "litellm",
|
||||
"server_url": "litellm_proxy",
|
||||
"require_approval": "never",
|
||||
"headers": {
|
||||
"x-litellm-api-key": "Bearer YOUR_LITELLM_API_KEY",
|
||||
"x-mcp-servers": "Zapier_MCP,dev-group",
|
||||
"x-mcp-weather2-authorization": "Bearer your-weather-api-token"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Using your MCP with client side credentials
|
||||
|
||||
Use this if you want to pass a client side authentication token to LiteLLM to then pass to your MCP to auth to your MCP.
|
||||
|
|
|
|||
|
|
@ -323,7 +323,7 @@ curl --location '<your-litellm-proxy-base-url>/v1/responses' \
|
|||
{
|
||||
"type": "mcp",
|
||||
"server_label": "litellm",
|
||||
"server_url": "<your-litellm-proxy-base-url>/dev_group/mcp",
|
||||
"server_url": "litellm_proxy",
|
||||
"require_approval": "never",
|
||||
"headers": {
|
||||
"x-litellm-api-key": "Bearer YOUR_LITELLM_API_KEY"
|
||||
|
|
@ -335,7 +335,7 @@ curl --location '<your-litellm-proxy-base-url>/v1/responses' \
|
|||
}'
|
||||
```
|
||||
|
||||
This example uses URL namespacing to access all servers in the "dev_group" access group.
|
||||
This example uses the `x-mcp-servers` header to access all servers in the "dev_group" access group. Use `server_url: "litellm_proxy"` when calling the proxy's `/v1/responses` endpoint—do not use the full proxy URL.
|
||||
|
||||
</TabItem>
|
||||
|
||||
|
|
@ -423,7 +423,7 @@ curl --location '<your-litellm-proxy-base-url>/v1/responses' \
|
|||
{
|
||||
"type": "mcp",
|
||||
"server_label": "litellm",
|
||||
"server_url": "<your-litellm-proxy-base-url>/mcp/",
|
||||
"server_url": "litellm_proxy",
|
||||
"require_approval": "never",
|
||||
"headers": {
|
||||
"x-litellm-api-key": "Bearer YOUR_LITELLM_API_KEY",
|
||||
|
|
@ -436,7 +436,7 @@ curl --location '<your-litellm-proxy-base-url>/v1/responses' \
|
|||
}'
|
||||
```
|
||||
|
||||
This configuration restricts the request to only use tools from the specified MCP servers.
|
||||
This configuration restricts the request to only use tools from the specified MCP servers. Use `server_url: "litellm_proxy"` when calling the proxy's `/v1/responses` endpoint.
|
||||
|
||||
</TabItem>
|
||||
|
||||
|
|
|
|||
|
|
@ -2,6 +2,32 @@
|
|||
|
||||
Azure Model Router is a feature in Azure AI Foundry that automatically routes your requests to the best available model based on your requirements. This allows you to use a single endpoint that intelligently selects the optimal model for each request.
|
||||
|
||||
## Quick Start
|
||||
|
||||
**Model pattern**: `azure_ai/model_router/<deployment-name>`
|
||||
|
||||
```python
|
||||
import litellm
|
||||
|
||||
response = litellm.completion(
|
||||
model="azure_ai/model_router/model-router", # Replace with your deployment name
|
||||
messages=[{"role": "user", "content": "Hello!"}],
|
||||
api_base="https://your-endpoint.cognitiveservices.azure.com/openai/v1/",
|
||||
api_key="your-api-key",
|
||||
)
|
||||
```
|
||||
|
||||
**Proxy config** (`config.yaml`):
|
||||
|
||||
```yaml
|
||||
model_list:
|
||||
- model_name: model-router
|
||||
litellm_params:
|
||||
model: azure_ai/model_router/model-router
|
||||
api_base: https://your-endpoint.cognitiveservices.azure.com/openai/deployments/model-router/chat/completions?api-version=2025-01-01-preview
|
||||
api_key: your-api-key
|
||||
```
|
||||
|
||||
## Key Features
|
||||
|
||||
- **Automatic Model Selection**: Azure Model Router dynamically selects the best model for your request
|
||||
|
|
@ -229,19 +255,51 @@ Cost is tracked based on the actual model used (e.g., `gpt-4.1-nano`), plus a fl
|
|||
|
||||
## Cost Tracking
|
||||
|
||||
LiteLLM automatically handles cost tracking for Azure Model Router by:
|
||||
LiteLLM automatically handles cost tracking for Azure Model Router. Understanding how this works helps you interpret spend and debug billing.
|
||||
|
||||
1. **Detecting the actual model**: When Azure Model Router routes your request to a specific model (e.g., `gpt-4.1-nano-2025-04-14`), LiteLLM extracts this from the response
|
||||
2. **Calculating accurate costs**: Costs are calculated based on:
|
||||
- The actual model used (e.g., `gpt-4.1-nano` token costs)
|
||||
- Plus a flat infrastructure cost of **$0.14 per million input tokens** for using the Model Router
|
||||
3. **Streaming support**: Cost tracking works correctly for both streaming and non-streaming requests
|
||||
### How LiteLLM Calculates Cost
|
||||
|
||||
When you use Azure Model Router, LiteLLM computes **two cost components**:
|
||||
|
||||
| Component | Description | When Applied |
|
||||
|-----------|-------------|--------------|
|
||||
| **Model Cost** | Token-based cost for the actual model that handled the request (e.g., `gpt-5-nano`, `gpt-4.1-nano`) | Always, when Azure returns the model in the response |
|
||||
| **Router Flat Cost** | $0.14 per million input tokens (Azure AI Foundry infrastructure fee) | When the **request** was made via a model router endpoint |
|
||||
|
||||
### Cost Calculation Flow
|
||||
|
||||
1. **Request model detection**: LiteLLM records the model you requested (e.g., `azure_ai/model_router/model-router`). If it contains `model_router` or `model-router`, the request is treated as a router request.
|
||||
|
||||
2. **Response model extraction**: Azure returns the actual model used in the response (e.g., `gpt-5-nano-2025-08-07`). LiteLLM uses this for the model cost lookup.
|
||||
|
||||
3. **Model cost**: LiteLLM looks up the response model in its pricing table and computes cost from prompt tokens and completion tokens.
|
||||
|
||||
4. **Router flat cost**: Because the original request was to a model router, LiteLLM adds the flat cost ($0.14 per M input tokens) on top of the model cost.
|
||||
|
||||
5. **Total cost**: `Total = Model Cost + Router Flat Cost`
|
||||
|
||||
### Configuration Requirements
|
||||
|
||||
For cost tracking to work correctly:
|
||||
|
||||
- **Use the full pattern**: `azure_ai/model_router/<deployment-name>` (e.g., `azure_ai/model_router/model-router`)
|
||||
- **Proxy config**: When using the LiteLLM proxy, set `model` in `litellm_params` to the full pattern so the request model is correctly identified as a router
|
||||
|
||||
```yaml
|
||||
# proxy_server_config.yaml
|
||||
model_list:
|
||||
- model_name: model-router
|
||||
litellm_params:
|
||||
model: azure_ai/model_router/model-router # Required for router cost detection
|
||||
api_base: https://your-endpoint.cognitiveservices.azure.com/openai/deployments/model-router/chat/completions?api-version=2025-01-01-preview
|
||||
api_key: your-api-key
|
||||
```
|
||||
|
||||
### Cost Breakdown
|
||||
|
||||
When you use Azure Model Router, the total cost includes:
|
||||
|
||||
- **Model Cost**: Based on the actual model that handled your request (e.g., `gpt-4.1-nano`)
|
||||
- **Model Cost**: Based on the actual model that handled your request (e.g., `gpt-5-nano`, `gpt-4.1-nano`)
|
||||
- **Router Flat Cost**: $0.14 per million input tokens (Azure AI Foundry infrastructure fee)
|
||||
|
||||
### Example Response with Cost
|
||||
|
|
|
|||
|
|
@ -4,12 +4,12 @@ Use ChatGPT Pro/Max subscription models through LiteLLM with OAuth device flow a
|
|||
|
||||
| Property | Details |
|
||||
|-------|-------|
|
||||
| Description | ChatGPT subscription access (Codex + GPT-5.2 family) via ChatGPT backend API |
|
||||
| Description | ChatGPT subscription access (Codex + GPT-5.3/5.4 family) via ChatGPT backend API |
|
||||
| Provider Route on LiteLLM | `chatgpt/` |
|
||||
| Supported Endpoints | `/responses`, `/chat/completions` (bridged to Responses for supported models) |
|
||||
| API Reference | https://chatgpt.com |
|
||||
|
||||
ChatGPT subscription access is native to the Responses API. Chat Completions requests are bridged to Responses for supported models (for example `chatgpt/gpt-5.2`).
|
||||
ChatGPT subscription access is native to the Responses API. Chat Completions requests are bridged to Responses for supported models (for example `chatgpt/gpt-5.4`).
|
||||
|
||||
Notes:
|
||||
- The ChatGPT subscription backend rejects token limit fields (`max_tokens`, `max_output_tokens`, `max_completion_tokens`) and `metadata`. LiteLLM strips these fields for this provider.
|
||||
|
|
@ -31,7 +31,7 @@ ChatGPT subscription access uses an OAuth device code flow:
|
|||
import litellm
|
||||
|
||||
response = litellm.responses(
|
||||
model="chatgpt/gpt-5.2-codex",
|
||||
model="chatgpt/gpt-5.3-codex",
|
||||
input="Write a Python hello world"
|
||||
)
|
||||
|
||||
|
|
@ -44,7 +44,7 @@ print(response)
|
|||
import litellm
|
||||
|
||||
response = litellm.completion(
|
||||
model="chatgpt/gpt-5.2",
|
||||
model="chatgpt/gpt-5.4",
|
||||
messages=[{"role": "user", "content": "Write a Python hello world"}]
|
||||
)
|
||||
|
||||
|
|
@ -55,16 +55,36 @@ print(response)
|
|||
|
||||
```yaml showLineNumbers title="config.yaml"
|
||||
model_list:
|
||||
- model_name: chatgpt/gpt-5.2
|
||||
- model_name: chatgpt/gpt-5.4
|
||||
model_info:
|
||||
mode: responses
|
||||
litellm_params:
|
||||
model: chatgpt/gpt-5.2
|
||||
- model_name: chatgpt/gpt-5.2-codex
|
||||
model: chatgpt/gpt-5.4
|
||||
- model_name: chatgpt/gpt-5.4-pro
|
||||
model_info:
|
||||
mode: responses
|
||||
litellm_params:
|
||||
model: chatgpt/gpt-5.2-codex
|
||||
model: chatgpt/gpt-5.4-pro
|
||||
- model_name: chatgpt/gpt-5.3-codex
|
||||
model_info:
|
||||
mode: responses
|
||||
litellm_params:
|
||||
model: chatgpt/gpt-5.3-codex
|
||||
- model_name: chatgpt/gpt-5.3-codex-spark
|
||||
model_info:
|
||||
mode: responses
|
||||
litellm_params:
|
||||
model: chatgpt/gpt-5.3-codex-spark
|
||||
- model_name: chatgpt/gpt-5.3-instant
|
||||
model_info:
|
||||
mode: responses
|
||||
litellm_params:
|
||||
model: chatgpt/gpt-5.3-instant
|
||||
- model_name: chatgpt/gpt-5.3-chat-latest
|
||||
model_info:
|
||||
mode: responses
|
||||
litellm_params:
|
||||
model: chatgpt/gpt-5.3-chat-latest
|
||||
```
|
||||
|
||||
```bash showLineNumbers title="Start LiteLLM Proxy"
|
||||
|
|
|
|||
|
|
@ -192,8 +192,12 @@ os.environ["OPENAI_BASE_URL"] = "https://your_host/v1" # OPTIONAL
|
|||
| gpt-5.2-2025-12-11 | `response = completion(model="gpt-5.2-2025-12-11", messages=messages)` |
|
||||
| gpt-5.2-chat-latest | `response = completion(model="gpt-5.2-chat-latest", messages=messages)` |
|
||||
| gpt-5.3-chat-latest | `response = completion(model="gpt-5.3-chat-latest", messages=messages)` |
|
||||
| gpt-5.4 | `response = completion(model="gpt-5.4", messages=messages)` |
|
||||
| gpt-5.4-2026-03-05 | `response = completion(model="gpt-5.4-2026-03-05", messages=messages)` |
|
||||
| gpt-5.2-pro | `response = completion(model="gpt-5.2-pro", messages=messages)` |
|
||||
| gpt-5.2-pro-2025-12-11 | `response = completion(model="gpt-5.2-pro-2025-12-11", messages=messages)` |
|
||||
| gpt-5.4-pro | `response = completion(model="gpt-5.4-pro", messages=messages)` |
|
||||
| gpt-5.4-pro-2026-03-05 | `response = completion(model="gpt-5.4-pro-2026-03-05", messages=messages)` |
|
||||
| gpt-5.1 | `response = completion(model="gpt-5.1", messages=messages)` |
|
||||
| gpt-5.1-codex | `response = completion(model="gpt-5.1-codex", messages=messages)` |
|
||||
| gpt-5.1-codex-mini | `response = completion(model="gpt-5.1-codex-mini", messages=messages)` |
|
||||
|
|
|
|||
|
|
@ -1472,6 +1472,82 @@ Your WIF credentials JSON file typically looks like this (for AWS federation):
|
|||
|
||||
For more details on setting up Workload Identity Federation, see [Google Cloud WIF documentation](https://cloud.google.com/iam/docs/workload-identity-federation).
|
||||
|
||||
#### Explicit AWS Credentials for WIF
|
||||
|
||||
By default, AWS-based WIF relies on the EC2 instance metadata service to obtain AWS credentials. This works when LiteLLM runs on an EC2 instance or ECS task with an IAM role attached.
|
||||
|
||||
If your environment **does not have access to the EC2 metadata service** (e.g., running on-premises, in a container without host networking, or in a different cloud with security restrictions), you can provide explicit AWS credentials directly in the WIF credential JSON file. LiteLLM will use these to authenticate to AWS before performing the GCP token exchange.
|
||||
|
||||
Add the `aws_*` keys at the **top level** of your WIF credential JSON (alongside `type`, `audience`, etc.):
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "external_account",
|
||||
"audience": "//iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/providers/PROVIDER_ID",
|
||||
"subject_token_type": "urn:ietf:params:aws:token-type:aws4_request",
|
||||
"service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/SERVICE_ACCOUNT_EMAIL:generateAccessToken",
|
||||
"token_url": "https://sts.googleapis.com/v1/token",
|
||||
"credential_source": {
|
||||
"environment_id": "aws1",
|
||||
"region_url": "http://169.254.169.254/latest/meta-data/placement/availability-zone",
|
||||
"url": "http://169.254.169.254/latest/meta-data/iam/security-credentials",
|
||||
"regional_cred_verification_url": "https://sts.{region}.amazonaws.com?Action=GetCallerIdentity&Version=2011-06-15"
|
||||
},
|
||||
"aws_role_name": "arn:aws:iam::123456789012:role/MyWifRole",
|
||||
"aws_region_name": "us-east-1"
|
||||
}
|
||||
```
|
||||
|
||||
**Supported `aws_*` parameters:**
|
||||
|
||||
| Parameter | Required | Description |
|
||||
|---|---|---|
|
||||
| `aws_region_name` | Yes | AWS region for credential verification (e.g. `us-east-1`) |
|
||||
| `aws_role_name` | No | IAM role ARN for STS AssumeRole |
|
||||
| `aws_access_key_id` | No | Static AWS access key ID |
|
||||
| `aws_secret_access_key` | No | Static AWS secret access key |
|
||||
| `aws_session_token` | No | Temporary session token |
|
||||
| `aws_profile_name` | No | AWS CLI profile name |
|
||||
| `aws_session_name` | No | Session name for AssumeRole |
|
||||
| `aws_web_identity_token` | No | Web identity token for STS |
|
||||
| `aws_sts_endpoint` | No | Custom STS endpoint URL |
|
||||
| `aws_external_id` | No | External ID for cross-account AssumeRole |
|
||||
|
||||
`aws_region_name` is always required when using explicit AWS credentials. The other parameters follow the same authentication flows as [Bedrock AWS auth](/docs/providers/bedrock#authentication) -- you can use role assumption, static keys, profiles, or web identity tokens.
|
||||
|
||||
<Tabs>
|
||||
<TabItem value="sdk" label="SDK">
|
||||
|
||||
```python
|
||||
from litellm import completion
|
||||
|
||||
response = completion(
|
||||
model="vertex_ai/gemini-1.5-pro",
|
||||
messages=[{"role": "user", "content": "Hello!"}],
|
||||
vertex_credentials="/path/to/wif-credentials-with-aws.json", # WIF JSON with aws_* keys
|
||||
vertex_project="your-gcp-project-id",
|
||||
vertex_location="us-central1"
|
||||
)
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
<TabItem value="proxy" label="PROXY">
|
||||
|
||||
```yaml
|
||||
model_list:
|
||||
- model_name: gemini-model
|
||||
litellm_params:
|
||||
model: vertex_ai/gemini-1.5-pro
|
||||
vertex_project: your-gcp-project-id
|
||||
vertex_location: us-central1
|
||||
vertex_credentials: /path/to/wif-credentials-with-aws.json # WIF JSON with aws_* keys
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
When `aws_*` keys are present in the JSON, LiteLLM automatically uses explicit AWS authentication instead of the EC2 metadata service. When they are absent, the standard metadata-based flow is used unchanged.
|
||||
|
||||
### **Environment Variables**
|
||||
|
||||
You can set:
|
||||
|
|
@ -1687,6 +1763,20 @@ litellm.vertex_location = "us-central1 # Your Location
|
|||
| gemini-2.5-flash-lite-preview-09-2025 | `completion('gemini-2.5-flash-lite-preview-09-2025', messages)`, `completion('vertex_ai/gemini-2.5-flash-lite-preview-09-2025', messages)` |
|
||||
| gemini-3.1-flash-lite-preview | `completion('gemini-3.1-flash-lite-preview', messages)`, `completion('vertex_ai/gemini-3.1-flash-lite-preview', messages)` |
|
||||
|
||||
## PayGo / Priority Cost Tracking
|
||||
|
||||
LiteLLM automatically tracks spend for Vertex AI Gemini models using the correct pricing tier based on the response's `usageMetadata.trafficType`:
|
||||
|
||||
| Vertex AI `trafficType` | LiteLLM `service_tier` | Pricing applied |
|
||||
|-------------------------|-------------------------|-----------------|
|
||||
| `ON_DEMAND_PRIORITY` | `priority` | PayGo / priority pricing (`input_cost_per_token_priority`, `output_cost_per_token_priority`) |
|
||||
| `ON_DEMAND` | standard | Default on-demand pricing |
|
||||
| `FLEX` / `BATCH` | `flex` | Batch/flex pricing |
|
||||
|
||||
When you use [Vertex AI PayGo](https://cloud.google.com/vertex-ai/generative-ai/pricing) (on-demand priority) or batch workloads, LiteLLM reads `trafficType` from the response and applies the matching cost per token from the [model cost map](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json). No configuration is required — spend tracking works out of the box for both standard and PayGo requests.
|
||||
|
||||
See [Spend Tracking](../proxy/cost_tracking.md) for general cost tracking setup.
|
||||
|
||||
## Private Service Connect (PSC) Endpoints
|
||||
|
||||
LiteLLM supports Vertex AI models deployed to Private Service Connect (PSC) endpoints, allowing you to use custom `api_base` URLs for private deployments.
|
||||
|
|
|
|||
|
|
@ -8,6 +8,8 @@ Track spend for keys, users, and teams across 100+ LLMs.
|
|||
|
||||
LiteLLM automatically tracks spend for all known models. See our [model cost map](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json)
|
||||
|
||||
Provider-specific cost tracking (e.g., [Vertex AI PayGo / priority pricing](../providers/vertex.md#paygo--priority-cost-tracking), [Bedrock service tiers](../providers/bedrock.md#usage---service-tier), [Azure base model mapping](./custom_pricing.md#set-base_model-for-cost-tracking-eg-azure-deployments)) is applied automatically when the response includes tier metadata.
|
||||
|
||||
:::tip Keep Pricing Data Updated
|
||||
[Sync model pricing data from GitHub](./sync_models_github.md) to ensure accurate cost tracking.
|
||||
:::
|
||||
|
|
|
|||
|
|
@ -104,9 +104,18 @@ There are other keys you can use to specify costs for different scenarios and mo
|
|||
- `input_cost_per_video_per_second` - Cost per second of video input
|
||||
- `input_cost_per_video_per_second_above_128k_tokens` - Video cost for large contexts
|
||||
- `input_cost_per_character` - Character-based pricing for some providers
|
||||
- `input_cost_per_token_priority` / `output_cost_per_token_priority` - Priority/PayGo pricing (Vertex AI Gemini, Bedrock)
|
||||
- `input_cost_per_token_flex` / `output_cost_per_token_flex` - Batch/flex pricing
|
||||
|
||||
These keys evolve based on how new models handle multimodality. The latest version can be found at [https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json).
|
||||
|
||||
### Service Tier / PayGo Pricing (Vertex AI, Bedrock)
|
||||
|
||||
For providers that support multiple pricing tiers (e.g., Vertex AI PayGo, Bedrock service tiers), LiteLLM automatically applies the correct cost based on the response:
|
||||
|
||||
- **Vertex AI Gemini**: Uses `usageMetadata.trafficType` (`ON_DEMAND_PRIORITY` → priority, `FLEX`/`BATCH` → flex). See [Vertex AI - PayGo / Priority Cost Tracking](../providers/vertex.md#paygo--priority-cost-tracking).
|
||||
- **Bedrock**: Uses `serviceTier` from the response. See [Bedrock - Usage - Service Tier](../providers/bedrock.md#usage---service-tier).
|
||||
|
||||
## Zero-Cost Models (Bypass Budget Checks)
|
||||
|
||||
**Use Case**: You have on-premises or free models that should be accessible even when users exceed their budget limits.
|
||||
|
|
|
|||
|
|
@ -112,6 +112,8 @@ general_settings:
|
|||
forward_llm_provider_auth_headers: true # Enable BYOK
|
||||
```
|
||||
|
||||
For **Claude Code** with `/login` and your own Anthropic key, see [Claude Code BYOK](../tutorials/claude_code_byok.md). Use `ANTHROPIC_CUSTOM_HEADERS="x-litellm-api-key: sk-12345"` to pass your LiteLLM key while your Anthropic key (from `/login`) is forwarded as `x-api-key`.
|
||||
|
||||
Client request:
|
||||
```bash
|
||||
curl -X POST "http://localhost:4000/v1/messages" \
|
||||
|
|
|
|||
123
docs/my-website/docs/tutorials/claude_code_byok.md
Normal file
123
docs/my-website/docs/tutorials/claude_code_byok.md
Normal file
|
|
@ -0,0 +1,123 @@
|
|||
# Claude Code with Bring Your Own Key (BYOK)
|
||||
|
||||
Use Claude Code with your own Anthropic API key through the LiteLLM proxy. When you use Claude's `/login` with your Anthropic account, your API key is sent as `x-api-key`. With BYOK enabled, LiteLLM forwards your key to Anthropic instead of using proxy-configured keys — so you pay Anthropic directly while still benefiting from LiteLLM's routing, logging, and guardrails.
|
||||
|
||||
## How It Works
|
||||
|
||||
1. **Claude Code `/login`** — You sign in with your Anthropic account; Claude Code sends your Anthropic API key as `x-api-key`.
|
||||
2. **LiteLLM authentication** — You pass your LiteLLM proxy key via `ANTHROPIC_CUSTOM_HEADERS` so the proxy can authenticate and track your usage.
|
||||
3. **Key forwarding** — With `forward_llm_provider_auth_headers: true`, LiteLLM forwards your `x-api-key` to Anthropic, giving it precedence over any proxy-configured keys.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- [Claude Code](https://docs.anthropic.com/en/docs/claude-code/overview) installed
|
||||
- Anthropic API key (from [console.anthropic.com](https://console.anthropic.com))
|
||||
- LiteLLM proxy with a virtual key for authentication
|
||||
|
||||
## Step 1: Configure LiteLLM Proxy
|
||||
|
||||
Enable forwarding of LLM provider auth headers so your Anthropic key takes precedence:
|
||||
|
||||
```yaml title="config.yaml"
|
||||
model_list:
|
||||
- model_name: claude-sonnet-4-5
|
||||
litellm_params:
|
||||
model: anthropic/claude-sonnet-4-5
|
||||
# No api_key needed — client's key will be used
|
||||
|
||||
litellm_settings:
|
||||
forward_llm_provider_auth_headers: true # Required for BYOK
|
||||
```
|
||||
|
||||
:::info Why `forward_llm_provider_auth_headers`?
|
||||
|
||||
By default, LiteLLM strips `x-api-key` from client requests for security. Setting this to `true` allows client-provided provider keys (like your Anthropic key from `/login`) to be forwarded to Anthropic, overriding any proxy-configured keys.
|
||||
|
||||
:::
|
||||
|
||||
## Step 2: Create a LiteLLM Virtual Key
|
||||
|
||||
Create a virtual key in the LiteLLM UI or via API.
|
||||
```bash
|
||||
# Example: Create key via API
|
||||
curl -X POST "http://localhost:4000/key/generate" \
|
||||
-H "Authorization: Bearer sk-your-master-key" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"key_alias": "claude-code-byok", "models": ["claude-sonnet-4-5"]}'
|
||||
```
|
||||
|
||||
## Step 3: Configure Claude Code
|
||||
|
||||
Set environment variables so Claude Code uses LiteLLM and sends your LiteLLM key for proxy auth:
|
||||
|
||||
```bash
|
||||
# Point Claude Code to your LiteLLM proxy
|
||||
export ANTHROPIC_BASE_URL="http://localhost:4000"
|
||||
|
||||
# Model name from your config
|
||||
export ANTHROPIC_MODEL="claude-sonnet-4-5"
|
||||
|
||||
# LiteLLM proxy auth — this is added to every request
|
||||
# Use x-litellm-api-key so the proxy authenticates you; your Anthropic key goes via x-api-key from /login
|
||||
export ANTHROPIC_CUSTOM_HEADERS="x-litellm-api-key: sk-12345"
|
||||
```
|
||||
|
||||
Replace `sk-12345` with your actual LiteLLM virtual key.
|
||||
|
||||
:::tip Multiple headers
|
||||
|
||||
For multiple headers, use newline-separated values:
|
||||
|
||||
```bash
|
||||
export ANTHROPIC_CUSTOM_HEADERS="x-litellm-api-key: sk-12345
|
||||
x-litellm-user-id: my-user-id"
|
||||
```
|
||||
|
||||
:::
|
||||
|
||||
## Step 4: Sign In with Claude Code
|
||||
|
||||
1. Launch Claude Code:
|
||||
|
||||
```bash
|
||||
claude
|
||||
```
|
||||
|
||||
2. Use **`/login`** and sign in with your Anthropic account (or use your API key directly).
|
||||
|
||||
3. Claude Code will send:
|
||||
- `x-api-key`: Your Anthropic API key (from `/login`)
|
||||
- `x-litellm-api-key`: Your LiteLLM key (from `ANTHROPIC_CUSTOM_HEADERS`)
|
||||
|
||||
4. LiteLLM authenticates you via `x-litellm-api-key`, then forwards `x-api-key` to Anthropic. Your Anthropic key takes precedence over any proxy-configured key.
|
||||
|
||||
## Summary
|
||||
|
||||
| Header | Source | Purpose |
|
||||
|--------|--------|---------|
|
||||
| `x-api-key` | Claude Code `/login` (Anthropic key) | Sent to Anthropic for API calls |
|
||||
| `x-litellm-api-key` | `ANTHROPIC_CUSTOM_HEADERS` | Proxy authentication, tracking, rate limits |
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Requests fail with "invalid x-api-key"
|
||||
|
||||
- Ensure `forward_llm_provider_auth_headers: true` is set in `litellm_settings` (or `general_settings`).
|
||||
- Restart the LiteLLM proxy after changing the config.
|
||||
- Verify you completed `/login` in Claude Code so your Anthropic key is being sent.
|
||||
|
||||
### Proxy returns 401
|
||||
|
||||
- Check that `ANTHROPIC_CUSTOM_HEADERS` includes `x-litellm-api-key: <your-key>`.
|
||||
- Ensure the LiteLLM key is valid and has access to the model.
|
||||
|
||||
### Proxy key is used instead of my Anthropic key
|
||||
|
||||
- Confirm `forward_llm_provider_auth_headers: true` is in your config.
|
||||
- The setting can be in `litellm_settings` or `general_settings` depending on your config structure.
|
||||
- Enable debug logging: `LITELLM_LOG=DEBUG` to see which key is being forwarded.
|
||||
|
||||
## Related
|
||||
|
||||
- [Forward Client Headers](./../proxy/forward_client_headers.md) — Full BYOK and header forwarding docs
|
||||
- [Claude Code Max Subscription](./claude_code_max_subscription.md) — Using Claude Code with OAuth/Max subscription through LiteLLM
|
||||
BIN
docs/my-website/img/claude_code_byok_screenshot.png
Normal file
BIN
docs/my-website/img/claude_code_byok_screenshot.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 99 KiB |
|
|
@ -154,6 +154,7 @@ const sidebars = {
|
|||
items: [
|
||||
"tutorials/claude_responses_api",
|
||||
"tutorials/claude_code_max_subscription",
|
||||
"tutorials/claude_code_byok",
|
||||
"tutorials/claude_code_customer_tracking",
|
||||
"tutorials/claude_code_prompt_cache_routing",
|
||||
"tutorials/claude_code_websearch",
|
||||
|
|
@ -538,6 +539,7 @@ const sidebars = {
|
|||
items: [
|
||||
"a2a",
|
||||
"a2a_invoking_agents",
|
||||
"a2a_agent_headers",
|
||||
"a2a_cost_tracking",
|
||||
"a2a_agent_permissions",
|
||||
"a2a_iteration_budgets"
|
||||
|
|
|
|||
|
|
@ -78,8 +78,6 @@ class CheckBatchCost:
|
|||
"status": {"not_in": ["failed", "expired", "cancelled"]}
|
||||
}
|
||||
)
|
||||
completed_jobs = []
|
||||
|
||||
for job in jobs:
|
||||
# get the model from the job
|
||||
unified_object_id = job.unified_object_id
|
||||
|
|
@ -237,10 +235,16 @@ class CheckBatchCost:
|
|||
)
|
||||
|
||||
# mark the job as complete
|
||||
completed_jobs.append(job)
|
||||
|
||||
if len(completed_jobs) > 0:
|
||||
await self.prisma_client.db.litellm_managedobjecttable.update_many(
|
||||
where={"id": {"in": [job.id for job in completed_jobs]}},
|
||||
data={"batch_processed": True, "status": "complete"},
|
||||
)
|
||||
try:
|
||||
await self.prisma_client.db.litellm_managedobjecttable.update(
|
||||
where={"id": job.id},
|
||||
data={
|
||||
"batch_processed": True,
|
||||
"status": "complete",
|
||||
"file_object": response.model_dump_json(),
|
||||
},
|
||||
)
|
||||
except Exception as db_err:
|
||||
verbose_proxy_logger.error(
|
||||
f"CheckBatchCost: failed to mark job {job.id} complete in DB: {db_err}"
|
||||
)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,5 @@
|
|||
-- Add static_headers and extra_headers to LiteLLM_AgentsTable
|
||||
|
||||
ALTER TABLE "LiteLLM_AgentsTable"
|
||||
ADD COLUMN IF NOT EXISTS "static_headers" JSONB DEFAULT '{}',
|
||||
ADD COLUMN IF NOT EXISTS "extra_headers" TEXT[] DEFAULT ARRAY[]::TEXT[];
|
||||
|
|
@ -9,6 +9,7 @@ import datetime
|
|||
import uuid
|
||||
from typing import TYPE_CHECKING, Any, AsyncIterator, Coroutine, Dict, Optional, Union
|
||||
|
||||
import httpx
|
||||
import litellm
|
||||
from litellm._logging import verbose_logger, verbose_proxy_logger
|
||||
from litellm.a2a_protocol.streaming_iterator import A2AStreamingIterator
|
||||
|
|
@ -212,6 +213,7 @@ async def asend_message(
|
|||
api_base: Optional[str] = None,
|
||||
litellm_params: Optional[Dict[str, Any]] = None,
|
||||
agent_id: Optional[str] = None,
|
||||
agent_extra_headers: Optional[Dict[str, str]] = None,
|
||||
**kwargs: Any,
|
||||
) -> LiteLLMSendMessageResponse:
|
||||
"""
|
||||
|
|
@ -293,9 +295,12 @@ async def asend_message(
|
|||
"Either a2a_client or api_base is required for standard A2A flow"
|
||||
)
|
||||
trace_id = trace_id or str(uuid.uuid4())
|
||||
extra_headers = {"X-LiteLLM-Trace-Id": trace_id}
|
||||
extra_headers: Dict[str, str] = {"X-LiteLLM-Trace-Id": trace_id}
|
||||
if agent_id:
|
||||
extra_headers["X-LiteLLM-Agent-Id"] = agent_id
|
||||
# Overlay agent-level headers (agent headers take precedence over LiteLLM internal ones)
|
||||
if agent_extra_headers:
|
||||
extra_headers.update(agent_extra_headers)
|
||||
a2a_client = await create_a2a_client(
|
||||
base_url=api_base, extra_headers=extra_headers
|
||||
)
|
||||
|
|
@ -442,6 +447,7 @@ async def asend_message_streaming(
|
|||
agent_id: Optional[str] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
proxy_server_request: Optional[Dict[str, Any]] = None,
|
||||
agent_extra_headers: Optional[Dict[str, str]] = None,
|
||||
) -> AsyncIterator[Any]:
|
||||
"""
|
||||
Async: Send a streaming message to an A2A agent.
|
||||
|
|
@ -523,7 +529,17 @@ async def asend_message_streaming(
|
|||
raise ValueError(
|
||||
"Either a2a_client or api_base is required for standard A2A flow"
|
||||
)
|
||||
a2a_client = await create_a2a_client(base_url=api_base)
|
||||
# Mirror the non-streaming path: always include trace and agent-id headers
|
||||
streaming_extra_headers: Dict[str, str] = {
|
||||
"X-LiteLLM-Trace-Id": str(request.id),
|
||||
}
|
||||
if agent_id:
|
||||
streaming_extra_headers["X-LiteLLM-Agent-Id"] = agent_id
|
||||
if agent_extra_headers:
|
||||
streaming_extra_headers.update(agent_extra_headers)
|
||||
a2a_client = await create_a2a_client(
|
||||
base_url=api_base, extra_headers=streaming_extra_headers
|
||||
)
|
||||
|
||||
# Type assertion: a2a_client is guaranteed to be non-None here
|
||||
assert a2a_client is not None
|
||||
|
|
@ -637,17 +653,17 @@ async def create_a2a_client(
|
|||
|
||||
verbose_logger.info(f"Creating A2A client for {base_url}")
|
||||
|
||||
# Use LiteLLM's cached httpx client
|
||||
http_handler = get_async_httpx_client(
|
||||
llm_provider=httpxSpecialProvider.A2A,
|
||||
params={"timeout": timeout},
|
||||
# Always create a fresh httpx client per A2A call so that per-agent auth
|
||||
# headers (extra_headers) are never shared across agents or requests.
|
||||
# Mutating a cached shared client would cause headers from one agent to
|
||||
# bleed into requests made to a different agent.
|
||||
httpx_client = httpx.AsyncClient(
|
||||
timeout=httpx.Timeout(timeout),
|
||||
headers=extra_headers or {},
|
||||
)
|
||||
httpx_client = http_handler.client
|
||||
|
||||
if extra_headers:
|
||||
httpx_client.headers.update(extra_headers)
|
||||
verbose_proxy_logger.debug(
|
||||
f"A2A client created with extra_headers={extra_headers}"
|
||||
f"A2A client created with extra_headers={list(extra_headers.keys())}"
|
||||
)
|
||||
|
||||
# Resolve agent card
|
||||
|
|
|
|||
|
|
@ -166,6 +166,14 @@ class Cache:
|
|||
None. Cache is set as a litellm param
|
||||
"""
|
||||
if type == LiteLLMCacheType.REDIS:
|
||||
# Check REDIS_CLUSTER_NODES env var if no explicit startup nodes
|
||||
if not redis_startup_nodes:
|
||||
_env_cluster_nodes = litellm.get_secret("REDIS_CLUSTER_NODES")
|
||||
if _env_cluster_nodes is not None and isinstance(
|
||||
_env_cluster_nodes, str
|
||||
):
|
||||
redis_startup_nodes = json.loads(_env_cluster_nodes)
|
||||
|
||||
if redis_startup_nodes:
|
||||
# Only pass GCP parameters if they are provided
|
||||
cluster_kwargs = {
|
||||
|
|
|
|||
|
|
@ -1242,6 +1242,11 @@ X_LITELLM_DISABLE_CALLBACKS = "x-litellm-disable-callbacks"
|
|||
LITELLM_METADATA_FIELD = "litellm_metadata"
|
||||
OLD_LITELLM_METADATA_FIELD = "metadata"
|
||||
LITELLM_TRUNCATED_PAYLOAD_FIELD = "litellm_truncated"
|
||||
LITELLM_TRUNCATION_DB_SAFEGUARD_NOTE = (
|
||||
"Truncation is a DB storage safeguard. "
|
||||
"Full, untruncated data is logged to logging callbacks (OTEL, Datadog, etc.). "
|
||||
"To increase the truncation limit, set `MAX_STRING_LENGTH_PROMPT_IN_DB` in your env."
|
||||
)
|
||||
|
||||
########################### LiteLLM Proxy Specific Constants ###########################
|
||||
########################################################################################
|
||||
|
|
|
|||
|
|
@ -272,6 +272,8 @@ def cost_per_token( # noqa: PLR0915
|
|||
### SERVICE TIER ###
|
||||
service_tier: Optional[str] = None, # for OpenAI service tier pricing
|
||||
response: Optional[Any] = None,
|
||||
### REQUEST MODEL ###
|
||||
request_model: Optional[str] = None, # original request model for router detection
|
||||
) -> Tuple[float, float]: # type: ignore
|
||||
"""
|
||||
Calculates the cost per token for a given model, prompt tokens, and completion tokens.
|
||||
|
|
@ -520,7 +522,7 @@ def cost_per_token( # noqa: PLR0915
|
|||
return dashscope_cost_per_token(model=model, usage=usage_block)
|
||||
elif custom_llm_provider == "azure_ai":
|
||||
return azure_ai_cost_per_token(
|
||||
model=model, usage=usage_block, response_time_ms=response_time_ms
|
||||
model=model, usage=usage_block, response_time_ms=response_time_ms, request_model=request_model
|
||||
)
|
||||
else:
|
||||
model_info = _cached_get_model_info_helper(
|
||||
|
|
@ -1457,6 +1459,11 @@ def completion_cost( # noqa: PLR0915
|
|||
text=completion_string
|
||||
)
|
||||
|
||||
# Get the original request model for router detection
|
||||
request_model_for_cost = None
|
||||
if litellm_logging_obj is not None:
|
||||
request_model_for_cost = litellm_logging_obj.model
|
||||
|
||||
(
|
||||
prompt_tokens_cost_usd_dollar,
|
||||
completion_tokens_cost_usd_dollar,
|
||||
|
|
@ -1479,6 +1486,7 @@ def completion_cost( # noqa: PLR0915
|
|||
rerank_billed_units=rerank_billed_units,
|
||||
service_tier=service_tier,
|
||||
response=completion_response,
|
||||
request_model=request_model_for_cost,
|
||||
)
|
||||
|
||||
# Get additional costs from provider (e.g., routing fees, infrastructure costs)
|
||||
|
|
|
|||
|
|
@ -46,7 +46,7 @@ class AzureOpenAIGPT5Config(AzureOpenAIConfig, OpenAIGPT5Config):
|
|||
# Only gpt-5.2+ has been verified to support logprobs on Azure.
|
||||
# The base OpenAI class includes logprobs for gpt-5.1+, but Azure
|
||||
# hasn't verified support for gpt-5.1, so remove them unless gpt-5.2/5.4+.
|
||||
if self.is_model_gpt_5_1_model(model) and not self.is_model_gpt_5_2_model(model):
|
||||
if self._supports_reasoning_effort_level(model, "none") and not self.is_model_gpt_5_2_model(model):
|
||||
params = [p for p in params if p not in ["logprobs", "top_logprobs"]]
|
||||
elif self.is_model_gpt_5_2_model(model):
|
||||
azure_supported_params = ["logprobs", "top_logprobs"]
|
||||
|
|
@ -69,9 +69,9 @@ class AzureOpenAIGPT5Config(AzureOpenAIConfig, OpenAIGPT5Config):
|
|||
|
||||
# gpt-5.1/5.2/5.4 support reasoning_effort='none', but other gpt-5 models don't
|
||||
# See: https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/reasoning
|
||||
is_gpt_5_1 = self.is_model_gpt_5_1_model(model)
|
||||
supports_none = self._supports_reasoning_effort_level(model, "none")
|
||||
|
||||
if reasoning_effort_value == "none" and not is_gpt_5_1:
|
||||
if reasoning_effort_value == "none" and not supports_none:
|
||||
if litellm.drop_params is True or (
|
||||
drop_params is not None and drop_params is True
|
||||
):
|
||||
|
|
@ -101,8 +101,8 @@ class AzureOpenAIGPT5Config(AzureOpenAIConfig, OpenAIGPT5Config):
|
|||
drop_params=drop_params,
|
||||
)
|
||||
|
||||
# Only drop reasoning_effort='none' for non-gpt-5.1/5.2/5.4 models
|
||||
if result.get("reasoning_effort") == "none" and not is_gpt_5_1:
|
||||
# Only drop reasoning_effort='none' for models that don't support it
|
||||
if result.get("reasoning_effort") == "none" and not supports_none:
|
||||
result.pop("reasoning_effort")
|
||||
|
||||
return result
|
||||
|
|
|
|||
|
|
@ -61,7 +61,10 @@ def calculate_azure_model_router_flat_cost(model: str, prompt_tokens: int) -> fl
|
|||
|
||||
|
||||
def cost_per_token(
|
||||
model: str, usage: Usage, response_time_ms: Optional[float] = 0.0
|
||||
model: str,
|
||||
usage: Usage,
|
||||
response_time_ms: Optional[float] = 0.0,
|
||||
request_model: Optional[str] = None,
|
||||
) -> Tuple[float, float]:
|
||||
"""
|
||||
Calculate the cost per token for Azure AI models.
|
||||
|
|
@ -71,9 +74,10 @@ def cost_per_token(
|
|||
- Plus the cost of the actual model used (handled by generic_cost_per_token)
|
||||
|
||||
Args:
|
||||
model: str, the model name without provider prefix
|
||||
model: str, the model name without provider prefix (from response)
|
||||
usage: LiteLLM Usage block
|
||||
response_time_ms: Optional response time in milliseconds
|
||||
request_model: Optional[str], the original request model name (to detect router usage)
|
||||
|
||||
Returns:
|
||||
Tuple[float, float] - prompt_cost_in_usd, completion_cost_in_usd
|
||||
|
|
@ -84,7 +88,13 @@ def cost_per_token(
|
|||
"""
|
||||
prompt_cost = 0.0
|
||||
completion_cost = 0.0
|
||||
|
||||
|
||||
# Determine if this was a model router request
|
||||
# Check both the response model and the request model
|
||||
is_router_request = _is_azure_model_router(model) or (
|
||||
request_model is not None and _is_azure_model_router(request_model)
|
||||
)
|
||||
|
||||
# Calculate base cost using generic cost calculator
|
||||
# This may raise an exception if the model is not in the cost map
|
||||
try:
|
||||
|
|
@ -103,19 +113,21 @@ def cost_per_token(
|
|||
verbose_logger.debug(
|
||||
f"Azure AI Model Router: model '{model}' not in cost map, calculating routing flat cost only. Error: {e}"
|
||||
)
|
||||
|
||||
|
||||
# Add flat cost for Azure Model Router
|
||||
# The flat cost is defined in model_prices_and_context_window.json for azure_ai/model_router
|
||||
if _is_azure_model_router(model):
|
||||
router_flat_cost = calculate_azure_model_router_flat_cost(model, usage.prompt_tokens)
|
||||
|
||||
if is_router_request:
|
||||
# Use the request model for flat cost calculation if available, otherwise use response model
|
||||
router_model_for_calc = request_model if request_model else model
|
||||
router_flat_cost = calculate_azure_model_router_flat_cost(router_model_for_calc, usage.prompt_tokens)
|
||||
|
||||
if router_flat_cost > 0:
|
||||
verbose_logger.debug(
|
||||
f"Azure AI Model Router flat cost: ${router_flat_cost:.6f} "
|
||||
f"({usage.prompt_tokens} tokens × ${router_flat_cost / usage.prompt_tokens:.9f}/token)"
|
||||
)
|
||||
|
||||
|
||||
# Add flat cost to prompt cost
|
||||
prompt_cost += router_flat_cost
|
||||
|
||||
|
||||
return prompt_cost, completion_cost
|
||||
|
|
|
|||
|
|
@ -1,12 +1,30 @@
|
|||
"""Support for OpenAI gpt-5 model family."""
|
||||
|
||||
from typing import Optional
|
||||
from typing import Optional, Union
|
||||
|
||||
import litellm
|
||||
from litellm.utils import _supports_factory
|
||||
|
||||
from .gpt_transformation import OpenAIGPTConfig
|
||||
|
||||
|
||||
def _normalize_reasoning_effort_for_chat_completion(
|
||||
value: Union[str, dict, None],
|
||||
) -> Optional[str]:
|
||||
"""Convert reasoning_effort to the string format expected by OpenAI chat completion API.
|
||||
|
||||
The chat completion API expects a simple string: 'none', 'low', 'medium', 'high', or 'xhigh'.
|
||||
Config/deployments may pass the Responses API format: {'effort': 'high', 'summary': 'detailed'}.
|
||||
"""
|
||||
if value is None:
|
||||
return None
|
||||
if isinstance(value, str):
|
||||
return value
|
||||
if isinstance(value, dict) and "effort" in value:
|
||||
return value["effort"]
|
||||
return None
|
||||
|
||||
|
||||
class OpenAIGPT5Config(OpenAIGPTConfig):
|
||||
"""Configuration for gpt-5 models including GPT-5-Codex variants.
|
||||
|
||||
|
|
@ -40,47 +58,32 @@ class OpenAIGPT5Config(OpenAIGPTConfig):
|
|||
"""Check if the model is specifically a GPT-5 Codex variant."""
|
||||
return "gpt-5-codex" in model
|
||||
|
||||
@classmethod
|
||||
def is_model_gpt_5_1_codex_max_model(cls, model: str) -> bool:
|
||||
"""Check if the model is the gpt-5.1-codex-max variant."""
|
||||
model_name = model.split("/")[-1] # handle provider prefixes
|
||||
return model_name == "gpt-5.1-codex-max"
|
||||
|
||||
@classmethod
|
||||
def is_model_gpt_5_1_model(cls, model: str) -> bool:
|
||||
"""Check if the model is a gpt-5.1, gpt-5.2, or gpt-5.4 chat variant.
|
||||
|
||||
gpt-5.1/5.2/5.4 support temperature when reasoning_effort="none",
|
||||
unlike base gpt-5 which only supports temperature=1. Excludes
|
||||
pro variants which keep stricter knobs and chat-only variants
|
||||
which only support temperature=1.
|
||||
"""
|
||||
model_name = model.split("/")[-1]
|
||||
is_gpt_5_1 = model_name.startswith("gpt-5.1")
|
||||
is_gpt_5_2 = (
|
||||
model_name.startswith("gpt-5.2")
|
||||
and "pro" not in model_name
|
||||
and not model_name.startswith("gpt-5.2-chat")
|
||||
)
|
||||
is_gpt_5_4 = (
|
||||
model_name.startswith("gpt-5.4")
|
||||
and "pro" not in model_name
|
||||
and not model_name.startswith("gpt-5.4-chat")
|
||||
)
|
||||
return is_gpt_5_1 or is_gpt_5_2 or is_gpt_5_4
|
||||
|
||||
@classmethod
|
||||
def is_model_gpt_5_2_pro_model(cls, model: str) -> bool:
|
||||
"""Check if the model is the gpt-5.2-pro snapshot/alias."""
|
||||
model_name = model.split("/")[-1]
|
||||
return model_name.startswith("gpt-5.2-pro")
|
||||
|
||||
@classmethod
|
||||
def is_model_gpt_5_2_model(cls, model: str) -> bool:
|
||||
"""Check if the model is a gpt-5.2 variant (including pro)."""
|
||||
model_name = model.split("/")[-1]
|
||||
return model_name.startswith("gpt-5.2") or model_name.startswith("gpt-5.4")
|
||||
|
||||
@classmethod
|
||||
def is_model_gpt_5_4_model(cls, model: str) -> bool:
|
||||
"""Check if the model is a gpt-5.4 variant (including pro)."""
|
||||
model_name = model.split("/")[-1]
|
||||
return model_name.startswith("gpt-5.4")
|
||||
|
||||
@classmethod
|
||||
def _supports_reasoning_effort_level(cls, model: str, level: str) -> bool:
|
||||
"""Check if the model supports a specific reasoning_effort level.
|
||||
|
||||
Looks up ``supports_{level}_reasoning_effort`` in the model map via
|
||||
the shared ``_supports_factory`` helper.
|
||||
Returns False for unknown models (safe fallback).
|
||||
"""
|
||||
return _supports_factory(
|
||||
model=model,
|
||||
custom_llm_provider=None,
|
||||
key=f"supports_{level}_reasoning_effort",
|
||||
)
|
||||
|
||||
def get_supported_openai_params(self, model: str) -> list:
|
||||
if self.is_model_gpt_5_search_model(model):
|
||||
return [
|
||||
|
|
@ -118,8 +121,8 @@ class OpenAIGPT5Config(OpenAIGPTConfig):
|
|||
"web_search_options",
|
||||
]
|
||||
|
||||
# gpt-5.1/5.2/5.4 support logprobs, top_p, top_logprobs when reasoning_effort="none"
|
||||
if not self.is_model_gpt_5_1_model(model):
|
||||
# gpt-5.1/5.2 support logprobs, top_p, top_logprobs when reasoning_effort="none"
|
||||
if not self._supports_reasoning_effort_level(model, "none"):
|
||||
non_supported_params.extend(["logprobs", "top_p", "top_logprobs"])
|
||||
|
||||
return [
|
||||
|
|
@ -147,15 +150,22 @@ class OpenAIGPT5Config(OpenAIGPTConfig):
|
|||
drop_params=drop_params,
|
||||
)
|
||||
|
||||
reasoning_effort = (
|
||||
# Normalize reasoning_effort: chat completion API expects a string, not a dict
|
||||
# (e.g. {'effort': 'high', 'summary': 'detailed'} -> 'high')
|
||||
raw_reasoning_effort = (
|
||||
non_default_params.get("reasoning_effort")
|
||||
or optional_params.get("reasoning_effort")
|
||||
)
|
||||
normalized = _normalize_reasoning_effort_for_chat_completion(raw_reasoning_effort)
|
||||
if raw_reasoning_effort is not None and normalized is not None:
|
||||
if "reasoning_effort" in non_default_params:
|
||||
non_default_params["reasoning_effort"] = normalized
|
||||
if "reasoning_effort" in optional_params:
|
||||
optional_params["reasoning_effort"] = normalized
|
||||
|
||||
reasoning_effort = normalized or raw_reasoning_effort
|
||||
if reasoning_effort is not None and reasoning_effort == "xhigh":
|
||||
if not (
|
||||
self.is_model_gpt_5_1_codex_max_model(model)
|
||||
or self.is_model_gpt_5_2_model(model)
|
||||
):
|
||||
if not self._supports_reasoning_effort_level(model, "xhigh"):
|
||||
if litellm.drop_params or drop_params:
|
||||
non_default_params.pop("reasoning_effort", None)
|
||||
else:
|
||||
|
|
@ -175,8 +185,20 @@ class OpenAIGPT5Config(OpenAIGPTConfig):
|
|||
"max_tokens"
|
||||
)
|
||||
|
||||
# gpt-5.1/5.2/5.4 support logprobs, top_p, top_logprobs only when reasoning_effort="none"
|
||||
if self.is_model_gpt_5_1_model(model):
|
||||
# gpt-5.4: function calls not supported when reasoning_effort != "none"
|
||||
# Drop reasoning_effort when tools are present (small minority of volume)
|
||||
if self.is_model_gpt_5_4_model(model):
|
||||
has_tools = bool(
|
||||
non_default_params.get("tools") or optional_params.get("tools")
|
||||
)
|
||||
if has_tools and reasoning_effort not in (None, "none"):
|
||||
non_default_params.pop("reasoning_effort", None)
|
||||
optional_params.pop("reasoning_effort", None)
|
||||
reasoning_effort = None
|
||||
|
||||
# gpt-5.1/5.2 support logprobs, top_p, top_logprobs only when reasoning_effort="none"
|
||||
supports_none = self._supports_reasoning_effort_level(model, "none")
|
||||
if supports_none:
|
||||
sampling_params = ["logprobs", "top_logprobs", "top_p"]
|
||||
has_sampling = any(p in non_default_params for p in sampling_params)
|
||||
if has_sampling and reasoning_effort not in (None, "none"):
|
||||
|
|
@ -196,10 +218,8 @@ class OpenAIGPT5Config(OpenAIGPTConfig):
|
|||
if "temperature" in non_default_params:
|
||||
temperature_value: Optional[float] = non_default_params.pop("temperature")
|
||||
if temperature_value is not None:
|
||||
is_gpt_5_1 = self.is_model_gpt_5_1_model(model)
|
||||
|
||||
# gpt-5.1 supports any temperature when reasoning_effort="none" (or not specified, as it defaults to "none")
|
||||
if is_gpt_5_1 and (reasoning_effort == "none" or reasoning_effort is None):
|
||||
# models supporting reasoning_effort="none" also support flexible temperature
|
||||
if supports_none and (reasoning_effort == "none" or reasoning_effort is None):
|
||||
optional_params["temperature"] = temperature_value
|
||||
elif temperature_value == 1:
|
||||
optional_params["temperature"] = temperature_value
|
||||
|
|
|
|||
|
|
@ -131,7 +131,10 @@ class OpenAIOSeriesConfig(OpenAIGPTConfig):
|
|||
|
||||
def is_model_o_series_model(self, model: str) -> bool:
|
||||
model = model.split("/")[-1] # could be "openai/o3" or "o3"
|
||||
return model.startswith(("o1", "o3", "o4")) and model in litellm.open_ai_chat_completion_models
|
||||
return (
|
||||
len(model) > 1 and model[0] == "o" and model[1].isdigit()
|
||||
and model in litellm.open_ai_chat_completion_models
|
||||
)
|
||||
|
||||
@overload
|
||||
def _transform_messages(
|
||||
|
|
|
|||
52
litellm/llms/vertex_ai/aws_credentials_supplier.py
Normal file
52
litellm/llms/vertex_ai/aws_credentials_supplier.py
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
"""
|
||||
Custom AWS Security Credentials Supplier for Vertex AI WIF.
|
||||
|
||||
Wraps boto3/botocore credentials so that google-auth can use them
|
||||
for the AWS-to-GCP Workload Identity Federation token exchange
|
||||
without hitting the EC2 instance metadata service.
|
||||
|
||||
Requires google-auth >= 2.29.0.
|
||||
"""
|
||||
|
||||
from typing import Callable
|
||||
|
||||
from google.auth import aws
|
||||
|
||||
|
||||
class AwsCredentialsSupplier(aws.AwsSecurityCredentialsSupplier):
|
||||
"""
|
||||
Supplies AWS credentials to google-auth's aws.Credentials for WIF
|
||||
token exchange.
|
||||
|
||||
This bypasses the default metadata-based credential retrieval,
|
||||
allowing WIF to work in environments where EC2 metadata is blocked.
|
||||
|
||||
Accepts a credentials_provider callable that is invoked on every
|
||||
get_aws_security_credentials() call, so that refreshed/rotated
|
||||
credentials are picked up automatically (important for temporary
|
||||
STS tokens).
|
||||
"""
|
||||
|
||||
def __init__(self, credentials_provider: Callable, aws_region: str):
|
||||
"""
|
||||
Args:
|
||||
credentials_provider: A zero-arg callable that returns a
|
||||
botocore.credentials.Credentials object (with access_key,
|
||||
secret_key, and token attributes).
|
||||
aws_region: The AWS region string (e.g. "us-east-1").
|
||||
"""
|
||||
self._credentials_provider = credentials_provider
|
||||
self._region = aws_region
|
||||
|
||||
def get_aws_security_credentials(self, context, request):
|
||||
"""Return current AWS credentials for the GCP token exchange."""
|
||||
current = self._credentials_provider()
|
||||
return aws.AwsSecurityCredentials(
|
||||
access_key_id=current.access_key,
|
||||
secret_access_key=current.secret_key,
|
||||
session_token=current.token,
|
||||
)
|
||||
|
||||
def get_aws_region(self, context, request):
|
||||
"""Return the AWS region for credential verification."""
|
||||
return self._region
|
||||
|
|
@ -800,9 +800,10 @@ class VertexGeminiConfig(VertexAIBaseConfig, BaseConfig):
|
|||
GeminiThinkingConfig with thinkingLevel and includeThoughts
|
||||
"""
|
||||
# Check if this is gemini-3-flash which supports MINIMAL thinking level
|
||||
# Covers gemini-3-flash, gemini-3-flash-preview, gemini-3.1-flash, gemini-3.1-flash-lite-preview, etc.
|
||||
is_gemini3flash = model and (
|
||||
"gemini-3-flash-preview" in model.lower()
|
||||
or "gemini-3-flash" in model.lower()
|
||||
"gemini-3-flash" in model.lower()
|
||||
or "gemini-3.1-flash" in model.lower()
|
||||
)
|
||||
is_gemini31pro = model and (
|
||||
"gemini-3.1-pro-preview" in model.lower()
|
||||
|
|
|
|||
125
litellm/llms/vertex_ai/vertex_ai_aws_wif.py
Normal file
125
litellm/llms/vertex_ai/vertex_ai_aws_wif.py
Normal file
|
|
@ -0,0 +1,125 @@
|
|||
"""
|
||||
AWS Workload Identity Federation (WIF) auth for Vertex AI.
|
||||
|
||||
Handles explicit AWS credentials for GCP WIF token exchange,
|
||||
bypassing the EC2 instance metadata service.
|
||||
|
||||
When aws_* keys are present in the WIF credential JSON, this module
|
||||
uses BaseAWSLLM to obtain AWS credentials and wraps them in a custom
|
||||
AwsSecurityCredentialsSupplier for google-auth.
|
||||
"""
|
||||
|
||||
from typing import Dict
|
||||
|
||||
GOOGLE_IMPORT_ERROR_MESSAGE = (
|
||||
"Google Cloud SDK not found. Install it with: pip install 'litellm[google]' "
|
||||
"or pip install google-cloud-aiplatform"
|
||||
)
|
||||
|
||||
# AWS params recognized in WIF credential JSON for explicit auth.
|
||||
# These match the kwargs accepted by BaseAWSLLM.get_credentials().
|
||||
_AWS_CREDENTIAL_KEYS = frozenset({
|
||||
"aws_access_key_id",
|
||||
"aws_secret_access_key",
|
||||
"aws_session_token",
|
||||
"aws_region_name",
|
||||
"aws_session_name",
|
||||
"aws_profile_name",
|
||||
"aws_role_name",
|
||||
"aws_web_identity_token",
|
||||
"aws_sts_endpoint",
|
||||
"aws_external_id",
|
||||
})
|
||||
|
||||
|
||||
class VertexAIAwsWifAuth:
|
||||
"""
|
||||
Handles AWS-to-GCP Workload Identity Federation credential creation
|
||||
for Vertex AI, using explicit AWS credentials rather than EC2 metadata.
|
||||
"""
|
||||
|
||||
@staticmethod
|
||||
def extract_aws_params(json_obj: dict) -> Dict[str, str]:
|
||||
"""
|
||||
Extract LiteLLM-specific aws_* keys from a WIF credential JSON dict.
|
||||
|
||||
Returns a dict of {param_name: value} for any recognized aws_* keys
|
||||
found in the JSON. Returns empty dict if none are present.
|
||||
"""
|
||||
return {
|
||||
key: json_obj[key]
|
||||
for key in _AWS_CREDENTIAL_KEYS
|
||||
if key in json_obj
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def credentials_from_explicit_aws(json_obj, aws_params, scopes):
|
||||
"""
|
||||
Create GCP credentials using explicit AWS credentials for WIF.
|
||||
|
||||
Uses BaseAWSLLM to obtain AWS credentials (via STS AssumeRole, profile,
|
||||
static keys, etc.), then wraps them in a custom AwsSecurityCredentialsSupplier
|
||||
so that google-auth bypasses the EC2 metadata service.
|
||||
|
||||
Args:
|
||||
json_obj: The WIF credential JSON dict (contains audience, token_url, etc.)
|
||||
aws_params: Dict of aws_* params extracted from json_obj
|
||||
scopes: OAuth scopes for the GCP credentials
|
||||
"""
|
||||
try:
|
||||
from google.auth import aws
|
||||
except ImportError:
|
||||
raise ImportError(GOOGLE_IMPORT_ERROR_MESSAGE)
|
||||
|
||||
from litellm.llms.bedrock.base_aws_llm import BaseAWSLLM
|
||||
from litellm.llms.vertex_ai.aws_credentials_supplier import (
|
||||
AwsCredentialsSupplier,
|
||||
)
|
||||
|
||||
# Validate region first — required for the GCP token exchange.
|
||||
# Check before get_credentials() to avoid unnecessary AWS API calls
|
||||
# (e.g. STS AssumeRole) on misconfiguration.
|
||||
aws_region = aws_params.get("aws_region_name")
|
||||
if not aws_region:
|
||||
raise ValueError(
|
||||
"aws_region_name is required in the WIF credential JSON "
|
||||
"when using explicit AWS authentication. Add "
|
||||
'"aws_region_name": "<your-region>" to your credential file.'
|
||||
)
|
||||
|
||||
# Build a credentials provider that re-resolves AWS creds on each call.
|
||||
# This ensures rotated/refreshed STS tokens are picked up during
|
||||
# long-running processes when google-auth refreshes the GCP token.
|
||||
base_aws = BaseAWSLLM()
|
||||
aws_params_copy = dict(aws_params) # avoid mutating caller's dict
|
||||
|
||||
def _get_aws_credentials():
|
||||
return base_aws.get_credentials(**aws_params_copy)
|
||||
|
||||
# Create the custom supplier with a lazy credentials provider
|
||||
supplier = AwsCredentialsSupplier(
|
||||
credentials_provider=_get_aws_credentials,
|
||||
aws_region=aws_region,
|
||||
)
|
||||
|
||||
# Build kwargs for aws.Credentials — forward optional fields from JSON
|
||||
creds_kwargs = dict(
|
||||
audience=json_obj.get("audience"),
|
||||
subject_token_type=json_obj.get("subject_token_type"),
|
||||
token_url=json_obj.get("token_url"),
|
||||
credential_source=None, # Not using metadata endpoints
|
||||
aws_security_credentials_supplier=supplier,
|
||||
service_account_impersonation_url=json_obj.get(
|
||||
"service_account_impersonation_url"
|
||||
),
|
||||
)
|
||||
# Forward universe_domain if present (defaults to googleapis.com)
|
||||
if "universe_domain" in json_obj:
|
||||
creds_kwargs["universe_domain"] = json_obj["universe_domain"]
|
||||
|
||||
creds = aws.Credentials(**creds_kwargs)
|
||||
|
||||
if scopes and hasattr(creds, "requires_scopes") and creds.requires_scopes:
|
||||
creds = creds.with_scopes(scopes)
|
||||
|
||||
return creds
|
||||
|
|
@ -96,10 +96,23 @@ class VertexBase:
|
|||
else ""
|
||||
)
|
||||
if isinstance(environment_id, str) and "aws" in environment_id:
|
||||
creds = self._credentials_from_identity_pool_with_aws(
|
||||
json_obj,
|
||||
scopes=["https://www.googleapis.com/auth/cloud-platform"],
|
||||
# Check if explicit AWS params are in the JSON (bypasses metadata)
|
||||
from litellm.llms.vertex_ai.vertex_ai_aws_wif import (
|
||||
VertexAIAwsWifAuth,
|
||||
)
|
||||
|
||||
aws_params = VertexAIAwsWifAuth.extract_aws_params(json_obj)
|
||||
if aws_params:
|
||||
creds = VertexAIAwsWifAuth.credentials_from_explicit_aws(
|
||||
json_obj,
|
||||
aws_params=aws_params,
|
||||
scopes=["https://www.googleapis.com/auth/cloud-platform"],
|
||||
)
|
||||
else:
|
||||
creds = self._credentials_from_identity_pool_with_aws(
|
||||
json_obj,
|
||||
scopes=["https://www.googleapis.com/auth/cloud-platform"],
|
||||
)
|
||||
else:
|
||||
creds = self._credentials_from_identity_pool(
|
||||
json_obj,
|
||||
|
|
|
|||
|
|
@ -1239,7 +1239,7 @@
|
|||
"supports_vision": true,
|
||||
"tool_use_system_prompt_tokens": 346
|
||||
},
|
||||
"apac.anthropic.claude-sonnet-4-6": {
|
||||
"au.anthropic.claude-sonnet-4-6": {
|
||||
"cache_creation_input_token_cost": 4.125e-06,
|
||||
"cache_creation_input_token_cost_above_200k_tokens": 8.25e-06,
|
||||
"cache_read_input_token_cost": 3.3e-07,
|
||||
|
|
@ -18437,6 +18437,93 @@
|
|||
"max_tokens": 8191,
|
||||
"mode": "embedding"
|
||||
},
|
||||
"chatgpt/gpt-5.4": {
|
||||
"litellm_provider": "chatgpt",
|
||||
"max_input_tokens": 1050000,
|
||||
"max_output_tokens": 128000,
|
||||
"max_tokens": 128000,
|
||||
"mode": "responses",
|
||||
"supported_endpoints": [
|
||||
"/v1/chat/completions",
|
||||
"/v1/responses"
|
||||
],
|
||||
"supports_function_calling": true,
|
||||
"supports_parallel_function_calling": true,
|
||||
"supports_response_schema": true,
|
||||
"supports_vision": true
|
||||
},
|
||||
"chatgpt/gpt-5.4-pro": {
|
||||
"litellm_provider": "chatgpt",
|
||||
"max_input_tokens": 1050000,
|
||||
"max_output_tokens": 128000,
|
||||
"max_tokens": 128000,
|
||||
"mode": "responses",
|
||||
"supported_endpoints": [
|
||||
"/v1/responses"
|
||||
],
|
||||
"supports_function_calling": true,
|
||||
"supports_parallel_function_calling": true,
|
||||
"supports_response_schema": true,
|
||||
"supports_vision": true
|
||||
},
|
||||
"chatgpt/gpt-5.3-codex": {
|
||||
"litellm_provider": "chatgpt",
|
||||
"max_input_tokens": 128000,
|
||||
"max_output_tokens": 128000,
|
||||
"max_tokens": 128000,
|
||||
"mode": "responses",
|
||||
"supported_endpoints": [
|
||||
"/v1/responses"
|
||||
],
|
||||
"supports_function_calling": true,
|
||||
"supports_parallel_function_calling": true,
|
||||
"supports_response_schema": true,
|
||||
"supports_vision": true
|
||||
},
|
||||
"chatgpt/gpt-5.3-codex-spark": {
|
||||
"litellm_provider": "chatgpt",
|
||||
"max_input_tokens": 128000,
|
||||
"max_output_tokens": 128000,
|
||||
"max_tokens": 128000,
|
||||
"mode": "responses",
|
||||
"supported_endpoints": [
|
||||
"/v1/responses"
|
||||
],
|
||||
"supports_function_calling": true,
|
||||
"supports_parallel_function_calling": true,
|
||||
"supports_response_schema": true,
|
||||
"supports_vision": true
|
||||
},
|
||||
"chatgpt/gpt-5.3-instant": {
|
||||
"litellm_provider": "chatgpt",
|
||||
"max_input_tokens": 128000,
|
||||
"max_output_tokens": 64000,
|
||||
"max_tokens": 64000,
|
||||
"mode": "responses",
|
||||
"supported_endpoints": [
|
||||
"/v1/chat/completions",
|
||||
"/v1/responses"
|
||||
],
|
||||
"supports_function_calling": true,
|
||||
"supports_parallel_function_calling": true,
|
||||
"supports_response_schema": true,
|
||||
"supports_vision": true
|
||||
},
|
||||
"chatgpt/gpt-5.3-chat-latest": {
|
||||
"litellm_provider": "chatgpt",
|
||||
"max_input_tokens": 128000,
|
||||
"max_output_tokens": 64000,
|
||||
"max_tokens": 64000,
|
||||
"mode": "responses",
|
||||
"supported_endpoints": [
|
||||
"/v1/chat/completions",
|
||||
"/v1/responses"
|
||||
],
|
||||
"supports_function_calling": true,
|
||||
"supports_parallel_function_calling": true,
|
||||
"supports_response_schema": true,
|
||||
"supports_vision": true
|
||||
},
|
||||
"chatgpt/gpt-5.2-codex": {
|
||||
"litellm_provider": "chatgpt",
|
||||
"max_input_tokens": 128000,
|
||||
|
|
@ -20506,7 +20593,9 @@
|
|||
"supports_system_messages": true,
|
||||
"supports_tool_choice": true,
|
||||
"supports_service_tier": true,
|
||||
"supports_vision": true
|
||||
"supports_vision": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": false
|
||||
},
|
||||
"gpt-5.1": {
|
||||
"cache_read_input_token_cost": 1.25e-07,
|
||||
|
|
@ -20542,7 +20631,10 @@
|
|||
"supports_system_messages": true,
|
||||
"supports_tool_choice": true,
|
||||
"supports_service_tier": true,
|
||||
"supports_vision": true
|
||||
"supports_vision": true,
|
||||
"supports_web_search": true,
|
||||
"supports_none_reasoning_effort": true,
|
||||
"supports_xhigh_reasoning_effort": false
|
||||
},
|
||||
"gpt-5.1-2025-11-13": {
|
||||
"cache_read_input_token_cost": 1.25e-07,
|
||||
|
|
@ -20578,7 +20670,10 @@
|
|||
"supports_system_messages": true,
|
||||
"supports_tool_choice": true,
|
||||
"supports_service_tier": true,
|
||||
"supports_vision": true
|
||||
"supports_vision": true,
|
||||
"supports_web_search": true,
|
||||
"supports_none_reasoning_effort": true,
|
||||
"supports_xhigh_reasoning_effort": false
|
||||
},
|
||||
"gpt-5.1-chat-latest": {
|
||||
"cache_read_input_token_cost": 1.25e-07,
|
||||
|
|
@ -20613,7 +20708,10 @@
|
|||
"supports_response_schema": true,
|
||||
"supports_system_messages": true,
|
||||
"supports_tool_choice": false,
|
||||
"supports_vision": true
|
||||
"supports_vision": true,
|
||||
"supports_web_search": true,
|
||||
"supports_none_reasoning_effort": true,
|
||||
"supports_xhigh_reasoning_effort": false
|
||||
},
|
||||
"gpt-5.2": {
|
||||
"cache_read_input_token_cost": 1.75e-07,
|
||||
|
|
@ -20650,7 +20748,10 @@
|
|||
"supports_system_messages": true,
|
||||
"supports_tool_choice": true,
|
||||
"supports_service_tier": true,
|
||||
"supports_vision": true
|
||||
"supports_vision": true,
|
||||
"supports_web_search": true,
|
||||
"supports_none_reasoning_effort": true,
|
||||
"supports_xhigh_reasoning_effort": true
|
||||
},
|
||||
"gpt-5.2-2025-12-11": {
|
||||
"cache_read_input_token_cost": 1.75e-07,
|
||||
|
|
@ -20687,7 +20788,10 @@
|
|||
"supports_system_messages": true,
|
||||
"supports_tool_choice": true,
|
||||
"supports_service_tier": true,
|
||||
"supports_vision": true
|
||||
"supports_vision": true,
|
||||
"supports_web_search": true,
|
||||
"supports_none_reasoning_effort": true,
|
||||
"supports_xhigh_reasoning_effort": true
|
||||
},
|
||||
"gpt-5.2-chat-latest": {
|
||||
"cache_read_input_token_cost": 1.75e-07,
|
||||
|
|
@ -20721,7 +20825,10 @@
|
|||
"supports_response_schema": true,
|
||||
"supports_system_messages": true,
|
||||
"supports_tool_choice": true,
|
||||
"supports_vision": true
|
||||
"supports_vision": true,
|
||||
"supports_web_search": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": false
|
||||
},
|
||||
"gpt-5.3-chat-latest": {
|
||||
"cache_read_input_token_cost": 1.75e-07,
|
||||
|
|
@ -20755,7 +20862,10 @@
|
|||
"supports_response_schema": true,
|
||||
"supports_system_messages": true,
|
||||
"supports_tool_choice": true,
|
||||
"supports_vision": true
|
||||
"supports_vision": true,
|
||||
"supports_web_search": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": false
|
||||
},
|
||||
"gpt-5.2-pro": {
|
||||
"input_cost_per_token": 2.1e-05,
|
||||
|
|
@ -20786,7 +20896,9 @@
|
|||
"supports_system_messages": true,
|
||||
"supports_tool_choice": true,
|
||||
"supports_vision": true,
|
||||
"supports_web_search": true
|
||||
"supports_web_search": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": true
|
||||
},
|
||||
"gpt-5.2-pro-2025-12-11": {
|
||||
"input_cost_per_token": 2.1e-05,
|
||||
|
|
@ -20817,20 +20929,82 @@
|
|||
"supports_system_messages": true,
|
||||
"supports_tool_choice": true,
|
||||
"supports_vision": true,
|
||||
"supports_web_search": true
|
||||
"supports_web_search": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": true
|
||||
},
|
||||
"gpt-5.4": {
|
||||
"cache_read_input_token_cost": 2.5e-07,
|
||||
"cache_read_input_token_cost_above_272k_tokens": 5e-07,
|
||||
"cache_read_input_token_cost_flex": 1.3e-07,
|
||||
"cache_read_input_token_cost_priority": 5e-07,
|
||||
"cache_read_input_token_cost_above_272k_tokens_priority": 1e-06,
|
||||
"input_cost_per_token": 2.5e-06,
|
||||
"input_cost_per_token_above_272k_tokens": 5e-06,
|
||||
"input_cost_per_token_flex": 1.25e-06,
|
||||
"input_cost_per_token_batches": 1.25e-06,
|
||||
"input_cost_per_token_priority": 5e-06,
|
||||
"input_cost_per_token_above_272k_tokens_priority": 1e-05,
|
||||
"litellm_provider": "openai",
|
||||
"max_input_tokens": 1050000,
|
||||
"max_output_tokens": 128000,
|
||||
"max_tokens": 128000,
|
||||
"mode": "responses",
|
||||
"mode": "chat",
|
||||
"output_cost_per_token": 1.5e-05,
|
||||
"output_cost_per_token_above_272k_tokens": 2.25e-05,
|
||||
"output_cost_per_token_flex": 7.5e-06,
|
||||
"output_cost_per_token_batches": 7.5e-06,
|
||||
"output_cost_per_token_priority": 2.25e-05,
|
||||
"output_cost_per_token_above_272k_tokens_priority": 3.375e-05,
|
||||
"supported_endpoints": [
|
||||
"/v1/chat/completions",
|
||||
"/v1/batch",
|
||||
"/v1/responses"
|
||||
],
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"text"
|
||||
],
|
||||
"supports_function_calling": true,
|
||||
"supports_native_streaming": true,
|
||||
"supports_parallel_function_calling": true,
|
||||
"supports_pdf_input": true,
|
||||
"supports_prompt_caching": true,
|
||||
"supports_reasoning": true,
|
||||
"supports_response_schema": true,
|
||||
"supports_system_messages": true,
|
||||
"supports_tool_choice": true,
|
||||
"supports_service_tier": true,
|
||||
"supports_vision": true,
|
||||
"supports_none_reasoning_effort": true,
|
||||
"supports_xhigh_reasoning_effort": true
|
||||
},
|
||||
"gpt-5.4-2026-03-05": {
|
||||
"cache_read_input_token_cost": 2.5e-07,
|
||||
"cache_read_input_token_cost_above_272k_tokens": 5e-07,
|
||||
"cache_read_input_token_cost_flex": 1.3e-07,
|
||||
"cache_read_input_token_cost_priority": 5e-07,
|
||||
"cache_read_input_token_cost_above_272k_tokens_priority": 1e-06,
|
||||
"input_cost_per_token": 2.5e-06,
|
||||
"input_cost_per_token_above_272k_tokens": 5e-06,
|
||||
"input_cost_per_token_flex": 1.25e-06,
|
||||
"input_cost_per_token_batches": 1.25e-06,
|
||||
"input_cost_per_token_priority": 5e-06,
|
||||
"input_cost_per_token_above_272k_tokens_priority": 1e-05,
|
||||
"litellm_provider": "openai",
|
||||
"max_input_tokens": 1050000,
|
||||
"max_output_tokens": 128000,
|
||||
"max_tokens": 128000,
|
||||
"mode": "chat",
|
||||
"output_cost_per_token": 1.5e-05,
|
||||
"output_cost_per_token_above_272k_tokens": 2.25e-05,
|
||||
"output_cost_per_token_flex": 7.5e-06,
|
||||
"output_cost_per_token_batches": 7.5e-06,
|
||||
"output_cost_per_token_priority": 2.25e-05,
|
||||
"output_cost_per_token_above_272k_tokens_priority": 3.375e-05,
|
||||
"supported_endpoints": [
|
||||
"/v1/chat/completions",
|
||||
"/v1/batch",
|
||||
|
|
@ -20855,18 +21029,28 @@
|
|||
"supports_service_tier": true,
|
||||
"supports_vision": true
|
||||
},
|
||||
"gpt-5.4-2026-03-05": {
|
||||
"cache_read_input_token_cost": 2.5e-07,
|
||||
"cache_read_input_token_cost_priority": 5e-07,
|
||||
"input_cost_per_token": 2.5e-06,
|
||||
"input_cost_per_token_priority": 5e-06,
|
||||
"gpt-5.4-pro": {
|
||||
"cache_read_input_token_cost": 3e-06,
|
||||
"cache_read_input_token_cost_above_272k_tokens": 6e-06,
|
||||
"cache_read_input_token_cost_priority": 6e-06,
|
||||
"cache_read_input_token_cost_above_272k_tokens_priority": 1.2e-05,
|
||||
"input_cost_per_token": 3e-05,
|
||||
"input_cost_per_token_above_272k_tokens": 6e-05,
|
||||
"input_cost_per_token_flex": 1.5e-05,
|
||||
"input_cost_per_token_batches": 1.5e-05,
|
||||
"input_cost_per_token_priority": 6e-05,
|
||||
"input_cost_per_token_above_272k_tokens_priority": 1.2e-04,
|
||||
"litellm_provider": "openai",
|
||||
"max_input_tokens": 1050000,
|
||||
"max_output_tokens": 128000,
|
||||
"max_tokens": 128000,
|
||||
"mode": "responses",
|
||||
"output_cost_per_token": 1.5e-05,
|
||||
"output_cost_per_token_priority": 2.25e-05,
|
||||
"mode": "chat",
|
||||
"output_cost_per_token": 1.8e-04,
|
||||
"output_cost_per_token_above_272k_tokens": 2.7e-04,
|
||||
"output_cost_per_token_flex": 9e-05,
|
||||
"output_cost_per_token_batches": 9e-05,
|
||||
"output_cost_per_token_priority": 2.7e-04,
|
||||
"output_cost_per_token_above_272k_tokens_priority": 4.05e-04,
|
||||
"supported_endpoints": [
|
||||
"/v1/chat/completions",
|
||||
"/v1/batch",
|
||||
|
|
@ -20885,11 +21069,63 @@
|
|||
"supports_pdf_input": true,
|
||||
"supports_prompt_caching": true,
|
||||
"supports_reasoning": true,
|
||||
"supports_response_schema": true,
|
||||
"supports_response_schema": false,
|
||||
"supports_system_messages": true,
|
||||
"supports_tool_choice": true,
|
||||
"supports_service_tier": true,
|
||||
"supports_vision": true
|
||||
"supports_vision": true,
|
||||
"supports_web_search": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": true
|
||||
},
|
||||
"gpt-5.4-pro-2026-03-05": {
|
||||
"cache_read_input_token_cost": 3e-06,
|
||||
"cache_read_input_token_cost_above_272k_tokens": 6e-06,
|
||||
"cache_read_input_token_cost_priority": 6e-06,
|
||||
"cache_read_input_token_cost_above_272k_tokens_priority": 1.2e-05,
|
||||
"input_cost_per_token": 3e-05,
|
||||
"input_cost_per_token_above_272k_tokens": 6e-05,
|
||||
"input_cost_per_token_flex": 1.5e-05,
|
||||
"input_cost_per_token_batches": 1.5e-05,
|
||||
"input_cost_per_token_priority": 6e-05,
|
||||
"input_cost_per_token_above_272k_tokens_priority": 1.2e-04,
|
||||
"litellm_provider": "openai",
|
||||
"max_input_tokens": 1050000,
|
||||
"max_output_tokens": 128000,
|
||||
"max_tokens": 128000,
|
||||
"mode": "chat",
|
||||
"output_cost_per_token": 1.8e-04,
|
||||
"output_cost_per_token_above_272k_tokens": 2.7e-04,
|
||||
"output_cost_per_token_flex": 9e-05,
|
||||
"output_cost_per_token_batches": 9e-05,
|
||||
"output_cost_per_token_priority": 2.7e-04,
|
||||
"output_cost_per_token_above_272k_tokens_priority": 4.05e-04,
|
||||
"supported_endpoints": [
|
||||
"/v1/chat/completions",
|
||||
"/v1/batch",
|
||||
"/v1/responses"
|
||||
],
|
||||
"supported_modalities": [
|
||||
"text",
|
||||
"image"
|
||||
],
|
||||
"supported_output_modalities": [
|
||||
"text"
|
||||
],
|
||||
"supports_function_calling": true,
|
||||
"supports_native_streaming": true,
|
||||
"supports_parallel_function_calling": true,
|
||||
"supports_pdf_input": true,
|
||||
"supports_prompt_caching": true,
|
||||
"supports_reasoning": true,
|
||||
"supports_response_schema": false,
|
||||
"supports_system_messages": true,
|
||||
"supports_tool_choice": true,
|
||||
"supports_service_tier": true,
|
||||
"supports_vision": true,
|
||||
"supports_web_search": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": true
|
||||
},
|
||||
"gpt-5-pro": {
|
||||
"input_cost_per_token": 1.5e-05,
|
||||
|
|
@ -20922,7 +21158,9 @@
|
|||
"supports_system_messages": true,
|
||||
"supports_tool_choice": true,
|
||||
"supports_vision": true,
|
||||
"supports_web_search": true
|
||||
"supports_web_search": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": false
|
||||
},
|
||||
"gpt-5-pro-2025-10-06": {
|
||||
"input_cost_per_token": 1.5e-05,
|
||||
|
|
@ -20955,7 +21193,9 @@
|
|||
"supports_system_messages": true,
|
||||
"supports_tool_choice": true,
|
||||
"supports_vision": true,
|
||||
"supports_web_search": true
|
||||
"supports_web_search": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": false
|
||||
},
|
||||
"gpt-5-2025-08-07": {
|
||||
"cache_read_input_token_cost": 1.25e-07,
|
||||
|
|
@ -20994,7 +21234,9 @@
|
|||
"supports_system_messages": true,
|
||||
"supports_tool_choice": true,
|
||||
"supports_service_tier": true,
|
||||
"supports_vision": true
|
||||
"supports_vision": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": false
|
||||
},
|
||||
"gpt-5-chat": {
|
||||
"cache_read_input_token_cost": 1.25e-07,
|
||||
|
|
@ -21026,7 +21268,9 @@
|
|||
"supports_response_schema": true,
|
||||
"supports_system_messages": true,
|
||||
"supports_tool_choice": false,
|
||||
"supports_vision": true
|
||||
"supports_vision": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": false
|
||||
},
|
||||
"gpt-5-chat-latest": {
|
||||
"cache_read_input_token_cost": 1.25e-07,
|
||||
|
|
@ -21058,7 +21302,9 @@
|
|||
"supports_response_schema": true,
|
||||
"supports_system_messages": true,
|
||||
"supports_tool_choice": false,
|
||||
"supports_vision": true
|
||||
"supports_vision": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": false
|
||||
},
|
||||
"gpt-5-codex": {
|
||||
"cache_read_input_token_cost": 1.25e-07,
|
||||
|
|
@ -21088,7 +21334,9 @@
|
|||
"supports_response_schema": true,
|
||||
"supports_system_messages": false,
|
||||
"supports_tool_choice": true,
|
||||
"supports_vision": true
|
||||
"supports_vision": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": false
|
||||
},
|
||||
"gpt-5.1-codex": {
|
||||
"cache_read_input_token_cost": 1.25e-07,
|
||||
|
|
@ -21121,7 +21369,9 @@
|
|||
"supports_response_schema": true,
|
||||
"supports_system_messages": false,
|
||||
"supports_tool_choice": true,
|
||||
"supports_vision": true
|
||||
"supports_vision": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": false
|
||||
},
|
||||
"gpt-5.1-codex-max": {
|
||||
"cache_read_input_token_cost": 1.25e-07,
|
||||
|
|
@ -21151,7 +21401,9 @@
|
|||
"supports_response_schema": true,
|
||||
"supports_system_messages": false,
|
||||
"supports_tool_choice": true,
|
||||
"supports_vision": true
|
||||
"supports_vision": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": true
|
||||
},
|
||||
"gpt-5.1-codex-mini": {
|
||||
"cache_read_input_token_cost": 2.5e-08,
|
||||
|
|
@ -21184,7 +21436,9 @@
|
|||
"supports_response_schema": true,
|
||||
"supports_system_messages": false,
|
||||
"supports_tool_choice": true,
|
||||
"supports_vision": true
|
||||
"supports_vision": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": false
|
||||
},
|
||||
"gpt-5.2-codex": {
|
||||
"cache_read_input_token_cost": 1.75e-07,
|
||||
|
|
@ -21217,7 +21471,9 @@
|
|||
"supports_response_schema": true,
|
||||
"supports_system_messages": false,
|
||||
"supports_tool_choice": true,
|
||||
"supports_vision": true
|
||||
"supports_vision": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": true
|
||||
},
|
||||
"gpt-5.3-codex": {
|
||||
"cache_read_input_token_cost": 1.75e-07,
|
||||
|
|
@ -21250,7 +21506,9 @@
|
|||
"supports_response_schema": true,
|
||||
"supports_system_messages": false,
|
||||
"supports_tool_choice": true,
|
||||
"supports_vision": true
|
||||
"supports_vision": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": false
|
||||
},
|
||||
"gpt-5-mini": {
|
||||
"cache_read_input_token_cost": 2.5e-08,
|
||||
|
|
@ -21289,7 +21547,9 @@
|
|||
"supports_system_messages": true,
|
||||
"supports_tool_choice": true,
|
||||
"supports_service_tier": true,
|
||||
"supports_vision": true
|
||||
"supports_vision": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": false
|
||||
},
|
||||
"gpt-5-mini-2025-08-07": {
|
||||
"cache_read_input_token_cost": 2.5e-08,
|
||||
|
|
@ -21328,7 +21588,9 @@
|
|||
"supports_system_messages": true,
|
||||
"supports_tool_choice": true,
|
||||
"supports_service_tier": true,
|
||||
"supports_vision": true
|
||||
"supports_vision": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": false
|
||||
},
|
||||
"gpt-5-nano": {
|
||||
"cache_read_input_token_cost": 5e-09,
|
||||
|
|
@ -21364,7 +21626,9 @@
|
|||
"supports_response_schema": true,
|
||||
"supports_system_messages": true,
|
||||
"supports_tool_choice": true,
|
||||
"supports_vision": true
|
||||
"supports_vision": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": false
|
||||
},
|
||||
"gpt-5-nano-2025-08-07": {
|
||||
"cache_read_input_token_cost": 5e-09,
|
||||
|
|
@ -21399,7 +21663,9 @@
|
|||
"supports_response_schema": true,
|
||||
"supports_system_messages": true,
|
||||
"supports_tool_choice": true,
|
||||
"supports_vision": true
|
||||
"supports_vision": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": false
|
||||
},
|
||||
"gpt-image-1": {
|
||||
"cache_read_input_image_token_cost": 2.5e-06,
|
||||
|
|
@ -38559,7 +38825,9 @@
|
|||
"supports_system_messages": true,
|
||||
"supports_tool_choice": true,
|
||||
"supports_vision": true,
|
||||
"supports_web_search": true
|
||||
"supports_web_search": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": false
|
||||
},
|
||||
"gpt-5-search-api-2025-10-14": {
|
||||
"cache_read_input_token_cost": 1.25e-07,
|
||||
|
|
@ -38578,7 +38846,9 @@
|
|||
"supports_system_messages": true,
|
||||
"supports_tool_choice": true,
|
||||
"supports_vision": true,
|
||||
"supports_web_search": true
|
||||
"supports_web_search": true,
|
||||
"supports_none_reasoning_effort": false,
|
||||
"supports_xhigh_reasoning_effort": false
|
||||
},
|
||||
"gpt-realtime-mini-2025-10-06": {
|
||||
"cache_creation_input_audio_token_cost": 3e-07,
|
||||
|
|
|
|||
|
|
@ -124,6 +124,8 @@ def extract_parameters(operation: Dict[str, Any]) -> tuple:
|
|||
# OpenAPI 3.x and 2.x parameters
|
||||
if "parameters" in operation:
|
||||
for param in operation["parameters"]:
|
||||
if "name" not in param:
|
||||
continue
|
||||
param_name = param["name"]
|
||||
if param.get("in") == "path":
|
||||
path_params.append(param_name)
|
||||
|
|
@ -147,6 +149,8 @@ def build_input_schema(operation: Dict[str, Any]) -> Dict[str, Any]:
|
|||
# Process parameters
|
||||
if "parameters" in operation:
|
||||
for param in operation["parameters"]:
|
||||
if "name" not in param:
|
||||
continue
|
||||
param_name = param["name"]
|
||||
param_schema = param.get("schema", {})
|
||||
param_type = param_schema.get("type", "string")
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -1,32 +1,29 @@
|
|||
1:"$Sreact.fragment"
|
||||
2:I[347257,["/litellm-asset-prefix/_next/static/chunks/d96012bcfc98706a.js","/litellm-asset-prefix/_next/static/chunks/dbca964212122d58.js"],"ClientPageRoot"]
|
||||
3:I[952683,["/litellm-asset-prefix/_next/static/chunks/26adfa4e8ffc85c7.js","/litellm-asset-prefix/_next/static/chunks/e8ed72789c2b42ff.js","/litellm-asset-prefix/_next/static/chunks/61f59596bf7f0628.js","/litellm-asset-prefix/_next/static/chunks/9f5ccd929375c1d6.js","/litellm-asset-prefix/_next/static/chunks/30539b80ac15aad2.js","/litellm-asset-prefix/_next/static/chunks/142704439974f6b3.js","/litellm-asset-prefix/_next/static/chunks/8b4fd43197a5dfc8.js","/litellm-asset-prefix/_next/static/chunks/1fe0596a309ad6cf.js","/litellm-asset-prefix/_next/static/chunks/7b788dd93ad868b3.js","/litellm-asset-prefix/_next/static/chunks/c979f252fa5ee77c.js","/litellm-asset-prefix/_next/static/chunks/50febaabfb896c46.js","/litellm-asset-prefix/_next/static/chunks/c4cc89d1b0a147d5.js","/litellm-asset-prefix/_next/static/chunks/fe5201571c777f09.js","/litellm-asset-prefix/_next/static/chunks/c748f222de4766db.js","/litellm-asset-prefix/_next/static/chunks/5f9c3b92a016f382.js","/litellm-asset-prefix/_next/static/chunks/fe750aa0bf04912c.js","/litellm-asset-prefix/_next/static/chunks/7d4cded1a1238581.js","/litellm-asset-prefix/_next/static/chunks/d64d74932cb225a3.js","/litellm-asset-prefix/_next/static/chunks/9ffb8ddd0c9a7c31.js","/litellm-asset-prefix/_next/static/chunks/c4fe728e74b52958.js","/litellm-asset-prefix/_next/static/chunks/1532edb438ed84bb.js","/litellm-asset-prefix/_next/static/chunks/d0d828f9a0668699.js","/litellm-asset-prefix/_next/static/chunks/88c74f8b4b20d25a.js","/litellm-asset-prefix/_next/static/chunks/25c5a08661ac2ec3.js","/litellm-asset-prefix/_next/static/chunks/134f728fa7099e3e.js","/litellm-asset-prefix/_next/static/chunks/9d6e5aad99b19216.js","/litellm-asset-prefix/_next/static/chunks/e8718f949e42598e.js","/litellm-asset-prefix/_next/static/chunks/6a51328383335d1e.js","/litellm-asset-prefix/_next/static/chunks/3b30ab8eaa03bc21.js","/litellm-asset-prefix/_next/static/chunks/99cf9cf99df5ccfc.js","/litellm-asset-prefix/_next/static/chunks/6a4eede876bb5c8f.js","/litellm-asset-prefix/_next/static/chunks/7e417dd24c8becd0.js","/litellm-asset-prefix/_next/static/chunks/c7b74067c01ee971.js","/litellm-asset-prefix/_next/static/chunks/a85adee4198d5478.js","/litellm-asset-prefix/_next/static/chunks/bff4459534c52f16.js","/litellm-asset-prefix/_next/static/chunks/39eb5927f56ccc0e.js","/litellm-asset-prefix/_next/static/chunks/ebb2b2d8175d3d2f.js","/litellm-asset-prefix/_next/static/chunks/5a5488ab3db0c3de.js","/litellm-asset-prefix/_next/static/chunks/0a65da2cd24e2ab6.js","/litellm-asset-prefix/_next/static/chunks/82dbdb17d57c6737.js","/litellm-asset-prefix/_next/static/chunks/b31a08272a82b84e.js","/litellm-asset-prefix/_next/static/chunks/496b84010c33cf69.js","/litellm-asset-prefix/_next/static/chunks/eba9ac65320061b1.js","/litellm-asset-prefix/_next/static/chunks/0eda6dc5d5f35d92.js","/litellm-asset-prefix/_next/static/chunks/96aed36e606e8582.js","/litellm-asset-prefix/_next/static/chunks/ad426ab08aee6c64.js","/litellm-asset-prefix/_next/static/chunks/2971c4658f1bcd7d.js","/litellm-asset-prefix/_next/static/chunks/06ebe9b0e9cdf241.js","/litellm-asset-prefix/_next/static/chunks/6c4c97f1ea6e7d77.js","/litellm-asset-prefix/_next/static/chunks/165d00848f04c4c9.js","/litellm-asset-prefix/_next/static/chunks/bb0a6e4a3a18721a.js","/litellm-asset-prefix/_next/static/chunks/4980372eaa37b78b.js","/litellm-asset-prefix/_next/static/chunks/8ab6e2ca95000c8e.js","/litellm-asset-prefix/_next/static/chunks/06cdd9bb80c63794.js","/litellm-asset-prefix/_next/static/chunks/66a190706fc6c35a.js","/litellm-asset-prefix/_next/static/chunks/54e29148cb2f2582.js","/litellm-asset-prefix/_next/static/chunks/fd9de419c8c0222e.js"],"default"]
|
||||
1c:I[897367,["/litellm-asset-prefix/_next/static/chunks/d96012bcfc98706a.js","/litellm-asset-prefix/_next/static/chunks/dbca964212122d58.js"],"OutletBoundary"]
|
||||
1d:"$Sreact.suspense"
|
||||
3:I[952683,["/litellm-asset-prefix/_next/static/chunks/a7f104aa2cc7f3f0.js","/litellm-asset-prefix/_next/static/chunks/e8ed72789c2b42ff.js","/litellm-asset-prefix/_next/static/chunks/db1ebd02d726c50f.js","/litellm-asset-prefix/_next/static/chunks/142704439974f6b3.js","/litellm-asset-prefix/_next/static/chunks/30539b80ac15aad2.js","/litellm-asset-prefix/_next/static/chunks/e55394619917c445.js","/litellm-asset-prefix/_next/static/chunks/df6665addf8c6036.js","/litellm-asset-prefix/_next/static/chunks/1fe0596a309ad6cf.js","/litellm-asset-prefix/_next/static/chunks/9d9e6235c06ebb90.js","/litellm-asset-prefix/_next/static/chunks/403c4d96324c23a6.js","/litellm-asset-prefix/_next/static/chunks/308c947b873bf49b.js","/litellm-asset-prefix/_next/static/chunks/88c74f8b4b20d25a.js","/litellm-asset-prefix/_next/static/chunks/2d471965761a22ff.js","/litellm-asset-prefix/_next/static/chunks/6eb81672801a9dec.js","/litellm-asset-prefix/_next/static/chunks/3b30ab8eaa03bc21.js","/litellm-asset-prefix/_next/static/chunks/7b788dd93ad868b3.js","/litellm-asset-prefix/_next/static/chunks/5f9c3b92a016f382.js","/litellm-asset-prefix/_next/static/chunks/38605193023f8e1c.js","/litellm-asset-prefix/_next/static/chunks/d0d828f9a0668699.js","/litellm-asset-prefix/_next/static/chunks/4d3d997560b322ca.js","/litellm-asset-prefix/_next/static/chunks/4e4d0f466b5c1780.js","/litellm-asset-prefix/_next/static/chunks/d64d74932cb225a3.js","/litellm-asset-prefix/_next/static/chunks/3fb7a83546b6aa35.js","/litellm-asset-prefix/_next/static/chunks/134f728fa7099e3e.js","/litellm-asset-prefix/_next/static/chunks/9d6e5aad99b19216.js","/litellm-asset-prefix/_next/static/chunks/bb02cdce134f811b.js","/litellm-asset-prefix/_next/static/chunks/0f93c304f38c63d0.js","/litellm-asset-prefix/_next/static/chunks/130b80d41c79d98e.js","/litellm-asset-prefix/_next/static/chunks/348b31083769a7c4.js","/litellm-asset-prefix/_next/static/chunks/759173c2a452c43c.js","/litellm-asset-prefix/_next/static/chunks/4980372eaa37b78b.js","/litellm-asset-prefix/_next/static/chunks/c81bdad246412686.js","/litellm-asset-prefix/_next/static/chunks/b6cdb9a433f054f3.js","/litellm-asset-prefix/_next/static/chunks/a85adee4198d5478.js","/litellm-asset-prefix/_next/static/chunks/fcdf7322b0aa3e2e.js","/litellm-asset-prefix/_next/static/chunks/0adb91ab5f3140d5.js","/litellm-asset-prefix/_next/static/chunks/675f6c62ddae3031.js","/litellm-asset-prefix/_next/static/chunks/7e417dd24c8becd0.js","/litellm-asset-prefix/_next/static/chunks/0a65da2cd24e2ab6.js","/litellm-asset-prefix/_next/static/chunks/86c1e01d849eed8e.js","/litellm-asset-prefix/_next/static/chunks/eba9ac65320061b1.js","/litellm-asset-prefix/_next/static/chunks/85238af541b170ca.js","/litellm-asset-prefix/_next/static/chunks/8c09c924a98654d1.js","/litellm-asset-prefix/_next/static/chunks/b13b0ddeb85b5333.js","/litellm-asset-prefix/_next/static/chunks/2971c4658f1bcd7d.js","/litellm-asset-prefix/_next/static/chunks/496b84010c33cf69.js","/litellm-asset-prefix/_next/static/chunks/06ebe9b0e9cdf241.js","/litellm-asset-prefix/_next/static/chunks/6c4c97f1ea6e7d77.js","/litellm-asset-prefix/_next/static/chunks/e8718f949e42598e.js","/litellm-asset-prefix/_next/static/chunks/99cf9cf99df5ccfc.js","/litellm-asset-prefix/_next/static/chunks/bb0a6e4a3a18721a.js","/litellm-asset-prefix/_next/static/chunks/cfca40b1a4a490bf.js","/litellm-asset-prefix/_next/static/chunks/54e29148cb2f2582.js","/litellm-asset-prefix/_next/static/chunks/37c03dc421ba81f8.js"],"default"]
|
||||
19:I[897367,["/litellm-asset-prefix/_next/static/chunks/d96012bcfc98706a.js","/litellm-asset-prefix/_next/static/chunks/dbca964212122d58.js"],"OutletBoundary"]
|
||||
1a:"$Sreact.suspense"
|
||||
:HL["/litellm-asset-prefix/_next/static/chunks/3f3fa56b5786d58c.css","style"]
|
||||
0:{"buildId":"U_YrOOnSehrpkdU42KJ-W","rsc":["$","$1","c",{"children":[["$","$L2",null,{"Component":"$3","serverProvidedParams":{"searchParams":{},"params":{},"promises":["$@4","$@5"]}}],[["$","link","0",{"rel":"stylesheet","href":"/litellm-asset-prefix/_next/static/chunks/3f3fa56b5786d58c.css","precedence":"next"}],["$","script","script-0",{"src":"/litellm-asset-prefix/_next/static/chunks/61f59596bf7f0628.js","async":true}],["$","script","script-1",{"src":"/litellm-asset-prefix/_next/static/chunks/9f5ccd929375c1d6.js","async":true}],["$","script","script-2",{"src":"/litellm-asset-prefix/_next/static/chunks/30539b80ac15aad2.js","async":true}],["$","script","script-3",{"src":"/litellm-asset-prefix/_next/static/chunks/142704439974f6b3.js","async":true}],["$","script","script-4",{"src":"/litellm-asset-prefix/_next/static/chunks/8b4fd43197a5dfc8.js","async":true}],["$","script","script-5",{"src":"/litellm-asset-prefix/_next/static/chunks/1fe0596a309ad6cf.js","async":true}],["$","script","script-6",{"src":"/litellm-asset-prefix/_next/static/chunks/7b788dd93ad868b3.js","async":true}],["$","script","script-7",{"src":"/litellm-asset-prefix/_next/static/chunks/c979f252fa5ee77c.js","async":true}],["$","script","script-8",{"src":"/litellm-asset-prefix/_next/static/chunks/50febaabfb896c46.js","async":true}],["$","script","script-9",{"src":"/litellm-asset-prefix/_next/static/chunks/c4cc89d1b0a147d5.js","async":true}],["$","script","script-10",{"src":"/litellm-asset-prefix/_next/static/chunks/fe5201571c777f09.js","async":true}],["$","script","script-11",{"src":"/litellm-asset-prefix/_next/static/chunks/c748f222de4766db.js","async":true}],["$","script","script-12",{"src":"/litellm-asset-prefix/_next/static/chunks/5f9c3b92a016f382.js","async":true}],["$","script","script-13",{"src":"/litellm-asset-prefix/_next/static/chunks/fe750aa0bf04912c.js","async":true}],["$","script","script-14",{"src":"/litellm-asset-prefix/_next/static/chunks/7d4cded1a1238581.js","async":true}],["$","script","script-15",{"src":"/litellm-asset-prefix/_next/static/chunks/d64d74932cb225a3.js","async":true}],["$","script","script-16",{"src":"/litellm-asset-prefix/_next/static/chunks/9ffb8ddd0c9a7c31.js","async":true}],["$","script","script-17",{"src":"/litellm-asset-prefix/_next/static/chunks/c4fe728e74b52958.js","async":true}],["$","script","script-18",{"src":"/litellm-asset-prefix/_next/static/chunks/1532edb438ed84bb.js","async":true}],["$","script","script-19",{"src":"/litellm-asset-prefix/_next/static/chunks/d0d828f9a0668699.js","async":true}],["$","script","script-20",{"src":"/litellm-asset-prefix/_next/static/chunks/88c74f8b4b20d25a.js","async":true}],["$","script","script-21",{"src":"/litellm-asset-prefix/_next/static/chunks/25c5a08661ac2ec3.js","async":true}],["$","script","script-22",{"src":"/litellm-asset-prefix/_next/static/chunks/134f728fa7099e3e.js","async":true}],["$","script","script-23",{"src":"/litellm-asset-prefix/_next/static/chunks/9d6e5aad99b19216.js","async":true}],["$","script","script-24",{"src":"/litellm-asset-prefix/_next/static/chunks/e8718f949e42598e.js","async":true}],["$","script","script-25",{"src":"/litellm-asset-prefix/_next/static/chunks/6a51328383335d1e.js","async":true}],["$","script","script-26",{"src":"/litellm-asset-prefix/_next/static/chunks/3b30ab8eaa03bc21.js","async":true}],["$","script","script-27",{"src":"/litellm-asset-prefix/_next/static/chunks/99cf9cf99df5ccfc.js","async":true}],["$","script","script-28",{"src":"/litellm-asset-prefix/_next/static/chunks/6a4eede876bb5c8f.js","async":true}],["$","script","script-29",{"src":"/litellm-asset-prefix/_next/static/chunks/7e417dd24c8becd0.js","async":true}],["$","script","script-30",{"src":"/litellm-asset-prefix/_next/static/chunks/c7b74067c01ee971.js","async":true}],["$","script","script-31",{"src":"/litellm-asset-prefix/_next/static/chunks/a85adee4198d5478.js","async":true}],["$","script","script-32",{"src":"/litellm-asset-prefix/_next/static/chunks/bff4459534c52f16.js","async":true}],["$","script","script-33",{"src":"/litellm-asset-prefix/_next/static/chunks/39eb5927f56ccc0e.js","async":true}],"$L6","$L7","$L8","$L9","$La","$Lb","$Lc","$Ld","$Le","$Lf","$L10","$L11","$L12","$L13","$L14","$L15","$L16","$L17","$L18","$L19","$L1a"],"$L1b"]}],"loading":null,"isPartial":false}
|
||||
0:{"buildId":"cbFGTIkRGp63usVNisey9","rsc":["$","$1","c",{"children":[["$","$L2",null,{"Component":"$3","serverProvidedParams":{"searchParams":{},"params":{},"promises":["$@4","$@5"]}}],[["$","link","0",{"rel":"stylesheet","href":"/litellm-asset-prefix/_next/static/chunks/3f3fa56b5786d58c.css","precedence":"next"}],["$","script","script-0",{"src":"/litellm-asset-prefix/_next/static/chunks/db1ebd02d726c50f.js","async":true}],["$","script","script-1",{"src":"/litellm-asset-prefix/_next/static/chunks/142704439974f6b3.js","async":true}],["$","script","script-2",{"src":"/litellm-asset-prefix/_next/static/chunks/30539b80ac15aad2.js","async":true}],["$","script","script-3",{"src":"/litellm-asset-prefix/_next/static/chunks/e55394619917c445.js","async":true}],["$","script","script-4",{"src":"/litellm-asset-prefix/_next/static/chunks/df6665addf8c6036.js","async":true}],["$","script","script-5",{"src":"/litellm-asset-prefix/_next/static/chunks/1fe0596a309ad6cf.js","async":true}],["$","script","script-6",{"src":"/litellm-asset-prefix/_next/static/chunks/9d9e6235c06ebb90.js","async":true}],["$","script","script-7",{"src":"/litellm-asset-prefix/_next/static/chunks/403c4d96324c23a6.js","async":true}],["$","script","script-8",{"src":"/litellm-asset-prefix/_next/static/chunks/308c947b873bf49b.js","async":true}],["$","script","script-9",{"src":"/litellm-asset-prefix/_next/static/chunks/88c74f8b4b20d25a.js","async":true}],["$","script","script-10",{"src":"/litellm-asset-prefix/_next/static/chunks/2d471965761a22ff.js","async":true}],["$","script","script-11",{"src":"/litellm-asset-prefix/_next/static/chunks/6eb81672801a9dec.js","async":true}],["$","script","script-12",{"src":"/litellm-asset-prefix/_next/static/chunks/3b30ab8eaa03bc21.js","async":true}],["$","script","script-13",{"src":"/litellm-asset-prefix/_next/static/chunks/7b788dd93ad868b3.js","async":true}],["$","script","script-14",{"src":"/litellm-asset-prefix/_next/static/chunks/5f9c3b92a016f382.js","async":true}],["$","script","script-15",{"src":"/litellm-asset-prefix/_next/static/chunks/38605193023f8e1c.js","async":true}],["$","script","script-16",{"src":"/litellm-asset-prefix/_next/static/chunks/d0d828f9a0668699.js","async":true}],["$","script","script-17",{"src":"/litellm-asset-prefix/_next/static/chunks/4d3d997560b322ca.js","async":true}],["$","script","script-18",{"src":"/litellm-asset-prefix/_next/static/chunks/4e4d0f466b5c1780.js","async":true}],["$","script","script-19",{"src":"/litellm-asset-prefix/_next/static/chunks/d64d74932cb225a3.js","async":true}],["$","script","script-20",{"src":"/litellm-asset-prefix/_next/static/chunks/3fb7a83546b6aa35.js","async":true}],["$","script","script-21",{"src":"/litellm-asset-prefix/_next/static/chunks/134f728fa7099e3e.js","async":true}],["$","script","script-22",{"src":"/litellm-asset-prefix/_next/static/chunks/9d6e5aad99b19216.js","async":true}],["$","script","script-23",{"src":"/litellm-asset-prefix/_next/static/chunks/bb02cdce134f811b.js","async":true}],["$","script","script-24",{"src":"/litellm-asset-prefix/_next/static/chunks/0f93c304f38c63d0.js","async":true}],["$","script","script-25",{"src":"/litellm-asset-prefix/_next/static/chunks/130b80d41c79d98e.js","async":true}],["$","script","script-26",{"src":"/litellm-asset-prefix/_next/static/chunks/348b31083769a7c4.js","async":true}],["$","script","script-27",{"src":"/litellm-asset-prefix/_next/static/chunks/759173c2a452c43c.js","async":true}],["$","script","script-28",{"src":"/litellm-asset-prefix/_next/static/chunks/4980372eaa37b78b.js","async":true}],["$","script","script-29",{"src":"/litellm-asset-prefix/_next/static/chunks/c81bdad246412686.js","async":true}],["$","script","script-30",{"src":"/litellm-asset-prefix/_next/static/chunks/b6cdb9a433f054f3.js","async":true}],["$","script","script-31",{"src":"/litellm-asset-prefix/_next/static/chunks/a85adee4198d5478.js","async":true}],["$","script","script-32",{"src":"/litellm-asset-prefix/_next/static/chunks/fcdf7322b0aa3e2e.js","async":true}],["$","script","script-33",{"src":"/litellm-asset-prefix/_next/static/chunks/0adb91ab5f3140d5.js","async":true}],"$L6","$L7","$L8","$L9","$La","$Lb","$Lc","$Ld","$Le","$Lf","$L10","$L11","$L12","$L13","$L14","$L15","$L16","$L17"],"$L18"]}],"loading":null,"isPartial":false}
|
||||
4:{}
|
||||
5:"$0:rsc:props:children:0:props:serverProvidedParams:params"
|
||||
6:["$","script","script-34",{"src":"/litellm-asset-prefix/_next/static/chunks/ebb2b2d8175d3d2f.js","async":true}]
|
||||
7:["$","script","script-35",{"src":"/litellm-asset-prefix/_next/static/chunks/5a5488ab3db0c3de.js","async":true}]
|
||||
6:["$","script","script-34",{"src":"/litellm-asset-prefix/_next/static/chunks/675f6c62ddae3031.js","async":true}]
|
||||
7:["$","script","script-35",{"src":"/litellm-asset-prefix/_next/static/chunks/7e417dd24c8becd0.js","async":true}]
|
||||
8:["$","script","script-36",{"src":"/litellm-asset-prefix/_next/static/chunks/0a65da2cd24e2ab6.js","async":true}]
|
||||
9:["$","script","script-37",{"src":"/litellm-asset-prefix/_next/static/chunks/82dbdb17d57c6737.js","async":true}]
|
||||
a:["$","script","script-38",{"src":"/litellm-asset-prefix/_next/static/chunks/b31a08272a82b84e.js","async":true}]
|
||||
b:["$","script","script-39",{"src":"/litellm-asset-prefix/_next/static/chunks/496b84010c33cf69.js","async":true}]
|
||||
c:["$","script","script-40",{"src":"/litellm-asset-prefix/_next/static/chunks/eba9ac65320061b1.js","async":true}]
|
||||
d:["$","script","script-41",{"src":"/litellm-asset-prefix/_next/static/chunks/0eda6dc5d5f35d92.js","async":true}]
|
||||
e:["$","script","script-42",{"src":"/litellm-asset-prefix/_next/static/chunks/96aed36e606e8582.js","async":true}]
|
||||
f:["$","script","script-43",{"src":"/litellm-asset-prefix/_next/static/chunks/ad426ab08aee6c64.js","async":true}]
|
||||
10:["$","script","script-44",{"src":"/litellm-asset-prefix/_next/static/chunks/2971c4658f1bcd7d.js","async":true}]
|
||||
11:["$","script","script-45",{"src":"/litellm-asset-prefix/_next/static/chunks/06ebe9b0e9cdf241.js","async":true}]
|
||||
12:["$","script","script-46",{"src":"/litellm-asset-prefix/_next/static/chunks/6c4c97f1ea6e7d77.js","async":true}]
|
||||
13:["$","script","script-47",{"src":"/litellm-asset-prefix/_next/static/chunks/165d00848f04c4c9.js","async":true}]
|
||||
9:["$","script","script-37",{"src":"/litellm-asset-prefix/_next/static/chunks/86c1e01d849eed8e.js","async":true}]
|
||||
a:["$","script","script-38",{"src":"/litellm-asset-prefix/_next/static/chunks/eba9ac65320061b1.js","async":true}]
|
||||
b:["$","script","script-39",{"src":"/litellm-asset-prefix/_next/static/chunks/85238af541b170ca.js","async":true}]
|
||||
c:["$","script","script-40",{"src":"/litellm-asset-prefix/_next/static/chunks/8c09c924a98654d1.js","async":true}]
|
||||
d:["$","script","script-41",{"src":"/litellm-asset-prefix/_next/static/chunks/b13b0ddeb85b5333.js","async":true}]
|
||||
e:["$","script","script-42",{"src":"/litellm-asset-prefix/_next/static/chunks/2971c4658f1bcd7d.js","async":true}]
|
||||
f:["$","script","script-43",{"src":"/litellm-asset-prefix/_next/static/chunks/496b84010c33cf69.js","async":true}]
|
||||
10:["$","script","script-44",{"src":"/litellm-asset-prefix/_next/static/chunks/06ebe9b0e9cdf241.js","async":true}]
|
||||
11:["$","script","script-45",{"src":"/litellm-asset-prefix/_next/static/chunks/6c4c97f1ea6e7d77.js","async":true}]
|
||||
12:["$","script","script-46",{"src":"/litellm-asset-prefix/_next/static/chunks/e8718f949e42598e.js","async":true}]
|
||||
13:["$","script","script-47",{"src":"/litellm-asset-prefix/_next/static/chunks/99cf9cf99df5ccfc.js","async":true}]
|
||||
14:["$","script","script-48",{"src":"/litellm-asset-prefix/_next/static/chunks/bb0a6e4a3a18721a.js","async":true}]
|
||||
15:["$","script","script-49",{"src":"/litellm-asset-prefix/_next/static/chunks/4980372eaa37b78b.js","async":true}]
|
||||
16:["$","script","script-50",{"src":"/litellm-asset-prefix/_next/static/chunks/8ab6e2ca95000c8e.js","async":true}]
|
||||
17:["$","script","script-51",{"src":"/litellm-asset-prefix/_next/static/chunks/06cdd9bb80c63794.js","async":true}]
|
||||
18:["$","script","script-52",{"src":"/litellm-asset-prefix/_next/static/chunks/66a190706fc6c35a.js","async":true}]
|
||||
19:["$","script","script-53",{"src":"/litellm-asset-prefix/_next/static/chunks/54e29148cb2f2582.js","async":true}]
|
||||
1a:["$","script","script-54",{"src":"/litellm-asset-prefix/_next/static/chunks/fd9de419c8c0222e.js","async":true}]
|
||||
1b:["$","$L1c",null,{"children":["$","$1d",null,{"name":"Next.MetadataOutlet","children":"$@1e"}]}]
|
||||
1e:null
|
||||
15:["$","script","script-49",{"src":"/litellm-asset-prefix/_next/static/chunks/cfca40b1a4a490bf.js","async":true}]
|
||||
16:["$","script","script-50",{"src":"/litellm-asset-prefix/_next/static/chunks/54e29148cb2f2582.js","async":true}]
|
||||
17:["$","script","script-51",{"src":"/litellm-asset-prefix/_next/static/chunks/37c03dc421ba81f8.js","async":true}]
|
||||
18:["$","$L19",null,{"children":["$","$1a",null,{"name":"Next.MetadataOutlet","children":"$@1b"}]}]
|
||||
1b:null
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
|
|
@ -3,4 +3,4 @@
|
|||
3:I[897367,["/litellm-asset-prefix/_next/static/chunks/d96012bcfc98706a.js","/litellm-asset-prefix/_next/static/chunks/dbca964212122d58.js"],"MetadataBoundary"]
|
||||
4:"$Sreact.suspense"
|
||||
5:I[27201,["/litellm-asset-prefix/_next/static/chunks/d96012bcfc98706a.js","/litellm-asset-prefix/_next/static/chunks/dbca964212122d58.js"],"IconMark"]
|
||||
0:{"buildId":"U_YrOOnSehrpkdU42KJ-W","rsc":["$","$1","h",{"children":[null,["$","$L2",null,{"children":[["$","meta","0",{"charSet":"utf-8"}],["$","meta","1",{"name":"viewport","content":"width=device-width, initial-scale=1"}]]}],["$","div",null,{"hidden":true,"children":["$","$L3",null,{"children":["$","$4",null,{"name":"Next.Metadata","children":[["$","title","0",{"children":"LiteLLM Dashboard"}],["$","meta","1",{"name":"description","content":"LiteLLM Proxy Admin UI"}],["$","link","2",{"rel":"icon","href":"/favicon.ico?favicon.1d32c690.ico","sizes":"48x48","type":"image/x-icon"}],["$","link","3",{"rel":"icon","href":"./favicon.ico"}],["$","$L5","4",{}]]}]}]}],["$","meta",null,{"name":"next-size-adjust","content":""}]]}],"loading":null,"isPartial":false}
|
||||
0:{"buildId":"cbFGTIkRGp63usVNisey9","rsc":["$","$1","h",{"children":[null,["$","$L2",null,{"children":[["$","meta","0",{"charSet":"utf-8"}],["$","meta","1",{"name":"viewport","content":"width=device-width, initial-scale=1"}]]}],["$","div",null,{"hidden":true,"children":["$","$L3",null,{"children":["$","$4",null,{"name":"Next.Metadata","children":[["$","title","0",{"children":"LiteLLM Dashboard"}],["$","meta","1",{"name":"description","content":"LiteLLM Proxy Admin UI"}],["$","link","2",{"rel":"icon","href":"/favicon.ico?favicon.1d32c690.ico","sizes":"48x48","type":"image/x-icon"}],["$","link","3",{"rel":"icon","href":"./favicon.ico"}],["$","$L5","4",{}]]}]}]}],["$","meta",null,{"name":"next-size-adjust","content":""}]]}],"loading":null,"isPartial":false}
|
||||
|
|
|
|||
|
|
@ -1,7 +1,8 @@
|
|||
1:"$Sreact.fragment"
|
||||
2:I[71195,["/litellm-asset-prefix/_next/static/chunks/26adfa4e8ffc85c7.js","/litellm-asset-prefix/_next/static/chunks/e8ed72789c2b42ff.js"],"default"]
|
||||
3:I[339756,["/litellm-asset-prefix/_next/static/chunks/d96012bcfc98706a.js","/litellm-asset-prefix/_next/static/chunks/dbca964212122d58.js"],"default"]
|
||||
4:I[837457,["/litellm-asset-prefix/_next/static/chunks/d96012bcfc98706a.js","/litellm-asset-prefix/_next/static/chunks/dbca964212122d58.js"],"default"]
|
||||
2:I[867271,["/litellm-asset-prefix/_next/static/chunks/a7f104aa2cc7f3f0.js","/litellm-asset-prefix/_next/static/chunks/e8ed72789c2b42ff.js"],"default"]
|
||||
3:I[71195,["/litellm-asset-prefix/_next/static/chunks/a7f104aa2cc7f3f0.js","/litellm-asset-prefix/_next/static/chunks/e8ed72789c2b42ff.js"],"default"]
|
||||
4:I[339756,["/litellm-asset-prefix/_next/static/chunks/d96012bcfc98706a.js","/litellm-asset-prefix/_next/static/chunks/dbca964212122d58.js"],"default"]
|
||||
5:I[837457,["/litellm-asset-prefix/_next/static/chunks/d96012bcfc98706a.js","/litellm-asset-prefix/_next/static/chunks/dbca964212122d58.js"],"default"]
|
||||
:HL["/litellm-asset-prefix/_next/static/chunks/4e20891f2fd03463.css","style"]
|
||||
:HL["/litellm-asset-prefix/_next/static/chunks/6fabf2cec1bd2d6e.css","style"]
|
||||
0:{"buildId":"U_YrOOnSehrpkdU42KJ-W","rsc":["$","$1","c",{"children":[[["$","link","0",{"rel":"stylesheet","href":"/litellm-asset-prefix/_next/static/chunks/4e20891f2fd03463.css","precedence":"next"}],["$","link","1",{"rel":"stylesheet","href":"/litellm-asset-prefix/_next/static/chunks/6fabf2cec1bd2d6e.css","precedence":"next"}],["$","script","script-0",{"src":"/litellm-asset-prefix/_next/static/chunks/26adfa4e8ffc85c7.js","async":true}],["$","script","script-1",{"src":"/litellm-asset-prefix/_next/static/chunks/e8ed72789c2b42ff.js","async":true}]],["$","html",null,{"lang":"en","children":["$","body",null,{"className":"inter_5972bc34-module__OU16Qa__className","children":["$","$L2",null,{"children":["$","$L3",null,{"parallelRouterKey":"children","template":["$","$L4",null,{}],"notFound":[[["$","title",null,{"children":"404: This page could not be found."}],["$","div",null,{"style":{"fontFamily":"system-ui,\"Segoe UI\",Roboto,Helvetica,Arial,sans-serif,\"Apple Color Emoji\",\"Segoe UI Emoji\"","height":"100vh","textAlign":"center","display":"flex","flexDirection":"column","alignItems":"center","justifyContent":"center"},"children":["$","div",null,{"children":[["$","style",null,{"dangerouslySetInnerHTML":{"__html":"body{color:#000;background:#fff;margin:0}.next-error-h1{border-right:1px solid rgba(0,0,0,.3)}@media (prefers-color-scheme:dark){body{color:#fff;background:#000}.next-error-h1{border-right:1px solid rgba(255,255,255,.3)}}"}}],["$","h1",null,{"className":"next-error-h1","style":{"display":"inline-block","margin":"0 20px 0 0","padding":"0 23px 0 0","fontSize":24,"fontWeight":500,"verticalAlign":"top","lineHeight":"49px"},"children":404}],["$","div",null,{"style":{"display":"inline-block"},"children":["$","h2",null,{"style":{"fontSize":14,"fontWeight":400,"lineHeight":"49px","margin":0},"children":"This page could not be found."}]}]]}]}]],[]]}]}]}]}]]}],"loading":null,"isPartial":false}
|
||||
:HL["/litellm-asset-prefix/_next/static/chunks/7936c9bd377ea4bf.css","style"]
|
||||
0:{"buildId":"cbFGTIkRGp63usVNisey9","rsc":["$","$1","c",{"children":[[["$","link","0",{"rel":"stylesheet","href":"/litellm-asset-prefix/_next/static/chunks/4e20891f2fd03463.css","precedence":"next"}],["$","link","1",{"rel":"stylesheet","href":"/litellm-asset-prefix/_next/static/chunks/7936c9bd377ea4bf.css","precedence":"next"}],["$","script","script-0",{"src":"/litellm-asset-prefix/_next/static/chunks/a7f104aa2cc7f3f0.js","async":true}],["$","script","script-1",{"src":"/litellm-asset-prefix/_next/static/chunks/e8ed72789c2b42ff.js","async":true}]],["$","html",null,{"lang":"en","children":["$","body",null,{"className":"inter_5972bc34-module__OU16Qa__className","children":["$","$L2",null,{"children":["$","$L3",null,{"children":["$","$L4",null,{"parallelRouterKey":"children","template":["$","$L5",null,{}],"notFound":[[["$","title",null,{"children":"404: This page could not be found."}],["$","div",null,{"style":{"fontFamily":"system-ui,\"Segoe UI\",Roboto,Helvetica,Arial,sans-serif,\"Apple Color Emoji\",\"Segoe UI Emoji\"","height":"100vh","textAlign":"center","display":"flex","flexDirection":"column","alignItems":"center","justifyContent":"center"},"children":["$","div",null,{"children":[["$","style",null,{"dangerouslySetInnerHTML":{"__html":"body{color:#000;background:#fff;margin:0}.next-error-h1{border-right:1px solid rgba(0,0,0,.3)}@media (prefers-color-scheme:dark){body{color:#fff;background:#000}.next-error-h1{border-right:1px solid rgba(255,255,255,.3)}}"}}],["$","h1",null,{"className":"next-error-h1","style":{"display":"inline-block","margin":"0 20px 0 0","padding":"0 23px 0 0","fontSize":24,"fontWeight":500,"verticalAlign":"top","lineHeight":"49px"},"children":404}],["$","div",null,{"style":{"display":"inline-block"},"children":["$","h2",null,{"style":{"fontSize":14,"fontWeight":400,"lineHeight":"49px","margin":0},"children":"This page could not be found."}]}]]}]}]],[]]}]}]}]}]}]]}],"loading":null,"isPartial":false}
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
:HL["/litellm-asset-prefix/_next/static/chunks/4e20891f2fd03463.css","style"]
|
||||
:HL["/litellm-asset-prefix/_next/static/chunks/6fabf2cec1bd2d6e.css","style"]
|
||||
:HL["/litellm-asset-prefix/_next/static/chunks/7936c9bd377ea4bf.css","style"]
|
||||
:HL["/litellm-asset-prefix/_next/static/media/83afe278b6a6bb3c-s.p.3a6ba036.woff2","font",{"crossOrigin":"","type":"font/woff2"}]
|
||||
:HL["/litellm-asset-prefix/_next/static/chunks/3f3fa56b5786d58c.css","style"]
|
||||
0:{"buildId":"U_YrOOnSehrpkdU42KJ-W","tree":{"name":"","paramType":null,"paramKey":"","hasRuntimePrefetch":false,"slots":{"children":{"name":"__PAGE__","paramType":null,"paramKey":"__PAGE__","hasRuntimePrefetch":false,"slots":null,"isRootLayout":false}},"isRootLayout":true},"staleTime":300}
|
||||
0:{"buildId":"cbFGTIkRGp63usVNisey9","tree":{"name":"","paramType":null,"paramKey":"","hasRuntimePrefetch":false,"slots":{"children":{"name":"__PAGE__","paramType":null,"paramKey":"__PAGE__","hasRuntimePrefetch":false,"slots":null,"isRootLayout":false}},"isRootLayout":true},"staleTime":300}
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
|
@ -0,0 +1 @@
|
|||
(globalThis.TURBOPACK||(globalThis.TURBOPACK=[])).push(["object"==typeof document?document.currentScript:void 0,233525,(e,c,u)=>{"use strict";Object.defineProperty(u,"__esModule",{value:!0}),Object.defineProperty(u,"warnOnce",{enumerable:!0,get:function(){return r}});let r=e=>{}},349356,e=>{e.v({AElig:"Æ",AMP:"&",Aacute:"Á",Acirc:"Â",Agrave:"À",Aring:"Å",Atilde:"Ã",Auml:"Ä",COPY:"©",Ccedil:"Ç",ETH:"Ð",Eacute:"É",Ecirc:"Ê",Egrave:"È",Euml:"Ë",GT:">",Iacute:"Í",Icirc:"Î",Igrave:"Ì",Iuml:"Ï",LT:"<",Ntilde:"Ñ",Oacute:"Ó",Ocirc:"Ô",Ograve:"Ò",Oslash:"Ø",Otilde:"Õ",Ouml:"Ö",QUOT:'"',REG:"®",THORN:"Þ",Uacute:"Ú",Ucirc:"Û",Ugrave:"Ù",Uuml:"Ü",Yacute:"Ý",aacute:"á",acirc:"â",acute:"´",aelig:"æ",agrave:"à",amp:"&",aring:"å",atilde:"ã",auml:"ä",brvbar:"¦",ccedil:"ç",cedil:"¸",cent:"¢",copy:"©",curren:"¤",deg:"°",divide:"÷",eacute:"é",ecirc:"ê",egrave:"è",eth:"ð",euml:"ë",frac12:"½",frac14:"¼",frac34:"¾",gt:">",iacute:"í",icirc:"î",iexcl:"¡",igrave:"ì",iquest:"¿",iuml:"ï",laquo:"«",lt:"<",macr:"¯",micro:"µ",middot:"·",nbsp:" ",not:"¬",ntilde:"ñ",oacute:"ó",ocirc:"ô",ograve:"ò",ordf:"ª",ordm:"º",oslash:"ø",otilde:"õ",ouml:"ö",para:"¶",plusmn:"±",pound:"£",quot:'"',raquo:"»",reg:"®",sect:"§",shy:"",sup1:"¹",sup2:"²",sup3:"³",szlig:"ß",thorn:"þ",times:"×",uacute:"ú",ucirc:"û",ugrave:"ù",uml:"¨",uuml:"ü",yacute:"ý",yen:"¥",yuml:"ÿ"})},137429,e=>{e.v({0:"<22>",128:"€",130:"‚",131:"ƒ",132:"„",133:"…",134:"†",135:"‡",136:"ˆ",137:"‰",138:"Š",139:"‹",140:"Œ",142:"Ž",145:"‘",146:"’",147:"“",148:"”",149:"•",150:"–",151:"—",152:"˜",153:"™",154:"š",155:"›",156:"œ",158:"ž",159:"Ÿ"})},928685,e=>{"use strict";var c=e.i(38953);e.s(["SearchOutlined",()=>c.default])},86408,e=>{"use strict";var c=e.i(843476),u=e.i(271645),r=e.i(618566),t=e.i(934879);function a(){let e=(0,r.useSearchParams)().get("key"),[a,i]=(0,u.useState)(null);return console.log("PublicModelHubTable accessToken:",a),(0,u.useEffect)(()=>{e&&i(e)},[e]),(0,c.jsx)(t.default,{accessToken:a,publicPage:!0,premiumUser:!1,userRole:null})}function i(){return(0,c.jsx)(u.Suspense,{fallback:(0,c.jsx)("div",{className:"flex items-center justify-center min-h-screen",children:"Loading..."}),children:(0,c.jsx)(a,{})})}e.s(["default",()=>i])}]);
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
|
@ -1 +0,0 @@
|
|||
(globalThis.TURBOPACK||(globalThis.TURBOPACK=[])).push(["object"==typeof document?document.currentScript:void 0,56456,e=>{"use strict";var s=e.i(739295);e.s(["LoadingOutlined",()=>s.default])},566606,e=>{"use strict";var s=e.i(843476),i=e.i(271645),r=e.i(618566),t=e.i(317751),a=e.i(912598),n=e.i(947293),l=e.i(764205),o=e.i(954616),d=e.i(266027),u=e.i(612256);let c=(0,e.i(243652).createQueryKeys)("onboarding");var m=e.i(482725),p=e.i(56456);function x(){return(0,s.jsx)("div",{className:"mx-auto w-full max-w-md mt-10 flex justify-center",children:(0,s.jsx)(m.Spin,{indicator:(0,s.jsx)(p.LoadingOutlined,{spin:!0}),size:"large"})})}var h=e.i(560445),g=e.i(464571);function j(){return(0,s.jsxs)("div",{className:"mx-auto w-full max-w-md mt-10",children:[(0,s.jsx)(h.Alert,{type:"error",message:"Failed to load invitation",description:"The invitation link may be invalid or expired.",showIcon:!0}),(0,s.jsx)("div",{className:"mt-4",children:(0,s.jsx)(g.Button,{href:"/ui/login",children:"Back to Login"})})]})}var y=e.i(175712),w=e.i(808613),f=e.i(311451),v=e.i(898586);function b({variant:e,userEmail:r,isPending:t,claimError:a,onSubmit:n}){let[l]=w.Form.useForm();return i.default.useEffect(()=>{r&&l.setFieldValue("user_email",r)},[r,l]),(0,s.jsx)("div",{className:"mx-auto w-full max-w-md mt-10",children:(0,s.jsxs)(y.Card,{children:[(0,s.jsx)(v.Typography.Title,{level:5,className:"text-center mb-5",children:"🚅 LiteLLM"}),(0,s.jsx)(v.Typography.Title,{level:3,children:"reset_password"===e?"Reset Password":"Sign Up"}),(0,s.jsx)(v.Typography.Text,{children:"reset_password"===e?"Reset your password to access Admin UI.":"Claim your user account to login to Admin UI."}),"signup"===e&&(0,s.jsx)(h.Alert,{className:"mt-4",type:"info",message:"SSO",description:(0,s.jsxs)("div",{className:"flex justify-between items-center",children:[(0,s.jsx)("span",{children:"SSO is under the Enterprise Tier."}),(0,s.jsx)(g.Button,{type:"primary",size:"small",href:"https://forms.gle/W3U4PZpJGFHWtHyA9",target:"_blank",children:"Get Free Trial"})]}),showIcon:!0}),(0,s.jsxs)(w.Form,{className:"mt-10 mb-5",layout:"vertical",form:l,onFinish:e=>n({password:e.password}),children:[(0,s.jsx)(w.Form.Item,{label:"Email Address",name:"user_email",children:(0,s.jsx)(f.Input,{type:"email",disabled:!0})}),(0,s.jsx)(w.Form.Item,{label:"Password",name:"password",rules:[{required:!0,message:"password required to sign up"}],help:"reset_password"===e?"Enter your new password":"Create a password for your account",children:(0,s.jsx)(f.Input.Password,{})}),a&&(0,s.jsx)(h.Alert,{type:"error",message:a,showIcon:!0,className:"mb-4"}),(0,s.jsx)("div",{className:"mt-10",children:(0,s.jsx)(g.Button,{htmlType:"submit",loading:t,children:"reset_password"===e?"Reset Password":"Sign Up"})})]})]})})}function S({variant:e}){let t=(0,r.useSearchParams)().get("invitation_id"),[a,m]=i.default.useState(null),{data:p,isLoading:h,isError:g}=(e=>{let{isLoading:s}=(0,u.useUIConfig)();return(0,d.useQuery)({queryKey:c.detail(e??""),queryFn:async()=>{if(!e)throw Error("inviteId is required");return(0,l.getOnboardingCredentials)(e)},enabled:!!e&&!s})})(t),{mutate:y,isPending:w}=(0,o.useMutation)({mutationFn:async({accessToken:e,inviteId:s,userId:i,password:r})=>await (0,l.claimOnboardingToken)(e,s,i,r)}),f=p?.token?(0,n.jwtDecode)(p.token):null,v=f?.user_email??"",S=f?.user_id??null,T=f?.key??null,F=p?.token??null;return h?(0,s.jsx)(x,{}):g?(0,s.jsx)(j,{}):(0,s.jsx)(b,{variant:e,userEmail:v,isPending:w,claimError:a,onSubmit:e=>{T&&F&&S&&t&&(m(null),y({accessToken:T,inviteId:t,userId:S,password:e.password},{onSuccess:()=>{document.cookie=`token=${F}; path=/; SameSite=Lax`;let e=(0,l.getProxyBaseUrl)();window.location.href=e?`${e}/ui/?login=success`:"/ui/?login=success"},onError:e=>{m(e.message||"Failed to submit. Please try again.")}}))}})}let T=new t.QueryClient;function F(){let e=(0,r.useSearchParams)().get("action");return(0,s.jsx)(S,{variant:"reset_password"===e?"reset_password":"signup"})}function P(){return(0,s.jsx)(a.QueryClientProvider,{client:T,children:(0,s.jsx)(i.Suspense,{fallback:(0,s.jsx)("div",{className:"flex items-center justify-center min-h-screen",children:"Loading..."}),children:(0,s.jsx)(F,{})})})}e.s(["default",()=>P],566606)}]);
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
|
@ -1 +0,0 @@
|
|||
(globalThis.TURBOPACK||(globalThis.TURBOPACK=[])).push(["object"==typeof document?document.currentScript:void 0,949616,t=>{"use strict";function r(t,r){(null==r||r>t.length)&&(r=t.length);for(var e=0,n=Array(r);e<r;e++)n[e]=t[e];return n}t.s(["default",()=>r])},713882,t=>{"use strict";var r=t.i(949616);function e(t,e){if(t){if("string"==typeof t)return(0,r.default)(t,e);var n=({}).toString.call(t).slice(8,-1);return"Object"===n&&t.constructor&&(n=t.constructor.name),"Map"===n||"Set"===n?Array.from(t):"Arguments"===n||/^(?:Ui|I)nt(?:8|16|32)(?:Clamped)?Array$/.test(n)?(0,r.default)(t,e):void 0}}t.s(["default",()=>e])},410160,t=>{"use strict";function r(t){return(r="function"==typeof Symbol&&"symbol"==typeof Symbol.iterator?function(t){return typeof t}:function(t){return t&&"function"==typeof Symbol&&t.constructor===Symbol&&t!==Symbol.prototype?"symbol":typeof t})(t)}t.s(["default",()=>r])},211577,394257,t=>{"use strict";var r=t.i(410160);function e(t){var e=function(t,e){if("object"!=(0,r.default)(t)||!t)return t;var n=t[Symbol.toPrimitive];if(void 0!==n){var i=n.call(t,e||"default");if("object"!=(0,r.default)(i))return i;throw TypeError("@@toPrimitive must return a primitive value.")}return("string"===e?String:Number)(t)}(t,"string");return"symbol"==(0,r.default)(e)?e:e+""}function n(t,r,n){return(r=e(r))in t?Object.defineProperty(t,r,{value:n,enumerable:!0,configurable:!0,writable:!0}):t[r]=n,t}t.s(["default",()=>e],394257),t.s(["default",()=>n],211577)},308665,962837,t=>{"use strict";var r=t.i(949616);function e(t){if(Array.isArray(t))return(0,r.default)(t)}function n(t){if("u">typeof Symbol&&null!=t[Symbol.iterator]||null!=t["@@iterator"])return Array.from(t)}t.s(["default",()=>e],308665),t.s(["default",()=>n],962837)},8211,t=>{"use strict";var r=t.i(308665),e=t.i(962837),n=t.i(713882);function i(t){return(0,r.default)(t)||(0,e.default)(t)||(0,n.default)(t)||function(){throw TypeError("Invalid attempt to spread non-iterable instance.\nIn order to be iterable, non-array objects must have a [Symbol.iterator]() method.")}()}t.s(["default",()=>i],8211)},915874,t=>{"use strict";function r(t,r){if(null==t)return{};var e={};for(var n in t)if(({}).hasOwnProperty.call(t,n)){if(-1!==r.indexOf(n))continue;e[n]=t[n]}return e}t.s(["default",()=>r])},703923,t=>{"use strict";var r=t.i(915874);function e(t,e){if(null==t)return{};var n,i,u=(0,r.default)(t,e);if(Object.getOwnPropertySymbols){var o=Object.getOwnPropertySymbols(t);for(i=0;i<o.length;i++)n=o[i],-1===e.indexOf(n)&&({}).propertyIsEnumerable.call(t,n)&&(u[n]=t[n])}return u}t.s(["default",()=>e])},931067,t=>{"use strict";function r(){return(r=Object.assign.bind()).apply(null,arguments)}t.s(["default",()=>r])},71195,t=>{"use strict";var r=t.i(843476),e=t.i(271645),n=t.i(698173),i=t.i(727749);function u({children:t}){let[u,o]=n.notification.useNotification(),a=(0,e.useRef)(!1);return(0,e.useEffect)(()=>{a.current||((0,i.setNotificationInstance)(u),a.current=!0)},[u]),(0,r.jsxs)(r.Fragment,{children:[o,t]})}t.s(["default",()=>u])}]);
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
Some files were not shown because too many files have changed in this diff Show more
Loading…
Add table
Reference in a new issue