litellm/litellm/proxy/client

LiteLLM Proxy Client

A Python client library for interacting with the LiteLLM proxy server. This client provides a clean, typed interface for managing models, keys, credentials, and making chat completions.

Installation

uv add litellm

Quick Start

from litellm.proxy.client import Client

# Initialize the client
client = Client(
    base_url="http://localhost:4000",  # Your LiteLLM proxy server URL
    api_key="sk-api-key"               # Optional: API key for authentication
)

# Make a chat completion request
response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[
        {"role": "user", "content": "Hello, how are you?"}
    ]
)
print(response.choices[0].message.content)

Features

The client is organized into several resource clients for different functionality:

• chat: Chat completions
• models: Model management
• model_groups: Model group management
• keys: API key management
• credentials: Credential management
• users: User management

Chat Completions

Make chat completion requests to your LiteLLM proxy:

# Basic chat completion
response = client.chat.completions.create(
    model="gpt-4",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What's the capital of France?"}
    ]
)

# Stream responses
for chunk in client.chat.completions.create(
    model="gpt-4",
    messages=[{"role": "user", "content": "Tell me a story"}],
    stream=True
):
    print(chunk.choices[0].delta.content or "", end="")

Model Management

Manage available models on your proxy:

# List available models
models = client.models.list()

# Add a new model
client.models.add(
    model_name="gpt-4",
    litellm_params={
        "api_key": "your-openai-key",
        "api_base": "https://api.openai.com/v1"
    }
)

# Delete a model
client.models.delete(model_name="gpt-4")

API Key Management

Manage virtual API keys:

# Generate a new API key
key = client.keys.generate(
    models=["gpt-4", "gpt-3.5-turbo"],
    aliases={"gpt4": "gpt-4"},
    duration="24h",
    key_alias="my-key",
    team_id="team123"
)

# List all keys
keys = client.keys.list(
    page=1,
    size=10,
    return_full_object=True
)

# Delete keys
client.keys.delete(
    keys=["sk-key1", "sk-key2"],
    key_aliases=["alias1", "alias2"]
)

Credential Management

Manage model credentials:

# Create new credentials
client.credentials.create(
    credential_name="azure1",
    credential_info={"api_type": "azure"},
    credential_values={
        "api_key": "your-azure-key",
        "api_base": "https://example.azure.openai.com"
    }
)

# List all credentials
credentials = client.credentials.list()

# Get a specific credential
credential = client.credentials.get(credential_name="azure1")

# Delete credentials
client.credentials.delete(credential_name="azure1")

Model Groups

Manage model groups for load balancing and fallbacks:

# Create a model group
client.model_groups.create(
    name="gpt4-group",
    models=[
        {"model_name": "gpt-4", "litellm_params": {"api_key": "key1"}},
        {"model_name": "gpt-4-backup", "litellm_params": {"api_key": "key2"}}
    ]
)

# List model groups
groups = client.model_groups.list()

# Delete a model group
client.model_groups.delete(name="gpt4-group")

Users Management

Manage users on your proxy:

from litellm.proxy.client import UsersManagementClient

users = UsersManagementClient(base_url="http://localhost:4000", api_key="sk-test")

# List users
user_list = users.list_users()

# Get user info
user_info = users.get_user(user_id="u1")

# Create a new user
created = users.create_user({
    "user_email": "a@b.com",
    "user_role": "internal_user",
    "user_alias": "Alice",
    "teams": ["team1"],
    "max_budget": 100.0
})

# Delete users
users.delete_user(["u1", "u2"])

Low-Level HTTP Client

The client provides access to a low-level HTTP client for making direct requests to the LiteLLM proxy server. This is useful when you need more control or when working with endpoints that don't yet have a high-level interface.

# Access the HTTP client
client = Client(
    base_url="http://localhost:4000",
    api_key="sk-api-key"
)

# Make a custom request
response = client.http.request(
    method="POST",
    uri="/health/test_connection",
    json={
        "litellm_params": {
            "model": "gpt-4",
            "api_key": "your-api-key",
            "api_base": "https://api.openai.com/v1"
        },
        "mode": "chat"
    }
)

# The response is automatically parsed from JSON
print(response)

HTTP Client Features

• Automatic URL handling (handles trailing/leading slashes)
• Built-in authentication (adds Bearer token if api_key is provided)
• JSON request/response handling
• Configurable timeout (default: 30 seconds)
• Comprehensive error handling
• Support for custom headers and request parameters

HTTP Client request method parameters

• method: HTTP method (GET, POST, PUT, DELETE, etc.)
• uri: URI path (will be appended to base_url)
• data: (optional) Data to send in the request body
• json: (optional) JSON data to send in the request body
• headers: (optional) Custom HTTP headers
• Additional keyword arguments are passed to the underlying requests library

Error Handling

The client provides clear error handling with custom exceptions:

from litellm.proxy.client.exceptions import UnauthorizedError

try:
    response = client.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": "Hello"}]
    )
except UnauthorizedError as e:
    print("Authentication failed:", e)
except Exception as e:
    print("Request failed:", e)

Advanced Usage

Request Customization

All methods support returning the raw request object for inspection or modification:

# Get the prepared request without sending it
request = client.models.list(return_request=True)
print(request.method)  # GET
print(request.url)     # http://localhost:8000/models
print(request.headers) # {'Content-Type': 'application/json', ...}

Pagination

Methods that return lists support pagination:

# Get the first page of keys
page1 = client.keys.list(page=1, size=10)

# Get the second page
page2 = client.keys.list(page=2, size=10)

Filtering

Many list methods support filtering:

# Filter keys by user and team
keys = client.keys.list(
    user_id="user123",
    team_id="team456",
    include_team_keys=True
)

Contributing

Contributions are welcome! Please check out our contributing guidelines for details.

License

This project is licensed under the MIT License - see the LICENSE file for details.

CLI Authentication Flow

The LiteLLM CLI supports SSO authentication through a polling-based approach that works with any OAuth-compatible SSO provider.

How CLI Authentication Works

sequenceDiagram
    participant CLI as CLI
    participant Browser as Browser
    participant Proxy as LiteLLM Proxy
    participant SSO as SSO Provider
    
    CLI->>Proxy: POST /sso/cli/start
    Proxy->>CLI: Return login_id, poll_secret, user_code
    CLI->>Browser: Open /sso/key/generate?source=litellm-cli&key=login_id
    
    Browser->>Proxy: GET /sso/key/generate?source=litellm-cli&key=login_id
    Proxy->>Proxy: Set cli_state = litellm-session-token:login_id
    Proxy->>SSO: Redirect with state=litellm-session-token:login_id
    
    SSO->>Browser: Show login page
    Browser->>SSO: User authenticates
    SSO->>Proxy: Redirect to /sso/callback?state=litellm-session-token:login_id
    
    Proxy->>Proxy: Check if state starts with "litellm-session-token:"
    Proxy->>Browser: Prompt for user_code
    Browser->>Proxy: POST /sso/cli/complete/login_id
    
    CLI->>Proxy: Poll /sso/cli/poll/login_id with poll_secret header
    Proxy->>CLI: Return {"status": "ready", "key": "jwt"}
    CLI->>CLI: Save the secret to the OS keychain (metadata to ~/.litellm/token.json)

Authentication Commands

The CLI provides these authentication commands:

• lite login - Start SSO authentication flow
• lite login --pkce - Sign in through the system browser with OAuth authorization code + PKCE; the key renews itself with a refresh token
• lite logout - Clear stored authentication token (and revoke a --pkce refresh token on the proxy)
• lite whoami - Show current authentication status
• lite auth print-token - Print the cached token (used as Claude Code's apiKeyHelper); renews a --pkce key first and fails once a classic token has expired

Authentication Flow Steps

1. Start Session: CLI creates a short-lived login session with /sso/cli/start
2. Open Browser: CLI opens browser to /sso/key/generate with CLI source and login ID parameters
3. SSO Redirect: Proxy sets the formatted state (litellm-session-token:{login_id}) as OAuth state parameter and redirects to SSO provider
4. User Authentication: User completes SSO authentication in browser
5. Callback Processing: SSO provider redirects back to proxy with state parameter
6. User Code Verification: Browser confirms the verification code shown in the CLI
7. Polling: CLI polls /sso/cli/poll/{login_id} with the polling secret header until the JWT is ready. When CLI_SSO_CLAIM_MAP is configured on the proxy, the poll response may include attribution_metadata (allowlisted scalar OIDC claims for client attribution).
8. Token Storage: CLI saves the key to the OS keychain and the non-secret session metadata to ~/.litellm/token.json

Benefits of This Approach

• No Local Server: No need to run a local callback server
• Standard OAuth: Uses OAuth 2.0 state parameter correctly
• Remote Compatible: Works with remote proxy servers
• Secure: Keeps the polling secret out of the browser handoff
• Simple Setup: No additional OAuth redirect URL configuration needed

Token Storage

The key itself, together with the refresh token that renews a --pkce credential, goes into the OS keychain (macOS Keychain, Windows Credential Manager, or the Linux Secret Service) under service litellm-cli, account credential. Only the non-secret session metadata is written to ~/.litellm/token.json, in a 0700 directory with 0600 file permissions:

{
  "base_url": "https://your-proxy.com",
  "user_id": "cli-user",
  "user_email": "user@example.com",
  "user_role": "cli",
  "auth_header_name": "Authorization",
  "timestamp": 1234567890
}

Keychain storage needs the keyring package, which ships with pip install 'litellm[cli]'. Headless boxes and CI runners usually have no keychain either. In all of those cases the key and the refresh token stay in the same 0600 file alongside the metadata, exactly as they did before, and lite login names which one applies: the package is missing, the machine has no keychain, or you set LITELLM_CLI_DISABLE_KEYRING=1 to force the file even where a keychain exists. A token.json written by an older lite keeps working and is moved into the keychain, and scrubbed from the file, the first time a keychain-capable lite reads it. That includes a refresh token left behind by the release that moved only the key.

lite logout clears both stores. If the keychain is locked at that moment it says so, and re-running it once the keychain is unlocked finishes the job.

The stored credential is a short-lived, per-session agent token, not a managed virtual key. It is scoped to the user and team you logged in as and inherits their models and budgets; spend is tracked against the shared team and user budgets rather than a separate per-session cap, so multiple logins or several concurrent agents all draw down the same allowance. It is short-lived by design (default 24h, configurable via LITELLM_CLI_JWT_EXPIRATION_HOURS); re-run lite login to refresh it and pick up your latest team and user settings. lite auth print-token (usable as Claude Code's apiKeyHelper) prints it while fresh and fails once it expires -- there is no silent renewal. It is accepted on a default deployment without EXPERIMENTAL_UI_LOGIN, does not appear in the Keys UI, and cannot be rotated or revoked mid-session. A credential from lite login --pkce is the exception: it carries a refresh token, so the CLI renews the key shortly before it expires and lite logout revokes the refresh token on the proxy (see Browser sign-in with PKCE). Only the holder can end a --pkce session early, with lite logout; an admin has no button for it, but every renewal re-reads the user on the proxy, so deactivating the user or removing them from the team makes the next renewal fail and the key runs out within LITELLM_CLI_JWT_EXPIRATION_HOURS. On a proxy with more than one worker or replica, configure Redis (litellm_settings.cache with Redis cache_params, or general_settings.coordination_redis) so a refresh token stays single-use and lite logout holds on every worker; without Redis each worker keeps its own record. For a long-lived, rotatable, Keys-UI-visible credential, create a dedicated virtual key in the dashboard and pass it via --api-key or LITELLM_PROXY_API_KEY.

Usage

Once authenticated, the CLI will automatically use the stored token for all requests. You no longer need to specify --api-key for subsequent commands.

# Login
lite login

# Use CLI without specifying API key
lite models list

# Check authentication status
lite whoami

# Logout
lite logout