mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-29 06:33:28 +00:00
feat(chatgpt): forward client subscriptions through the gateway (#41522)
* feat(chatgpt): forward each Codex user's ChatGPT OAuth login through the proxy Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(router): cover _model_group_llm_provider metadata resolution Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(chatgpt): keep the gateway ChatGPT login bound to the authenticator api_base Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(chatgpt): move client credential header selection into the chatgpt provider Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(chatgpt): cover credential recognition and stored account id fallbacks Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(chatgpt): support per-model chatgpt_auth_file for multiple ChatGPT accounts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(chatgpt): resolve bare chatgpt_auth_file paths and drop redundant test docstring Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(chatgpt): pass per-model auth file to early provider detection Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(chatgpt): normalize chatgpt_auth_file before caching the authenticator Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(chatgpt): type account selection parameters * fix(chatgpt): preserve account selection through saved prompts * fix(chatgpt): allow administrator account configuration through the proxy * test(chatgpt): construct immutable request parameters * fix(router): preserve typed deployment credential resolution * fix(chatgpt): allow admins to probe unsaved account configurations * fix(chatgpt): retain SDK login headers during provider resolution * fix(chatgpt): allow admins to probe unsaved account configurations * fix(chatgpt): normalize subscription health probe input * fix(chatgpt): normalize health probe input and cover credential destinations * test(chatgpt): keep destination rejection assertions focused * feat(proxy): forward client ChatGPT subscriptions on native Codex routes * docs: lead ChatGPT forwarding with standard model setup * fix(chatgpt): preserve selected credentials in health provider lookup * fix(chatgpt): preserve gateway credentials and workspace selection * refactor(chatgpt): avoid rebinding streaming metadata inputs * fix(chatgpt): keep account lookup token argument immutable * fix(chatgpt): scope forwarded credential routing to ChatGPT * refactor: keep forwarded credential isolation opt-in by provider --------- Co-authored-by: Moe Khalil <moe@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Urbanonymous <ugutitej@gmail.com> Co-authored-by: mateo <mateo@berri.ai>
This commit is contained in:
devin-ai-integration[bot]
GitHub
parent
4da24fa01e
commit
2e27a02a9f
43 changed files with 2547 additions and 150 deletions
|
|
@ -0,0 +1,76 @@
|
|||
# Use your ChatGPT subscription through LiteLLM
|
||||
|
||||
Sign into ChatGPT in Codex or Pi and send model requests through your existing LiteLLM proxy. The client keeps your login and refreshes it. Each request uses that client's ChatGPT account, while a separate LiteLLM virtual key controls gateway access and usage attribution.
|
||||
|
||||
An administrator adds a `chatgpt/` model using the usual model configuration. No ChatGPT login needs to be added to the proxy. This uses the same separate-key convention as provider passthrough, including Claude Code subscription forwarding.
|
||||
|
||||
## Add a model to the proxy
|
||||
|
||||
Reuse your existing proxy. If you need one, follow the [proxy quickstart](https://docs.litellm.ai/docs/proxy/docker_quick_start). Merge this entry into its model list and restart the proxy:
|
||||
|
||||
```yaml
|
||||
model_list:
|
||||
- model_name: gpt-6-luna
|
||||
litellm_params:
|
||||
model: chatgpt/gpt-6-luna
|
||||
model_info:
|
||||
mode: responses
|
||||
```
|
||||
|
||||
Use a model available to the caller's subscription. The virtual key must allow the exposed model name. Leave saved ChatGPT logins and API credentials unset for a client-only deployment. Explicit cross-provider fallbacks can incur separate API charges; do not configure them for subscription-only traffic.
|
||||
|
||||
## Configure Codex
|
||||
|
||||
Use your existing LiteLLM URL and a virtual key assigned to your gateway user. Export that key as `LITELLM_API_KEY` in the environment that launches Codex. Keep the value out of committed configuration.
|
||||
|
||||
Add this provider to `~/.codex/config.toml`, preserving your other settings:
|
||||
|
||||
```toml
|
||||
model = "gpt-6-luna"
|
||||
model_provider = "litellm_subscription"
|
||||
|
||||
[model_providers.litellm_subscription]
|
||||
name = "LiteLLM subscription"
|
||||
base_url = "https://litellm.example.com/v1"
|
||||
wire_api = "responses"
|
||||
supports_websockets = false
|
||||
requires_openai_auth = true
|
||||
env_http_headers = { "x-litellm-api-key" = "LITELLM_API_KEY" }
|
||||
```
|
||||
|
||||
Replace `https://litellm.example.com` with your proxy URL. A local proxy normally uses `http://localhost:4000/v1`. Launch Codex, sign into ChatGPT if needed, and send a prompt with a model available to your subscription. The selected model must match the name exposed by the gateway.
|
||||
|
||||
For a managed workspace rollout, use OpenAI's [gateway configuration guide](https://learn.chatgpt.com/docs/enterprise/sign-in-with-chatgpt-through-a-gateway).
|
||||
|
||||
## Native compatibility and Pi
|
||||
|
||||
For a native Codex model catalog and request format, change Codex's `base_url` to `https://litellm.example.com/backend-api/codex`. This route uses the same credentials but does not need a model deployment or resolve model aliases. Restricted keys must allow the upstream model name.
|
||||
|
||||
### Configure Pi
|
||||
|
||||
Sign into the built-in `openai-codex` provider in Pi. Merge this override into `~/.pi/agent/models.json` and export your LiteLLM virtual key as `LITELLM_API_KEY` before launching Pi:
|
||||
|
||||
```json
|
||||
{
|
||||
"providers": {
|
||||
"openai-codex": {
|
||||
"baseUrl": "https://litellm.example.com/backend-api/codex",
|
||||
"headers": {
|
||||
"x-litellm-api-key": "LITELLM_API_KEY"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Keep Pi's subscription login as the provider credential. The header value names an environment variable, not a key to paste into the file.
|
||||
|
||||
## Verify account usage
|
||||
|
||||
Send a prompt and check the completed request in LiteLLM's Logs page. Its gateway user/key and token counts should match the caller. Give each person their own LiteLLM virtual key for separate attribution. The gateway credential and ChatGPT login are independently authenticated; LiteLLM does not establish that both identities belong to the same person.
|
||||
|
||||
The native routes are `GET /backend-api/codex/models` and `POST /backend-api/codex/responses`. Model requests use the upstream model name, so a restricted LiteLLM key must allow that name. OpenAI's model catalog is returned in its native format; seeing a model there does not override gateway permissions.
|
||||
|
||||
The proxy forwards the ChatGPT bearer and account header to the fixed OpenAI Codex endpoint. It removes gateway credentials and cookies from the upstream request and redacts credential headers in logging callbacks. The client remains responsible for refresh and reauthentication. Native passthrough returns authentication and quota errors without switching subscriptions. Standard model routes retain the caller login across ChatGPT retries; explicitly configured cross-provider fallbacks use their own credentials. An explicit workspace header is preserved for OpenAI to validate.
|
||||
|
||||
This flow does not connect accounts through the dashboard or pool subscription allowances. Use a Codex build that supports ChatGPT authentication with a custom model provider. The integration forwards HTTP and SSE requests; it does not add native WebSocket or other Codex backend routes.
|
||||
|
|
@ -76,6 +76,7 @@ OPTIONAL_KWARGS_KEYS: Final = (
|
|||
"itpm",
|
||||
"otpm",
|
||||
"use_xai_oauth",
|
||||
"chatgpt_auth_file",
|
||||
"github_copilot_auth_type",
|
||||
"github_copilot_host",
|
||||
"github_copilot_user_session",
|
||||
|
|
|
|||
|
|
@ -158,6 +158,14 @@ def declared_authenticating_provider(model: str | None, custom_llm_provider: str
|
|||
return declared if declared in PROVIDERS_THAT_AUTHENTICATE_ON_PROVIDER_INFO else None
|
||||
|
||||
|
||||
def resolve_model_and_provider_for_metadata(model: str, custom_llm_provider: str | None = None) -> tuple[str, str]:
|
||||
declared: Final = declared_authenticating_provider(model, custom_llm_provider)
|
||||
if declared is not None:
|
||||
return model.removeprefix(f"{declared}/"), declared
|
||||
resolved_model, resolved_provider, _, _ = get_llm_provider(model=model, custom_llm_provider=custom_llm_provider)
|
||||
return resolved_model, resolved_provider
|
||||
|
||||
|
||||
def inferred_provider(model: str | None) -> str | None:
|
||||
if not model:
|
||||
return None
|
||||
|
|
@ -842,7 +850,9 @@ def _get_openai_compatible_provider_info(
|
|||
api_base,
|
||||
dynamic_api_key,
|
||||
custom_llm_provider,
|
||||
) = litellm.ChatGPTConfig().get_openai_compatible_provider_info(model, api_base, api_key, custom_llm_provider)
|
||||
) = litellm.ChatGPTConfig().get_openai_compatible_provider_info(
|
||||
model, api_base, api_key, custom_llm_provider, litellm_params
|
||||
)
|
||||
elif custom_llm_provider == "novita":
|
||||
api_base = api_base or get_secret("NOVITA_API_BASE") or "https://api.novita.ai/v3/openai"
|
||||
dynamic_api_key = api_key or get_secret_str("NOVITA_API_KEY")
|
||||
|
|
|
|||
|
|
@ -1,10 +1,23 @@
|
|||
from collections.abc import Mapping, Sequence
|
||||
from typing import Final
|
||||
|
||||
from pydantic import TypeAdapter
|
||||
|
||||
from litellm.types.utils import ProviderSpecificHeader
|
||||
|
||||
_PROVIDER_HEADERS: Final[TypeAdapter[ProviderSpecificHeader | Sequence[ProviderSpecificHeader] | None]] = TypeAdapter(
|
||||
ProviderSpecificHeader | Sequence[ProviderSpecificHeader] | None
|
||||
)
|
||||
|
||||
|
||||
class ProviderSpecificHeaderUtils:
|
||||
@staticmethod
|
||||
def forwarded_authorization(provider_specific_header: object, custom_llm_provider: str | None) -> str | None:
|
||||
headers: Final = ProviderSpecificHeaderUtils.get_provider_specific_headers(
|
||||
_PROVIDER_HEADERS.validate_python(provider_specific_header), custom_llm_provider
|
||||
)
|
||||
return next((value for key, value in headers.items() if key.lower() == "authorization"), None)
|
||||
|
||||
@staticmethod
|
||||
def get_provider_specific_headers(
|
||||
provider_specific_header: ProviderSpecificHeader | Sequence[ProviderSpecificHeader] | None,
|
||||
|
|
|
|||
|
|
@ -1,9 +1,9 @@
|
|||
import base64
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
from collections.abc import Mapping
|
||||
from typing import Final, TypeAlias
|
||||
from functools import lru_cache
|
||||
from typing import TYPE_CHECKING, Final
|
||||
|
||||
import httpx
|
||||
from pydantic import JsonValue, TypeAdapter, ValidationError
|
||||
|
|
@ -24,19 +24,21 @@ from .common_utils import (
|
|||
CHATGPT_OAUTH_TOKEN_URL,
|
||||
GetAccessTokenError,
|
||||
GetDeviceCodeError,
|
||||
JsonObject,
|
||||
RefreshAccessTokenError,
|
||||
decode_jwt_claims,
|
||||
extract_chatgpt_account_id,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from litellm.types.router import GenericLiteLLMParams
|
||||
|
||||
TOKEN_EXPIRY_SKEW_SECONDS: Final = 60
|
||||
TOKEN_REFRESH_TIMEOUT_SECONDS: Final = 30
|
||||
DEVICE_CODE_TIMEOUT_SECONDS: Final = 15 * 60
|
||||
DEVICE_CODE_COOLDOWN_SECONDS: Final = 5 * 60
|
||||
DEVICE_CODE_POLL_SLEEP_SECONDS: Final = 5
|
||||
|
||||
OPENAI_AUTH_CLAIM_KEY: Final = "https://api.openai.com/auth"
|
||||
|
||||
JsonObject: TypeAlias = Mapping[str, JsonValue]
|
||||
|
||||
_JSON_OBJECT_ADAPTER: Final = TypeAdapter(JsonObject)
|
||||
|
||||
|
||||
|
|
@ -44,6 +46,26 @@ def _optional_str(value: JsonValue | None) -> str | None:
|
|||
return value if isinstance(value, str) else None
|
||||
|
||||
|
||||
def get_chatgpt_auth_file(
|
||||
litellm_params: "Mapping[str, object] | GenericLiteLLMParams | None",
|
||||
) -> str | None:
|
||||
if litellm_params is None:
|
||||
return None
|
||||
value: Final = (
|
||||
litellm_params.get("chatgpt_auth_file")
|
||||
if isinstance(litellm_params, Mapping)
|
||||
else litellm_params.chatgpt_auth_file
|
||||
)
|
||||
if not isinstance(value, str) or not value:
|
||||
return None
|
||||
return os.path.abspath(os.path.expanduser(value))
|
||||
|
||||
|
||||
@lru_cache(maxsize=128)
|
||||
def get_cached_authenticator(auth_file: str) -> "Authenticator":
|
||||
return Authenticator(auth_file=auth_file)
|
||||
|
||||
|
||||
def _token_refresh_timeout() -> httpx.Timeout:
|
||||
configured: Final = get_configured_request_timeout()
|
||||
seconds: Final = (
|
||||
|
|
@ -53,13 +75,13 @@ def _token_refresh_timeout() -> httpx.Timeout:
|
|||
|
||||
|
||||
class Authenticator:
|
||||
def __init__(self) -> None:
|
||||
self.token_dir = os.getenv(
|
||||
"CHATGPT_TOKEN_DIR",
|
||||
os.path.expanduser("~/.config/litellm/chatgpt"),
|
||||
def __init__(self, auth_file: str | None = None) -> None:
|
||||
default_auth_file: Final = os.path.join(
|
||||
os.getenv("CHATGPT_TOKEN_DIR", os.path.expanduser("~/.config/litellm/chatgpt")),
|
||||
os.getenv("CHATGPT_AUTH_FILE", "auth.json"),
|
||||
)
|
||||
self.auth_file = os.path.join(self.token_dir, os.getenv("CHATGPT_AUTH_FILE", "auth.json"))
|
||||
self._ensure_token_dir()
|
||||
self.auth_file = os.path.abspath(os.path.expanduser(auth_file or default_auth_file))
|
||||
self.token_dir = os.path.dirname(self.auth_file)
|
||||
|
||||
def get_api_base(self) -> str:
|
||||
return os.getenv("CHATGPT_API_BASE") or os.getenv("OPENAI_CHATGPT_API_BASE") or CHATGPT_API_BASE
|
||||
|
|
@ -99,16 +121,18 @@ class Authenticator:
|
|||
tokens: Final = self._login_device_code()
|
||||
return tokens["access_token"]
|
||||
|
||||
def get_account_id(self) -> str | None:
|
||||
def get_account_id(self, access_token: str | None = None) -> str | None:
|
||||
auth_data: Final = self._read_auth_file()
|
||||
if access_token and (not auth_data or auth_data.get("access_token") != access_token):
|
||||
return extract_chatgpt_account_id(access_token)
|
||||
if not auth_data:
|
||||
return None
|
||||
account_id: Final = _optional_str(auth_data.get("account_id"))
|
||||
if account_id:
|
||||
return account_id
|
||||
id_token: Final = auth_data.get("id_token")
|
||||
access_token: Final = auth_data.get("access_token")
|
||||
derived: Final = self._extract_account_id(_optional_str(id_token or access_token))
|
||||
saved_access_token: Final = auth_data.get("access_token")
|
||||
derived: Final = extract_chatgpt_account_id(_optional_str(id_token or saved_access_token))
|
||||
if derived:
|
||||
self._write_auth_file({**auth_data, "account_id": derived})
|
||||
return derived
|
||||
|
|
@ -129,6 +153,7 @@ class Authenticator:
|
|||
|
||||
def _write_auth_file(self, data: JsonObject) -> None:
|
||||
try:
|
||||
self._ensure_token_dir()
|
||||
with open(self.auth_file, "w") as f:
|
||||
json.dump(data, f)
|
||||
except OSError as exc:
|
||||
|
|
@ -146,34 +171,11 @@ class Authenticator:
|
|||
return time.time() >= float(derived_expires_at) - TOKEN_EXPIRY_SKEW_SECONDS
|
||||
|
||||
def _get_expires_at(self, token: str) -> int | None:
|
||||
claims: Final = self._decode_jwt_claims(token)
|
||||
exp: Final = claims.get("exp")
|
||||
exp: Final = decode_jwt_claims(token).get("exp")
|
||||
if isinstance(exp, (int, float)):
|
||||
return int(exp)
|
||||
return None
|
||||
|
||||
def _decode_jwt_claims(self, token: str) -> JsonObject:
|
||||
try:
|
||||
parts: Final = token.split(".")
|
||||
if len(parts) < 2:
|
||||
return {}
|
||||
payload_b64: Final = parts[1] + "=" * (-len(parts[1]) % 4)
|
||||
payload_bytes: Final = base64.urlsafe_b64decode(payload_b64)
|
||||
return _JSON_OBJECT_ADAPTER.validate_python(json.loads(payload_bytes.decode("utf-8")))
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
def _extract_account_id(self, token: str | None) -> str | None:
|
||||
if not token:
|
||||
return None
|
||||
claims: Final = self._decode_jwt_claims(token)
|
||||
auth_claims: Final = claims.get(OPENAI_AUTH_CLAIM_KEY)
|
||||
if isinstance(auth_claims, dict):
|
||||
account_id: Final = auth_claims.get("chatgpt_account_id")
|
||||
if isinstance(account_id, str) and account_id:
|
||||
return account_id
|
||||
return None
|
||||
|
||||
def _login_device_code(self) -> dict[str, str]:
|
||||
cooldown_remaining: Final = self._get_device_code_cooldown_remaining(self._read_auth_file())
|
||||
if cooldown_remaining > 0:
|
||||
|
|
@ -361,15 +363,16 @@ class Authenticator:
|
|||
"refresh_token": _optional_str(data.get("refresh_token")) or refresh_token,
|
||||
"id_token": id_token,
|
||||
}
|
||||
account_id: Final = self.get_account_id()
|
||||
auth_data: Final = self._build_auth_record(refreshed)
|
||||
self._write_auth_file(auth_data)
|
||||
self._write_auth_file({**auth_data, "account_id": account_id} if account_id else auth_data)
|
||||
return refreshed
|
||||
|
||||
def _build_auth_record(self, tokens: dict[str, str]) -> JsonObject:
|
||||
access_token: Final = tokens.get("access_token")
|
||||
id_token: Final = tokens.get("id_token")
|
||||
expires_at: Final = self._get_expires_at(access_token) if access_token else None
|
||||
account_id: Final = self._extract_account_id(id_token or access_token)
|
||||
account_id: Final = extract_chatgpt_account_id(id_token or access_token)
|
||||
return {
|
||||
"access_token": access_token,
|
||||
"refresh_token": tokens.get("refresh_token"),
|
||||
|
|
|
|||
|
|
@ -1,14 +1,20 @@
|
|||
from collections.abc import Mapping
|
||||
from typing import Any, Final
|
||||
|
||||
from litellm.exceptions import AuthenticationError
|
||||
from litellm.llms.openai.openai import OpenAIConfig
|
||||
from litellm.types.llms.openai import AllMessageValues
|
||||
from litellm.types.router import GenericLiteLLMParams
|
||||
|
||||
from ..authenticator import Authenticator
|
||||
from ..authenticator import Authenticator, get_cached_authenticator, get_chatgpt_auth_file
|
||||
from ..common_utils import (
|
||||
GetAccessTokenError,
|
||||
chatgpt_client_api_base,
|
||||
ensure_chatgpt_session_id,
|
||||
get_chatgpt_client_credential,
|
||||
get_chatgpt_default_headers,
|
||||
get_chatgpt_request_credential,
|
||||
strip_chatgpt_client_credential_headers,
|
||||
)
|
||||
from .streaming_utils import ChatGPTToolCallNormalizer
|
||||
|
||||
|
|
@ -19,9 +25,27 @@ class ChatGPTConfig(OpenAIConfig):
|
|||
api_key: str | None = None,
|
||||
api_base: str | None = None,
|
||||
custom_llm_provider: str = "openai",
|
||||
authenticator: Authenticator | None = None,
|
||||
) -> None:
|
||||
super().__init__()
|
||||
self.authenticator = Authenticator()
|
||||
self.authenticator = authenticator if authenticator is not None else Authenticator()
|
||||
|
||||
def _resolve_authenticator(
|
||||
self, litellm_params: Mapping[str, object] | GenericLiteLLMParams | None
|
||||
) -> Authenticator:
|
||||
auth_file: Final = get_chatgpt_auth_file(litellm_params)
|
||||
return get_cached_authenticator(auth_file) if auth_file else self.authenticator
|
||||
|
||||
@staticmethod
|
||||
def _get_access_token_or_raise(authenticator: Authenticator, model: str, llm_provider: str) -> str:
|
||||
try:
|
||||
return authenticator.get_access_token()
|
||||
except GetAccessTokenError as e:
|
||||
raise AuthenticationError(
|
||||
model=model,
|
||||
llm_provider=llm_provider,
|
||||
message=str(e),
|
||||
)
|
||||
|
||||
def api_base_without_login(self) -> str:
|
||||
return self.authenticator.get_api_base()
|
||||
|
|
@ -32,16 +56,16 @@ class ChatGPTConfig(OpenAIConfig):
|
|||
api_base: str | None,
|
||||
api_key: str | None,
|
||||
custom_llm_provider: str,
|
||||
litellm_params: Mapping[str, object] | GenericLiteLLMParams | None = None,
|
||||
) -> tuple[str | None, str | None, str]:
|
||||
dynamic_api_base: Final = self.api_base_without_login()
|
||||
try:
|
||||
dynamic_api_key: Final = self.authenticator.get_access_token()
|
||||
except GetAccessTokenError as e:
|
||||
raise AuthenticationError(
|
||||
model=model,
|
||||
llm_provider=custom_llm_provider,
|
||||
message=str(e),
|
||||
)
|
||||
client_credential: Final = get_chatgpt_request_credential(litellm_params)
|
||||
if client_credential is not None:
|
||||
return chatgpt_client_api_base(api_base), client_credential.access_token, custom_llm_provider
|
||||
if api_key:
|
||||
return api_base or self.api_base_without_login(), api_key, custom_llm_provider
|
||||
authenticator: Final = self._resolve_authenticator(litellm_params)
|
||||
dynamic_api_base: Final = authenticator.get_api_base()
|
||||
dynamic_api_key: Final = self._get_access_token_or_raise(authenticator, model, custom_llm_provider)
|
||||
return dynamic_api_base, dynamic_api_key, custom_llm_provider
|
||||
|
||||
def get_openai_compatible_provider_info(
|
||||
|
|
@ -50,26 +74,49 @@ class ChatGPTConfig(OpenAIConfig):
|
|||
api_base: str | None,
|
||||
api_key: str | None,
|
||||
custom_llm_provider: str,
|
||||
litellm_params: Mapping[str, object] | GenericLiteLLMParams | None = None,
|
||||
) -> tuple[str | None, str | None, str]:
|
||||
return self._get_openai_compatible_provider_info(model, api_base, api_key, custom_llm_provider)
|
||||
return self._get_openai_compatible_provider_info(model, api_base, api_key, custom_llm_provider, litellm_params)
|
||||
|
||||
def validate_environment(
|
||||
self,
|
||||
headers: dict,
|
||||
headers: dict[str, str],
|
||||
model: str,
|
||||
messages: list[AllMessageValues],
|
||||
optional_params: dict,
|
||||
litellm_params: dict,
|
||||
litellm_params: dict[str, object],
|
||||
api_key: str | None = None,
|
||||
api_base: str | None = None,
|
||||
) -> dict:
|
||||
) -> dict[str, str]:
|
||||
client_credential: Final = get_chatgpt_client_credential(headers)
|
||||
if client_credential is not None:
|
||||
return dict(
|
||||
get_chatgpt_default_headers(
|
||||
client_credential.access_token,
|
||||
client_credential.account_id,
|
||||
ensure_chatgpt_session_id(litellm_params),
|
||||
),
|
||||
**strip_chatgpt_client_credential_headers(headers),
|
||||
)
|
||||
authenticator: Final = self._resolve_authenticator(litellm_params)
|
||||
resolved_api_key: Final = (
|
||||
self._get_access_token_or_raise(authenticator, model, "chatgpt")
|
||||
if get_chatgpt_auth_file(litellm_params)
|
||||
else api_key
|
||||
)
|
||||
validated_headers: Final = super().validate_environment(
|
||||
headers, model, messages, optional_params, litellm_params, api_key, api_base
|
||||
dict(strip_chatgpt_client_credential_headers(headers)),
|
||||
model,
|
||||
messages,
|
||||
optional_params,
|
||||
litellm_params,
|
||||
resolved_api_key,
|
||||
api_base,
|
||||
)
|
||||
|
||||
account_id: Final = self.authenticator.get_account_id()
|
||||
account_id: Final = authenticator.get_account_id(resolved_api_key)
|
||||
session_id: Final = ensure_chatgpt_session_id(litellm_params)
|
||||
default_headers: Final = get_chatgpt_default_headers(api_key or "", account_id, session_id)
|
||||
default_headers: Final = get_chatgpt_default_headers(resolved_api_key or "", account_id, session_id)
|
||||
return {**default_headers, **validated_headers}
|
||||
|
||||
def post_stream_processing(self, stream: Any) -> ChatGPTToolCallNormalizer:
|
||||
|
|
|
|||
|
|
@ -2,14 +2,32 @@
|
|||
Constants and helpers for ChatGPT subscription OAuth.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import platform
|
||||
from typing import Any, Final
|
||||
from collections.abc import Mapping, Sequence
|
||||
from dataclasses import dataclass
|
||||
from types import MappingProxyType
|
||||
from typing import TYPE_CHECKING, Final, TypeAlias
|
||||
from urllib.parse import urlsplit
|
||||
from uuid import uuid4
|
||||
|
||||
import httpx
|
||||
from pydantic import JsonValue, TypeAdapter
|
||||
|
||||
from litellm.litellm_core_utils.get_provider_specific_headers import ProviderSpecificHeaderUtils
|
||||
from litellm.llms.base_llm.chat.transformation import BaseLLMException
|
||||
from litellm.types.utils import ProviderSpecificHeader
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from litellm.types.router import GenericLiteLLMParams
|
||||
|
||||
_SCOPED_HEADERS: Final[TypeAdapter[ProviderSpecificHeader | Sequence[ProviderSpecificHeader] | None]] = TypeAdapter(
|
||||
ProviderSpecificHeader | Sequence[ProviderSpecificHeader] | None
|
||||
)
|
||||
_STRING_HEADERS: Final = TypeAdapter(Mapping[str, str])
|
||||
_PARAMS: Final = TypeAdapter(Mapping[str, object])
|
||||
|
||||
# OAuth + API constants (derived from openai/codex)
|
||||
CHATGPT_AUTH_BASE: Final = "https://auth.openai.com"
|
||||
|
|
@ -19,6 +37,12 @@ CHATGPT_OAUTH_TOKEN_URL: Final = f"{CHATGPT_AUTH_BASE}/oauth/token"
|
|||
CHATGPT_DEVICE_VERIFY_URL: Final = f"{CHATGPT_AUTH_BASE}/codex/device"
|
||||
CHATGPT_API_BASE: Final = "https://chatgpt.com/backend-api/codex"
|
||||
CHATGPT_CLIENT_ID: Final = "app_EMoamEEZ73f0CkXaXp7hrann"
|
||||
CHATGPT_ACCOUNT_ID_HEADER: Final = "ChatGPT-Account-Id"
|
||||
OPENAI_AUTH_CLAIM_KEY: Final = "https://api.openai.com/auth"
|
||||
|
||||
JsonObject: TypeAlias = Mapping[str, JsonValue]
|
||||
_JSON_OBJECT_ADAPTER: Final = TypeAdapter(JsonObject)
|
||||
_EMPTY_CLAIMS: Final[JsonObject] = MappingProxyType({})
|
||||
|
||||
DEFAULT_ORIGINATOR: Final = "codex_cli_rs"
|
||||
DEFAULT_USER_AGENT: Final = "codex_cli_rs/0.0.0 (Unknown 0; unknown) unknown"
|
||||
|
|
@ -229,7 +253,7 @@ def get_chatgpt_default_headers(
|
|||
access_token: str,
|
||||
account_id: str | None,
|
||||
session_id: str | None = None,
|
||||
) -> dict:
|
||||
) -> dict[str, str]:
|
||||
originator: Final = get_chatgpt_originator()
|
||||
user_agent: Final = get_chatgpt_user_agent(originator)
|
||||
headers: Final = {
|
||||
|
|
@ -242,30 +266,133 @@ def get_chatgpt_default_headers(
|
|||
if session_id:
|
||||
headers["session_id"] = session_id
|
||||
if account_id:
|
||||
headers["ChatGPT-Account-Id"] = account_id
|
||||
headers[CHATGPT_ACCOUNT_ID_HEADER] = account_id
|
||||
return headers
|
||||
|
||||
|
||||
def decode_jwt_claims(token: str) -> JsonObject:
|
||||
try:
|
||||
parts: Final = token.split(".")
|
||||
if len(parts) < 2:
|
||||
return _EMPTY_CLAIMS
|
||||
payload_b64: Final = parts[1] + "=" * (-len(parts[1]) % 4)
|
||||
payload_bytes: Final = base64.urlsafe_b64decode(payload_b64)
|
||||
return _JSON_OBJECT_ADAPTER.validate_python(json.loads(payload_bytes.decode("utf-8")))
|
||||
except Exception:
|
||||
return _EMPTY_CLAIMS
|
||||
|
||||
|
||||
def extract_chatgpt_account_id(token: str | None) -> str | None:
|
||||
if not token:
|
||||
return None
|
||||
auth_claims: Final = decode_jwt_claims(token).get(OPENAI_AUTH_CLAIM_KEY)
|
||||
if not isinstance(auth_claims, dict):
|
||||
return None
|
||||
account_id: Final = auth_claims.get("chatgpt_account_id")
|
||||
return account_id if isinstance(account_id, str) and account_id else None
|
||||
|
||||
|
||||
def _bearer_token(value: str) -> str:
|
||||
scheme, separator, token = value.strip().partition(" ")
|
||||
if not separator:
|
||||
return scheme
|
||||
return token.strip() if scheme.lower() == "bearer" else ""
|
||||
|
||||
|
||||
def is_chatgpt_oauth_key(value: str | None) -> bool:
|
||||
if value is None:
|
||||
return False
|
||||
return OPENAI_AUTH_CLAIM_KEY in decode_jwt_claims(_bearer_token(value))
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ChatGPTClientCredential:
|
||||
access_token: str
|
||||
account_id: str | None
|
||||
|
||||
|
||||
def _header_value(headers: Mapping[str, str], name: str) -> str | None:
|
||||
return next((value for key, value in headers.items() if key.lower() == name), None)
|
||||
|
||||
|
||||
def get_chatgpt_client_credential(headers: Mapping[str, str]) -> ChatGPTClientCredential | None:
|
||||
auth_header: Final = _header_value(headers, "authorization")
|
||||
if auth_header is None or not is_chatgpt_oauth_key(auth_header):
|
||||
return None
|
||||
access_token: Final = _bearer_token(auth_header)
|
||||
account_id: Final = _header_value(headers, CHATGPT_ACCOUNT_ID_HEADER.lower()) or extract_chatgpt_account_id(
|
||||
access_token
|
||||
)
|
||||
return ChatGPTClientCredential(access_token=access_token, account_id=account_id)
|
||||
|
||||
|
||||
def select_chatgpt_client_credential_headers(headers: Mapping[str, str]) -> Mapping[str, str]:
|
||||
if get_chatgpt_client_credential(headers) is None:
|
||||
return MappingProxyType({})
|
||||
return MappingProxyType(
|
||||
{
|
||||
key: value
|
||||
for key, value in headers.items()
|
||||
if (key.lower() == "authorization" and is_chatgpt_oauth_key(value))
|
||||
or key.lower() == CHATGPT_ACCOUNT_ID_HEADER.lower()
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def strip_chatgpt_client_credential_headers(headers: Mapping[str, str]) -> Mapping[str, str]:
|
||||
return MappingProxyType(
|
||||
{
|
||||
key: value
|
||||
for key, value in headers.items()
|
||||
if key.lower() not in ("authorization", CHATGPT_ACCOUNT_ID_HEADER.lower())
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def get_chatgpt_request_credential(
|
||||
params: "Mapping[str, object] | GenericLiteLLMParams | None",
|
||||
) -> ChatGPTClientCredential | None:
|
||||
normalized: Final = _normalize_litellm_params(params)
|
||||
scoped: Final = normalized.get("provider_specific_header")
|
||||
extra: Final = normalized.get("extra_headers")
|
||||
headers: Final = ProviderSpecificHeaderUtils.get_provider_specific_headers(
|
||||
provider_specific_header=_SCOPED_HEADERS.validate_python(scoped), custom_llm_provider="chatgpt"
|
||||
)
|
||||
return get_chatgpt_client_credential(headers) or get_chatgpt_client_credential(
|
||||
_STRING_HEADERS.validate_python(extra) if extra is not None else MappingProxyType({})
|
||||
)
|
||||
|
||||
|
||||
def chatgpt_client_api_base(api_base: str | None) -> str:
|
||||
base: Final = api_base or CHATGPT_API_BASE
|
||||
parsed: Final = urlsplit(base)
|
||||
if (
|
||||
parsed.scheme != "https"
|
||||
or parsed.hostname != "chatgpt.com"
|
||||
or parsed.port not in (None, 443)
|
||||
or parsed.username
|
||||
or parsed.password
|
||||
):
|
||||
raise ChatGPTAuthError(
|
||||
status_code=400, message="Client ChatGPT credentials require the official HTTPS ChatGPT endpoint"
|
||||
)
|
||||
return base
|
||||
|
||||
|
||||
def get_chatgpt_default_instructions() -> str:
|
||||
return os.getenv("CHATGPT_DEFAULT_INSTRUCTIONS") or CHATGPT_DEFAULT_INSTRUCTIONS
|
||||
|
||||
|
||||
def _normalize_litellm_params(litellm_params: Any | None) -> dict:
|
||||
if litellm_params is None:
|
||||
return {}
|
||||
if isinstance(litellm_params, dict):
|
||||
return litellm_params
|
||||
if hasattr(litellm_params, "model_dump"):
|
||||
try:
|
||||
return litellm_params.model_dump()
|
||||
except Exception:
|
||||
return {}
|
||||
if hasattr(litellm_params, "dict"):
|
||||
try:
|
||||
return litellm_params.dict()
|
||||
except Exception:
|
||||
return {}
|
||||
return {}
|
||||
def _normalize_litellm_params(litellm_params: object) -> Mapping[str, object]:
|
||||
if isinstance(litellm_params, Mapping):
|
||||
return _PARAMS.validate_python(litellm_params)
|
||||
serializer: Final = getattr(litellm_params, "model_dump", None) or getattr(litellm_params, "dict", None)
|
||||
if not callable(serializer):
|
||||
return MappingProxyType({})
|
||||
try:
|
||||
return _PARAMS.validate_python(serializer())
|
||||
except Exception:
|
||||
return MappingProxyType({})
|
||||
|
||||
|
||||
def get_chatgpt_session_id(litellm_params: object) -> str | None:
|
||||
|
|
@ -274,8 +401,8 @@ def get_chatgpt_session_id(litellm_params: object) -> str | None:
|
|||
value = params.get(key)
|
||||
if value:
|
||||
return str(value)
|
||||
metadata: Final = params.get("metadata")
|
||||
if isinstance(metadata, dict):
|
||||
metadata: Final = _normalize_litellm_params(params.get("metadata"))
|
||||
if metadata:
|
||||
value = metadata.get("session_id")
|
||||
if value:
|
||||
return str(value)
|
||||
|
|
|
|||
|
|
@ -25,13 +25,15 @@ from litellm.types.llms.openai import (
|
|||
from litellm.types.router import GenericLiteLLMParams
|
||||
from litellm.types.utils import LlmProviders
|
||||
|
||||
from ..authenticator import Authenticator
|
||||
from ..authenticator import Authenticator, get_cached_authenticator, get_chatgpt_auth_file
|
||||
from ..common_utils import (
|
||||
CHATGPT_API_BASE,
|
||||
GetAccessTokenError,
|
||||
ensure_chatgpt_session_id,
|
||||
get_chatgpt_client_credential,
|
||||
get_chatgpt_default_headers,
|
||||
get_chatgpt_default_instructions,
|
||||
strip_chatgpt_client_credential_headers,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
|
@ -49,14 +51,35 @@ class ChatGPTResponsesAPIConfig(OpenAIResponsesAPIConfig):
|
|||
def custom_llm_provider(self) -> LlmProviders:
|
||||
return LlmProviders.CHATGPT
|
||||
|
||||
def _resolve_authenticator(self, litellm_params: GenericLiteLLMParams | None) -> Authenticator:
|
||||
auth_file: Final = get_chatgpt_auth_file(litellm_params)
|
||||
return get_cached_authenticator(auth_file) if auth_file else self.authenticator
|
||||
|
||||
def validate_environment(
|
||||
self,
|
||||
headers: dict,
|
||||
headers: dict[str, str],
|
||||
model: str,
|
||||
litellm_params: GenericLiteLLMParams | None,
|
||||
) -> dict:
|
||||
) -> dict[str, str]:
|
||||
session_id: Final = ensure_chatgpt_session_id(litellm_params)
|
||||
client_credential: Final = get_chatgpt_client_credential(headers)
|
||||
if client_credential is not None:
|
||||
return {
|
||||
**get_chatgpt_default_headers(client_credential.access_token, client_credential.account_id, session_id),
|
||||
**strip_chatgpt_client_credential_headers(headers),
|
||||
}
|
||||
authenticator: Final = self._resolve_authenticator(litellm_params)
|
||||
safe_headers: Final = strip_chatgpt_client_credential_headers(headers)
|
||||
configured_api_key: Final = litellm_params.api_key if litellm_params is not None else None
|
||||
if configured_api_key and not get_chatgpt_auth_file(litellm_params):
|
||||
return {
|
||||
**get_chatgpt_default_headers(
|
||||
configured_api_key, authenticator.get_account_id(configured_api_key), session_id
|
||||
),
|
||||
**safe_headers,
|
||||
}
|
||||
try:
|
||||
access_token: Final = self.authenticator.get_access_token()
|
||||
access_token: Final = authenticator.get_access_token()
|
||||
except GetAccessTokenError as e:
|
||||
raise AuthenticationError(
|
||||
model=model,
|
||||
|
|
@ -64,10 +87,9 @@ class ChatGPTResponsesAPIConfig(OpenAIResponsesAPIConfig):
|
|||
message=str(e),
|
||||
)
|
||||
|
||||
account_id: Final = self.authenticator.get_account_id()
|
||||
session_id: Final = ensure_chatgpt_session_id(litellm_params)
|
||||
account_id: Final = authenticator.get_account_id()
|
||||
default_headers: Final = get_chatgpt_default_headers(access_token, account_id, session_id)
|
||||
return {**default_headers, **headers}
|
||||
return {**default_headers, **safe_headers}
|
||||
|
||||
def transform_responses_api_request(
|
||||
self,
|
||||
|
|
@ -79,7 +101,7 @@ class ChatGPTResponsesAPIConfig(OpenAIResponsesAPIConfig):
|
|||
) -> dict:
|
||||
request: Final = super().transform_responses_api_request(
|
||||
model,
|
||||
input,
|
||||
[{"role": "user", "content": input}] if isinstance(input, str) else input,
|
||||
response_api_optional_request_params,
|
||||
litellm_params,
|
||||
headers,
|
||||
|
|
|
|||
|
|
@ -198,6 +198,26 @@ from litellm.utils import (
|
|||
)
|
||||
|
||||
|
||||
def _merge_headers_case_insensitive(
|
||||
headers: Mapping[str, object], extra_headers: Mapping[str, object] | None
|
||||
) -> dict[str, str]: # mutable-ok: legacy HTTP handler methods require a dict of headers
|
||||
overridden: Final = frozenset(name.lower() for name in extra_headers or {})
|
||||
return {
|
||||
**{name: str(value) for name, value in headers.items() if name.lower() not in overridden},
|
||||
**{name: str(value) for name, value in (extra_headers or {}).items()},
|
||||
}
|
||||
|
||||
|
||||
def _client_request_headers(
|
||||
response_api_optional_request_params: Mapping[str, object],
|
||||
extra_headers: Mapping[str, object] | None,
|
||||
) -> dict[str, str]: # mutable-ok: provider validate_environment methods require a dict
|
||||
from litellm.litellm_core_utils.core_helpers import as_str_mapping
|
||||
|
||||
optional_headers: Final = as_str_mapping(response_api_optional_request_params.get("extra_headers"))
|
||||
return _merge_headers_case_insensitive(optional_headers or {}, extra_headers)
|
||||
|
||||
|
||||
def _rust_responses_websocket_enabled(
|
||||
custom_llm_provider: str | None,
|
||||
) -> bool:
|
||||
|
|
@ -2906,14 +2926,18 @@ class BaseLLMHTTPHandler:
|
|||
else:
|
||||
sync_httpx_client = client
|
||||
|
||||
headers = responses_api_provider_config.validate_environment(
|
||||
headers=response_api_optional_request_params.get("extra_headers", {}) or {},
|
||||
model=model,
|
||||
litellm_params=litellm_params,
|
||||
)
|
||||
|
||||
if extra_headers:
|
||||
headers.update(extra_headers)
|
||||
headers = { # rebind-ok: validation replaces the method's incoming headers
|
||||
**responses_api_provider_config.validate_environment(
|
||||
headers=(
|
||||
_client_request_headers(response_api_optional_request_params, extra_headers)
|
||||
if custom_llm_provider == "chatgpt"
|
||||
else response_api_optional_request_params.get("extra_headers", {}) or {}
|
||||
),
|
||||
model=model,
|
||||
litellm_params=litellm_params,
|
||||
),
|
||||
**((extra_headers or {}) if custom_llm_provider != "chatgpt" else {}),
|
||||
}
|
||||
|
||||
# Check if streaming is requested
|
||||
stream = response_api_optional_request_params.get("stream", False)
|
||||
|
|
@ -3104,11 +3128,15 @@ class BaseLLMHTTPHandler:
|
|||
validated_headers: Final = {
|
||||
**await asyncio.to_thread(
|
||||
responses_api_provider_config.validate_environment,
|
||||
headers=response_api_optional_request_params.get("extra_headers", {}) or {},
|
||||
headers=(
|
||||
_client_request_headers(response_api_optional_request_params, extra_headers)
|
||||
if custom_llm_provider == "chatgpt"
|
||||
else response_api_optional_request_params.get("extra_headers", {}) or {}
|
||||
),
|
||||
model=model,
|
||||
litellm_params=litellm_params,
|
||||
),
|
||||
**(extra_headers or {}),
|
||||
**((extra_headers or {}) if custom_llm_provider != "chatgpt" else {}),
|
||||
}
|
||||
|
||||
# Check if streaming is requested
|
||||
|
|
|
|||
|
|
@ -677,7 +677,11 @@ async def acompletion(
|
|||
}
|
||||
if custom_llm_provider is None:
|
||||
_supplemental: Final = cast( # cast-ok: kwargs values are request-scoped objects
|
||||
"dict[str, object]", {k: kwargs[k] for k in OPTIONAL_KWARGS_KEYS if k in kwargs}
|
||||
"dict[str, object]",
|
||||
{
|
||||
**{k: kwargs[k] for k in OPTIONAL_KWARGS_KEYS | {"provider_specific_header"} if k in kwargs},
|
||||
"extra_headers": extra_headers,
|
||||
},
|
||||
)
|
||||
_, custom_llm_provider, _, _ = get_llm_provider(
|
||||
model=model,
|
||||
|
|
@ -5678,7 +5682,10 @@ def completion(
|
|||
if deployment_id is not None: # azure llms
|
||||
model = deployment_id
|
||||
custom_llm_provider = "azure"
|
||||
_supplemental_provider_params: Final = {k: kwargs[k] for k in OPTIONAL_KWARGS_KEYS if k in kwargs}
|
||||
_supplemental_provider_params: Final = {
|
||||
**{k: kwargs[k] for k in OPTIONAL_KWARGS_KEYS | {"provider_specific_header"} if k in kwargs},
|
||||
"extra_headers": extra_headers,
|
||||
}
|
||||
model, custom_llm_provider, dynamic_api_key, api_base = get_llm_provider(
|
||||
model=model,
|
||||
custom_llm_provider=custom_llm_provider,
|
||||
|
|
@ -5998,6 +6005,7 @@ def completion(
|
|||
tpm=kwargs.get("tpm"),
|
||||
rpm=kwargs.get("rpm"),
|
||||
use_xai_oauth=kwargs.get("use_xai_oauth", False),
|
||||
chatgpt_auth_file=routing_kwargs.get("chatgpt_auth_file"),
|
||||
gigachat_scope=kwargs.get("gigachat_scope"),
|
||||
gigachat_auth_url=kwargs.get("gigachat_auth_url"),
|
||||
gigachat_access_token=kwargs.get("gigachat_access_token"),
|
||||
|
|
@ -9140,6 +9148,7 @@ async def ahealth_check(
|
|||
custom_llm_provider=custom_llm_provider_from_params,
|
||||
api_base=api_base_from_params,
|
||||
api_key=api_key_from_params,
|
||||
litellm_params=GenericLiteLLMParams(**model_params),
|
||||
)
|
||||
|
||||
model_params["cache"] = {"no-cache": True} # don't used cached responses for making health check calls
|
||||
|
|
|
|||
|
|
@ -207,6 +207,11 @@ LAZY_FEATURES: Final[tuple[LazyFeature, ...]] = (
|
|||
module_path="litellm.proxy.management_endpoints.config_override_endpoints",
|
||||
path_prefixes=("/config_overrides",),
|
||||
),
|
||||
LazyFeature(
|
||||
name="codex_passthrough",
|
||||
module_path="litellm.proxy.pass_through_endpoints.codex_passthrough_endpoints",
|
||||
path_prefixes=("/backend-api/codex/",),
|
||||
),
|
||||
LazyFeature(
|
||||
name="llm_passthrough",
|
||||
module_path="litellm.proxy.pass_through_endpoints.llm_passthrough_endpoints",
|
||||
|
|
|
|||
|
|
@ -8673,6 +8673,184 @@
|
|||
}
|
||||
}
|
||||
},
|
||||
"codex_passthrough": {
|
||||
"components": {
|
||||
"schemas": {
|
||||
"HTTPValidationError": {
|
||||
"properties": {
|
||||
"detail": {
|
||||
"items": {
|
||||
"$ref": "#/components/schemas/ValidationError"
|
||||
},
|
||||
"title": "Detail",
|
||||
"type": "array"
|
||||
}
|
||||
},
|
||||
"title": "HTTPValidationError",
|
||||
"type": "object"
|
||||
},
|
||||
"ValidationError": {
|
||||
"properties": {
|
||||
"ctx": {
|
||||
"title": "Context",
|
||||
"type": "object"
|
||||
},
|
||||
"input": {
|
||||
"title": "Input"
|
||||
},
|
||||
"loc": {
|
||||
"items": {
|
||||
"anyOf": [
|
||||
{
|
||||
"type": "string"
|
||||
},
|
||||
{
|
||||
"type": "integer"
|
||||
}
|
||||
]
|
||||
},
|
||||
"title": "Location",
|
||||
"type": "array"
|
||||
},
|
||||
"msg": {
|
||||
"title": "Message",
|
||||
"type": "string"
|
||||
},
|
||||
"type": {
|
||||
"title": "Error Type",
|
||||
"type": "string"
|
||||
}
|
||||
},
|
||||
"required": [
|
||||
"loc",
|
||||
"msg",
|
||||
"type"
|
||||
],
|
||||
"title": "ValidationError",
|
||||
"type": "object"
|
||||
}
|
||||
}
|
||||
},
|
||||
"paths": {
|
||||
"/backend-api/codex/{endpoint}": {
|
||||
"get": {
|
||||
"description": "Forward a client's ChatGPT subscription with separate LiteLLM authentication.",
|
||||
"operationId": "codex_passthrough_route_backend_api_codex__endpoint__get",
|
||||
"parameters": [
|
||||
{
|
||||
"in": "path",
|
||||
"name": "endpoint",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"enum": [
|
||||
"models",
|
||||
"responses"
|
||||
],
|
||||
"title": "Endpoint",
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
],
|
||||
"responses": {
|
||||
"200": {
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {}
|
||||
}
|
||||
},
|
||||
"description": "Successful Response"
|
||||
},
|
||||
"422": {
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/HTTPValidationError"
|
||||
}
|
||||
}
|
||||
},
|
||||
"description": "Validation Error"
|
||||
}
|
||||
},
|
||||
"security": [
|
||||
{
|
||||
"APIKeyHeader": []
|
||||
}
|
||||
],
|
||||
"summary": "Codex Passthrough Route",
|
||||
"tags": [
|
||||
"codex_passthrough"
|
||||
]
|
||||
},
|
||||
"post": {
|
||||
"description": "Forward a client's ChatGPT subscription with separate LiteLLM authentication.",
|
||||
"operationId": "codex_passthrough_route_backend_api_codex__endpoint__post",
|
||||
"parameters": [
|
||||
{
|
||||
"in": "path",
|
||||
"name": "endpoint",
|
||||
"required": true,
|
||||
"schema": {
|
||||
"enum": [
|
||||
"models",
|
||||
"responses"
|
||||
],
|
||||
"title": "Endpoint",
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
],
|
||||
"requestBody": {
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"additionalProperties": true,
|
||||
"type": "object"
|
||||
}
|
||||
}
|
||||
},
|
||||
"required": true
|
||||
},
|
||||
"responses": {
|
||||
"200": {
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"additionalProperties": true,
|
||||
"type": "object"
|
||||
}
|
||||
},
|
||||
"text/event-stream": {
|
||||
"schema": {
|
||||
"type": "string"
|
||||
}
|
||||
}
|
||||
},
|
||||
"description": "Unmodified Codex response or server-sent events"
|
||||
},
|
||||
"422": {
|
||||
"content": {
|
||||
"application/json": {
|
||||
"schema": {
|
||||
"$ref": "#/components/schemas/HTTPValidationError"
|
||||
}
|
||||
}
|
||||
},
|
||||
"description": "Validation Error"
|
||||
}
|
||||
},
|
||||
"security": [
|
||||
{
|
||||
"APIKeyHeader": []
|
||||
}
|
||||
],
|
||||
"summary": "Codex Passthrough Route",
|
||||
"tags": [
|
||||
"codex_passthrough"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
"compliance": {
|
||||
"components": {
|
||||
"schemas": {
|
||||
|
|
|
|||
|
|
@ -391,6 +391,7 @@ class LiteLLMRoutes(enum.Enum):
|
|||
]
|
||||
|
||||
mapped_pass_through_routes = [
|
||||
"/backend-api/codex",
|
||||
"/bedrock",
|
||||
"/comprehendmedical",
|
||||
"/azure_speech",
|
||||
|
|
|
|||
|
|
@ -736,7 +736,9 @@ def is_request_body_safe(
|
|||
|
||||
def _reject_request_credential_params(litellm_params: Mapping[str, object], *, manages_deployments: bool) -> None:
|
||||
if manages_deployments:
|
||||
reject_server_owned_wif_fields(litellm_params)
|
||||
reject_server_owned_wif_fields(
|
||||
MappingProxyType({key: value for key, value in litellm_params.items() if key != "chatgpt_auth_file"})
|
||||
)
|
||||
else:
|
||||
reject_server_owned_wif_params(litellm_params)
|
||||
reject_federated_credential_reference(litellm_params)
|
||||
|
|
|
|||
|
|
@ -2304,7 +2304,15 @@ async def test_model_connection(
|
|||
)
|
||||
|
||||
selected_by_id: Final = deployment_by_id is not None
|
||||
reject_server_owned_wif_fields(request_litellm_params)
|
||||
reject_server_owned_wif_fields(
|
||||
MappingProxyType(
|
||||
{
|
||||
key: value
|
||||
for key, value in _OBJECT_MAPPING.validate_python(request_litellm_params).items()
|
||||
if key != "chatgpt_auth_file"
|
||||
}
|
||||
)
|
||||
)
|
||||
# Merge: config params (from proxy config) as base, request params override
|
||||
litellm_params = {
|
||||
**_config_base_for_health_check(
|
||||
|
|
|
|||
|
|
@ -1417,13 +1417,14 @@ def clean_headers(
|
|||
Cleaned headers dict
|
||||
"""
|
||||
from litellm.llms.anthropic.common_utils import is_anthropic_oauth_key
|
||||
from litellm.llms.chatgpt.common_utils import is_chatgpt_oauth_key
|
||||
|
||||
clean_headers: Final = {}
|
||||
litellm_key_lower: Final = litellm_key_header_name.lower() if litellm_key_header_name is not None else None
|
||||
for header, value in headers.items():
|
||||
header_lower = header.lower()
|
||||
|
||||
if header_lower == "authorization" and is_anthropic_oauth_key(value):
|
||||
if header_lower == "authorization" and (is_anthropic_oauth_key(value) or is_chatgpt_oauth_key(value)):
|
||||
if authenticated_with_header is None or authenticated_with_header.lower() != "authorization":
|
||||
clean_headers[header] = value
|
||||
continue
|
||||
|
|
@ -1452,7 +1453,7 @@ def clean_headers(
|
|||
|
||||
def _is_credential_header(header: str) -> bool:
|
||||
"""Whether `header` carries a caller credential rather than request context."""
|
||||
return header.lower() in _CREDENTIAL_HEADER_NAMES
|
||||
return header.lower().removeprefix("x-pass-") in _CREDENTIAL_HEADER_NAMES
|
||||
|
||||
|
||||
def redact_credential_headers(headers: Mapping[str, str]) -> Mapping[str, str]:
|
||||
|
|
@ -4282,6 +4283,7 @@ _NATIVE_MESSAGES_ANTHROPIC_API_HEADER_PROVIDERS: Final = ",".join(
|
|||
(LlmProviders.AZURE_AI.value, LlmProviders.DATABRICKS.value)
|
||||
)
|
||||
_ANTHROPIC_OAUTH_CREDENTIAL_PROVIDERS: Final = ",".join(sorted(ANTHROPIC_OAUTH_FORWARD_PROVIDERS))
|
||||
_CHATGPT_OAUTH_CREDENTIAL_PROVIDERS: Final = LlmProviders.CHATGPT.value
|
||||
|
||||
|
||||
def add_provider_specific_headers_to_request(
|
||||
|
|
@ -4289,6 +4291,7 @@ def add_provider_specific_headers_to_request(
|
|||
headers: Mapping[str, str],
|
||||
) -> bool:
|
||||
from litellm.llms.anthropic.common_utils import is_anthropic_oauth_key
|
||||
from litellm.llms.chatgpt.common_utils import select_chatgpt_client_credential_headers
|
||||
|
||||
anthropic_api_headers: Final = {header: headers[header] for header in ANTHROPIC_API_HEADERS if header in headers}
|
||||
anthropic_oauth_credential_headers: Final = {
|
||||
|
|
@ -4296,6 +4299,8 @@ def add_provider_specific_headers_to_request(
|
|||
for header, value in headers.items()
|
||||
if header.lower() == "authorization" and is_anthropic_oauth_key(value)
|
||||
}
|
||||
chatgpt_oauth_credential_headers: Final = dict(select_chatgpt_client_credential_headers(headers))
|
||||
|
||||
typesafe_danger_zone_headers: Final = {
|
||||
header: value
|
||||
for header, value in headers.items()
|
||||
|
|
@ -4320,13 +4325,17 @@ def add_provider_specific_headers_to_request(
|
|||
ProviderSpecificHeader(
|
||||
custom_llm_provider=LlmProviders.TYPESAFE.value, extra_headers=typesafe_danger_zone_headers
|
||||
),
|
||||
ProviderSpecificHeader(
|
||||
custom_llm_provider=_CHATGPT_OAUTH_CREDENTIAL_PROVIDERS,
|
||||
extra_headers=chatgpt_oauth_credential_headers,
|
||||
),
|
||||
)
|
||||
if scoped_header["extra_headers"]
|
||||
]
|
||||
|
||||
if scoped_headers:
|
||||
data["provider_specific_header"] = scoped_headers[0] if len(scoped_headers) == 1 else scoped_headers
|
||||
return bool(anthropic_oauth_credential_headers)
|
||||
return bool(anthropic_oauth_credential_headers or chatgpt_oauth_credential_headers)
|
||||
|
||||
|
||||
def _add_otel_traceparent_to_data(data: dict, request: Request):
|
||||
|
|
|
|||
|
|
@ -0,0 +1,87 @@
|
|||
import hmac
|
||||
from types import MappingProxyType
|
||||
from typing import Annotated, Final, Literal
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request, Response
|
||||
|
||||
from litellm.llms.chatgpt.common_utils import CHATGPT_API_BASE
|
||||
from litellm.proxy._types import UserAPIKeyAuth
|
||||
from litellm.proxy.auth.user_api_key_auth import user_api_key_auth
|
||||
from litellm.proxy.pass_through_endpoints.pass_through_endpoints import pass_through_request
|
||||
|
||||
router: Final = APIRouter()
|
||||
_HEADERS: Final = frozenset(
|
||||
{
|
||||
"authorization",
|
||||
"chatgpt-account-id",
|
||||
"content-type",
|
||||
"accept",
|
||||
"user-agent",
|
||||
"originator",
|
||||
"openai-beta",
|
||||
"session-id",
|
||||
"thread-id",
|
||||
"session_id",
|
||||
"conversation_id",
|
||||
"x-codex-turn-metadata",
|
||||
"x-codex-turn-state",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@router.get("/backend-api/codex/{endpoint}", tags=["ChatGPT Pass-through", "pass-through"])
|
||||
@router.post(
|
||||
"/backend-api/codex/{endpoint}",
|
||||
tags=["ChatGPT Pass-through", "pass-through"],
|
||||
openapi_extra={
|
||||
"requestBody": {
|
||||
"required": True,
|
||||
"content": {"application/json": {"schema": {"type": "object", "additionalProperties": True}}},
|
||||
}
|
||||
},
|
||||
responses={
|
||||
200: {
|
||||
"description": "Unmodified Codex response or server-sent events",
|
||||
"content": {
|
||||
"application/json": {"schema": {"type": "object", "additionalProperties": True}},
|
||||
"text/event-stream": {"schema": {"type": "string"}},
|
||||
},
|
||||
}
|
||||
},
|
||||
)
|
||||
async def codex_passthrough_route(
|
||||
endpoint: Literal["models", "responses"],
|
||||
request: Request,
|
||||
user_api_key_dict: Annotated[UserAPIKeyAuth, Depends(user_api_key_auth)],
|
||||
) -> Response:
|
||||
"""Forward a client's ChatGPT subscription with separate LiteLLM authentication."""
|
||||
expected_method: Final = "GET" if endpoint == "models" else "POST"
|
||||
if request.method != expected_method:
|
||||
raise HTTPException(status_code=405, detail="Method not allowed", headers={"Allow": expected_method})
|
||||
|
||||
authorization: Final = request.headers.get("authorization", "")
|
||||
scheme, _, token = authorization.partition(" ")
|
||||
gateway_key: Final = request.headers.get("x-litellm-api-key", "")
|
||||
gateway_parts: Final = gateway_key.split()
|
||||
gateway_token: Final = (
|
||||
gateway_parts[-1]
|
||||
if len(gateway_parts) == 1 or (len(gateway_parts) == 2 and gateway_parts[0].lower() == "bearer")
|
||||
else ""
|
||||
)
|
||||
if scheme.lower() != "bearer" or not token.strip() or not gateway_token:
|
||||
raise HTTPException(
|
||||
status_code=401,
|
||||
detail="Send the ChatGPT bearer in Authorization and the LiteLLM key in x-litellm-api-key",
|
||||
)
|
||||
if hmac.compare_digest(token.strip().encode(), gateway_token.encode()):
|
||||
raise HTTPException(status_code=401, detail="ChatGPT and LiteLLM credentials must be separate")
|
||||
|
||||
headers: Final = MappingProxyType({name: value for name, value in request.headers.items() if name in _HEADERS})
|
||||
return await pass_through_request(
|
||||
request=request,
|
||||
target=f"{CHATGPT_API_BASE}/{endpoint}",
|
||||
custom_headers=dict(headers),
|
||||
user_api_key_dict=user_api_key_dict,
|
||||
custom_llm_provider="chatgpt",
|
||||
upstream_headers_filter=lambda _: headers,
|
||||
)
|
||||
|
|
@ -10,6 +10,7 @@ from typing import Final, Literal, TypeAlias
|
|||
from urllib.parse import urlparse
|
||||
|
||||
import httpx
|
||||
from pydantic import TypeAdapter
|
||||
|
||||
import litellm
|
||||
from litellm._logging import verbose_proxy_logger
|
||||
|
|
@ -95,6 +96,8 @@ def is_openai_compatible_url(url_route: str | None) -> bool:
|
|||
hostname: Final = parsed_url.hostname
|
||||
if not hostname:
|
||||
return False
|
||||
if hostname == "chatgpt.com" and parsed_url.path.startswith("/backend-api/codex/"):
|
||||
return True
|
||||
if _hostname_matches(hostname, _OPENAI_HOSTNAMES):
|
||||
return True
|
||||
if _hostname_matches(hostname, _AZURE_OPENAI_HOSTNAMES):
|
||||
|
|
@ -198,7 +201,7 @@ class OpenAIPassthroughLoggingHandler(BasePassthroughLoggingHandler):
|
|||
if not url_route:
|
||||
return False
|
||||
parsed_url: Final = urlparse(url_route)
|
||||
return _is_openai_compatible_host(parsed_url.hostname) and (
|
||||
return is_openai_compatible_url(url_route) and (
|
||||
"/v1/responses" in parsed_url.path or "/responses" in parsed_url.path
|
||||
)
|
||||
|
||||
|
|
@ -424,7 +427,9 @@ class OpenAIPassthroughLoggingHandler(BasePassthroughLoggingHandler):
|
|||
) = None
|
||||
handler_instance: Final = OpenAIPassthroughLoggingHandler()
|
||||
|
||||
custom_llm_provider: Final = kwargs.get("custom_llm_provider", "openai")
|
||||
custom_llm_provider: Final = TypeAdapter(str).validate_python(
|
||||
kwargs.get("custom_llm_provider", logging_obj.model_call_details.get("custom_llm_provider", "openai"))
|
||||
)
|
||||
|
||||
if is_chat_completions:
|
||||
# Handle chat completions with existing logic
|
||||
|
|
|
|||
|
|
@ -118,6 +118,7 @@ from litellm.proxy.litellm_pre_call_utils import ( # noqa: F401 # legacy modul
|
|||
_key_or_team_allows_client_pricing_override, # pyright: ignore[reportPrivateUsage] # reuse the proxy's pricing trust policy
|
||||
_strip_client_pricing_overrides, # pyright: ignore[reportPrivateUsage] # sanitize before trusted hooks add guardrail costs
|
||||
get_dynamic_logging_metadata, # pyright: ignore[reportPrivateUsage] # shared proxy helper, same import style as _read_request_body above
|
||||
redact_credential_headers,
|
||||
)
|
||||
from litellm.proxy.route_llm_request import ProxyModelNotFoundError
|
||||
from litellm.proxy.route_priority import (
|
||||
|
|
@ -409,6 +410,8 @@ class HttpPassThroughEndpointHelpers(BasePassthroughUtils):
|
|||
@staticmethod
|
||||
def get_endpoint_type(url: str, custom_llm_provider: str | None = None) -> EndpointType:
|
||||
parsed_url: Final = urlparse(url)
|
||||
if custom_llm_provider == "chatgpt":
|
||||
return EndpointType.OPENAI
|
||||
if custom_llm_provider in ("typesafe", "laya", "bespoke") and parsed_url.path.removesuffix("/").endswith(
|
||||
"/v1/systemone"
|
||||
):
|
||||
|
|
@ -723,7 +726,7 @@ class HttpPassThroughEndpointHelpers(BasePassthroughUtils):
|
|||
"url": str(request.url),
|
||||
"method": request.method,
|
||||
"body": copy.copy(_parsed_body), # use copy instead of deepcopy
|
||||
"headers": request.headers,
|
||||
"headers": redact_credential_headers(request.headers),
|
||||
},
|
||||
},
|
||||
"call_type": "pass_through_endpoint",
|
||||
|
|
@ -1490,7 +1493,7 @@ async def pass_through_request(
|
|||
additional_args={
|
||||
"complete_input_dict": _parsed_body,
|
||||
"api_base": str(logging_url),
|
||||
"headers": upstream_headers,
|
||||
"headers": redact_credential_headers(upstream_headers),
|
||||
},
|
||||
)
|
||||
stream = HttpPassThroughEndpointHelpers.update_stream_param_based_on_request_body(
|
||||
|
|
@ -1561,7 +1564,7 @@ async def pass_through_request(
|
|||
data=_parsed_body or {},
|
||||
user_api_key_dict=user_api_key_dict,
|
||||
response=relay_response,
|
||||
request_headers=dict(request.headers),
|
||||
request_headers=dict(redact_credential_headers(request.headers)),
|
||||
)
|
||||
if callback_headers:
|
||||
_response_headers.update(callback_headers)
|
||||
|
|
@ -1653,7 +1656,7 @@ async def pass_through_request(
|
|||
data=_parsed_body or {},
|
||||
user_api_key_dict=user_api_key_dict,
|
||||
response=detected_relay_response,
|
||||
request_headers=dict(request.headers),
|
||||
request_headers=dict(redact_credential_headers(request.headers)),
|
||||
)
|
||||
if callback_headers:
|
||||
_response_headers.update(callback_headers)
|
||||
|
|
@ -1703,7 +1706,7 @@ async def pass_through_request(
|
|||
data=_parsed_body or {},
|
||||
user_api_key_dict=user_api_key_dict,
|
||||
response=response,
|
||||
request_headers=dict(request.headers),
|
||||
request_headers=dict(redact_credential_headers(request.headers)),
|
||||
)
|
||||
if relay_callback_headers:
|
||||
relay_custom_headers.update(relay_callback_headers)
|
||||
|
|
@ -1878,7 +1881,7 @@ async def pass_through_request(
|
|||
data=_parsed_body or {},
|
||||
user_api_key_dict=user_api_key_dict,
|
||||
response=response,
|
||||
request_headers=dict(request.headers),
|
||||
request_headers=dict(redact_credential_headers(request.headers)),
|
||||
)
|
||||
if callback_headers:
|
||||
custom_headers.update(callback_headers)
|
||||
|
|
|
|||
|
|
@ -19,6 +19,7 @@ from pydantic import ConfigDict, Field
|
|||
import litellm
|
||||
from litellm._logging import verbose_proxy_logger
|
||||
from litellm.constants import INTERNAL_CALL_ORIGIN_METADATA_KEY
|
||||
from litellm.litellm_core_utils.get_llm_provider_logic import resolve_model_and_provider_for_metadata
|
||||
from litellm.litellm_core_utils.llm_cost_calc.utils import (
|
||||
calculate_prompt_caching_savings,
|
||||
generic_cost_per_token,
|
||||
|
|
@ -69,7 +70,9 @@ def _resolve_model(model: str | None, custom_llm_provider: str | None) -> _Model
|
|||
if not model:
|
||||
return None
|
||||
try:
|
||||
resolved_model, provider, _, _ = litellm.get_llm_provider(model=model, custom_llm_provider=custom_llm_provider)
|
||||
resolved_model, provider = resolve_model_and_provider_for_metadata(
|
||||
model=model, custom_llm_provider=custom_llm_provider
|
||||
)
|
||||
except Exception as e: # noqa: BLE001 # get_llm_provider raises for unroutable names; degrade to an unavailable estimate
|
||||
verbose_proxy_logger.debug(
|
||||
"savings: cannot resolve provider for model=%s custom_llm_provider=%s (%s)", model, custom_llm_provider, e
|
||||
|
|
|
|||
|
|
@ -705,10 +705,12 @@ def _api_base_kwarg(kwargs: Mapping[str, object]) -> str | None:
|
|||
return api_base if isinstance(api_base, str) else None
|
||||
|
||||
|
||||
def _dispatched_model_name(model: str, custom_llm_provider: str, api_base: str | None) -> str:
|
||||
provider_model, _, _, _ = litellm.get_llm_provider(
|
||||
model=model, custom_llm_provider=custom_llm_provider, api_base=api_base
|
||||
)
|
||||
def _dispatched_model_name(
|
||||
model: str, custom_llm_provider: str, api_base: str | None, litellm_params: GenericLiteLLMParams | None = None
|
||||
) -> str:
|
||||
from litellm.litellm_core_utils.get_llm_provider_logic import resolve_model_and_provider_for_metadata
|
||||
|
||||
provider_model, _ = resolve_model_and_provider_for_metadata(model=model, custom_llm_provider=custom_llm_provider)
|
||||
return _strip_responses_routing_prefix(provider_model)
|
||||
|
||||
|
||||
|
|
@ -718,6 +720,7 @@ def _will_bridge_to_chat_completions(
|
|||
use_chat_completions_api: bool,
|
||||
model_info: object,
|
||||
api_base: str | None,
|
||||
litellm_params: GenericLiteLLMParams | None = None,
|
||||
) -> bool:
|
||||
"""``_bridges_to_chat_completions`` for callers running before the provider config is resolved.
|
||||
|
||||
|
|
@ -734,7 +737,7 @@ def _will_bridge_to_chat_completions(
|
|||
return True
|
||||
return _bridges_to_chat_completions(
|
||||
_resolve_responses_api_provider_config(
|
||||
_dispatched_model_name(normalized_model[0], custom_llm_provider, api_base),
|
||||
_dispatched_model_name(normalized_model[0], custom_llm_provider, api_base, litellm_params),
|
||||
custom_llm_provider,
|
||||
model_info,
|
||||
api_base,
|
||||
|
|
@ -750,12 +753,15 @@ def provider_stores_the_conversation(model: str, custom_llm_provider: str | None
|
|||
first MCP round runs as an internal call that writes no spend log at all.
|
||||
"""
|
||||
api_base: Final = _api_base_kwarg(kwargs)
|
||||
litellm_params: Final = GenericLiteLLMParams(**kwargs)
|
||||
return not _will_bridge_to_chat_completions(
|
||||
model,
|
||||
custom_llm_provider or litellm.get_llm_provider(model=model, api_base=api_base)[1],
|
||||
custom_llm_provider
|
||||
or litellm.get_llm_provider(model=model, api_base=api_base, litellm_params=litellm_params)[1],
|
||||
bool(kwargs.get("use_chat_completions_api")),
|
||||
kwargs.get("model_info"),
|
||||
api_base,
|
||||
litellm_params,
|
||||
)
|
||||
|
||||
|
||||
|
|
@ -905,7 +911,8 @@ async def aresponses(
|
|||
model=model,
|
||||
api_base=local_vars.get("base_url", None),
|
||||
litellm_params=GenericLiteLLMParams(
|
||||
**cast("dict[str, object]", kwargs) # cast-ok: kwargs is the untyped request dict
|
||||
extra_headers=extra_headers,
|
||||
**cast("dict[str, object]", kwargs), # cast-ok: kwargs is the untyped request dict
|
||||
),
|
||||
)
|
||||
# Update local_vars with detected provider (fixes #19782)
|
||||
|
|
@ -936,6 +943,7 @@ async def aresponses(
|
|||
bool(kwargs.get("use_chat_completions_api")),
|
||||
kwargs.get("model_info"),
|
||||
_api_base_kwarg(kwargs),
|
||||
GenericLiteLLMParams(**kwargs),
|
||||
),
|
||||
):
|
||||
_stamp_injection_points_for_the_hook(kwargs, model, custom_llm_provider, input, tools)
|
||||
|
|
@ -967,7 +975,7 @@ async def aresponses(
|
|||
original_model=original_model,
|
||||
swapped_model=model,
|
||||
custom_llm_provider=custom_llm_provider,
|
||||
kwargs=kwargs,
|
||||
kwargs={**kwargs, "extra_headers": extra_headers},
|
||||
prompt_id=prompt_id,
|
||||
)
|
||||
kwargs.pop("prompt_id", None)
|
||||
|
|
@ -1051,11 +1059,14 @@ def _resolve_prompt_swapped_provider(
|
|||
kwargs: Mapping[str, object],
|
||||
prompt_id: str | None,
|
||||
) -> str:
|
||||
swapped_provider: Final = litellm.get_llm_provider(model=swapped_model)[1]
|
||||
litellm_params: Final = GenericLiteLLMParams(**kwargs)
|
||||
swapped_provider: Final = litellm.get_llm_provider(model=swapped_model, litellm_params=litellm_params)[1]
|
||||
if kwargs.get("api_key") is None and kwargs.get("api_base") is None:
|
||||
return swapped_provider
|
||||
try:
|
||||
original_provider: Final = custom_llm_provider or litellm.get_llm_provider(model=original_model)[1]
|
||||
original_provider: Final = (
|
||||
custom_llm_provider or litellm.get_llm_provider(model=original_model, litellm_params=litellm_params)[1]
|
||||
)
|
||||
except litellm.BadRequestError:
|
||||
return swapped_provider
|
||||
if swapped_provider == original_provider:
|
||||
|
|
@ -1108,6 +1119,7 @@ def _apply_prompt_management_to_responses_call(
|
|||
use_chat_completions_api,
|
||||
kwargs.get("model_info"),
|
||||
_api_base_kwarg(kwargs),
|
||||
GenericLiteLLMParams(**kwargs),
|
||||
),
|
||||
):
|
||||
_stamp_injection_points_for_the_hook(kwargs, model, custom_llm_provider, input, tools)
|
||||
|
|
@ -1142,7 +1154,7 @@ def _apply_prompt_management_to_responses_call(
|
|||
original_model=original_model,
|
||||
swapped_model=model,
|
||||
custom_llm_provider=custom_llm_provider,
|
||||
kwargs=kwargs,
|
||||
kwargs={**kwargs, "extra_headers": local_vars.get("extra_headers")},
|
||||
prompt_id=prompt_id,
|
||||
)
|
||||
)
|
||||
|
|
@ -1498,7 +1510,8 @@ def responses(
|
|||
model=model,
|
||||
api_base=local_vars.get("base_url", None),
|
||||
litellm_params=GenericLiteLLMParams(
|
||||
**cast("dict[str, object]", kwargs) # cast-ok: kwargs is the untyped request dict
|
||||
extra_headers=extra_headers,
|
||||
**cast("dict[str, object]", kwargs), # cast-ok: kwargs is the untyped request dict
|
||||
),
|
||||
)
|
||||
local_vars["custom_llm_provider"] = custom_llm_provider
|
||||
|
|
@ -1514,8 +1527,16 @@ def responses(
|
|||
use_chat_completions_api=use_chat_completions_api,
|
||||
)
|
||||
|
||||
# get llm provider logic
|
||||
litellm_params: Final = GenericLiteLLMParams(**kwargs)
|
||||
from litellm.litellm_core_utils.core_helpers import attribute_or_none
|
||||
|
||||
litellm_params: Final = GenericLiteLLMParams(extra_headers=extra_headers, **kwargs)
|
||||
provider_headers: Final = ResponsesAPIRequestUtils.merge_provider_specific_headers(
|
||||
extra_headers=extra_headers,
|
||||
provider_specific_header=attribute_or_none(litellm_params, "provider_specific_header"),
|
||||
custom_llm_provider=custom_llm_provider,
|
||||
)
|
||||
local_vars["extra_headers"] = provider_headers
|
||||
|
||||
affinity_api_base: Final = (
|
||||
get_openai_credentials(api_base=litellm_params.api_base).api_base
|
||||
if custom_llm_provider == "openai"
|
||||
|
|
@ -1525,7 +1546,7 @@ def responses(
|
|||
try:
|
||||
effective_extra_headers: Final = (
|
||||
add_provider_affinity_header(
|
||||
headers=extra_headers or MappingProxyType({}),
|
||||
headers=provider_headers or MappingProxyType({}),
|
||||
litellm_params=MappingProxyType(
|
||||
{
|
||||
"provider_affinity_header": litellm_params.provider_affinity_header,
|
||||
|
|
@ -1538,7 +1559,7 @@ def responses(
|
|||
default_header=default_affinity_header,
|
||||
)
|
||||
if litellm_params.provider_affinity_header is not None or default_affinity_header is not None
|
||||
else extra_headers
|
||||
else provider_headers
|
||||
)
|
||||
except ValueError as affinity_error:
|
||||
raise litellm.BadRequestError(
|
||||
|
|
|
|||
|
|
@ -12,6 +12,7 @@ import litellm
|
|||
from litellm._logging import verbose_logger
|
||||
from litellm.litellm_core_utils.core_helpers import as_str_mapping, attribute_or_none, iter_items
|
||||
from litellm.litellm_core_utils.dot_notation_indexing import delete_nested_value, is_nested_path
|
||||
from litellm.litellm_core_utils.get_provider_specific_headers import ProviderSpecificHeaderUtils
|
||||
from litellm.llms.base_llm.responses.transformation import BaseResponsesAPIConfig, ReasoningSummaryLevel
|
||||
from litellm.types.llms.openai import (
|
||||
AllMessageValues,
|
||||
|
|
@ -29,11 +30,16 @@ from litellm.types.utils import (
|
|||
CacheCreationTokenDetails,
|
||||
CompletionTokensDetailsWrapper,
|
||||
PromptTokensDetailsWrapper,
|
||||
ProviderSpecificHeader,
|
||||
SpecialEnums,
|
||||
Usage,
|
||||
text_tokens_without_nested_reasoning,
|
||||
)
|
||||
|
||||
_PROVIDER_SPECIFIC_HEADER_ADAPTER: Final[
|
||||
TypeAdapter[ProviderSpecificHeader | Sequence[ProviderSpecificHeader] | None]
|
||||
] = TypeAdapter(ProviderSpecificHeader | Sequence[ProviderSpecificHeader] | None)
|
||||
|
||||
|
||||
class _UsageExtraFields(ExtraItemsTypedDict, total=False, extra_items=object):
|
||||
pass
|
||||
|
|
@ -233,6 +239,20 @@ class ResponsesAPIRequestUtils:
|
|||
**extra_headers,
|
||||
}
|
||||
|
||||
@staticmethod
|
||||
def merge_provider_specific_headers(
|
||||
extra_headers: Mapping[str, object] | None,
|
||||
provider_specific_header: object,
|
||||
custom_llm_provider: str | None,
|
||||
) -> dict[str, object] | None: # mutable-ok: legacy Responses request fields require a dict
|
||||
scoped_headers: Final = ProviderSpecificHeaderUtils.get_provider_specific_headers(
|
||||
provider_specific_header=_PROVIDER_SPECIFIC_HEADER_ADAPTER.validate_python(provider_specific_header),
|
||||
custom_llm_provider=custom_llm_provider,
|
||||
)
|
||||
if extra_headers is None and not scoped_headers:
|
||||
return None
|
||||
return {**(extra_headers or {}), **scoped_headers}
|
||||
|
||||
@staticmethod
|
||||
def _check_valid_arg(
|
||||
supported_params: list[str] | None,
|
||||
|
|
|
|||
|
|
@ -98,7 +98,9 @@ from litellm.litellm_core_utils.credential_accessor import CredentialAccessor, r
|
|||
from litellm.litellm_core_utils.dd_tracing import tracer
|
||||
from litellm.litellm_core_utils.get_llm_provider_logic import (
|
||||
declared_authenticating_provider,
|
||||
inferred_provider,
|
||||
is_registered_custom_provider,
|
||||
resolve_model_and_provider_for_metadata,
|
||||
)
|
||||
from litellm.litellm_core_utils.hidden_params import (
|
||||
HIDDEN_PARAMS_ATTR as _HIDDEN_PARAMS_ATTR,
|
||||
|
|
@ -561,6 +563,7 @@ _ALIAS_PARAMS_NEVER_FORWARDED: Final = frozenset(
|
|||
{"model", "api_base", "api_key", "api_version", "responses_api_version"}
|
||||
)
|
||||
_ALIAS_MARKER_FORWARDED_PARAMS_KWARG: Final = "_alias_marker_forwarded_params"
|
||||
_FORWARDED_CREDENTIAL_ISOLATION_PROVIDERS: Final = frozenset({"chatgpt"})
|
||||
_CLAUDE_CODE_SESSION_ID_RE: Final = re.compile(r"^[a-zA-Z0-9_\-]{8,}$")
|
||||
_CLAUDE_CODE_SESSION_ROUTER_TTL_SECONDS: Final = 3600
|
||||
CLAUDE_CODE_SESSION_ROUTER_BINDING_TARGET: Final = "claude_code_session_router_binding"
|
||||
|
|
@ -4801,9 +4804,39 @@ class Router:
|
|||
deployment_litellm_model_name = deployment["litellm_params"]["model"]
|
||||
deployment_api_base = deployment["litellm_params"].get("api_base")
|
||||
deployment_model_name: Final = deployment["model_name"]
|
||||
if is_clientside_credential(request_kwargs=kwargs):
|
||||
from litellm.litellm_core_utils.get_provider_specific_headers import ProviderSpecificHeaderUtils
|
||||
|
||||
request_context: Final = cast( # cast-ok: the router's legacy request dict has string keys
|
||||
Mapping[str, object], kwargs
|
||||
)
|
||||
raw_deployment_params: Final[object] = deployment["litellm_params"]
|
||||
deployment_context: Final = _LITELLM_PARAMS_ADAPTER.validate_python(
|
||||
raw_deployment_params.model_dump(exclude_none=True)
|
||||
if isinstance(raw_deployment_params, LiteLLM_Params)
|
||||
else raw_deployment_params
|
||||
)
|
||||
declared_provider: Final = deployment_context.get("custom_llm_provider")
|
||||
declared_model: Final = deployment_context.get("model")
|
||||
deployment_provider: Final = (
|
||||
declared_provider
|
||||
if isinstance(declared_provider, str)
|
||||
else inferred_provider(declared_model if isinstance(declared_model, str) else None)
|
||||
)
|
||||
forwarded_authorization: Final = (
|
||||
ProviderSpecificHeaderUtils.forwarded_authorization(
|
||||
request_context.get("provider_specific_header"), deployment_provider
|
||||
)
|
||||
if deployment_provider in _FORWARDED_CREDENTIAL_ISOLATION_PROVIDERS
|
||||
else None
|
||||
)
|
||||
if is_clientside_credential(request_kwargs=kwargs) or forwarded_authorization is not None:
|
||||
deployment_pydantic_obj: Final = self._handle_clientside_credential(
|
||||
deployment=deployment, kwargs=kwargs, function_name=function_name
|
||||
deployment=deployment,
|
||||
kwargs={
|
||||
**kwargs,
|
||||
**({"api_key": forwarded_authorization} if forwarded_authorization is not None else {}),
|
||||
},
|
||||
function_name=function_name,
|
||||
)
|
||||
model_info = deployment_pydantic_obj.model_info.model_dump()
|
||||
deployment_litellm_model_name = deployment_pydantic_obj.litellm_params.model
|
||||
|
|
@ -11048,6 +11081,10 @@ class Router:
|
|||
custom_llm_provider = None
|
||||
dynamic_api_key = None
|
||||
api_base = None
|
||||
elif declared_authenticating_provider(litellm_model, deployment.litellm_params.custom_llm_provider) is not None:
|
||||
_model, custom_llm_provider = ( # rebind-ok: each branch resolves the deployment provider
|
||||
self._model_group_llm_provider(deployment.litellm_params)
|
||||
)
|
||||
else:
|
||||
# check if model provider in supported providers
|
||||
(
|
||||
|
|
@ -12401,6 +12438,17 @@ class Router:
|
|||
)
|
||||
return model_info
|
||||
|
||||
@staticmethod
|
||||
def _model_group_llm_provider(litellm_params: LiteLLM_Params) -> tuple[str, str]:
|
||||
try:
|
||||
return resolve_model_and_provider_for_metadata(
|
||||
model=litellm_params.model,
|
||||
custom_llm_provider=litellm_params.custom_llm_provider,
|
||||
)
|
||||
except litellm.exceptions.BadRequestError as e:
|
||||
verbose_router_logger.error("litellm.router.py::get_model_group_info() - %s", e)
|
||||
return "", ""
|
||||
|
||||
def _set_model_group_info(
|
||||
self, model_group: str, user_facing_model_group_name: str, team_id: str | None = None
|
||||
) -> ModelGroupInfo | None:
|
||||
|
|
@ -12489,15 +12537,7 @@ class Router:
|
|||
deployment_is_mapped = deployment_is_catalog_mapped(model_info, model_info_dict)
|
||||
|
||||
# get llm provider
|
||||
litellm_model, llm_provider = "", ""
|
||||
try:
|
||||
litellm_model, llm_provider, _, _ = litellm.get_llm_provider(
|
||||
model=litellm_params.model,
|
||||
custom_llm_provider=litellm_params.custom_llm_provider,
|
||||
litellm_params=litellm_params,
|
||||
)
|
||||
except litellm.exceptions.BadRequestError as e:
|
||||
verbose_router_logger.error("litellm.router.py::get_model_group_info() - %s", e)
|
||||
litellm_model, llm_provider = self._model_group_llm_provider(litellm_params)
|
||||
|
||||
if model_info is None:
|
||||
supported_openai_params = litellm.get_supported_openai_params(
|
||||
|
|
|
|||
|
|
@ -91,6 +91,7 @@ class ProviderConnection:
|
|||
litellm_credential_name: str | None = None
|
||||
configurable_clientside_auth_params: "Sequence[str | ConfigurableClientsideParamsCustomAuth] | None" = None
|
||||
use_xai_oauth: bool | None = None
|
||||
chatgpt_auth_file: str | None = None
|
||||
github_copilot_auth_type: str | None = None
|
||||
github_copilot_host: str | None = None
|
||||
github_copilot_user_session: object | None = None
|
||||
|
|
|
|||
|
|
@ -572,6 +572,10 @@ class GenericLiteLLMParams(CredentialLiteLLMParams, CustomPricingLiteLLMParams):
|
|||
default=False,
|
||||
description="Use stored xAI OAuth credentials when no xAI API key is configured.",
|
||||
)
|
||||
chatgpt_auth_file: str | None = Field(
|
||||
default=None,
|
||||
description="Path to a ChatGPT OAuth auth.json for this deployment.",
|
||||
)
|
||||
fireworks_forward_user_id: bool | None = Field(
|
||||
default=None,
|
||||
description="Send the LiteLLM user id of the calling key as the `user` field on Fireworks AI chat, responses and messages requests.",
|
||||
|
|
|
|||
|
|
@ -4360,6 +4360,7 @@ server_owned_wif_litellm_params: Final = (
|
|||
+ openai_wif_litellm_params
|
||||
+ oauth_token_exchange_litellm_params
|
||||
+ github_copilot_oauth_litellm_params
|
||||
+ ("chatgpt_auth_file",)
|
||||
)
|
||||
secret_bearing_wif_litellm_params: Final = tuple(sorted(WIF_SECRET_BEARING_KEYS))
|
||||
|
||||
|
|
|
|||
|
|
@ -2964,12 +2964,14 @@ def supports_native_streaming(model: str, custom_llm_provider: str | None) -> bo
|
|||
Raises:
|
||||
Exception: If the given model is not found in model_prices_and_context_window.json.
|
||||
"""
|
||||
from litellm.litellm_core_utils.get_llm_provider_logic import resolve_model_and_provider_for_metadata
|
||||
|
||||
try:
|
||||
model, custom_llm_provider, _, _ = litellm.get_llm_provider(
|
||||
resolved_model, resolved_provider = resolve_model_and_provider_for_metadata(
|
||||
model=model, custom_llm_provider=custom_llm_provider
|
||||
)
|
||||
|
||||
model_info: Final = get_model_info_helper(model=model, custom_llm_provider=custom_llm_provider)
|
||||
model_info: Final = get_model_info_helper(model=resolved_model, custom_llm_provider=resolved_provider)
|
||||
supports_native_streaming = model_info.get("supports_native_streaming", True)
|
||||
if supports_native_streaming is None:
|
||||
supports_native_streaming = True
|
||||
|
|
|
|||
|
|
@ -1,5 +1,9 @@
|
|||
from collections.abc import Iterator
|
||||
from pathlib import Path
|
||||
from typing import Final
|
||||
|
||||
import respx
|
||||
|
||||
import asyncio, httpx, importlib, os
|
||||
import pytest
|
||||
|
||||
|
|
@ -8,6 +12,7 @@ from litellm import CustomLLM
|
|||
from litellm.litellm_core_utils import get_llm_provider_logic
|
||||
from litellm.litellm_core_utils.get_llm_provider_logic import (
|
||||
get_llm_provider,
|
||||
resolve_model_and_provider_for_metadata,
|
||||
get_provider_affinity_header,
|
||||
inferred_provider,
|
||||
is_registered_custom_provider,
|
||||
|
|
@ -715,3 +720,47 @@ def test_opencode_affinity_resolution_preserves_existing_transport(prefix: str)
|
|||
model, provider, _, destination = get_llm_provider(model=f"{prefix}/test-model", api_base=api_base)
|
||||
assert (model, provider, destination) == ("test-model", prefix, api_base)
|
||||
assert get_provider_affinity_header(destination) == "x-opencode-session"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def chatgpt_login_tripwire(monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> Iterator[None]:
|
||||
blocked: Final = tmp_path / "blocked"
|
||||
blocked.write_text("ordinary file")
|
||||
monkeypatch.setenv("CHATGPT_TOKEN_DIR", str(blocked / "unused"))
|
||||
monkeypatch.delenv("CHATGPT_AUTH_FILE", raising=False)
|
||||
with respx.mock as transport:
|
||||
yield
|
||||
assert len(transport.calls) == 0
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("model", "custom_llm_provider", "expected"),
|
||||
[
|
||||
("chatgpt/gpt-5.4", None, ("gpt-5.4", "chatgpt")),
|
||||
("gpt-5.4", "chatgpt", ("gpt-5.4", "chatgpt")),
|
||||
("github_copilot/gpt-4o", None, ("gpt-4o", "github_copilot")),
|
||||
("openai/gpt-5.4", None, ("gpt-5.4", "openai")),
|
||||
],
|
||||
)
|
||||
def test_resolve_model_and_provider_for_metadata_never_logs_in(
|
||||
chatgpt_login_tripwire: None, model: str, custom_llm_provider: str | None, expected: tuple[str, str]
|
||||
) -> None:
|
||||
assert resolve_model_and_provider_for_metadata(model, custom_llm_provider) == expected
|
||||
|
||||
|
||||
def test_metadata_callers_never_log_in_for_a_declared_chatgpt_deployment(
|
||||
chatgpt_login_tripwire: None, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
from litellm.litellm_core_utils.llm_response_utils.get_api_base import get_api_base
|
||||
from litellm.proxy.spend_tracking.savings import _resolve_model
|
||||
from litellm.router import Router
|
||||
from litellm.types.router import LiteLLM_Params
|
||||
from litellm.utils import supports_native_streaming
|
||||
|
||||
monkeypatch.setenv("LITELLM_LOCAL_MODEL_COST_MAP", "True")
|
||||
|
||||
assert supports_native_streaming("chatgpt/gpt-5.4", None) is True
|
||||
assert get_api_base("chatgpt/gpt-5.4", {"model": "chatgpt/gpt-5.4"}) == "https://chatgpt.com/backend-api/codex"
|
||||
identity = _resolve_model("chatgpt/gpt-5.4", None)
|
||||
assert identity is not None and identity.provider == "chatgpt"
|
||||
assert Router._model_group_llm_provider(LiteLLM_Params(model="chatgpt/gpt-5.4")) == ("gpt-5.4", "chatgpt")
|
||||
|
|
|
|||
638
tests/unit/llms/chatgpt/chat/test_chatgpt_chat_transformation.py
Normal file
638
tests/unit/llms/chatgpt/chat/test_chatgpt_chat_transformation.py
Normal file
|
|
@ -0,0 +1,638 @@
|
|||
from unittest.mock import MagicMock
|
||||
|
||||
import pytest
|
||||
|
||||
from litellm.exceptions import AuthenticationError
|
||||
from litellm.llms.chatgpt.chat.transformation import ChatGPTConfig
|
||||
from litellm.llms.chatgpt.common_utils import GetAccessTokenError
|
||||
|
||||
GATEWAY_BASE = "https://chatgpt.example/backend-api/codex"
|
||||
|
||||
|
||||
def _gateway_authenticator() -> MagicMock:
|
||||
authenticator = MagicMock()
|
||||
authenticator.get_api_base.return_value = GATEWAY_BASE
|
||||
authenticator.get_access_token.return_value = "gateway-access-token"
|
||||
return authenticator
|
||||
|
||||
|
||||
def _provider_info(
|
||||
config: ChatGPTConfig, api_base: str | None, api_key: str | None
|
||||
) -> tuple[str | None, str | None, str]:
|
||||
return config._get_openai_compatible_provider_info(
|
||||
model="gpt-5.4", api_base=api_base, api_key=api_key, custom_llm_provider="chatgpt"
|
||||
)
|
||||
|
||||
|
||||
class TestChatGPTChatProviderInfo:
|
||||
def test_configured_key_uses_configured_base_without_gateway_login(self):
|
||||
authenticator = _gateway_authenticator()
|
||||
config = ChatGPTConfig(authenticator=authenticator)
|
||||
|
||||
assert _provider_info(config, "http://127.0.0.1:9999", "configured-key") == (
|
||||
"http://127.0.0.1:9999",
|
||||
"configured-key",
|
||||
"chatgpt",
|
||||
)
|
||||
authenticator.get_access_token.assert_not_called()
|
||||
|
||||
def test_configured_key_without_base_uses_gateway_base(self):
|
||||
config = ChatGPTConfig(authenticator=_gateway_authenticator())
|
||||
|
||||
assert _provider_info(config, None, "configured-key") == (GATEWAY_BASE, "configured-key", "chatgpt")
|
||||
|
||||
def test_gateway_login_is_never_sent_to_a_custom_base(self):
|
||||
config = ChatGPTConfig(authenticator=_gateway_authenticator())
|
||||
|
||||
assert _provider_info(config, "http://attacker.example", None) == (
|
||||
GATEWAY_BASE,
|
||||
"gateway-access-token",
|
||||
"chatgpt",
|
||||
)
|
||||
|
||||
def test_missing_gateway_login_raises_authentication_error(self):
|
||||
authenticator = _gateway_authenticator()
|
||||
authenticator.get_access_token.side_effect = GetAccessTokenError(status_code=401, message="not logged in")
|
||||
config = ChatGPTConfig(authenticator=authenticator)
|
||||
|
||||
with pytest.raises(AuthenticationError, match="not logged in"):
|
||||
_provider_info(config, None, None)
|
||||
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
from pathlib import Path
|
||||
from types import MappingProxyType
|
||||
from typing import Final
|
||||
|
||||
import httpx
|
||||
import respx
|
||||
|
||||
import litellm
|
||||
from litellm.integrations.dotprompt.dotprompt_manager import DotpromptManager
|
||||
from litellm.litellm_core_utils.get_llm_provider_logic import get_llm_provider
|
||||
from litellm.llms.chatgpt.chat.transformation import ChatGPTConfig
|
||||
from litellm.types.router import GenericLiteLLMParams
|
||||
|
||||
|
||||
def _write_auth_record(path: Path, token: str, account_id: str) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"access_token": token,
|
||||
"account_id": account_id,
|
||||
"expires_at": 4102444800,
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def auth_files(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> tuple[Path, Path]:
|
||||
monkeypatch.setattr("time.time", lambda: 1700000000)
|
||||
default_dir: Final = tmp_path / "default"
|
||||
monkeypatch.setenv("CHATGPT_TOKEN_DIR", str(default_dir))
|
||||
monkeypatch.delenv("CHATGPT_AUTH_FILE", raising=False)
|
||||
_write_auth_record(default_dir / "auth.json", "token-default", "acct-default")
|
||||
file_a: Final = tmp_path / "account-a" / "auth.json"
|
||||
file_b: Final = tmp_path / "account-b" / "auth.json"
|
||||
_write_auth_record(file_a, "token-a", "acct-a")
|
||||
_write_auth_record(file_b, "token-b", "acct-b")
|
||||
return file_a, file_b
|
||||
|
||||
|
||||
class TestChatGPTMultiAccountChat:
|
||||
def test_validate_environment_uses_per_model_auth_file(self, auth_files: tuple[Path, Path]) -> None:
|
||||
file_a, file_b = auth_files
|
||||
config: Final = ChatGPTConfig()
|
||||
|
||||
headers_a: Final = config.validate_environment(
|
||||
headers={},
|
||||
model="gpt-6.1-sol",
|
||||
messages=[],
|
||||
optional_params={},
|
||||
litellm_params={"chatgpt_auth_file": str(file_a)},
|
||||
api_key="token-default",
|
||||
)
|
||||
headers_b: Final = config.validate_environment(
|
||||
headers={},
|
||||
model="gpt-6.1-sol",
|
||||
messages=[],
|
||||
optional_params={},
|
||||
litellm_params={"chatgpt_auth_file": str(file_b)},
|
||||
api_key="token-default",
|
||||
)
|
||||
|
||||
assert headers_a["Authorization"] == "Bearer token-a"
|
||||
assert headers_a["ChatGPT-Account-Id"] == "acct-a"
|
||||
assert headers_b["Authorization"] == "Bearer token-b"
|
||||
assert headers_b["ChatGPT-Account-Id"] == "acct-b"
|
||||
|
||||
def test_validate_environment_defaults_to_global_auth_file(self, auth_files: tuple[Path, Path]) -> None:
|
||||
config: Final = ChatGPTConfig()
|
||||
|
||||
headers: Final = config.validate_environment(
|
||||
headers={},
|
||||
model="gpt-6.1-sol",
|
||||
messages=[],
|
||||
optional_params={},
|
||||
litellm_params={},
|
||||
api_key="token-default",
|
||||
)
|
||||
|
||||
assert headers["Authorization"] == "Bearer token-default"
|
||||
assert headers["ChatGPT-Account-Id"] == "acct-default"
|
||||
|
||||
def test_provider_info_uses_per_model_auth_file(self, auth_files: tuple[Path, Path]) -> None:
|
||||
file_a, _ = auth_files
|
||||
config: Final = ChatGPTConfig()
|
||||
|
||||
_, per_model_key, _ = config._get_openai_compatible_provider_info(
|
||||
"gpt-6.1-sol", None, None, "chatgpt", {"chatgpt_auth_file": str(file_a)}
|
||||
)
|
||||
_, default_key, _ = config._get_openai_compatible_provider_info("gpt-6.1-sol", None, None, "chatgpt")
|
||||
|
||||
assert per_model_key == "token-a"
|
||||
assert default_key == "token-default"
|
||||
|
||||
def test_get_llm_provider_threads_auth_file(self, auth_files: tuple[Path, Path]) -> None:
|
||||
file_a, _ = auth_files
|
||||
|
||||
model, provider, dynamic_api_key, _ = get_llm_provider(
|
||||
model="chatgpt/gpt-6.1-sol",
|
||||
litellm_params=GenericLiteLLMParams(chatgpt_auth_file=str(file_a)),
|
||||
)
|
||||
|
||||
assert model == "gpt-6.1-sol"
|
||||
assert provider == "chatgpt"
|
||||
assert dynamic_api_key == "token-a"
|
||||
|
||||
|
||||
def _subscription_response(request: httpx.Request) -> httpx.Response:
|
||||
account: Final = request.headers["ChatGPT-Account-Id"]
|
||||
assert request.headers["Authorization"] in (
|
||||
f"Bearer token-{account.removeprefix('acct-')}",
|
||||
f"Bearer {_client_token(account)}",
|
||||
)
|
||||
assert "chatgpt_auth_file" not in request.content.decode()
|
||||
message: Final = {
|
||||
"id": "msg_test",
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"status": "completed",
|
||||
"content": [{"type": "output_text", "text": account, "annotations": []}],
|
||||
}
|
||||
response: Final = {
|
||||
"id": "resp_test",
|
||||
"object": "response",
|
||||
"created_at": 1700000000,
|
||||
"status": "completed",
|
||||
"model": "gpt-6.1-sol",
|
||||
"output": [message],
|
||||
"usage": {"input_tokens": 1, "output_tokens": 1, "total_tokens": 2},
|
||||
}
|
||||
events: Final = (
|
||||
{"type": "response.created", "response": {**response, "status": "in_progress", "output": []}},
|
||||
{
|
||||
"type": "response.output_text.delta",
|
||||
"item_id": "msg_test",
|
||||
"output_index": 0,
|
||||
"content_index": 0,
|
||||
"delta": account,
|
||||
},
|
||||
{"type": "response.output_item.done", "output_index": 0, "item": message},
|
||||
{"type": "response.completed", "response": response},
|
||||
)
|
||||
return httpx.Response(
|
||||
200,
|
||||
text="".join(f"data: {json.dumps(event)}\n\n" for event in events),
|
||||
headers={"content-type": "text/event-stream"},
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("blocked_default", [False, True])
|
||||
@pytest.mark.parametrize("with_prompt", [False, True])
|
||||
@pytest.mark.parametrize("endpoint", ["chat", "responses"])
|
||||
@pytest.mark.parametrize("through_router", [False, True])
|
||||
@pytest.mark.asyncio
|
||||
async def test_requests_use_selected_subscription(
|
||||
auth_files: tuple[Path, Path],
|
||||
endpoint: str,
|
||||
through_router: bool,
|
||||
with_prompt: bool,
|
||||
blocked_default: bool,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
monkeypatch.setattr(litellm, "disable_aiohttp_transport", True)
|
||||
default_dir: Final = auth_files[0].parent / "missing-default"
|
||||
if blocked_default:
|
||||
default_dir.write_text("ordinary file")
|
||||
monkeypatch.setenv("CHATGPT_TOKEN_DIR", str(default_dir / "unused"))
|
||||
if with_prompt:
|
||||
monkeypatch.setattr(
|
||||
litellm,
|
||||
"callbacks",
|
||||
[
|
||||
DotpromptManager(
|
||||
prompt_id="account-check",
|
||||
prompt_data={"content": "Saved account prompt", "metadata": {"model": "chatgpt/gpt-6.1-sol"}},
|
||||
)
|
||||
],
|
||||
)
|
||||
router: Final = (
|
||||
litellm.Router(
|
||||
model_list=[
|
||||
{
|
||||
"model_name": f"account-{index}",
|
||||
"litellm_params": {"model": "chatgpt/gpt-6.1-sol", "chatgpt_auth_file": str(path)},
|
||||
}
|
||||
for index, path in enumerate(auth_files)
|
||||
]
|
||||
)
|
||||
if through_router
|
||||
else None
|
||||
)
|
||||
|
||||
async def request(index: int, path: Path) -> str:
|
||||
params: Final = MappingProxyType(
|
||||
{
|
||||
"model": f"account-{index}" if router else "chatgpt/gpt-6.1-sol",
|
||||
**({"chatgpt_auth_file": str(path)} if router is None else {}),
|
||||
**({"prompt_id": "account-check"} if with_prompt else {}),
|
||||
}
|
||||
)
|
||||
if endpoint == "chat":
|
||||
result: Final = await (router.acompletion if router else litellm.acompletion)(
|
||||
**params, messages=[{"role": "user", "content": "Hello"}]
|
||||
)
|
||||
return result.choices[0].message.content
|
||||
response: Final = await (router.aresponses if router else litellm.aresponses)(
|
||||
**params, input=[{"role": "user", "content": "Hello"}]
|
||||
)
|
||||
return response.output[0].content[0].text
|
||||
|
||||
with respx.mock as transport:
|
||||
transport.post("https://chatgpt.com/backend-api/codex/responses").mock(side_effect=_subscription_response)
|
||||
results: Final = await asyncio.gather(*(request(index, path) for index, path in enumerate(auth_files)))
|
||||
assert results == ["acct-a", "acct-b"]
|
||||
if with_prompt:
|
||||
assert all("Saved account prompt" in call.request.content.decode() for call in transport.calls)
|
||||
|
||||
|
||||
def _client_token(account: str) -> str:
|
||||
import base64
|
||||
|
||||
payload: Final = (
|
||||
base64.urlsafe_b64encode(json.dumps({"https://api.openai.com/auth": {"chatgpt_account_id": account}}).encode())
|
||||
.rstrip(b"=")
|
||||
.decode()
|
||||
)
|
||||
return f"header.{payload}.signature"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("endpoint", ["chat", "responses"])
|
||||
@pytest.mark.parametrize("configured_login", [False, True])
|
||||
@pytest.mark.parametrize("with_prompt", [False, True])
|
||||
@pytest.mark.parametrize("stream", [False, True])
|
||||
@pytest.mark.asyncio
|
||||
async def test_client_login_keeps_its_account_across_gateway_deployments(
|
||||
auth_files: tuple[Path, Path],
|
||||
endpoint: str,
|
||||
configured_login: bool,
|
||||
with_prompt: bool,
|
||||
stream: bool,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
) -> None:
|
||||
monkeypatch.setattr(litellm, "disable_aiohttp_transport", True)
|
||||
monkeypatch.setenv("CHATGPT_TOKEN_DIR", str(auth_files[0].parent / "no-gateway-login"))
|
||||
if with_prompt:
|
||||
monkeypatch.setattr(
|
||||
litellm,
|
||||
"callbacks",
|
||||
[
|
||||
DotpromptManager(
|
||||
prompt_id="client-account-check",
|
||||
prompt_data={"content": "Saved client prompt", "metadata": {"model": "chatgpt/gpt-6.1-sol"}},
|
||||
)
|
||||
],
|
||||
)
|
||||
router: Final = litellm.Router(
|
||||
model_list=[
|
||||
{
|
||||
"model_name": "subscriptions",
|
||||
"litellm_params": {
|
||||
"model": "chatgpt/gpt-6.1-sol",
|
||||
**({"chatgpt_auth_file": str(path)} if configured_login else {}),
|
||||
},
|
||||
}
|
||||
for path in auth_files
|
||||
],
|
||||
num_retries=0,
|
||||
)
|
||||
|
||||
def reply(request: httpx.Request) -> httpx.Response:
|
||||
account: Final = request.headers["ChatGPT-Account-Id"]
|
||||
assert request.headers["Authorization"] == f"Bearer {_client_token(account)}"
|
||||
return _subscription_response(
|
||||
httpx.Request(
|
||||
"POST",
|
||||
request.url,
|
||||
content=request.content,
|
||||
headers={
|
||||
**{key: value for key, value in request.headers.items() if key.lower() != "authorization"},
|
||||
"Authorization": f"Bearer token-{account.removeprefix('acct-')}",
|
||||
},
|
||||
)
|
||||
)
|
||||
|
||||
async def call(account: str) -> str:
|
||||
params: Final = MappingProxyType(
|
||||
{
|
||||
"model": "subscriptions",
|
||||
"stream": stream,
|
||||
**({"prompt_id": "client-account-check"} if with_prompt else {}),
|
||||
"provider_specific_header": {
|
||||
"custom_llm_provider": "chatgpt",
|
||||
"extra_headers": {"Authorization": f"Bearer {_client_token(account)}"},
|
||||
},
|
||||
}
|
||||
)
|
||||
response: Final = await (
|
||||
router.acompletion(**params, messages=[{"role": "user", "content": "Hello"}])
|
||||
if endpoint == "chat"
|
||||
else router.aresponses(**params, input=[{"role": "user", "content": "Hello"}])
|
||||
)
|
||||
if stream:
|
||||
chunks: Final = [part async for part in response]
|
||||
return (
|
||||
"".join(part.choices[0].delta.content or "" for part in chunks)
|
||||
if endpoint == "chat"
|
||||
else "".join(
|
||||
getattr(part, "delta", "")
|
||||
for part in chunks
|
||||
if getattr(part, "type", "") == "response.output_text.delta"
|
||||
)
|
||||
)
|
||||
return response.choices[0].message.content if endpoint == "chat" else response.output[0].content[0].text
|
||||
|
||||
with respx.mock as transport:
|
||||
transport.post("https://chatgpt.com/backend-api/codex/responses").mock(side_effect=reply)
|
||||
results: Final = await asyncio.gather(call("acct-client-a"), call("acct-client-b"))
|
||||
assert results == ["acct-client-a", "acct-client-b"]
|
||||
assert len(transport.calls) == 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_one_client_limit_does_not_cool_down_other_subscriptions(
|
||||
auth_files: tuple[Path, Path], monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
monkeypatch.setattr(litellm, "disable_aiohttp_transport", True)
|
||||
router: Final = litellm.Router(
|
||||
model_list=[
|
||||
{"model_name": "pool", "litellm_params": {"model": "chatgpt/gpt-6.1-sol", "chatgpt_auth_file": str(path)}}
|
||||
for path in auth_files
|
||||
],
|
||||
num_retries=0,
|
||||
allowed_fails=0,
|
||||
cooldown_time=60,
|
||||
)
|
||||
headers: Final = {
|
||||
"custom_llm_provider": "chatgpt",
|
||||
"extra_headers": {"Authorization": f"Bearer {_client_token('acct-limited')}"},
|
||||
}
|
||||
with respx.mock as transport:
|
||||
route: Final = transport.post("https://chatgpt.com/backend-api/codex/responses")
|
||||
route.respond(
|
||||
429,
|
||||
headers={"retry-after": "60"},
|
||||
json={"error": {"message": "Usage limit reached", "type": "rate_limit_error"}},
|
||||
)
|
||||
for _ in range(2):
|
||||
with pytest.raises(litellm.RateLimitError):
|
||||
await router.aresponses(model="pool", input="Hello", provider_specific_header=headers)
|
||||
await asyncio.sleep(0)
|
||||
route.mock(side_effect=_subscription_response)
|
||||
response: Final = await router.aresponses(model="pool", input="Hello")
|
||||
assert response.output[0].content[0].text in ("acct-a", "acct-b")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("endpoint", ["chat", "responses"])
|
||||
@pytest.mark.parametrize("async_call", [False, True])
|
||||
@pytest.mark.parametrize("direct_headers", [False, True])
|
||||
@pytest.mark.parametrize("configured_key", [False, True])
|
||||
@pytest.mark.parametrize(
|
||||
"target", ["https://example.com/v1", "http://chatgpt.com/backend-api/codex", "https://chatgpt.com.example.com/v1"]
|
||||
)
|
||||
@pytest.mark.asyncio
|
||||
async def test_client_login_cannot_be_redirected_to_another_host(
|
||||
endpoint: str, target: str, async_call: bool, direct_headers: bool, configured_key: bool
|
||||
) -> None:
|
||||
headers: Final = {"Authorization": f"Bearer {_client_token('acct-client')}"}
|
||||
credential: Final = (
|
||||
{"extra_headers": headers}
|
||||
if direct_headers
|
||||
else {"provider_specific_header": {"custom_llm_provider": "chatgpt", "extra_headers": headers}}
|
||||
)
|
||||
params: Final = MappingProxyType(
|
||||
{
|
||||
"model": "chatgpt/gpt-6.1-sol",
|
||||
"api_base": target,
|
||||
**({"api_key": "configured-test-key"} if configured_key else {}),
|
||||
**credential,
|
||||
}
|
||||
)
|
||||
|
||||
async def request() -> None:
|
||||
if endpoint == "chat":
|
||||
if async_call:
|
||||
await litellm.acompletion(**params, messages=[{"role": "user", "content": "Hello"}])
|
||||
else:
|
||||
litellm.completion(**params, messages=[{"role": "user", "content": "Hello"}])
|
||||
elif async_call:
|
||||
await litellm.aresponses(**params, input="Hello")
|
||||
else:
|
||||
litellm.responses(**params, input="Hello")
|
||||
|
||||
with respx.mock as transport:
|
||||
with pytest.raises(litellm.BadRequestError, match="official HTTPS ChatGPT endpoint"):
|
||||
await request()
|
||||
assert len(transport.calls) == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_cross_provider_fallback_does_not_receive_the_chatgpt_login(
|
||||
auth_files: tuple[Path, Path], monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
monkeypatch.setattr(litellm, "disable_aiohttp_transport", True)
|
||||
router: Final = litellm.Router(
|
||||
model_list=[
|
||||
{
|
||||
"model_name": "subscription",
|
||||
"litellm_params": {"model": "chatgpt/gpt-6.1-sol", "chatgpt_auth_file": str(auth_files[0])},
|
||||
},
|
||||
{
|
||||
"model_name": "api-fallback",
|
||||
"litellm_params": {"model": "openai/gpt-6.1-sol", "api_key": "sk-example-fallback"},
|
||||
},
|
||||
],
|
||||
fallbacks=[{"subscription": ["api-fallback"]}],
|
||||
num_retries=0,
|
||||
)
|
||||
with respx.mock as transport:
|
||||
transport.post("https://chatgpt.com/backend-api/codex/responses").respond(
|
||||
429, json={"error": {"message": "Usage limit reached", "type": "rate_limit_error"}}
|
||||
)
|
||||
fallback: Final = transport.post("https://api.openai.com/v1/responses").respond(
|
||||
200,
|
||||
json={
|
||||
"id": "resp_fallback",
|
||||
"object": "response",
|
||||
"created_at": 1700000000,
|
||||
"status": "completed",
|
||||
"model": "gpt-6.1-sol",
|
||||
"output": [
|
||||
{
|
||||
"id": "msg_fallback",
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"status": "completed",
|
||||
"content": [{"type": "output_text", "text": "fallback", "annotations": []}],
|
||||
}
|
||||
],
|
||||
"usage": {"input_tokens": 1, "output_tokens": 1, "total_tokens": 2},
|
||||
},
|
||||
)
|
||||
response: Final = await router.aresponses(
|
||||
model="subscription",
|
||||
input="Hello",
|
||||
provider_specific_header={
|
||||
"custom_llm_provider": "chatgpt",
|
||||
"extra_headers": {
|
||||
"Authorization": f"Bearer {_client_token('acct-client')}",
|
||||
"ChatGPT-Account-Id": "acct-client",
|
||||
},
|
||||
},
|
||||
)
|
||||
assert response.output[0].content[0].text == "fallback"
|
||||
assert fallback.calls.last.request.headers["Authorization"] == "Bearer sk-example-fallback"
|
||||
assert "ChatGPT-Account-Id" not in fallback.calls.last.request.headers
|
||||
|
||||
|
||||
@pytest.mark.parametrize("endpoint", ["chat", "responses"])
|
||||
@pytest.mark.parametrize("async_call", [False, True])
|
||||
@pytest.mark.parametrize("with_prompt", [False, True])
|
||||
@pytest.mark.asyncio
|
||||
async def test_sdk_extra_headers_use_client_login_without_gateway_auth(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch, endpoint: str, async_call: bool, with_prompt: bool
|
||||
) -> None:
|
||||
blocked: Final = tmp_path / "blocked"
|
||||
blocked.write_text("ordinary file")
|
||||
monkeypatch.setenv("CHATGPT_TOKEN_DIR", str(blocked / "unused"))
|
||||
monkeypatch.delenv("CHATGPT_AUTH_FILE", raising=False)
|
||||
monkeypatch.setattr(litellm, "disable_aiohttp_transport", True)
|
||||
headers: Final = {"Authorization": f"Bearer {_client_token('acct-client-a')}"}
|
||||
prompt_params: Final = {"prompt_id": "sdk-client"} if with_prompt else {}
|
||||
if with_prompt:
|
||||
monkeypatch.setattr(
|
||||
litellm,
|
||||
"callbacks",
|
||||
[
|
||||
DotpromptManager(
|
||||
prompt_id="sdk-client",
|
||||
prompt_data={"content": "Hello", "metadata": {"model": "chatgpt/gpt-6-luna"}},
|
||||
)
|
||||
],
|
||||
)
|
||||
with respx.mock as transport:
|
||||
transport.post("https://chatgpt.com/backend-api/codex/responses").mock(side_effect=_subscription_response)
|
||||
if endpoint == "chat":
|
||||
params: Final = {
|
||||
"model": "chatgpt/gpt-6.1-sol",
|
||||
"messages": [{"role": "user", "content": "Hello"}],
|
||||
"extra_headers": headers,
|
||||
**prompt_params,
|
||||
}
|
||||
result: Final = await litellm.acompletion(**params) if async_call else litellm.completion(**params)
|
||||
assert result.choices[0].message.content == "acct-client-a"
|
||||
else:
|
||||
response: Final = (
|
||||
await litellm.aresponses(
|
||||
model="chatgpt/gpt-6.1-sol", input="Hello", extra_headers=headers, **prompt_params
|
||||
)
|
||||
if async_call
|
||||
else litellm.responses(
|
||||
model="chatgpt/gpt-6.1-sol", input="Hello", extra_headers=headers, **prompt_params
|
||||
)
|
||||
)
|
||||
assert response.output[0].content[0].text == "acct-client-a"
|
||||
assert transport.calls.last.request.headers["Authorization"] == headers["Authorization"]
|
||||
assert len(transport.calls) == 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize("endpoint", ["chat", "responses"])
|
||||
@pytest.mark.parametrize("sync", [False, True])
|
||||
@pytest.mark.parametrize("credential_source", ["default", "file", "api_key"])
|
||||
@pytest.mark.parametrize("authorization_header", [None, "Authorization", "authorization"])
|
||||
@pytest.mark.asyncio
|
||||
async def test_requests_preserve_gateway_token_and_selected_workspace(
|
||||
tmp_path: Path,
|
||||
monkeypatch: pytest.MonkeyPatch,
|
||||
endpoint: str,
|
||||
sync: bool,
|
||||
credential_source: str,
|
||||
authorization_header: str | None,
|
||||
) -> None:
|
||||
monkeypatch.setattr("time.time", lambda: 1700000000)
|
||||
monkeypatch.setattr(litellm, "disable_aiohttp_transport", True)
|
||||
monkeypatch.setenv("CHATGPT_AUTH_FILE", str(tmp_path / "default.json"))
|
||||
token: Final = _client_token("acct-token-default")
|
||||
auth_file: Final = tmp_path / ("selected.json" if credential_source == "file" else "default.json")
|
||||
_write_auth_record(auth_file, token, "acct-saved-workspace")
|
||||
configured_token: Final = _client_token("acct-configured")
|
||||
expected_token: Final = configured_token if credential_source == "api_key" else token
|
||||
expected_account: Final = "acct-configured" if credential_source == "api_key" else "acct-saved-workspace"
|
||||
|
||||
def reply(request: httpx.Request) -> httpx.Response:
|
||||
assert request.headers.get_list("authorization") == [f"Bearer {expected_token}"]
|
||||
assert request.headers.get_list("chatgpt-account-id") == [expected_account]
|
||||
assert request.headers["originator"] == "codex_cli_rs"
|
||||
return _subscription_response(
|
||||
httpx.Request(
|
||||
"POST",
|
||||
request.url,
|
||||
content=request.content,
|
||||
headers={
|
||||
"Authorization": f"Bearer token-{expected_account.removeprefix('acct-')}",
|
||||
"ChatGPT-Account-Id": expected_account,
|
||||
},
|
||||
)
|
||||
)
|
||||
|
||||
params: Final = MappingProxyType(
|
||||
{
|
||||
"model": "chatgpt/gpt-6.1-sol",
|
||||
**({"chatgpt_auth_file": str(auth_file)} if credential_source == "file" else {}),
|
||||
**({"api_key": configured_token} if credential_source == "api_key" else {}),
|
||||
"extra_headers": {
|
||||
"originator": "codex_cli_rs",
|
||||
**({authorization_header: "Bearer sk-unrelated-example"} if authorization_header else {}),
|
||||
},
|
||||
}
|
||||
)
|
||||
with respx.mock as transport:
|
||||
transport.post("https://chatgpt.com/backend-api/codex/responses").mock(side_effect=reply)
|
||||
if endpoint == "chat":
|
||||
result: Final = (
|
||||
await asyncio.to_thread(litellm.completion, **params, messages=[{"role": "user", "content": "Hello"}])
|
||||
if sync
|
||||
else await litellm.acompletion(**params, messages=[{"role": "user", "content": "Hello"}])
|
||||
)
|
||||
assert result.choices[0].message.content == expected_account
|
||||
else:
|
||||
response: Final = (
|
||||
await asyncio.to_thread(litellm.responses, **params, input=[{"role": "user", "content": "Hello"}])
|
||||
if sync
|
||||
else await litellm.aresponses(**params, input=[{"role": "user", "content": "Hello"}])
|
||||
)
|
||||
assert response.output[0].content[0].text == expected_account
|
||||
|
|
@ -4,8 +4,10 @@ Tests for ChatGPT subscription Responses API transformation
|
|||
Source: litellm/llms/chatgpt/responses/transformation.py
|
||||
"""
|
||||
|
||||
import base64
|
||||
import json
|
||||
from collections.abc import Generator
|
||||
from pathlib import Path
|
||||
from typing import Final
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
|
|
@ -13,6 +15,7 @@ import httpx
|
|||
import pytest
|
||||
|
||||
import litellm
|
||||
from litellm.llms.chatgpt.authenticator import Authenticator
|
||||
from litellm.llms.chatgpt.responses.transformation import ChatGPTResponsesAPIConfig
|
||||
from litellm.llms.openai.common_utils import OpenAIError
|
||||
from litellm.main import responses_api_bridge_check
|
||||
|
|
@ -395,3 +398,165 @@ class TestChatGPTResponsesAPITransformation:
|
|||
|
||||
assert exc_info.value.message == "ChatGPT upstream failed"
|
||||
assert exc_info.value.status_code == 502
|
||||
|
||||
|
||||
def _fake_chatgpt_oauth_token(account_id: str) -> str:
|
||||
header = base64.urlsafe_b64encode(b'{"alg":"RS256"}').rstrip(b"=").decode()
|
||||
payload = (
|
||||
base64.urlsafe_b64encode(
|
||||
json.dumps({"https://api.openai.com/auth": {"chatgpt_account_id": account_id}}).encode()
|
||||
)
|
||||
.rstrip(b"=")
|
||||
.decode()
|
||||
)
|
||||
return f"{header}.{payload}.fake-signature"
|
||||
|
||||
|
||||
def _gateway_authenticator() -> MagicMock:
|
||||
authenticator = MagicMock()
|
||||
authenticator.get_access_token.return_value = "gateway-access-token"
|
||||
authenticator.get_account_id.return_value = "acct-gateway"
|
||||
return authenticator
|
||||
|
||||
|
||||
def _config_with(authenticator: MagicMock) -> ChatGPTResponsesAPIConfig:
|
||||
return ChatGPTResponsesAPIConfig(authenticator=authenticator)
|
||||
|
||||
|
||||
class TestChatGPTClientCredentialForwarding:
|
||||
def test_client_chatgpt_oauth_wins_over_configured_key_and_gateway_login(self):
|
||||
authenticator = _gateway_authenticator()
|
||||
token = _fake_chatgpt_oauth_token("acct-alice")
|
||||
|
||||
headers = _config_with(authenticator).validate_environment(
|
||||
headers={"Authorization": f"Bearer {token}", "originator": "codex_cli_rs"},
|
||||
model="gpt-5.4",
|
||||
litellm_params=GenericLiteLLMParams(api_key="configured-gateway-key", litellm_session_id="session-1"),
|
||||
)
|
||||
|
||||
assert headers["Authorization"] == f"Bearer {token}"
|
||||
assert headers["ChatGPT-Account-Id"] == "acct-alice"
|
||||
assert headers["originator"] == "codex_cli_rs"
|
||||
assert headers["accept"] == "text/event-stream"
|
||||
assert [name for name in headers if name.lower() == "authorization"] == ["Authorization"]
|
||||
authenticator.get_access_token.assert_not_called()
|
||||
|
||||
def test_explicit_account_header_wins_over_jwt_claim(self):
|
||||
token = _fake_chatgpt_oauth_token("acct-from-jwt")
|
||||
|
||||
headers = _config_with(_gateway_authenticator()).validate_environment(
|
||||
headers={"authorization": f"Bearer {token}", "chatgpt-account-id": "acct-from-header"},
|
||||
model="gpt-5.4",
|
||||
litellm_params=GenericLiteLLMParams(),
|
||||
)
|
||||
|
||||
assert headers["ChatGPT-Account-Id"] == "acct-from-header"
|
||||
assert [name for name in headers if name.lower() == "chatgpt-account-id"] == ["ChatGPT-Account-Id"]
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"bearer", ["Bearer sk-proj-fake-platform-key", "Bearer not-a-jwt", "Bearer a.bm90LWpzb24.c"]
|
||||
)
|
||||
def test_non_chatgpt_bearer_falls_back_to_configured_key(self, bearer, tmp_path):
|
||||
authenticator = Authenticator(auth_file=str(tmp_path / "missing-default.json"))
|
||||
|
||||
headers = ChatGPTResponsesAPIConfig(authenticator=authenticator).validate_environment(
|
||||
headers={"Authorization": bearer},
|
||||
model="gpt-5.4",
|
||||
litellm_params=GenericLiteLLMParams(api_key="configured-gateway-key"),
|
||||
)
|
||||
|
||||
assert "ChatGPT-Account-Id" not in headers
|
||||
assert headers["Authorization"] == "Bearer configured-gateway-key"
|
||||
|
||||
def test_configured_key_is_used_without_client_credential(self):
|
||||
authenticator = _gateway_authenticator()
|
||||
|
||||
headers = _config_with(authenticator).validate_environment(
|
||||
headers={},
|
||||
model="gpt-5.4",
|
||||
litellm_params=GenericLiteLLMParams(api_key="configured-gateway-key"),
|
||||
)
|
||||
|
||||
assert headers["Authorization"] == "Bearer configured-gateway-key"
|
||||
authenticator.get_access_token.assert_not_called()
|
||||
|
||||
def test_gateway_login_is_used_without_client_credential_or_configured_key(self):
|
||||
headers = _config_with(_gateway_authenticator()).validate_environment(
|
||||
headers={},
|
||||
model="gpt-5.4",
|
||||
litellm_params=GenericLiteLLMParams(),
|
||||
)
|
||||
|
||||
assert headers["Authorization"] == "Bearer gateway-access-token"
|
||||
assert headers["ChatGPT-Account-Id"] == "acct-gateway"
|
||||
|
||||
def test_two_clients_never_share_a_credential(self):
|
||||
config = _config_with(_gateway_authenticator())
|
||||
alice = _fake_chatgpt_oauth_token("acct-alice")
|
||||
bob = _fake_chatgpt_oauth_token("acct-bob")
|
||||
|
||||
forwarded = [
|
||||
config.validate_environment(
|
||||
headers={"Authorization": f"Bearer {token}"}, model="gpt-5.4", litellm_params=GenericLiteLLMParams()
|
||||
)
|
||||
for token in (alice, bob, alice)
|
||||
]
|
||||
|
||||
assert [h["Authorization"] for h in forwarded] == [f"Bearer {alice}", f"Bearer {bob}", f"Bearer {alice}"]
|
||||
assert [h["ChatGPT-Account-Id"] for h in forwarded] == ["acct-alice", "acct-bob", "acct-alice"]
|
||||
|
||||
|
||||
class TestChatGPTResponsesMultiAccount:
|
||||
@pytest.fixture(autouse=True)
|
||||
def fixed_clock(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setattr("time.time", lambda: 1700000000)
|
||||
|
||||
@staticmethod
|
||||
def _write_auth_record(path: Path, token: str, account_id: str) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"access_token": token,
|
||||
"account_id": account_id,
|
||||
"expires_at": 4102444800,
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
def test_validate_environment_uses_per_model_auth_file(
|
||||
self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
default_dir: Final = tmp_path / "default"
|
||||
monkeypatch.setenv("CHATGPT_TOKEN_DIR", str(default_dir))
|
||||
monkeypatch.delenv("CHATGPT_AUTH_FILE", raising=False)
|
||||
self._write_auth_record(default_dir / "auth.json", "token-default", "acct-default")
|
||||
file_a: Final = tmp_path / "account-a" / "auth.json"
|
||||
file_b: Final = tmp_path / "account-b" / "auth.json"
|
||||
self._write_auth_record(file_a, "token-a", "acct-a")
|
||||
self._write_auth_record(file_b, "token-b", "acct-b")
|
||||
|
||||
config: Final = ChatGPTResponsesAPIConfig()
|
||||
|
||||
headers_a: Final = config.validate_environment(
|
||||
headers={},
|
||||
model="gpt-6.1-sol",
|
||||
litellm_params=GenericLiteLLMParams(chatgpt_auth_file=str(file_a)),
|
||||
)
|
||||
headers_b: Final = config.validate_environment(
|
||||
headers={},
|
||||
model="gpt-6.1-sol",
|
||||
litellm_params=GenericLiteLLMParams(chatgpt_auth_file=str(file_b)),
|
||||
)
|
||||
headers_default: Final = config.validate_environment(
|
||||
headers={},
|
||||
model="gpt-6.1-sol",
|
||||
litellm_params=GenericLiteLLMParams(),
|
||||
)
|
||||
|
||||
assert headers_a["Authorization"] == "Bearer token-a"
|
||||
assert headers_a["ChatGPT-Account-Id"] == "acct-a"
|
||||
assert headers_b["Authorization"] == "Bearer token-b"
|
||||
assert headers_b["ChatGPT-Account-Id"] == "acct-b"
|
||||
assert headers_default["Authorization"] == "Bearer token-default"
|
||||
assert headers_default["ChatGPT-Account-Id"] == "acct-default"
|
||||
|
|
|
|||
|
|
@ -2,17 +2,24 @@ import base64
|
|||
import json
|
||||
import time
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from pathlib import Path
|
||||
from typing import Final
|
||||
from unittest.mock import MagicMock, mock_open, patch
|
||||
|
||||
import pytest
|
||||
import httpx
|
||||
import respx
|
||||
|
||||
import litellm
|
||||
from litellm.constants import DEFAULT_REQUEST_TIMEOUT_SECONDS, HTTP_HANDLER_CONNECT_TIMEOUT_SECONDS
|
||||
from litellm.llms.chatgpt.authenticator import (
|
||||
TOKEN_REFRESH_TIMEOUT_SECONDS,
|
||||
Authenticator,
|
||||
get_cached_authenticator,
|
||||
get_chatgpt_auth_file,
|
||||
)
|
||||
from litellm.llms.chatgpt.common_utils import GetAccessTokenError
|
||||
from litellm.types.router import GenericLiteLLMParams
|
||||
|
||||
|
||||
def _make_jwt(payload: dict) -> str:
|
||||
|
|
@ -161,3 +168,144 @@ class TestChatGPTAuthenticator:
|
|||
assert account_id == "acct-123"
|
||||
mock_write.assert_called_once()
|
||||
assert mock_write.call_args[0][0]["account_id"] == "acct-123"
|
||||
|
||||
def test_get_account_id_prefers_stored_value_without_rewriting_file(self, authenticator):
|
||||
id_token = _make_jwt({"https://api.openai.com/auth": {"chatgpt_account_id": "acct-from-token"}})
|
||||
auth_data = json.dumps({"account_id": "acct-stored", "id_token": id_token})
|
||||
|
||||
with (
|
||||
patch("builtins.open", mock_open(read_data=auth_data)),
|
||||
patch.object(authenticator, "_write_auth_file") as mock_write,
|
||||
):
|
||||
assert authenticator.get_account_id() == "acct-stored"
|
||||
mock_write.assert_not_called()
|
||||
|
||||
def test_get_account_id_without_login_returns_none(self, authenticator):
|
||||
with (
|
||||
patch("builtins.open", mock_open(read_data="{}")),
|
||||
patch.object(authenticator, "_write_auth_file") as mock_write,
|
||||
):
|
||||
assert authenticator.get_account_id() is None
|
||||
mock_write.assert_not_called()
|
||||
|
||||
|
||||
def _write_auth_record(path: Path, token: str, account_id: str) -> None:
|
||||
path.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"access_token": token,
|
||||
"account_id": account_id,
|
||||
"expires_at": 4102444800,
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
class TestChatGPTMultiAccountAuthenticator:
|
||||
@pytest.fixture(autouse=True)
|
||||
def fixed_clock(self, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setattr("time.time", lambda: 1700000000)
|
||||
|
||||
def test_explicit_auth_file_overrides_env(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setenv("CHATGPT_TOKEN_DIR", str(tmp_path / "env-dir"))
|
||||
monkeypatch.setenv("CHATGPT_AUTH_FILE", "env.json")
|
||||
custom_file: Final = tmp_path / "account-a" / "auth.json"
|
||||
|
||||
authenticator: Final = Authenticator(auth_file=str(custom_file))
|
||||
|
||||
assert authenticator.auth_file == str(custom_file)
|
||||
assert authenticator.token_dir == str(custom_file.parent)
|
||||
assert not custom_file.parent.exists()
|
||||
authenticator._write_auth_file({"access_token": "token-a"})
|
||||
assert json.loads(custom_file.read_text())["access_token"] == "token-a"
|
||||
|
||||
def test_default_auth_file_from_env(self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setenv("CHATGPT_TOKEN_DIR", str(tmp_path / "env-dir"))
|
||||
monkeypatch.setenv("CHATGPT_AUTH_FILE", "env.json")
|
||||
|
||||
authenticator: Final = Authenticator()
|
||||
|
||||
assert authenticator.auth_file == str(tmp_path / "env-dir" / "env.json")
|
||||
|
||||
def test_bare_filename_auth_file_resolves_against_cwd(
|
||||
self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
monkeypatch.chdir(tmp_path)
|
||||
|
||||
authenticator: Final = Authenticator(auth_file="auth.json")
|
||||
|
||||
assert authenticator.auth_file == str(tmp_path / "auth.json")
|
||||
assert authenticator.token_dir == str(tmp_path)
|
||||
|
||||
def test_two_auth_files_are_isolated(self, tmp_path: Path) -> None:
|
||||
file_a: Final = tmp_path / "account-a.json"
|
||||
file_b: Final = tmp_path / "account-b.json"
|
||||
_write_auth_record(file_a, "token-a", "acct-a")
|
||||
_write_auth_record(file_b, "token-b", "acct-b")
|
||||
|
||||
authenticator_a: Final = Authenticator(auth_file=str(file_a))
|
||||
authenticator_b: Final = Authenticator(auth_file=str(file_b))
|
||||
|
||||
assert authenticator_a.get_access_token() == "token-a"
|
||||
assert authenticator_a.get_account_id() == "acct-a"
|
||||
assert authenticator_b.get_access_token() == "token-b"
|
||||
assert authenticator_b.get_account_id() == "acct-b"
|
||||
|
||||
def test_get_cached_authenticator_reuses_instance_per_path(self, tmp_path: Path) -> None:
|
||||
file_a: Final = tmp_path / "account-a.json"
|
||||
file_b: Final = tmp_path / "account-b.json"
|
||||
|
||||
authenticator_a: Final = get_cached_authenticator(str(file_a))
|
||||
|
||||
assert get_cached_authenticator(str(file_a)) is authenticator_a
|
||||
assert get_cached_authenticator(str(file_b)) is not authenticator_a
|
||||
|
||||
def test_get_chatgpt_auth_file(self) -> None:
|
||||
assert get_chatgpt_auth_file(None) is None
|
||||
assert get_chatgpt_auth_file({}) is None
|
||||
assert get_chatgpt_auth_file({"chatgpt_auth_file": ""}) is None
|
||||
assert get_chatgpt_auth_file({"chatgpt_auth_file": "/a/auth.json"}) == "/a/auth.json"
|
||||
assert get_chatgpt_auth_file(GenericLiteLLMParams()) is None
|
||||
assert get_chatgpt_auth_file(GenericLiteLLMParams(chatgpt_auth_file="/b/auth.json")) == "/b/auth.json"
|
||||
|
||||
def test_relative_and_absolute_auth_file_share_cached_authenticator(
|
||||
self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
monkeypatch.chdir(tmp_path)
|
||||
get_cached_authenticator.cache_clear()
|
||||
auth_file: Final = tmp_path / "x" / "auth.json"
|
||||
auth_file.parent.mkdir(parents=True)
|
||||
auth_file.write_text("{}")
|
||||
|
||||
relative_key: Final = get_chatgpt_auth_file({"chatgpt_auth_file": "x/auth.json"})
|
||||
absolute_key: Final = get_chatgpt_auth_file({"chatgpt_auth_file": str(auth_file)})
|
||||
assert relative_key is not None and absolute_key is not None
|
||||
|
||||
assert get_cached_authenticator(relative_key) is get_cached_authenticator(absolute_key)
|
||||
|
||||
|
||||
def test_refresh_keeps_saved_workspace(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setattr("time.time", lambda: 1700000000)
|
||||
auth_file: Final = tmp_path / "selected.json"
|
||||
auth_file.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"access_token": "expired-example-token",
|
||||
"refresh_token": "example-refresh-token",
|
||||
"expires_at": 1,
|
||||
"account_id": "acct-selected-workspace",
|
||||
}
|
||||
)
|
||||
)
|
||||
token: Final = _make_jwt({"exp": 4102444800, "https://api.openai.com/auth": {"chatgpt_account_id": "acct-default"}})
|
||||
authenticator: Final = Authenticator(auth_file=str(auth_file))
|
||||
with respx.mock as transport:
|
||||
transport.post("https://auth.openai.com/oauth/token").mock(
|
||||
return_value=httpx.Response(
|
||||
200,
|
||||
json={"access_token": token, "id_token": token, "refresh_token": "example-rotated-refresh"},
|
||||
)
|
||||
)
|
||||
assert authenticator.get_access_token() == token
|
||||
assert authenticator.get_account_id(token) == "acct-selected-workspace"
|
||||
assert json.loads(auth_file.read_text())["refresh_token"] == "example-rotated-refresh"
|
||||
|
|
|
|||
90
tests/unit/llms/chatgpt/test_chatgpt_common_utils.py
Normal file
90
tests/unit/llms/chatgpt/test_chatgpt_common_utils.py
Normal file
|
|
@ -0,0 +1,90 @@
|
|||
import base64
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from litellm.llms.chatgpt.common_utils import (
|
||||
extract_chatgpt_account_id,
|
||||
get_chatgpt_client_credential,
|
||||
is_chatgpt_oauth_key,
|
||||
select_chatgpt_client_credential_headers,
|
||||
)
|
||||
|
||||
|
||||
def _make_jwt(payload: dict) -> str:
|
||||
def _b64(obj: dict) -> str:
|
||||
raw = json.dumps(obj, separators=(",", ":")).encode("utf-8")
|
||||
return base64.urlsafe_b64encode(raw).decode("utf-8").rstrip("=")
|
||||
|
||||
return f"{_b64({'alg': 'none', 'typ': 'JWT'})}.{_b64(payload)}."
|
||||
|
||||
|
||||
CHATGPT_JWT = _make_jwt({"https://api.openai.com/auth": {"chatgpt_account_id": "acct-alice"}})
|
||||
CHATGPT_JWT_WITHOUT_ACCOUNT = _make_jwt({"https://api.openai.com/auth": {"user_id": "user-1"}})
|
||||
NON_CHATGPT_JWT = _make_jwt({"sub": "someone", "https://api.openai.com/auth": "not-an-object"})
|
||||
|
||||
|
||||
class TestExtractChatGPTAccountId:
|
||||
def test_reads_account_id_from_auth_claim(self):
|
||||
assert extract_chatgpt_account_id(CHATGPT_JWT) == "acct-alice"
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"token",
|
||||
[None, "", "sk-proj-not-a-jwt", "header.only", NON_CHATGPT_JWT, CHATGPT_JWT_WITHOUT_ACCOUNT],
|
||||
)
|
||||
def test_returns_none_without_a_usable_account_claim(self, token):
|
||||
assert extract_chatgpt_account_id(token) is None
|
||||
|
||||
|
||||
class TestIsChatGPTOAuthKey:
|
||||
def test_recognizes_bearer_and_raw_chatgpt_jwt(self):
|
||||
assert is_chatgpt_oauth_key(CHATGPT_JWT)
|
||||
assert is_chatgpt_oauth_key(f"Bearer {CHATGPT_JWT}")
|
||||
assert is_chatgpt_oauth_key(f"Bearer {CHATGPT_JWT_WITHOUT_ACCOUNT}")
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"value", [None, "", "Bearer sk-proj-platform-key", "Bearer not.a.jwt", _make_jwt({"sub": "x"})]
|
||||
)
|
||||
def test_rejects_non_chatgpt_credentials(self, value):
|
||||
assert not is_chatgpt_oauth_key(value)
|
||||
|
||||
|
||||
class TestSelectChatGPTClientCredentialHeaders:
|
||||
def test_account_header_alone_is_not_forwarded(self):
|
||||
selected = select_chatgpt_client_credential_headers(
|
||||
{"chatgpt-account-id": "acct-alice", "authorization": "Bearer sk-proj-platform-key"}
|
||||
)
|
||||
assert dict(selected) == {}
|
||||
assert get_chatgpt_client_credential({"chatgpt-account-id": "acct-alice"}) is None
|
||||
|
||||
def test_forwards_only_bearer_and_account_context(self):
|
||||
selected = select_chatgpt_client_credential_headers(
|
||||
{
|
||||
"Authorization": f"Bearer {CHATGPT_JWT}",
|
||||
"ChatGPT-Account-Id": "acct-explicit",
|
||||
"x-litellm-api-key": "sk-gateway",
|
||||
"user-agent": "codex",
|
||||
}
|
||||
)
|
||||
assert dict(selected) == {"Authorization": f"Bearer {CHATGPT_JWT}", "ChatGPT-Account-Id": "acct-explicit"}
|
||||
credential = get_chatgpt_client_credential(selected)
|
||||
assert credential is not None
|
||||
assert credential.access_token == CHATGPT_JWT
|
||||
assert credential.account_id == "acct-explicit"
|
||||
|
||||
def test_account_id_falls_back_to_jwt_claim(self):
|
||||
credential = get_chatgpt_client_credential({"authorization": f"Bearer {CHATGPT_JWT}"})
|
||||
assert credential is not None
|
||||
assert credential.account_id == "acct-alice"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("scheme", ["Bearer", "bearer", "BEARER"])
|
||||
def test_bearer_scheme_is_case_insensitive(scheme):
|
||||
credential = get_chatgpt_client_credential({"authorization": f"{scheme} {CHATGPT_JWT}"})
|
||||
assert credential is not None
|
||||
assert credential.access_token == CHATGPT_JWT
|
||||
assert credential.account_id == "acct-alice"
|
||||
|
||||
|
||||
def test_other_authentication_schemes_are_not_chatgpt_logins():
|
||||
assert not is_chatgpt_oauth_key(f"Basic {CHATGPT_JWT}")
|
||||
|
|
@ -5654,3 +5654,44 @@ def test_vector_store_retrieve_handler_sends_id_before_azure_api_version_query()
|
|||
sent: Final = received[0]
|
||||
assert str(sent.url) == "https://x.openai.azure.com/openai/vector_stores/vs_abc?api-version=2025-04-01-preview"
|
||||
assert len(received) == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("is_async", (False, True))
|
||||
@pytest.mark.parametrize("provider", ("openai", "azure"))
|
||||
async def test_non_chatgpt_responses_preserves_caller_headers_after_validation(
|
||||
is_async: bool, provider: str, respx_mock: respx.MockRouter,
|
||||
) -> None:
|
||||
route: Final = respx_mock.post(url__regex=r"https://responses\.example\.com/.*").respond(
|
||||
200,
|
||||
json={
|
||||
"id": "resp_headers", "object": "response", "created_at": 1,
|
||||
"model": "gpt-6-luna", "status": "completed", "output": [],
|
||||
"parallel_tool_calls": False, "tool_choice": "auto", "tools": [],
|
||||
},
|
||||
)
|
||||
client: Final = AsyncHTTPHandler(transport=httpx.AsyncHTTPTransport()) if is_async else HTTPHandler(client=httpx.Client())
|
||||
kwargs: Final = {
|
||||
"model": f"{provider}/gpt-6-luna",
|
||||
"input": "hello",
|
||||
"api_key": "fake-configured-key",
|
||||
"api_base": "https://responses.example.com/v1",
|
||||
"api_version": "2025-04-01-preview",
|
||||
"extra_headers": {"Authorization": "Bearer fake-caller-key", "content-type": "application/vnd.example+json"},
|
||||
"client": client,
|
||||
}
|
||||
try:
|
||||
response: Final = await litellm.aresponses(**kwargs) if is_async else litellm.responses(**kwargs)
|
||||
assert response.status == "completed"
|
||||
assert route.calls[0].request.headers["authorization"] == "Bearer fake-caller-key"
|
||||
expected_content_types: Final = (
|
||||
["application/json", "application/vnd.example+json"]
|
||||
if provider == "openai"
|
||||
else ["application/vnd.example+json"]
|
||||
)
|
||||
assert route.calls[0].request.headers.get_list("content-type") == expected_content_types
|
||||
finally:
|
||||
if isinstance(client, AsyncHTTPHandler):
|
||||
await client.close()
|
||||
else:
|
||||
client.client.close()
|
||||
|
|
|
|||
|
|
@ -3626,6 +3626,18 @@ class TestIsRequestBodySafeBlocksEndpointTargetingFields:
|
|||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("allow_client_side_credentials", [False, True])
|
||||
@pytest.mark.parametrize("nested", [False, True])
|
||||
def test_request_cannot_select_chatgpt_auth_file(allow_client_side_credentials: bool, nested: bool) -> None:
|
||||
selector: Final = {"chatgpt_auth_file": "/tokens/account-b/auth.json"}
|
||||
with pytest.raises(ValueError, match="chatgpt_auth_file"):
|
||||
is_request_body_safe(
|
||||
request_body={"model": "chatgpt-account-a", **({"extra_body": selector} if nested else selector)},
|
||||
general_settings={"allow_client_side_credentials": allow_client_side_credentials},
|
||||
llm_router=None,
|
||||
model="chatgpt-account-a",
|
||||
)
|
||||
|
||||
class TestIsRequestBodySafeBlocksBedrockProjectOverride:
|
||||
"""``aws_bedrock_project_id`` pins a deployment to a Bedrock project so
|
||||
that project's data-retention policy applies to its requests. A
|
||||
|
|
@ -5304,3 +5316,27 @@ async def test_a_non_numeric_max_request_size_mb_is_ignored_without_premium():
|
|||
patch("litellm.proxy.proxy_server.premium_user", False),
|
||||
):
|
||||
assert await check_if_request_size_is_safe(request=_request_with_content_length(1024)) is True
|
||||
|
||||
|
||||
@pytest.mark.parametrize("allow_client_side_credentials", [False, True])
|
||||
@pytest.mark.parametrize("route", ["/model/new", "/model/update", "/model/account-a/update", "/health/test_connection"])
|
||||
def test_chatgpt_model_configuration_reaches_role_authorization(
|
||||
allow_client_side_credentials: bool, route: str
|
||||
) -> None:
|
||||
body: Final = {"litellm_params": {"model": "chatgpt/gpt-6-luna", "chatgpt_auth_file": "/tokens/account-a/auth.json"}}
|
||||
assert is_request_body_safe(
|
||||
request_body=body,
|
||||
general_settings={"allow_client_side_credentials": allow_client_side_credentials},
|
||||
llm_router=None,
|
||||
model="",
|
||||
route=route,
|
||||
)
|
||||
for inference_route in ("/v1/chat/completions", "/v1/responses"):
|
||||
with pytest.raises(ValueError, match="chatgpt_auth_file"):
|
||||
is_request_body_safe(
|
||||
request_body=body,
|
||||
general_settings={"allow_client_side_credentials": allow_client_side_credentials},
|
||||
llm_router=None,
|
||||
model="",
|
||||
route=inference_route,
|
||||
)
|
||||
|
|
|
|||
|
|
@ -5437,3 +5437,64 @@ async def test_health_services_endpoint_langfuse_missing_keys_errors(monkeypatch
|
|||
with pytest.raises(ProxyException, match="auth_check failed") as raised:
|
||||
await health_services_endpoint(service="langfuse")
|
||||
assert "connection refused by lf.internal.example" in str(raised.value.message)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("admin", [False, True])
|
||||
async def test_unsaved_chatgpt_account_probe_requires_proxy_admin(
|
||||
tmp_path: Path, monkeypatch: pytest.MonkeyPatch, admin: bool
|
||||
) -> None:
|
||||
from starlette.requests import Request
|
||||
from litellm.proxy import proxy_server
|
||||
|
||||
monkeypatch.setenv("CHATGPT_AUTH_FILE", str(tmp_path / "missing-default-auth.json"))
|
||||
auth_file: Final = tmp_path / "auth.json"
|
||||
auth_file.write_text(
|
||||
json.dumps({"access_token": "token-probe", "account_id": "acct-probe", "expires_at": 4102444800})
|
||||
)
|
||||
monkeypatch.setattr(litellm, "disable_aiohttp_transport", True)
|
||||
monkeypatch.setattr(proxy_server, "prisma_client", MagicMock())
|
||||
monkeypatch.setattr(proxy_server, "llm_router", None)
|
||||
monkeypatch.setattr(proxy_server, "general_settings", {})
|
||||
output: Final = {
|
||||
"id": "msg_probe",
|
||||
"type": "message",
|
||||
"role": "assistant",
|
||||
"status": "completed",
|
||||
"content": [{"type": "output_text", "text": "ok", "annotations": []}],
|
||||
}
|
||||
response: Final = {
|
||||
"id": "resp_probe",
|
||||
"object": "response",
|
||||
"created_at": 1700000000,
|
||||
"status": "completed",
|
||||
"model": "gpt-6.1-sol",
|
||||
"output": [output],
|
||||
"usage": {"input_tokens": 1, "output_tokens": 1, "total_tokens": 2},
|
||||
}
|
||||
with respx.mock(assert_all_called=False) as upstream:
|
||||
upstream.post("https://chatgpt.com/backend-api/codex/responses").respond(
|
||||
200,
|
||||
text="data: " + json.dumps({"type": "response.completed", "response": response}) + "\n\n",
|
||||
headers={"content-type": "text/event-stream"},
|
||||
)
|
||||
call: Final = health_test_model_connection(
|
||||
request=Request({"type": "http", "headers": []}),
|
||||
mode="responses",
|
||||
litellm_params={"model": "chatgpt/gpt-6.1-sol", "chatgpt_auth_file": str(auth_file)},
|
||||
model_info={},
|
||||
user_api_key_dict=UserAPIKeyAuth(
|
||||
user_role=LitellmUserRoles.PROXY_ADMIN if admin else LitellmUserRoles.INTERNAL_USER, token="test-key"
|
||||
),
|
||||
)
|
||||
if admin:
|
||||
result: Final = await call
|
||||
assert result["status"] == "success"
|
||||
assert upstream.calls.last.request.headers["Authorization"] == "Bearer token-probe"
|
||||
assert upstream.calls.last.request.headers["ChatGPT-Account-Id"] == "acct-probe"
|
||||
assert isinstance(json.loads(upstream.calls.last.request.content)["input"], list)
|
||||
else:
|
||||
with pytest.raises(ProxyException) as error:
|
||||
await call
|
||||
assert error.value.code == "403"
|
||||
assert len(upstream.calls) == 0
|
||||
|
|
|
|||
|
|
@ -8933,7 +8933,14 @@ class TestNonAdminCannotPersistWifFieldsOnModel:
|
|||
mock_prisma.db.litellm_proxymodeltable.update.assert_not_called()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_patch_model_non_admin_cannot_set_wif_field(self):
|
||||
@pytest.mark.parametrize(
|
||||
"field,value",
|
||||
[
|
||||
("anthropic_keycloak_token_url", "https://identity.example.com/token"),
|
||||
("chatgpt_auth_file", "/tokens/account-b/auth.json"),
|
||||
],
|
||||
)
|
||||
async def test_patch_model_non_admin_cannot_set_wif_field(self, field: str, value: str) -> None:
|
||||
from litellm.proxy.management_endpoints.model_management_endpoints import (
|
||||
patch_model,
|
||||
)
|
||||
|
|
@ -8979,13 +8986,13 @@ class TestNonAdminCannotPersistWifFieldsOnModel:
|
|||
model_id="m1",
|
||||
patch_data=updateDeployment(
|
||||
litellm_params=updateLiteLLMParams(
|
||||
anthropic_keycloak_token_url="https://attacker.example/token",
|
||||
**{field: value},
|
||||
)
|
||||
),
|
||||
user_api_key_dict=non_admin,
|
||||
)
|
||||
err = exc_info.value
|
||||
assert getattr(err, "param", "") == "anthropic_keycloak_token_url"
|
||||
assert getattr(err, "param", "") == field
|
||||
mock_prisma.db.litellm_proxymodeltable.update.assert_not_called()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
|
|
@ -9044,7 +9051,14 @@ class TestNonAdminCannotPersistWifFieldsOnModel:
|
|||
mock_prisma.db.litellm_proxymodeltable.update.assert_not_called()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_patch_model_admin_can_set_wif_field(self):
|
||||
@pytest.mark.parametrize(
|
||||
"field,value",
|
||||
[
|
||||
("anthropic_keycloak_token_url", "https://identity.example.com/token"),
|
||||
("chatgpt_auth_file", "/tokens/account-b/auth.json"),
|
||||
],
|
||||
)
|
||||
async def test_patch_model_admin_can_set_wif_field(self, field: str, value: str) -> None:
|
||||
from litellm.proxy.management_endpoints.model_management_endpoints import (
|
||||
patch_model,
|
||||
)
|
||||
|
|
@ -9095,7 +9109,7 @@ class TestNonAdminCannotPersistWifFieldsOnModel:
|
|||
model_id="m1",
|
||||
patch_data=updateDeployment(
|
||||
litellm_params=updateLiteLLMParams(
|
||||
anthropic_keycloak_token_url="https://keycloak.internal/token",
|
||||
**{field: value},
|
||||
)
|
||||
),
|
||||
user_api_key_dict=admin,
|
||||
|
|
|
|||
|
|
@ -0,0 +1,185 @@
|
|||
import asyncio
|
||||
import json
|
||||
from collections.abc import Iterator
|
||||
from datetime import datetime
|
||||
from typing import Final
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
import respx
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
import litellm
|
||||
from litellm.integrations.custom_logger import CustomLogger
|
||||
from litellm.proxy import proxy_server
|
||||
from tests._master_key import MASTER_KEY
|
||||
|
||||
_BASE: Final = "https://chatgpt.com/backend-api/codex"
|
||||
_TOKEN: Final = "test-subscription-token"
|
||||
_HEADERS: Final = {"authorization": f"Bearer {_TOKEN}", "x-litellm-api-key": MASTER_KEY}
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def client(monkeypatch: pytest.MonkeyPatch, httpx_transport: None) -> Iterator[TestClient]:
|
||||
monkeypatch.setattr(proxy_server, "master_key", MASTER_KEY)
|
||||
yield TestClient(proxy_server.app)
|
||||
|
||||
|
||||
def test_native_catalog_preserves_account_and_filters_gateway_credentials(
|
||||
client: TestClient, respx_mock: respx.MockRouter
|
||||
) -> None:
|
||||
catalog: Final = {"models": [{"slug": "subscription-model", "supported_reasoning_levels": ["high"]}]}
|
||||
upstream: Final = respx_mock.get(f"{_BASE}/models", params={"client_version": "1.0.0"}).respond(json=catalog)
|
||||
response: Final = client.get(
|
||||
"/backend-api/codex/models?client_version=1.0.0",
|
||||
headers={
|
||||
**_HEADERS,
|
||||
"chatgpt-account-id": "account-a",
|
||||
"cookie": "session=test-cookie",
|
||||
"x-pass-authorization": "Bearer test-injected-token",
|
||||
"x-pass-x-litellm-api-key": "test-injected-key",
|
||||
},
|
||||
)
|
||||
assert response.status_code == 200
|
||||
assert response.json() == catalog
|
||||
assert upstream.calls.last.request.headers["authorization"] == f"Bearer {_TOKEN}"
|
||||
assert upstream.calls.last.request.headers["chatgpt-account-id"] == "account-a"
|
||||
assert "x-litellm-api-key" not in upstream.calls.last.request.headers
|
||||
assert "cookie" not in upstream.calls.last.request.headers
|
||||
|
||||
|
||||
@pytest.mark.parametrize("account", ["account-a", "account-b"])
|
||||
def test_native_responses_preserves_body_and_stream(
|
||||
client: TestClient, respx_mock: respx.MockRouter, account: str
|
||||
) -> None:
|
||||
body: Final = {
|
||||
"model": "subscription-model",
|
||||
"input": [{"role": "user", "content": "Hello"}],
|
||||
"reasoning": {"effort": "high"},
|
||||
"store": False,
|
||||
"stream": True,
|
||||
}
|
||||
stream: Final = (
|
||||
b'event: response.output_text.delta\ndata: {"type":"response.output_text.delta","delta":"Hello"}\n\n'
|
||||
)
|
||||
upstream: Final = respx_mock.post(f"{_BASE}/responses").respond(
|
||||
content=stream, headers={"content-type": "text/event-stream"}
|
||||
)
|
||||
response: Final = client.post(
|
||||
"/backend-api/codex/responses",
|
||||
json=body,
|
||||
headers={**_HEADERS, "authorization": f"Bearer test-{account}", "chatgpt-account-id": account},
|
||||
)
|
||||
assert response.status_code == 200
|
||||
assert response.content == stream
|
||||
assert response.headers["content-type"].startswith("text/event-stream")
|
||||
assert json.loads(upstream.calls.last.request.content) == body
|
||||
assert upstream.calls.last.request.headers["authorization"] == f"Bearer test-{account}"
|
||||
assert upstream.calls.last.request.headers["chatgpt-account-id"] == account
|
||||
|
||||
|
||||
@pytest.mark.parametrize("gateway_key", [None, "sk-invalid-test-key"])
|
||||
def test_gateway_auth_rejects_before_upstream(
|
||||
client: TestClient, respx_mock: respx.MockRouter, gateway_key: str | None
|
||||
) -> None:
|
||||
headers: Final = {
|
||||
"authorization": f"Bearer {_TOKEN}",
|
||||
**({"x-litellm-api-key": gateway_key} if gateway_key else {}),
|
||||
}
|
||||
response: Final = client.get("/backend-api/codex/models", headers=headers)
|
||||
assert response.status_code in (400, 401, 403)
|
||||
assert not respx_mock.calls
|
||||
|
||||
|
||||
@pytest.mark.parametrize("authorization", ["", "Basic test", "Bearer ", f"Bearer {MASTER_KEY}"])
|
||||
def test_subscription_credential_required(client: TestClient, respx_mock: respx.MockRouter, authorization: str) -> None:
|
||||
response: Final = client.get("/backend-api/codex/models", headers={**_HEADERS, "authorization": authorization})
|
||||
assert response.status_code == 401
|
||||
assert not respx_mock.calls
|
||||
|
||||
|
||||
@pytest.mark.parametrize("status", [401, 403, 429])
|
||||
def test_provider_errors_do_not_fall_back(client: TestClient, respx_mock: respx.MockRouter, status: int) -> None:
|
||||
upstream: Final = respx_mock.post(f"{_BASE}/responses").respond(
|
||||
status, json={"error": {"message": "subscription unavailable"}}, headers={"retry-after": "60"}
|
||||
)
|
||||
response: Final = client.post(
|
||||
"/backend-api/codex/responses", headers=_HEADERS, json={"model": "subscription-model", "stream": True}
|
||||
)
|
||||
assert response.status_code == status
|
||||
assert upstream.call_count == 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize("method,path", [("POST", "models"), ("GET", "responses"), ("GET", "deviceauth")])
|
||||
def test_only_native_inference_routes_are_forwarded(
|
||||
client: TestClient, respx_mock: respx.MockRouter, method: str, path: str
|
||||
) -> None:
|
||||
response: Final = client.request(method, f"/backend-api/codex/{path}", headers=_HEADERS)
|
||||
assert response.status_code in (405, 422)
|
||||
assert not respx_mock.calls
|
||||
|
||||
|
||||
class UsageRecorder(CustomLogger):
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
self.events: Final[asyncio.Queue[dict[str, object]]] = asyncio.Queue()
|
||||
|
||||
async def async_log_success_event(
|
||||
self, kwargs: dict[str, object], response_obj: object, start_time: datetime, end_time: datetime
|
||||
) -> None:
|
||||
await self.events.put(kwargs)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize("stream", [False, True])
|
||||
async def test_usage_logged_without_credentials(
|
||||
monkeypatch: pytest.MonkeyPatch, httpx_transport: None, respx_mock: respx.MockRouter, stream: bool
|
||||
) -> None:
|
||||
monkeypatch.setattr(proxy_server, "master_key", MASTER_KEY)
|
||||
recorder: Final = UsageRecorder()
|
||||
monkeypatch.setattr(litellm, "_async_success_callback", [recorder])
|
||||
litellm.register_model(
|
||||
{
|
||||
"chatgpt/subscription-model": {
|
||||
"litellm_provider": "chatgpt",
|
||||
"mode": "responses",
|
||||
"input_cost_per_token": 0.0,
|
||||
"output_cost_per_token": 0.0,
|
||||
}
|
||||
}
|
||||
)
|
||||
result: Final = {
|
||||
"id": "resp_test",
|
||||
"object": "response",
|
||||
"created_at": 1700000000,
|
||||
"status": "completed",
|
||||
"model": "subscription-model",
|
||||
"output": [],
|
||||
"usage": {"input_tokens": 12, "output_tokens": 3, "total_tokens": 15},
|
||||
}
|
||||
event: Final = {"type": "response.completed", "response": result}
|
||||
response_body: Final = (
|
||||
httpx.Response(
|
||||
200,
|
||||
content=f"event: response.completed\ndata: {json.dumps(event)}\n\n",
|
||||
headers={"content-type": "text/event-stream"},
|
||||
)
|
||||
if stream
|
||||
else httpx.Response(200, json=result)
|
||||
)
|
||||
respx_mock.post(f"{_BASE}/responses").mock(return_value=response_body)
|
||||
async with httpx.AsyncClient(transport=httpx.ASGITransport(app=proxy_server.app), base_url="http://test") as caller:
|
||||
response: Final = await caller.post(
|
||||
"/backend-api/codex/responses",
|
||||
json={"model": "subscription-model", "stream": stream, "input": "Hello"},
|
||||
headers={**_HEADERS, "x-pass-authorization": "Bearer test-hidden-token", "cookie": "test-hidden-cookie"},
|
||||
)
|
||||
assert response.status_code == 200
|
||||
logged: Final = await asyncio.wait_for(recorder.events.get(), timeout=5)
|
||||
payload: Final = logged["standard_logging_object"]
|
||||
assert isinstance(payload, dict)
|
||||
assert payload["prompt_tokens"] == 12
|
||||
assert payload["completion_tokens"] == 3
|
||||
serialized: Final = json.dumps(logged, default=str)
|
||||
for secret in (_TOKEN, MASTER_KEY, "test-hidden-token", "test-hidden-cookie"):
|
||||
assert secret not in serialized
|
||||
|
|
@ -1,4 +1,5 @@
|
|||
import asyncio
|
||||
import base64
|
||||
import copy
|
||||
import json
|
||||
import os
|
||||
|
|
@ -9208,6 +9209,97 @@ def test_default_team_settings_bool_turn_off_message_logging_redacts():
|
|||
)
|
||||
|
||||
|
||||
def _fake_chatgpt_oauth_token(account_id: str) -> str:
|
||||
header = base64.urlsafe_b64encode(b'{"alg":"RS256"}').rstrip(b"=").decode()
|
||||
payload = (
|
||||
base64.urlsafe_b64encode(
|
||||
json.dumps({"https://api.openai.com/auth": {"chatgpt_account_id": account_id}, "exp": 4102444800}).encode()
|
||||
)
|
||||
.rstrip(b"=")
|
||||
.decode()
|
||||
)
|
||||
return f"{header}.{payload}.fake-signature"
|
||||
|
||||
|
||||
CHATGPT_OAUTH_TOKEN = f"Bearer {_fake_chatgpt_oauth_token('acct-alice')}"
|
||||
CHATGPT_ACCOUNT_ID_HEADER_NAME = "ChatGPT-Account-Id"
|
||||
PLATFORM_API_KEY_BEARER = "Bearer sk-proj-fake-platform-key-0123456789"
|
||||
CHATGPT_LEAK_TARGET_PROVIDERS = ["openai", "azure", "anthropic", "bedrock", "vertex_ai"]
|
||||
|
||||
|
||||
def _codex_client_headers(
|
||||
authorization_header_name: str = "authorization", account_id: str | None = "acct-alice"
|
||||
) -> dict:
|
||||
return {
|
||||
"content-type": "application/json",
|
||||
"originator": "codex_cli_rs",
|
||||
authorization_header_name: CHATGPT_OAUTH_TOKEN,
|
||||
**({CHATGPT_ACCOUNT_ID_HEADER_NAME: account_id} if account_id is not None else {}),
|
||||
}
|
||||
|
||||
|
||||
def test_clean_headers_keeps_chatgpt_oauth_bearer_when_gateway_key_is_separate():
|
||||
result = clean_headers(
|
||||
headers=Headers({**_codex_client_headers(), "x-litellm-api-key": "sk-proxy-virtual-key"}),
|
||||
litellm_key_header_name="x-litellm-api-key",
|
||||
forward_llm_provider_auth_headers=False,
|
||||
authenticated_with_header="x-litellm-api-key",
|
||||
)
|
||||
|
||||
assert result["authorization"] == CHATGPT_OAUTH_TOKEN
|
||||
assert "x-litellm-api-key" not in result
|
||||
|
||||
|
||||
def test_clean_headers_drops_bearer_that_authenticated_to_the_gateway():
|
||||
result = clean_headers(
|
||||
headers=Headers({"authorization": CHATGPT_OAUTH_TOKEN, "content-type": "application/json"}),
|
||||
litellm_key_header_name=None,
|
||||
forward_llm_provider_auth_headers=False,
|
||||
authenticated_with_header="authorization",
|
||||
)
|
||||
|
||||
assert _authorization_values(result) == []
|
||||
|
||||
|
||||
@pytest.mark.parametrize("bearer", [PLATFORM_API_KEY_BEARER, "Bearer not.a-jwt", "Bearer a.bm90LWpzb24.c"])
|
||||
def test_clean_headers_drops_non_chatgpt_bearers(bearer):
|
||||
result = clean_headers(
|
||||
headers=Headers({"authorization": bearer, "x-litellm-api-key": "sk-proxy-virtual-key"}),
|
||||
litellm_key_header_name="x-litellm-api-key",
|
||||
forward_llm_provider_auth_headers=False,
|
||||
authenticated_with_header="x-litellm-api-key",
|
||||
)
|
||||
|
||||
assert _authorization_values(result) == []
|
||||
|
||||
|
||||
@pytest.mark.parametrize("authorization_header_name", AUTHORIZATION_HEADER_CASINGS)
|
||||
def test_chatgpt_oauth_credential_and_account_reach_chatgpt_provider(authorization_header_name):
|
||||
forwarded = _headers_forwarded_to(_codex_client_headers(authorization_header_name), "chatgpt")
|
||||
|
||||
assert _authorization_values(forwarded) == [CHATGPT_OAUTH_TOKEN]
|
||||
assert forwarded[CHATGPT_ACCOUNT_ID_HEADER_NAME] == "acct-alice"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("custom_llm_provider", CHATGPT_LEAK_TARGET_PROVIDERS)
|
||||
def test_chatgpt_oauth_credential_is_never_forwarded_to_other_providers(custom_llm_provider):
|
||||
forwarded = _headers_forwarded_to(_codex_client_headers(), custom_llm_provider)
|
||||
|
||||
assert _authorization_values(forwarded) == []
|
||||
assert CHATGPT_OAUTH_TOKEN not in forwarded.values()
|
||||
assert CHATGPT_ACCOUNT_ID_HEADER_NAME not in forwarded
|
||||
|
||||
|
||||
def test_chatgpt_account_header_is_not_forwarded_without_chatgpt_oauth_bearer():
|
||||
headers = {**_codex_client_headers(), "authorization": PLATFORM_API_KEY_BEARER}
|
||||
|
||||
assert _headers_forwarded_to(headers, "chatgpt") == {}
|
||||
|
||||
|
||||
def test_anthropic_oauth_bearer_does_not_reach_chatgpt_provider():
|
||||
assert _authorization_values(_headers_forwarded_to(_client_headers(), "chatgpt")) == []
|
||||
|
||||
|
||||
def test_add_user_api_key_auth_to_request_metadata_attributes_a_cli_session_to_its_alias():
|
||||
data = {"model": "gpt-5.4-nano", "messages": [{"role": "user", "content": "hi"}], "litellm_metadata": {}}
|
||||
session = UserAPIKeyAuth(
|
||||
|
|
|
|||
|
|
@ -933,6 +933,42 @@ class TestResponsesInputToChatMessages:
|
|||
) == [user_message]
|
||||
|
||||
|
||||
class TestMergeProviderSpecificHeaders:
|
||||
def test_scoped_headers_merge_only_for_matching_provider(self):
|
||||
scoped = {"custom_llm_provider": "chatgpt", "extra_headers": {"Authorization": "Bearer client-token"}}
|
||||
|
||||
merged = ResponsesAPIRequestUtils.merge_provider_specific_headers(
|
||||
extra_headers={"originator": "codex_cli_rs"},
|
||||
provider_specific_header=scoped,
|
||||
custom_llm_provider="chatgpt",
|
||||
)
|
||||
untouched = ResponsesAPIRequestUtils.merge_provider_specific_headers(
|
||||
extra_headers={"originator": "codex_cli_rs"},
|
||||
provider_specific_header=scoped,
|
||||
custom_llm_provider="openai",
|
||||
)
|
||||
|
||||
assert merged == {"originator": "codex_cli_rs", "Authorization": "Bearer client-token"}
|
||||
assert untouched == {"originator": "codex_cli_rs"}
|
||||
|
||||
def test_sequence_of_scoped_entries_each_keep_their_own_scope(self):
|
||||
entries = [
|
||||
{"custom_llm_provider": "anthropic", "extra_headers": {"Authorization": "Bearer anthropic-token"}},
|
||||
{"custom_llm_provider": "chatgpt", "extra_headers": {"Authorization": "Bearer chatgpt-token"}},
|
||||
]
|
||||
|
||||
merged = ResponsesAPIRequestUtils.merge_provider_specific_headers(
|
||||
extra_headers=None, provider_specific_header=entries, custom_llm_provider="chatgpt"
|
||||
)
|
||||
|
||||
assert merged == {"Authorization": "Bearer chatgpt-token"}
|
||||
|
||||
def test_no_scoped_headers_returns_extra_headers_unchanged(self):
|
||||
assert ResponsesAPIRequestUtils.merge_provider_specific_headers(
|
||||
extra_headers=None, provider_specific_header=None, custom_llm_provider="chatgpt"
|
||||
) is None
|
||||
|
||||
|
||||
def test_responses_object_bills_an_explicit_cache_hit_at_the_explicit_rate():
|
||||
model = "dashscope/qwen-responses-explicit-cache-test"
|
||||
previous_row = litellm.model_cost.get(model)
|
||||
|
|
|
|||
|
|
@ -283,6 +283,15 @@ def test_update_kwargs_does_not_mutate_defaults_and_merges_metadata():
|
|||
assert kwargs["litellm_metadata"] == {"baz": 123}
|
||||
|
||||
|
||||
def test_router_model_group_llm_provider_resolves_chatgpt_without_login(tmp_path, monkeypatch: pytest.MonkeyPatch):
|
||||
monkeypatch.setenv("CHATGPT_TOKEN_DIR", str(tmp_path))
|
||||
params: Final = LiteLLM_Params(model="chatgpt/gpt-5.4")
|
||||
|
||||
assert Router._model_group_llm_provider(params) == ("gpt-5.4", "chatgpt")
|
||||
assert Router._model_group_llm_provider(LiteLLM_Params(model="no-such-provider-model-xyz")) == ("", "")
|
||||
assert not any(tmp_path.iterdir())
|
||||
|
||||
|
||||
def test_router_with_model_info_and_model_group():
|
||||
"""
|
||||
Test edge case where user specifies model_group in model_info
|
||||
|
|
|
|||
|
|
@ -94,6 +94,7 @@ CONNECTION_NAMES: Final = (
|
|||
"github_copilot_host",
|
||||
"configurable_clientside_auth_params",
|
||||
"use_xai_oauth",
|
||||
"chatgpt_auth_file",
|
||||
"fireworks_forward_user_id",
|
||||
"aws_batch_role_arn",
|
||||
"aws_bedrock_control_plane_endpoint",
|
||||
|
|
|
|||
|
|
@ -16,8 +16,8 @@ from litellm.types.router import (
|
|||
holds_secret_pointer,
|
||||
reject_server_owned_wif_params,
|
||||
server_owned_wif_fields_named,
|
||||
server_owned_wif_fields_written,
|
||||
server_owned_wif_fields_present,
|
||||
server_owned_wif_fields_written,
|
||||
)
|
||||
from litellm.types.utils import (
|
||||
CustomPricingLiteLLMParams,
|
||||
|
|
@ -367,6 +367,7 @@ def test_server_owned_registry_includes_anthropic_openai_and_oauth_token_exchang
|
|||
+ openai_wif_litellm_params
|
||||
+ oauth_token_exchange_litellm_params
|
||||
+ github_copilot_oauth_litellm_params
|
||||
+ ("chatgpt_auth_file",)
|
||||
)
|
||||
assert set(openai_wif_litellm_params) == {
|
||||
"openai_identity_provider_id",
|
||||
|
|
|
|||
105
ui/litellm-dashboard/src/lib/http/schema.d.ts
generated
vendored
105
ui/litellm-dashboard/src/lib/http/schema.d.ts
generated
vendored
|
|
@ -1792,6 +1792,30 @@ export interface paths {
|
|||
patch: operations["azure_speech_proxy_route_azure_speech__endpoint__patch"];
|
||||
trace?: never;
|
||||
};
|
||||
"/backend-api/codex/{endpoint}": {
|
||||
parameters: {
|
||||
query?: never;
|
||||
header?: never;
|
||||
path?: never;
|
||||
cookie?: never;
|
||||
};
|
||||
/**
|
||||
* Codex Passthrough Route
|
||||
* @description Forward a client's ChatGPT subscription with separate LiteLLM authentication.
|
||||
*/
|
||||
get: operations["codex_passthrough_route_backend_api_codex__endpoint__get"];
|
||||
put?: never;
|
||||
/**
|
||||
* Codex Passthrough Route
|
||||
* @description Forward a client's ChatGPT subscription with separate LiteLLM authentication.
|
||||
*/
|
||||
post: operations["codex_passthrough_route_backend_api_codex__endpoint__post"];
|
||||
delete?: never;
|
||||
options?: never;
|
||||
head?: never;
|
||||
patch?: never;
|
||||
trace?: never;
|
||||
};
|
||||
"/batches": {
|
||||
parameters: {
|
||||
query?: never;
|
||||
|
|
@ -35309,6 +35333,11 @@ export interface components {
|
|||
cache_read_input_token_cost_priority?: number | null;
|
||||
/** Cache Read Input Token Cost Ultrafast */
|
||||
cache_read_input_token_cost_ultrafast?: number | null;
|
||||
/**
|
||||
* Chatgpt Auth File
|
||||
* @description Path to a ChatGPT OAuth auth.json for this deployment.
|
||||
*/
|
||||
chatgpt_auth_file?: string | null;
|
||||
/** Citation Cost Per Token */
|
||||
citation_cost_per_token?: number | null;
|
||||
/** Client Id */
|
||||
|
|
@ -50660,6 +50689,11 @@ export interface components {
|
|||
cache_read_input_token_cost_priority?: number | null;
|
||||
/** Cache Read Input Token Cost Ultrafast */
|
||||
cache_read_input_token_cost_ultrafast?: number | null;
|
||||
/**
|
||||
* Chatgpt Auth File
|
||||
* @description Path to a ChatGPT OAuth auth.json for this deployment.
|
||||
*/
|
||||
chatgpt_auth_file?: string | null;
|
||||
/** Citation Cost Per Token */
|
||||
citation_cost_per_token?: number | null;
|
||||
/** Client Id */
|
||||
|
|
@ -53737,6 +53771,77 @@ export interface operations {
|
|||
};
|
||||
};
|
||||
};
|
||||
codex_passthrough_route_backend_api_codex__endpoint__get: {
|
||||
parameters: {
|
||||
query?: never;
|
||||
header?: never;
|
||||
path: {
|
||||
endpoint: "models" | "responses";
|
||||
};
|
||||
cookie?: never;
|
||||
};
|
||||
requestBody?: never;
|
||||
responses: {
|
||||
/** @description Successful Response */
|
||||
200: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": unknown;
|
||||
};
|
||||
};
|
||||
/** @description Validation Error */
|
||||
422: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": components["schemas"]["HTTPValidationError"];
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
codex_passthrough_route_backend_api_codex__endpoint__post: {
|
||||
parameters: {
|
||||
query?: never;
|
||||
header?: never;
|
||||
path: {
|
||||
endpoint: "models" | "responses";
|
||||
};
|
||||
cookie?: never;
|
||||
};
|
||||
requestBody: {
|
||||
content: {
|
||||
"application/json": {
|
||||
[key: string]: unknown;
|
||||
};
|
||||
};
|
||||
};
|
||||
responses: {
|
||||
/** @description Unmodified Codex response or server-sent events */
|
||||
200: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": {
|
||||
[key: string]: unknown;
|
||||
};
|
||||
"text/event-stream": string;
|
||||
};
|
||||
};
|
||||
/** @description Validation Error */
|
||||
422: {
|
||||
headers: {
|
||||
[name: string]: unknown;
|
||||
};
|
||||
content: {
|
||||
"application/json": components["schemas"]["HTTPValidationError"];
|
||||
};
|
||||
};
|
||||
};
|
||||
};
|
||||
list_batches_batches_get: {
|
||||
parameters: {
|
||||
query?: {
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue