scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-08 03:08:45 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 114

feat(github_copilot): replace file-based auth with session key credentials

Browse source 
Move GitHub Copilot authentication from a file-based device-code flow
(which printed credentials to stdio on startup) to a named credential /
session key model suitable for the proxy.

Key changes:
- Authenticator accepts optional access_token for credential mode with
  a module-level cache to share copilot inference tokens across requests
- Add stateless OAuth device code SSO endpoints (initiate + status)
- Early credential resolution in the router before client selection
- Always send static Copilot headers (editor-version, user-agent, etc.)
  even before API key exchange, fixing "missing Editor-Version" errors
- Fix UI polling to ratchet up the interval on GitHub slow_down instead
  of resetting to 5s on the next pending response
- Narrow bare except to (GetAPIKeyError, GetAccessTokenError) in main.py
- Remove dead config.authenticator mocks from tests
- Update docs with credential-based proxy auth flow

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Hunter Wittenborn 2026-03-30 01:05:49 -05:00
parent d8e4fc4dd0
commit 3cd86f3982
18 changed files with 817 additions and 271 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

136
docs/my-website/docs/providers/github_copilot.md
View file

@ -20,14 +20,27 @@ https://docs.github.com/en/copilot
## Authentication
GitHub Copilot uses OAuth device flow for authentication. On first use, you'll be prompted to authenticate via GitHub:
GitHub Copilot uses OAuth Device Code flow for authentication. LiteLLM supports two authentication modes:
1. LiteLLM will display a device code and verification URL
2. Visit the URL and enter the code to authenticate
3. Your credentials will be stored locally for future use
- **LiteLLM Proxy**: Use named credentials created via the credential API or UI. The interactive device flow is handled by the proxy on your behalf — credentials are stored and reused automatically.
- **Python SDK**: Credentials are read from `~/.config/litellm/github_copilot/access-token` on disk (file-based, for backward compatibility).
:::info
If you hit a GitHub Copilot model without a configured credential (proxy) or access-token file (SDK), LiteLLM returns an `AuthenticationError` pointing to this page.
:::
## Usage - LiteLLM Python SDK
### Setup (first time)
Authenticate once by running the login command. This stores your access token in `~/.config/litellm/github_copilot/access-token` for future use.
```bash showLineNumbers title="Authenticate via device flow (SDK only)"
litellm --login github_copilot
```
### Chat Completion
```python showLineNumbers title="GitHub Copilot Chat Completion"
@ -87,6 +100,8 @@ print(response)
## Usage - LiteLLM Proxy
The proxy requires a named credential. See [Credential-Based Authentication](#credential-based-authentication-proxy) below.
Add the following to your LiteLLM Proxy configuration file:
```yaml showLineNumbers title="config.yaml"
@ -94,16 +109,19 @@ model_list:
- model_name: github_copilot/gpt-4
litellm_params:
model: github_copilot/gpt-4
litellm_credential_name: my-copilot # named credential (required for proxy)
- model_name: github_copilot/gpt-5.1-codex
model_info:
mode: responses
litellm_params:
model: github_copilot/gpt-5.1-codex
litellm_credential_name: my-copilot
- model_name: github_copilot/text-embedding-ada-002
model_info:
mode: embedding
litellm_params:
model: github_copilot/text-embedding-ada-002
litellm_credential_name: my-copilot
```
Start your LiteLLM Proxy server:
@ -170,18 +188,115 @@ curl http://localhost:4000/v1/chat/completions \
</TabItem>
</Tabs>
## Getting Started
## Credential-Based Authentication (Proxy)
1. Ensure you have GitHub Copilot access (paid GitHub subscription required)
2. Run your first LiteLLM request - you'll be prompted to authenticate
3. Follow the device flow authentication process
4. Start making requests to GitHub Copilot through LiteLLM
The LiteLLM Proxy uses a stateless OAuth Device Code flow. Nothing is stored in the database until the GitHub token is successfully obtained — the client holds the `device_code` between API calls.
You can complete the flow via the **LiteLLM UI** (Models → Credentials → Add Credential → GitHub Copilot) or with the curl steps below.
### Step 1: Initiate the Device Code Flow
```bash showLineNumbers title="Start GitHub OAuth"
curl -X POST http://localhost:4000/credentials/github_copilot/initiate \
-H "Authorization: Bearer your-proxy-api-key"
```
Response:
```json
{
"device_code": "xxx",
"user_code": "ABCD-1234",
"verification_uri": "https://github.com/login/device",
"poll_interval_ms": 5000,
"expires_in": 900
}
```
### Step 2: Authorize on GitHub
Visit the `verification_uri` and enter the `user_code` to authorize LiteLLM.
### Step 3: Poll for Completion
Poll the status endpoint until the flow completes. Use `device_code` from step 1.
Use `poll_interval_ms` from the initiate response as your default polling interval.
```bash showLineNumbers title="Check authorization status"
curl -X POST http://localhost:4000/credentials/github_copilot/status \
-H "Authorization: Bearer your-proxy-api-key" \
-H "Content-Type: application/json" \
-d '{"device_code": "xxx"}'
```
Possible responses:
```json title="Still waiting for user"
{"status": "pending"}
```
```json title="GitHub is being polled too fast — wait before retrying"
{"status": "pending", "retry_after_ms": 10000}
```
If `retry_after_ms` is present, you **must** wait that many milliseconds before calling `/status` again. Ignoring it causes GitHub to keep increasing the required interval.
```json title="User authorized successfully"
{"status": "complete", "access_token": "ghu_xxx"}
```
```json title="Flow expired or denied"
{"status": "failed", "error": "The device code has expired."}
```
### Step 4: Store as a Named Credential
Once you have the `access_token`, store it as a named credential:
```bash showLineNumbers title="Save credential"
curl -X POST http://localhost:4000/credentials \
-H "Authorization: Bearer your-proxy-api-key" \
-H "Content-Type: application/json" \
-d '{
"credential_name": "my-copilot",
"credential_values": {"api_key": "ghu_xxx"},
"credential_info": {"custom_llm_provider": "github_copilot"}
}'
```
### Step 5: Attach Credential to a Model
Reference the credential in your config (or via the UI under Models → Add Model → Existing Credentials):
```yaml showLineNumbers title="config.yaml"
model_list:
- model_name: copilot-gpt4
litellm_params:
model: github_copilot/gpt-4
litellm_credential_name: my-copilot
```
### Multiple Accounts
Create multiple credentials with different names to support different GitHub accounts:
```yaml showLineNumbers title="config.yaml - Multiple accounts"
model_list:
- model_name: copilot-team-a
litellm_params:
model: github_copilot/gpt-4
litellm_credential_name: team-a-copilot
- model_name: copilot-team-b
litellm_params:
model: github_copilot/gpt-4
litellm_credential_name: team-b-copilot
```
## Configuration
### Environment Variables
You can customize token storage locations:
You can customize token storage locations (SDK / file-based mode):
```bash showLineNumbers title="Environment Variables"
# Optional: Custom token directory
@ -208,4 +323,3 @@ extra_headers = {
"user-agent": "GithubCopilot/1.155.0" # User agent
}
```

129
litellm/llms/github_copilot/authenticator.py
View file

@ -19,27 +19,44 @@ from .common_utils import (
# Constants
GITHUB_CLIENT_ID = "Iv1.b507a08c87ecfe98"
# Module-level cache for copilot inference tokens in credential mode.
# Key = GitHub access_token, value = api_key_info dict (token + expires_at).
# Mirrors the file-based api-key.json pattern but kept in memory so that
# per-request Authenticator instances share the cached token.
_credential_api_key_cache: Dict[str, Dict[str, Any]] = {}
GITHUB_DEVICE_CODE_URL = "https://github.com/login/device/code"
GITHUB_ACCESS_TOKEN_URL = "https://github.com/login/oauth/access_token"
GITHUB_API_KEY_URL = "https://api.github.com/copilot_internal/v2/token"
class Authenticator:
def __init__(self) -> None:
"""Initialize the GitHub Copilot authenticator with configurable token paths."""
# Token storage paths
self.token_dir = os.getenv(
"GITHUB_COPILOT_TOKEN_DIR",
os.path.expanduser("~/.config/litellm/github_copilot"),
)
self.access_token_file = os.path.join(
self.token_dir,
os.getenv("GITHUB_COPILOT_ACCESS_TOKEN_FILE", "access-token"),
)
self.api_key_file = os.path.join(
self.token_dir, os.getenv("GITHUB_COPILOT_API_KEY_FILE", "api-key.json")
)
self._ensure_token_dir()
def __init__(self, access_token: Optional[str] = None) -> None:
"""Initialize the GitHub Copilot authenticator.
Args:
access_token: If provided, the authenticator operates in
*credential mode* — it uses this token directly instead of
the file-based device-code flow. When ``None`` (the
default), the existing file-based behaviour is preserved.
"""
self._injected_access_token = access_token
if access_token is None:
# File-based mode (backward compatible)
self.token_dir = os.getenv(
"GITHUB_COPILOT_TOKEN_DIR",
os.path.expanduser("~/.config/litellm/github_copilot"),
)
self.access_token_file = os.path.join(
self.token_dir,
os.getenv("GITHUB_COPILOT_ACCESS_TOKEN_FILE", "access-token"),
)
self.api_key_file = os.path.join(
self.token_dir,
os.getenv("GITHUB_COPILOT_API_KEY_FILE", "api-key.json"),
)
self._ensure_token_dir()
def get_access_token(self) -> str:
"""
@ -51,32 +68,23 @@ class Authenticator:
Raises:
GetAccessTokenError: If unable to obtain an access token after retries.
"""
if self._injected_access_token is not None:
return self._injected_access_token
try:
with open(self.access_token_file, "r") as f:
access_token = f.read().strip()
if access_token:
return access_token
except IOError:
verbose_logger.warning(
"No existing access token found or error reading file"
)
for attempt in range(3):
verbose_logger.debug(f"Access token acquisition attempt {attempt + 1}/3")
try:
access_token = self._login()
try:
with open(self.access_token_file, "w") as f:
f.write(access_token)
except IOError:
verbose_logger.error("Error saving access token to file")
return access_token
except (GetDeviceCodeError, GetAccessTokenError, RefreshAPIKeyError) as e:
verbose_logger.warning(f"Failed attempt {attempt + 1}: {str(e)}")
continue
pass # No file — fall through to auth error below
raise GetAccessTokenError(
message="Failed to get access token after 3 attempts",
message=(
"No GitHub Copilot access token configured. "
"Use a named credential via the LiteLLM proxy or UI before making requests. "
"See: https://docs.litellm.ai/docs/providers/github_copilot"
),
status_code=401,
)
@ -90,6 +98,9 @@ class Authenticator:
Raises:
GetAPIKeyError: If unable to obtain an API key.
"""
if self._injected_access_token is not None:
return self._get_api_key_credential_mode()
try:
with open(self.api_key_file, "r") as f:
api_key_info = json.load(f)
@ -139,6 +150,13 @@ class Authenticator:
Returns:
Optional[str]: The GitHub Copilot API endpoint, or None if not found.
"""
if self._injected_access_token is not None:
cached = _credential_api_key_cache.get(self._injected_access_token)
if cached:
endpoints = cached.get("endpoints", {})
return endpoints.get("api")
return None
try:
with open(self.api_key_file, "r") as f:
api_key_info = json.load(f)
@ -149,6 +167,35 @@ class Authenticator:
verbose_logger.warning(f"Error reading API endpoint from file: {str(e)}")
return None
def _get_api_key_credential_mode(self) -> str:
"""Get API key when operating in credential mode (injected access token).
Uses a module-level cache keyed by access_token so that multiple
per-request Authenticator instances share the same cached copilot
inference token and avoid redundant GitHub API calls.
"""
cached = _credential_api_key_cache.get(self._injected_access_token) # type: ignore[arg-type]
if cached and cached.get("expires_at", 0) > datetime.now().timestamp():
token = cached.get("token")
if token:
return token
try:
api_key_info = self._refresh_api_key()
_credential_api_key_cache[self._injected_access_token] = api_key_info # type: ignore[index]
token = api_key_info.get("token")
if not token:
raise GetAPIKeyError(
message="API key response missing token",
status_code=401,
)
return token
except RefreshAPIKeyError as e:
raise GetAPIKeyError(
message=f"Failed to refresh API key: {str(e)}",
status_code=401,
)
def _refresh_api_key(self) -> Dict[str, Any]:
"""
Refresh the API key using the access token.
@ -177,6 +224,8 @@ class Authenticator:
verbose_logger.warning(
f"API key response missing token: {response_json}"
)
except GetAccessTokenError:
raise # Re-raise with the original helpful message (docs link etc.)
except httpx.HTTPStatusError as e:
verbose_logger.error(
f"HTTP error refreshing API key (attempt {attempt+1}/{max_retries}): {str(e)}"
@ -194,10 +243,14 @@ class Authenticator:
if not os.path.exists(self.token_dir):
os.makedirs(self.token_dir, exist_ok=True)
def _get_github_headers(self, access_token: Optional[str] = None) -> Dict[str, str]:
@staticmethod
def get_github_headers(access_token: Optional[str] = None) -> Dict[str, str]:
"""
Generate standard GitHub headers for API requests.
This is a static method so it can be imported and used by the SSO
endpoint module without instantiating an Authenticator.
Args:
access_token: Optional access token to include in the headers.
@ -210,16 +263,18 @@ class Authenticator:
"editor-plugin-version": "copilot/1.155.0",
"user-agent": "GithubCopilot/1.155.0",
"accept-encoding": "gzip,deflate,br",
"content-type": "application/json",
}
if access_token:
headers["authorization"] = f"token {access_token}"
if "content-type" not in headers:
headers["content-type"] = "application/json"
return headers
# Backward-compatible instance alias
def _get_github_headers(self, access_token: Optional[str] = None) -> Dict[str, str]:
return Authenticator.get_github_headers(access_token)
def _get_device_code(self) -> Dict[str, str]:
"""
Get a device code for GitHub authentication.

36
litellm/llms/github_copilot/chat/transformation.py
View file

@ -8,8 +8,10 @@ from litellm.types.llms.openai import AllMessageValues
from ..authenticator import Authenticator
from ..common_utils import (
GITHUB_COPILOT_API_BASE,
GetAccessTokenError,
GetAPIKeyError,
get_copilot_default_headers,
get_copilot_static_headers,
)
@ -21,7 +23,6 @@ class GithubCopilotConfig(OpenAIConfig):
custom_llm_provider: str = "openai",
) -> None:
super().__init__()
self.authenticator = Authenticator()
def _get_openai_compatible_provider_info(
self,
@ -30,10 +31,17 @@ class GithubCopilotConfig(OpenAIConfig):
api_key: Optional[str],
custom_llm_provider: str,
) -> Tuple[Optional[str], Optional[str], str]:
dynamic_api_base = self.authenticator.get_api_base() or GITHUB_COPILOT_API_BASE
# If no api_key is provided we're being called at router registration time
# (before litellm_credential_name has been resolved). Return the default
# base and defer auth to request time rather than raising here and causing
# the deployment to be silently dropped from the router.
if not api_key:
return GITHUB_COPILOT_API_BASE, None, custom_llm_provider
authenticator = Authenticator(access_token=api_key)
dynamic_api_base = authenticator.get_api_base() or GITHUB_COPILOT_API_BASE
try:
dynamic_api_key = self.authenticator.get_api_key()
except GetAPIKeyError as e:
dynamic_api_key = authenticator.get_api_key()
except (GetAPIKeyError, GetAccessTokenError) as e:
raise AuthenticationError(
model=model,
llm_provider=custom_llm_provider,
@ -82,13 +90,19 @@ class GithubCopilotConfig(OpenAIConfig):
headers, model, messages, optional_params, litellm_params, api_key, api_base
)
# Add Copilot-specific headers (editor-version, user-agent, etc.)
try:
copilot_api_key = self.authenticator.get_api_key()
copilot_headers = get_copilot_default_headers(copilot_api_key)
validated_headers = {**copilot_headers, **validated_headers}
except GetAPIKeyError:
pass # Will be handled later in the request flow
# Always add static Copilot headers (editor-version, user-agent, etc.)
# These are required by the GitHub Copilot API on every request.
validated_headers = {**get_copilot_static_headers(), **validated_headers}
# If we have an api_key (GitHub access token), exchange it for a
# copilot inference token and set the Authorization header.
if api_key:
try:
copilot_api_key = Authenticator(access_token=api_key).get_api_key()
copilot_headers = get_copilot_default_headers(copilot_api_key)
validated_headers = {**copilot_headers, **validated_headers}
except (GetAPIKeyError, GetAccessTokenError):
pass # Will be handled later in the request flow
# Add X-Initiator header based on message roles
initiator = self._determine_initiator(messages)

20
litellm/llms/github_copilot/common_utils.py
View file

@ -56,14 +56,14 @@ class GetAPIKeyError(GithubCopilotError):
pass
def get_copilot_default_headers(api_key: str) -> dict:
def get_copilot_static_headers() -> dict:
"""
Get default headers for GitHub Copilot Responses API.
Get static headers required by the GitHub Copilot API.
Based on copilot-api's header configuration.
These headers (editor-version, user-agent, etc.) must be present on every
request regardless of whether the API key has been resolved yet.
"""
return {
"Authorization": f"Bearer {api_key}",
"content-type": "application/json",
"copilot-integration-id": "vscode-chat",
"editor-version": "vscode/1.95.0", # Fixed version for stability
@ -74,3 +74,15 @@ def get_copilot_default_headers(api_key: str) -> dict:
"x-request-id": str(uuid4()),
"x-vscode-user-agent-library-version": "electron-fetch",
}
def get_copilot_default_headers(api_key: str) -> dict:
"""
Get default headers for GitHub Copilot Responses API.
Based on copilot-api's header configuration.
"""
return {
**get_copilot_static_headers(),
"Authorization": f"Bearer {api_key}",
}

19
litellm/llms/github_copilot/embedding/transformation.py
View file

@ -19,6 +19,7 @@ from litellm.utils import convert_to_model_response_object
from ..authenticator import Authenticator
from ..common_utils import (
GetAccessTokenError,
GetAPIKeyError,
GITHUB_COPILOT_API_BASE,
get_copilot_default_headers,
@ -41,7 +42,6 @@ class GithubCopilotEmbeddingConfig(BaseEmbeddingConfig):
def __init__(self) -> None:
super().__init__()
self.authenticator = Authenticator()
def validate_environment(
self,
@ -57,9 +57,6 @@ class GithubCopilotEmbeddingConfig(BaseEmbeddingConfig):
Validate environment and set up headers for GitHub Copilot API.
"""
try:
# Get GitHub Copilot API key via OAuth
api_key = self.authenticator.get_api_key()
if not api_key:
raise AuthenticationError(
model=model,
@ -67,6 +64,9 @@ class GithubCopilotEmbeddingConfig(BaseEmbeddingConfig):
message="GitHub Copilot API key is required. Please authenticate via OAuth Device Flow.",
)
# Get GitHub Copilot API key via OAuth
api_key = Authenticator(access_token=api_key).get_api_key()
# Get default headers
default_headers = get_copilot_default_headers(api_key)
@ -79,7 +79,7 @@ class GithubCopilotEmbeddingConfig(BaseEmbeddingConfig):
return merged_headers
except GetAPIKeyError as e:
except (GetAPIKeyError, GetAccessTokenError) as e:
raise AuthenticationError(
model=model,
llm_provider="github_copilot",
@ -98,10 +98,11 @@ class GithubCopilotEmbeddingConfig(BaseEmbeddingConfig):
"""
Get the complete URL for GitHub Copilot Embedding API endpoint.
"""
# Use provided api_base or fall back to authenticator's base or default
api_base = (
self.authenticator.get_api_base() or api_base or GITHUB_COPILOT_API_BASE
)
# Use provided api_base or fall back to credential-resolved base or default
if api_key:
api_base = Authenticator(access_token=api_key).get_api_base() or api_base or GITHUB_COPILOT_API_BASE
else:
api_base = api_base or GITHUB_COPILOT_API_BASE
# Remove trailing slashes
api_base = api_base.rstrip("/")

26
litellm/llms/github_copilot/responses/transformation.py
View file

@ -23,6 +23,7 @@ from litellm.types.utils import LlmProviders
from ..authenticator import Authenticator
from ..common_utils import (
GITHUB_COPILOT_API_BASE,
GetAccessTokenError,
GetAPIKeyError,
get_copilot_default_headers,
)
@ -54,7 +55,6 @@ class GithubCopilotResponsesAPIConfig(OpenAIResponsesAPIConfig):
def __init__(self) -> None:
super().__init__()
self.authenticator = Authenticator()
@property
def custom_llm_provider(self) -> LlmProviders:
@ -103,16 +103,20 @@ class GithubCopilotResponsesAPIConfig(OpenAIResponsesAPIConfig):
- User-provided extra_headers (merged with priority)
"""
try:
# Get GitHub Copilot API key via OAuth
api_key = self.authenticator.get_api_key()
if not api_key:
if isinstance(litellm_params, dict):
_api_key = litellm_params.get("api_key")
else:
_api_key = getattr(litellm_params, "api_key", None) if litellm_params else None
if not _api_key:
raise AuthenticationError(
model=model,
llm_provider="github_copilot",
message="GitHub Copilot API key is required. Please authenticate via OAuth Device Flow.",
)
# Get GitHub Copilot API key via OAuth
api_key = Authenticator(access_token=_api_key).get_api_key()
# Get default headers (from copilot-api configuration)
default_headers = get_copilot_default_headers(api_key)
@ -143,7 +147,7 @@ class GithubCopilotResponsesAPIConfig(OpenAIResponsesAPIConfig):
return merged_headers
except GetAPIKeyError as e:
except (GetAPIKeyError, GetAccessTokenError) as e:
raise AuthenticationError(
model=model,
llm_provider="github_copilot",
@ -164,10 +168,12 @@ class GithubCopilotResponsesAPIConfig(OpenAIResponsesAPIConfig):
Business/enterprise accounts (api.business.githubcopilot.com) can be
added in the future by detecting account type.
"""
# Use provided api_base or fall back to authenticator's base or default
api_base = (
api_base or self.authenticator.get_api_base() or GITHUB_COPILOT_API_BASE
)
# Use provided api_base or fall back to credential-resolved base or default
_api_key = litellm_params.get("api_key") if isinstance(litellm_params, dict) else None
if _api_key:
api_base = api_base or Authenticator(access_token=_api_key).get_api_base() or GITHUB_COPILOT_API_BASE
else:
api_base = api_base or GITHUB_COPILOT_API_BASE
# Remove trailing slashes
api_base = api_base.rstrip("/")

15
litellm/main.py
View file

@ -2609,12 +2609,21 @@ def completion( # type: ignore # noqa: PLR0915
if custom_llm_provider == "github_copilot":
from litellm.llms.github_copilot.authenticator import Authenticator
from litellm.llms.github_copilot.common_utils import (
GetAccessTokenError,
GetAPIKeyError,
get_copilot_default_headers,
get_copilot_static_headers,
)
copilot_auth = Authenticator()
copilot_api_key = copilot_auth.get_api_key()
copilot_headers = get_copilot_default_headers(copilot_api_key)
# Always add static headers (editor-version, user-agent, etc.)
# — the Copilot API requires these on every request.
copilot_headers = get_copilot_static_headers()
if api_key:
try:
copilot_api_key = Authenticator(access_token=api_key).get_api_key()
copilot_headers = get_copilot_default_headers(copilot_api_key)
except (GetAPIKeyError, GetAccessTokenError):
pass # auth failure handled downstream
if extra_headers:
copilot_headers.update(extra_headers)
extra_headers = copilot_headers

50
litellm/proxy/credential_endpoints/endpoints.py
View file

@ -113,6 +113,29 @@ async def create_credential(
raise handle_exception_on_proxy(e)
def _fetch_github_login(api_key: str) -> Optional[str]:
"""
Call GET https://api.github.com/user with the given GitHub access token
and return the login name, or None if the call fails.
"""
from litellm.llms.custom_httpx.http_handler import _get_httpx_client
try:
sync_client = _get_httpx_client()
resp = sync_client.get(
"https://api.github.com/user",
headers={
"Authorization": f"token {api_key}",
"Accept": "application/json",
},
)
if resp.status_code == 200:
return resp.json().get("login")
except Exception as e:
verbose_proxy_logger.warning(f"Could not fetch GitHub user info: {e}")
return None
@router.get(
"/credentials",
dependencies=[Depends(user_api_key_auth)],
@ -127,14 +150,25 @@ async def get_credentials(
[BETA] endpoint. This might change unexpectedly.
"""
try:
masked_credentials = [
{
"credential_name": credential.credential_name,
"credential_values": _get_masked_values(credential.credential_values),
"credential_info": credential.credential_info,
}
for credential in litellm.credential_list
]
masked_credentials = []
for credential in litellm.credential_list:
credential_info = dict(credential.credential_info or {})
# For GitHub Copilot credentials, inject runtime github_login from the API.
# The login is NOT stored in the DB — it's fetched live and added to the
# response only so the UI can display it.
if credential_info.get("custom_llm_provider") == "github_copilot":
api_key = (credential.credential_values or {}).get("api_key")
if api_key:
github_login = _fetch_github_login(api_key)
if github_login:
credential_info = {**credential_info, "github_login": github_login}
masked_credentials.append(
{
"credential_name": credential.credential_name,
"credential_values": _get_masked_values(credential.credential_values),
"credential_info": credential_info,
}
)
return {"success": True, "credentials": masked_credentials}
except Exception as e:
return handle_exception_on_proxy(e)

7
litellm/proxy/proxy_cli.py
View file

@ -324,7 +324,12 @@ class ProxyInitializationHelpers:
"""Helper function to determine the event loop type based on platform"""
if sys.platform in ("win32", "cygwin", "cli"):
return None # Let uvicorn choose the default loop on Windows
return "uvloop"
try:
import uvloop # noqa: F401
return "uvloop"
except (ImportError, Exception):
return "asyncio"
@staticmethod
def _maybe_setup_prometheus_multiproc_dir(

4
litellm/proxy/proxy_server.py
View file

@ -319,6 +319,9 @@ from litellm.proxy.common_utils.reset_budget_job import ResetBudgetJob
from litellm.proxy.common_utils.swagger_utils import ERROR_RESPONSES
from litellm.proxy.container_endpoints.endpoints import router as container_router
from litellm.proxy.credential_endpoints.endpoints import router as credential_router
from litellm.proxy.credential_endpoints.github_copilot_sso import (
router as github_copilot_sso_router,
)
from litellm.proxy.db.db_transaction_queue.spend_log_cleanup import SpendLogCleanup
from litellm.proxy.db.exception_handler import PrismaDBExceptionHandler
from litellm.proxy.discovery_endpoints import ui_discovery_endpoints_router
@ -13447,6 +13450,7 @@ app.include_router(vector_store_router)
app.include_router(vector_store_management_router)
app.include_router(vector_store_files_router)
app.include_router(credential_router)
app.include_router(github_copilot_sso_router)
app.include_router(llm_passthrough_router)
app.include_router(webrtc_router)
app.include_router(mcp_management_router)

24
litellm/proxy/public_endpoints/provider_create_fields.json
View file

@ -1174,28 +1174,8 @@
"provider": "GITHUB_COPILOT",
"provider_display_name": "Github Copilot",
"litellm_provider": "github_copilot",
"credential_fields": [
{
"key": "api_base",
"label": "API Base",
"placeholder": null,
"tooltip": null,
"required": false,
"field_type": "text",
"options": null,
"default_value": null
},
{
"key": "api_key",
"label": "API Key",
"placeholder": null,
"tooltip": null,
"required": false,
"field_type": "password",
"options": null,
"default_value": null
}
],
"auth_flow": "device_code",
"credential_fields": [],
"default_model_placeholder": "gpt-3.5-turbo"
},
{

12
litellm/router.py
View file

@ -1455,6 +1455,7 @@ class Router:
self._update_kwargs_with_deployment(deployment=deployment, kwargs=kwargs)
kwargs.pop("silent_model", None) # Ensure it's not in kwargs either
model_name = litellm_params["model"]
potential_model_client = self._get_client(
deployment=deployment, kwargs=kwargs
@ -2365,6 +2366,17 @@ class Router:
existing_tags.append(credential_tag)
kwargs[metadata_variable_name]["tags"] = existing_tags
## EARLY CREDENTIAL RESOLUTION
# Resolve api_key from litellm_credential_name before client selection
# or function invocation. Without this, deployments using named
# credentials pass api_key=None to cached-client checks and to
# downstream litellm functions (whose @client decorator skips
# load_credentials_from_list for async requests).
if credential_name and not kwargs.get("api_key"):
_cred = CredentialAccessor.get_credential_values(credential_name)
if _cred.get("api_key"):
kwargs["api_key"] = _cred["api_key"]
kwargs["model_info"] = model_info
kwargs["timeout"] = self._get_timeout(

1
litellm/types/proxy/public_endpoints/public_endpoints.py
View file

@ -28,6 +28,7 @@ class ProviderCreateInfo(BaseModel):
provider: str
provider_display_name: str
litellm_provider: str
auth_flow: Optional[Literal["device_code"]] = None
credential_fields: List[ProviderCredentialField]
default_model_placeholder: Optional[str] = None

63
tests/test_litellm/llms/github_copilot/embedding/test_github_copilot_embedding_transformation.py
View file

@ -10,26 +10,24 @@ from litellm.exceptions import AuthenticationError
from litellm.llms.github_copilot.embedding.transformation import GithubCopilotEmbeddingConfig
from litellm.llms.github_copilot.common_utils import GetAPIKeyError
def test_github_copilot_embedding_config_validate_environment():
@patch("litellm.llms.github_copilot.embedding.transformation.Authenticator")
def test_github_copilot_embedding_config_validate_environment(mock_authenticator_class):
"""Test the GitHub Copilot embedding configuration environment validation."""
config = GithubCopilotEmbeddingConfig()
# Mock the authenticator
mock_api_key = "gh.test-key-123456789"
config.authenticator = MagicMock()
config.authenticator.get_api_key.return_value = mock_api_key
mock_auth_instance = MagicMock()
mock_auth_instance.get_api_key.return_value = mock_api_key
mock_authenticator_class.return_value = mock_auth_instance
# Test with valid API key
headers = {}
config = GithubCopilotEmbeddingConfig()
model = "github_copilot/text-embedding-3-small"
validated_headers = config.validate_environment(
headers=headers,
headers={},
model=model,
messages=[],
optional_params={},
litellm_params={},
api_key=None,
api_key="gh-access-token",
)
assert validated_headers["Authorization"] == f"Bearer {mock_api_key}"
@ -37,12 +35,7 @@ def test_github_copilot_embedding_config_validate_environment():
assert validated_headers["editor-version"] == "vscode/1.95.0"
assert "x-request-id" in validated_headers
# Test with authentication failure
config.authenticator.get_api_key.side_effect = GetAPIKeyError(
message="Failed to get API key",
status_code=401,
)
# Test with no api_key → immediate AuthenticationError
with pytest.raises(AuthenticationError) as excinfo:
config.validate_environment(
headers={},
@ -52,16 +45,33 @@ def test_github_copilot_embedding_config_validate_environment():
litellm_params={},
api_key=None,
)
assert "required" in str(excinfo.value).lower()
# Test with authentication failure from GitHub
mock_auth_instance.get_api_key.side_effect = GetAPIKeyError(
message="Failed to get API key",
status_code=401,
)
with pytest.raises(AuthenticationError) as excinfo:
config.validate_environment(
headers={},
model=model,
messages=[],
optional_params={},
litellm_params={},
api_key="gh-access-token",
)
assert "Failed to get API key" in str(excinfo.value)
def test_github_copilot_embedding_config_get_complete_url():
@patch("litellm.llms.github_copilot.embedding.transformation.Authenticator")
def test_github_copilot_embedding_config_get_complete_url(mock_authenticator_class):
"""Test the GitHub Copilot embedding configuration URL generation."""
mock_auth_instance = MagicMock()
mock_authenticator_class.return_value = mock_auth_instance
config = GithubCopilotEmbeddingConfig()
config.authenticator = MagicMock()
# Test with default API base
config.authenticator.get_api_base.return_value = None
# No api_key → always default base
url = config.get_complete_url(
api_base=None,
api_key=None,
@ -71,19 +81,18 @@ def test_github_copilot_embedding_config_get_complete_url():
)
assert url == "https://api.githubcopilot.com/embeddings"
# Test with custom API base from authenticator
config.authenticator.get_api_base.return_value = "https://api.enterprise.githubcopilot.com"
# api_key + authenticator returns custom base
mock_auth_instance.get_api_base.return_value = "https://api.enterprise.githubcopilot.com"
url = config.get_complete_url(
api_base=None,
api_key=None,
api_key="gh-access-token",
model="github_copilot/text-embedding-3-small",
optional_params={},
litellm_params={},
)
assert url == "https://api.enterprise.githubcopilot.com/embeddings"
# Test with custom API base from params
config.authenticator.get_api_base.return_value = None
# Explicit api_base always wins
url = config.get_complete_url(
api_base="https://custom.api.com",
api_key=None,

23
tests/test_litellm/llms/github_copilot/responses/test_github_copilot_responses_transformation.py
View file

@ -44,7 +44,6 @@ class TestGithubCopilotResponsesAPITransformation:
@patch("litellm.llms.github_copilot.responses.transformation.Authenticator")
def test_github_copilot_responses_endpoint_url(self, mock_authenticator_class):
"""Test that get_complete_url returns correct GitHub Copilot endpoint"""
# Mock authenticator to return default base
mock_auth_instance = MagicMock()
mock_auth_instance.get_api_base.return_value = (
"https://api.individual.githubcopilot.com"
@ -53,13 +52,21 @@ class TestGithubCopilotResponsesAPITransformation:
config = GithubCopilotResponsesAPIConfig()
# Test with default GitHub Copilot API base (from authenticator)
# No api_key in litellm_params → default base
url = config.get_complete_url(api_base=None, litellm_params={})
assert url == "https://api.individual.githubcopilot.com/responses", (
f"Expected GitHub Copilot responses endpoint, got {url}"
assert url == "https://api.githubcopilot.com/responses", (
f"Expected default endpoint when no api_key, got {url}"
)
# Test with custom api_base (overrides authenticator)
# api_key present → authenticator resolves custom base
url = config.get_complete_url(
api_base=None, litellm_params={"api_key": "gh-access-token"}
)
assert url == "https://api.individual.githubcopilot.com/responses", (
f"Expected authenticator-resolved endpoint, got {url}"
)
# Explicit api_base always wins regardless of api_key
custom_url = config.get_complete_url(
api_base="https://custom.githubcopilot.com", litellm_params={}
)
@ -67,7 +74,7 @@ class TestGithubCopilotResponsesAPITransformation:
f"Expected custom endpoint, got {custom_url}"
)
# Test with trailing slash
# Trailing slash stripped
url_with_slash = config.get_complete_url(
api_base="https://api.githubcopilot.com/", litellm_params={}
)
@ -86,7 +93,7 @@ class TestGithubCopilotResponsesAPITransformation:
config = GithubCopilotResponsesAPIConfig()
headers = config.validate_environment(
headers={}, model="gpt-5.1-codex", litellm_params={}
headers={}, model="gpt-5.1-codex", litellm_params={"api_key": "gh-access-token"}
)
# Check required headers
@ -115,7 +122,7 @@ class TestGithubCopilotResponsesAPITransformation:
}
headers = config.validate_environment(
headers=custom_headers, model="gpt-5.1-codex", litellm_params={}
headers=custom_headers, model="gpt-5.1-codex", litellm_params={"api_key": "gh-access-token"}
)
# User header should override default

139
tests/test_litellm/llms/github_copilot/test_github_copilot_authenticator.py
View file

@ -67,24 +67,18 @@ class TestGitHubCopilotAuthenticator:
token = authenticator.get_access_token()
assert token == mock_token
def test_get_access_token_login(self, authenticator):
"""Test logging in to get an access token."""
mock_token = "mock-access-token"
with patch.object(authenticator, "_login", return_value=mock_token), \
patch("builtins.open", mock_open()), \
patch("builtins.open", side_effect=IOError) as mock_read:
token = authenticator.get_access_token()
assert token == mock_token
authenticator._login.assert_called_once()
def test_get_access_token_no_file_raises(self, authenticator):
"""Test that GetAccessTokenError is raised when no access-token file exists."""
with patch("builtins.open", side_effect=IOError):
with pytest.raises(GetAccessTokenError) as exc_info:
authenticator.get_access_token()
assert "https://docs.litellm.ai/docs/providers/github_copilot" in str(exc_info.value)
def test_get_access_token_failure(self, authenticator):
"""Test that an exception is raised after multiple login failures."""
with patch.object(authenticator, "_login", side_effect=GetDeviceCodeError(message="Test error", status_code=400)), \
patch("builtins.open", side_effect=IOError):
def test_get_access_token_empty_file_raises(self, authenticator):
"""Test that GetAccessTokenError is raised when the access-token file is empty."""
with patch("builtins.open", mock_open(read_data="")):
with pytest.raises(GetAccessTokenError):
authenticator.get_access_token()
assert authenticator._login.call_count == 3
def test_get_api_key_from_file(self, authenticator):
"""Test retrieving an API key from a file."""
@ -179,6 +173,16 @@ class TestGitHubCopilotAuthenticator:
authenticator._poll_for_access_token.assert_called_once_with("mock-device-code")
mock_print.assert_called_once()
def test_get_github_headers_static(self):
"""Test that get_github_headers works as a static method."""
headers = Authenticator.get_github_headers()
assert "accept" in headers
assert "content-type" in headers
assert "authorization" not in headers
headers_with_token = Authenticator.get_github_headers("my-token")
assert headers_with_token["authorization"] == "token my-token"
def test_get_api_base_from_file(self, authenticator):
"""Test retrieving the API base endpoint from a file."""
mock_api_key_data = json.dumps({
@ -189,3 +193,108 @@ class TestGitHubCopilotAuthenticator:
with patch("builtins.open", mock_open(read_data=mock_api_key_data)):
api_base = authenticator.get_api_base()
assert api_base == "https://api.enterprise.githubcopilot.com"
class TestAuthenticatorCredentialMode:
"""Tests for credential mode (injected access token)."""
def test_init_credential_mode_no_file_io(self):
"""Credential mode should not create any directories."""
auth = Authenticator(access_token="test-token")
assert auth._injected_access_token == "test-token"
assert not hasattr(auth, "token_dir")
def test_get_access_token_returns_injected(self):
"""get_access_token returns the injected token directly."""
auth = Authenticator(access_token="my-github-token")
assert auth.get_access_token() == "my-github-token"
def test_get_api_key_credential_mode(self):
"""get_api_key in credential mode calls _refresh_api_key and caches."""
import litellm.llms.github_copilot.authenticator as auth_module
access_token = "my-github-token-caching-test"
# Clear any stale cache entry before the test
auth_module._credential_api_key_cache.pop(access_token, None)
auth = Authenticator(access_token=access_token)
future_time = (datetime.now() + timedelta(hours=1)).timestamp()
mock_api_key_info = {"token": "copilot-api-key", "expires_at": future_time}
mock_client = MagicMock()
mock_response = MagicMock()
mock_response.raise_for_status.return_value = None
mock_response.json.return_value = mock_api_key_info
mock_client.get.return_value = mock_response
with patch(
"litellm.llms.github_copilot.authenticator._get_httpx_client",
return_value=mock_client,
):
api_key = auth.get_api_key()
assert api_key == "copilot-api-key"
# Second call should use cache (no additional HTTP calls)
api_key2 = auth.get_api_key()
assert api_key2 == "copilot-api-key"
assert mock_client.get.call_count == 1 # Only called once
# Cleanup
auth_module._credential_api_key_cache.pop(access_token, None)
def test_get_api_key_credential_mode_expired_cache(self):
"""get_api_key re-fetches when cached token is expired."""
import litellm.llms.github_copilot.authenticator as auth_module
past_time = (datetime.now() - timedelta(hours=1)).timestamp()
future_time = (datetime.now() + timedelta(hours=1)).timestamp()
access_token = "my-github-token-expired-test"
auth = Authenticator(access_token=access_token)
# Pre-populate module-level cache with an expired entry
auth_module._credential_api_key_cache[access_token] = {
"token": "old-key",
"expires_at": past_time,
}
mock_client = MagicMock()
mock_response = MagicMock()
mock_response.raise_for_status.return_value = None
mock_response.json.return_value = {"token": "new-key", "expires_at": future_time}
mock_client.get.return_value = mock_response
with patch(
"litellm.llms.github_copilot.authenticator._get_httpx_client",
return_value=mock_client,
):
api_key = auth.get_api_key()
assert api_key == "new-key"
assert mock_client.get.call_count == 1
# Cleanup
auth_module._credential_api_key_cache.pop(access_token, None)
def test_get_api_base_credential_mode_no_cache(self):
"""get_api_base returns None when no cache is available."""
import litellm.llms.github_copilot.authenticator as auth_module
access_token = "my-github-token-no-cache-test"
auth = Authenticator(access_token=access_token)
# Ensure no stale cache
auth_module._credential_api_key_cache.pop(access_token, None)
assert auth.get_api_base() is None
def test_get_api_base_credential_mode_with_cache(self):
"""get_api_base returns endpoint from module-level cache."""
import litellm.llms.github_copilot.authenticator as auth_module
access_token = "my-github-token-cache-test"
auth = Authenticator(access_token=access_token)
auth_module._credential_api_key_cache[access_token] = {
"token": "test",
"endpoints": {"api": "https://custom.copilot.api"},
}
assert auth.get_api_base() == "https://custom.copilot.api"
# Cleanup
auth_module._credential_api_key_cache.pop(access_token, None)

114
tests/test_litellm/llms/github_copilot/test_github_copilot_transformation.py
View file

@ -31,63 +31,62 @@ from litellm.llms.github_copilot.common_utils import (
)
def test_github_copilot_config_get_openai_compatible_provider_info():
@patch("litellm.llms.github_copilot.chat.transformation.Authenticator")
def test_github_copilot_config_get_openai_compatible_provider_info(mock_authenticator_class):
"""Test the GitHub Copilot configuration provider info retrieval."""
mock_api_key = "gh.test-key-123456789"
mock_auth_instance = MagicMock()
mock_auth_instance.get_api_key.return_value = mock_api_key
mock_auth_instance.get_api_base.return_value = "https://api.enterprise.githubcopilot.com"
mock_authenticator_class.return_value = mock_auth_instance
config = GithubCopilotConfig()
# Mock the authenticator to avoid actual API calls
mock_api_key = "gh.test-key-123456789"
config.authenticator = MagicMock()
config.authenticator.get_api_key.return_value = mock_api_key
# Test with dynamic endpoint
config.authenticator.get_api_base.return_value = "https://api.enterprise.githubcopilot.com"
# Test with default values
model = "github_copilot/gpt-4"
(
api_base,
dynamic_api_key,
custom_llm_provider,
) = config._get_openai_compatible_provider_info(
model=model,
api_base=None,
api_key=None,
custom_llm_provider="github_copilot",
)
assert api_base == "https://api.enterprise.githubcopilot.com"
assert dynamic_api_key == mock_api_key
assert custom_llm_provider == "github_copilot"
# Test fallback to default if no dynamic endpoint
config.authenticator.get_api_base.return_value = None
(
api_base,
dynamic_api_key,
custom_llm_provider,
) = config._get_openai_compatible_provider_info(
# No api_key → returns defaults immediately without calling authenticator
api_base, dynamic_api_key, custom_llm_provider = config._get_openai_compatible_provider_info(
model=model,
api_base=None,
api_key=None,
custom_llm_provider="github_copilot",
)
assert api_base == "https://api.githubcopilot.com"
assert dynamic_api_key is None
assert custom_llm_provider == "github_copilot"
# Test with authentication failure
config.authenticator.get_api_key.side_effect = GetAPIKeyError(
# With api_key → uses authenticator's dynamic base
api_base, dynamic_api_key, custom_llm_provider = config._get_openai_compatible_provider_info(
model=model,
api_base=None,
api_key="gh-access-token",
custom_llm_provider="github_copilot",
)
assert api_base == "https://api.enterprise.githubcopilot.com"
assert dynamic_api_key == mock_api_key
assert custom_llm_provider == "github_copilot"
# Fallback to default when authenticator returns no base
mock_auth_instance.get_api_base.return_value = None
api_base, _, _ = config._get_openai_compatible_provider_info(
model=model,
api_base=None,
api_key="gh-access-token",
custom_llm_provider="github_copilot",
)
assert api_base == "https://api.githubcopilot.com"
# Authentication failure
mock_auth_instance.get_api_key.side_effect = GetAPIKeyError(
message="Failed to get API key",
status_code=401,
)
with pytest.raises(AuthenticationError) as excinfo:
config._get_openai_compatible_provider_info(
model=model,
api_base=None,
api_key=None,
api_key="gh-access-token",
custom_llm_provider="github_copilot",
)
assert "Failed to get API key" in str(excinfo.value)
@ -121,6 +120,7 @@ def test_completion_github_copilot_mock_response(mock_completion, mock_get_api_k
response = completion(
model="github_copilot/gpt-4",
messages=messages,
api_key="gh-access-token",
extra_headers=headers,
)
@ -181,10 +181,6 @@ def test_x_initiator_header_user_request():
"""Test that user-only messages result in X-Initiator: user header"""
config = GithubCopilotConfig()
# Mock the authenticator
config.authenticator = MagicMock()
config.authenticator.get_api_key.return_value = "gh.test-key-123"
config.authenticator.get_api_base.return_value = None
messages = [
{"role": "system", "content": "You are an assistant."},
@ -208,10 +204,6 @@ def test_x_initiator_header_agent_request_with_assistant():
"""Test that messages with assistant role result in X-Initiator: agent header"""
config = GithubCopilotConfig()
# Mock the authenticator
config.authenticator = MagicMock()
config.authenticator.get_api_key.return_value = "gh.test-key-123"
config.authenticator.get_api_base.return_value = None
messages = [
{"role": "system", "content": "You are an assistant."},
@ -235,10 +227,6 @@ def test_x_initiator_header_agent_request_with_tool():
"""Test that messages with tool role result in X-Initiator: agent header"""
config = GithubCopilotConfig()
# Mock the authenticator
config.authenticator = MagicMock()
config.authenticator.get_api_key.return_value = "gh.test-key-123"
config.authenticator.get_api_base.return_value = None
messages = [
{"role": "system", "content": "You are an assistant."},
@ -261,11 +249,6 @@ def test_x_initiator_header_agent_request_with_tool():
def test_x_initiator_header_mixed_messages_with_agent_roles():
"""Test that mixed messages with agent roles (assistant/tool) result in X-Initiator: agent header"""
config = GithubCopilotConfig()
# Mock the authenticator
config.authenticator = MagicMock()
config.authenticator.get_api_key.return_value = "gh.test-key-123"
config.authenticator.get_api_base.return_value = None
messages = [
{"role": "user", "content": "Hello"},
@ -289,11 +272,6 @@ def test_x_initiator_header_mixed_messages_with_agent_roles():
def test_x_initiator_header_user_only_messages():
"""Test that user + system only messages result in X-Initiator: user header"""
config = GithubCopilotConfig()
# Mock the authenticator
config.authenticator = MagicMock()
config.authenticator.get_api_key.return_value = "gh.test-key-123"
config.authenticator.get_api_base.return_value = None
messages = [
{"role": "system", "content": "You are an assistant."},
@ -318,10 +296,6 @@ def test_x_initiator_header_empty_messages():
"""Test that empty messages result in X-Initiator: user header"""
config = GithubCopilotConfig()
# Mock the authenticator
config.authenticator = MagicMock()
config.authenticator.get_api_key.return_value = "gh.test-key-123"
config.authenticator.get_api_base.return_value = None
messages = []
@ -342,10 +316,6 @@ def test_x_initiator_header_system_only_messages():
"""Test that system-only messages result in X-Initiator: user header"""
config = GithubCopilotConfig()
# Mock the authenticator
config.authenticator = MagicMock()
config.authenticator.get_api_key.return_value = "gh.test-key-123"
config.authenticator.get_api_base.return_value = None
messages = [
{"role": "system", "content": "You are an assistant."},
@ -419,10 +389,6 @@ def test_copilot_vision_request_header_with_image():
"""Test that Copilot-Vision-Request header is added when messages contain images"""
config = GithubCopilotConfig()
# Mock the authenticator
config.authenticator = MagicMock()
config.authenticator.get_api_key.return_value = "gh.test-key-123"
config.authenticator.get_api_base.return_value = None
messages = [
{
@ -455,10 +421,6 @@ def test_copilot_vision_request_header_text_only():
"""Test that Copilot-Vision-Request header is not added for text-only messages"""
config = GithubCopilotConfig()
# Mock the authenticator
config.authenticator = MagicMock()
config.authenticator.get_api_key.return_value = "gh.test-key-123"
config.authenticator.get_api_base.return_value = None
messages = [
{"role": "user", "content": "Just a text message"},
@ -482,10 +444,6 @@ def test_copilot_vision_request_header_with_type_image_url():
"""Test that Copilot-Vision-Request header is added for content with type: image_url"""
config = GithubCopilotConfig()
# Mock the authenticator
config.authenticator = MagicMock()
config.authenticator.get_api_key.return_value = "gh.test-key-123"
config.authenticator.get_api_base.return_value = None
messages = [
{

270
ui/litellm-dashboard/src/components/model_add/AddCredentialModal.tsx
View file

@ -1,21 +1,76 @@
import { TextInput } from "@tremor/react";
import { Select as AntdSelect, Button, Form, Modal, Tooltip, Typography } from "antd";
import { Select as AntdSelect, Button, Form, Modal, Spin, Tooltip, Typography } from "antd";
import type { UploadProps } from "antd/es/upload";
import React, { useState } from "react";
import React, { useCallback, useEffect, useRef, useState } from "react";
import {
credentialCreateCall,
githubCopilotInitiateAuth,
githubCopilotCheckStatus,
} from "@/components/networking";
import NotificationsManager from "../molecules/notifications_manager";
import ProviderSpecificFields from "../add_model/provider_specific_fields";
import { Providers, providerLogoMap } from "../provider_info_helpers";
const { Link } = Typography;
import useAuthorized from "@/app/(dashboard)/hooks/useAuthorized";
import { useProviderFields } from "@/app/(dashboard)/hooks/providers/useProviderFields";
const { Link, Text } = Typography;
interface AddCredentialsModalProps {
open: boolean;
onCancel: () => void;
onAddCredential: (values: any) => void;
uploadProps: UploadProps;
initialCredentialName?: string;
initialProvider?: string;
}
const AddCredentialsModal: React.FC<AddCredentialsModalProps> = ({ open, onCancel, onAddCredential, uploadProps }) => {
const AddCredentialsModal: React.FC<AddCredentialsModalProps> = ({ open, onCancel, onAddCredential, uploadProps, initialCredentialName, initialProvider }) => {
const [form] = Form.useForm();
const [selectedProvider, setSelectedProvider] = useState<Providers>(Providers.OpenAI);
const { accessToken } = useAuthorized();
const { data: providerMetadata } = useProviderFields();
// Device code flow state
const [deviceCodeState, setDeviceCodeState] = useState<
| { phase: "idle" }
| { phase: "polling"; deviceCode: string; userCode: string; verificationUri: string }
| { phase: "success"; credentialName: string }
| { phase: "error"; message: string }
>({ phase: "idle" });
// Hold access_token in a ref — never rendered, never put in form fields
const accessTokenRef = useRef<string | null>(null);
const pollingRef = useRef<ReturnType<typeof setInterval> | null>(null);
// Determine if the selected provider uses device_code auth flow
const isDeviceCodeProvider = React.useMemo(() => {
if (!providerMetadata) return false;
const info = providerMetadata.find(
(p) =>
p.provider === selectedProvider ||
p.provider_display_name === Providers[selectedProvider as keyof typeof Providers],
);
return info?.auth_flow === "device_code";
}, [selectedProvider, providerMetadata]);
// Cleanup polling on unmount or modal close
const stopPolling = useCallback(() => {
if (pollingRef.current) {
clearInterval(pollingRef.current);
pollingRef.current = null;
}
}, []);
useEffect(() => {
return stopPolling;
}, [stopPolling]);
const handleCancel = () => {
stopPolling();
setDeviceCodeState({ phase: "idle" });
accessTokenRef.current = null;
onCancel();
form.resetFields();
};
const handleSubmit = (values: any) => {
const filteredValues = Object.entries(values).reduce((acc, [key, value]) => {
@ -28,14 +83,168 @@ const AddCredentialsModal: React.FC<AddCredentialsModalProps> = ({ open, onCance
form.resetFields();
};
const handleStartDeviceCode = async () => {
const credentialName = form.getFieldValue("credential_name");
if (!credentialName) {
form.validateFields(["credential_name"]);
return;
}
if (!accessToken) return;
try {
const result = await githubCopilotInitiateAuth(accessToken);
setDeviceCodeState({
phase: "polling",
deviceCode: result.device_code,
userCode: result.user_code,
verificationUri: result.verification_uri,
});
if (!result.poll_interval_ms) throw new Error("GitHub initiate response missing poll_interval_ms");
// Mutable baseline — ratchets up when GitHub sends slow_down so that
// subsequent normal "pending" responses keep using the increased interval.
let currentPollInterval = result.poll_interval_ms;
// setTimeout-based loop so each poll fires only after the previous one
// completes, and slow_down's retry_after_ms is respected exactly.
const schedulePoll = (delayMs: number) => {
pollingRef.current = setTimeout(async () => {
try {
const status = await githubCopilotCheckStatus(accessToken, result.device_code);
console.log("[GH Copilot AddCredential] poll response:", status);
if (status.status === "complete" && status.access_token) {
stopPolling();
accessTokenRef.current = status.access_token;
// Store as named credential
try {
await credentialCreateCall(accessToken, {
credential_name: credentialName,
credential_values: { api_key: status.access_token },
credential_info: { custom_llm_provider: "github_copilot" },
});
setDeviceCodeState({ phase: "success", credentialName });
} catch (e) {
console.error("[GH Copilot AddCredential] credentialCreateCall failed:", e);
NotificationsManager.error(
`Failed to save credential: ${e instanceof Error ? e.message : "Unknown error"}`,
);
setDeviceCodeState({ phase: "error", message: "Failed to save credential" });
}
} else if (status.status === "failed") {
stopPolling();
setDeviceCodeState({ phase: "error", message: status.error || "Authorization failed" });
} else {
// pending — ratchet up the baseline if GitHub requested slower
if (status.retry_after_ms != null) {
currentPollInterval = status.retry_after_ms;
}
schedulePoll(currentPollInterval);
}
} catch (e) {
console.error("[GH Copilot AddCredential] poll error:", e);
stopPolling();
setDeviceCodeState({ phase: "error", message: "Failed to check authorization status" });
}
}, delayMs);
};
schedulePoll(currentPollInterval);
} catch {
setDeviceCodeState({ phase: "error", message: "Failed to start GitHub authorization" });
}
};
const handleSuccessClose = () => {
stopPolling();
setDeviceCodeState({ phase: "idle" });
accessTokenRef.current = null;
onCancel();
form.resetFields();
};
const renderDeviceCodeFlow = () => {
switch (deviceCodeState.phase) {
case "idle":
return (
<div className="text-center py-4">
<Text className="block mb-4">
GitHub Copilot uses OAuth Device Code authorization. Click below to start.
</Text>
<Button type="primary" onClick={handleStartDeviceCode}>
Start GitHub Authorization
</Button>
</div>
);
case "polling":
return (
<div className="text-center py-4">
<Text className="block mb-2">
Enter this code on GitHub:
</Text>
<div
style={{
fontSize: "2rem",
fontWeight: "bold",
fontFamily: "monospace",
letterSpacing: "0.3em",
margin: "16px 0",
padding: "12px 24px",
background: "#f5f5f5",
borderRadius: 8,
display: "inline-block",
userSelect: "all",
}}
>
{deviceCodeState.userCode}
</div>
<div className="mb-4">
<Button
type="link"
onClick={() => window.open(deviceCodeState.verificationUri, "_blank")}
>
Open {deviceCodeState.verificationUri}
</Button>
</div>
<Spin />
<Text className="block mt-2 mb-4" type="secondary">
Waiting for GitHub authorization...
</Text>
<Button onClick={handleCancel}>Cancel</Button>
</div>
);
case "success":
return (
<div className="text-center py-4">
<Text className="block mb-4" type="success" style={{ fontSize: "1.1rem" }}>
GitHub Copilot credential &quot;{deviceCodeState.credentialName}&quot; created successfully!
</Text>
<Button type="primary" onClick={handleSuccessClose}>
Done
</Button>
</div>
);
case "error":
return (
<div className="text-center py-4">
<Text className="block mb-4" type="danger">
{deviceCodeState.message}
</Text>
<Button
onClick={() => setDeviceCodeState({ phase: "idle" })}
style={{ marginRight: 8 }}
>
Retry
</Button>
<Button onClick={handleCancel}>Cancel</Button>
</div>
);
}
};
return (
<Modal
title="Add New Credential"
open={open}
onCancel={() => {
onCancel();
form.resetFields();
}}
onCancel={handleCancel}
footer={null}
width={600}
>
@ -61,6 +270,10 @@ const AddCredentialsModal: React.FC<AddCredentialsModalProps> = ({ open, onCance
onChange={(value) => {
setSelectedProvider(value as Providers);
form.setFieldValue("custom_llm_provider", value);
// Reset device code state when provider changes
stopPolling();
setDeviceCodeState({ phase: "idle" });
accessTokenRef.current = null;
}}
>
{Object.entries(Providers).map(([providerEnum, providerDisplayName]) => (
@ -89,27 +302,30 @@ const AddCredentialsModal: React.FC<AddCredentialsModalProps> = ({ open, onCance
</AntdSelect>
</Form.Item>
<ProviderSpecificFields selectedProvider={selectedProvider} uploadProps={uploadProps} />
{isDeviceCodeProvider ? (
renderDeviceCodeFlow()
) : (
<>
<ProviderSpecificFields selectedProvider={selectedProvider} uploadProps={uploadProps} />
{/* Modal Footer */}
<div className="flex justify-between items-center">
<Tooltip title="Get help on our github">
<Link href="https://github.com/BerriAI/litellm/issues">Need Help?</Link>
</Tooltip>
{/* Modal Footer */}
<div className="flex justify-between items-center">
<Tooltip title="Get help on our github">
<Link href="https://github.com/BerriAI/litellm/issues">Need Help?</Link>
</Tooltip>
<div>
<Button
onClick={() => {
onCancel();
form.resetFields();
}}
style={{ marginRight: 10 }}
>
Cancel
</Button>
<Button htmlType="submit">{"Add Credential"}</Button>
</div>
</div>
<div>
<Button
onClick={handleCancel}
style={{ marginRight: 10 }}
>
Cancel
</Button>
<Button htmlType="submit">{"Add Credential"}</Button>
</div>
</div>
</>
)}
</Form>
</Modal>
);