mirror of
https://github.com/BerriAI/litellm.git
synced 2026-08-28 05:25:59 +00:00
[Feat] Add github co-pilot as a new LLM API provider (#12325)
* Litellm dev 03 05 2025 contributor prs (#9079) * feat: add support for copilot provider * test: add tests for github copilot * chore: clean up github copilot authenticator * test: add test for github copilot authenticator * test: add test for github copilot for sonnet 3.7 thought model * Fix #7629 - Add tzdata package to Dockerfile (#8915) * Add tzdata package to Dockerfile * Move tzdata to python requirement.txt * feat: add support for copilot provider (#8577) * feat: add support for copilot provider * test: add tests for github copilot * chore: clean up github copilot authenticator * test: add test for github copilot authenticator * test: add test for github copilot for sonnet 3.7 thought model --------- Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com> * feat: add model information for copilot models * fix: fix linting errors * test: remove integration test for github_copilot + fix misisng mock * fix: use print to make sure the logger message shown * test: remove debug print * fix lint (#11112) * Add init files to make test directories Python packages and update import paths in test_token_counter.py (#11119) * Update litellm/model_prices_and_context_window_backup.json Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com> --------- Co-authored-by: Son H. Nguyen <nhs.000.dev@gmail.com> Co-authored-by: subnet.dev <50828879+subnet-dev@users.noreply.github.com> Co-authored-by: Son H. Nguyen <33925625+nhs000@users.noreply.github.com> Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com> Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> * refactor github copilot * test_github_copilot_transformation.py * test_github_copilot_authenticator.py * add GitHub Copilot * fix order * doc fix --------- Co-authored-by: Krish Dholakia <krrishdholakia@gmail.com> Co-authored-by: Son H. Nguyen <nhs.000.dev@gmail.com> Co-authored-by: subnet.dev <50828879+subnet-dev@users.noreply.github.com> Co-authored-by: Son H. Nguyen <33925625+nhs000@users.noreply.github.com> Co-authored-by: மனோஜ்குமார் பழனிச்சாமி <smartmanoj42857@gmail.com>
This commit is contained in:
Ishaan Jaff
GitHub
parent
453591ed7c
commit
c3e673b627
14 changed files with 942 additions and 1 deletions
186
docs/my-website/docs/providers/github_copilot.md
Normal file
186
docs/my-website/docs/providers/github_copilot.md
Normal file
|
|
@ -0,0 +1,186 @@
|
|||
import Tabs from '@theme/Tabs';
|
||||
import TabItem from '@theme/TabItem';
|
||||
|
||||
# GitHub Copilot
|
||||
|
||||
https://docs.github.com/en/copilot
|
||||
|
||||
:::tip
|
||||
|
||||
**We support GitHub Copilot Chat API with automatic authentication handling**
|
||||
|
||||
:::
|
||||
|
||||
| Property | Details |
|
||||
|-------|-------|
|
||||
| Description | GitHub Copilot Chat API provides access to GitHub's AI-powered coding assistant. |
|
||||
| Provider Route on LiteLLM | `github_copilot/` |
|
||||
| Supported Endpoints | `/chat/completions` |
|
||||
| API Reference | [GitHub Copilot docs](https://docs.github.com/en/copilot) |
|
||||
|
||||
## Authentication
|
||||
|
||||
GitHub Copilot uses OAuth device flow for authentication. On first use, you'll be prompted to authenticate via GitHub:
|
||||
|
||||
1. LiteLLM will display a device code and verification URL
|
||||
2. Visit the URL and enter the code to authenticate
|
||||
3. Your credentials will be stored locally for future use
|
||||
|
||||
## Usage - LiteLLM Python SDK
|
||||
|
||||
### Chat Completion
|
||||
|
||||
```python showLineNumbers title="GitHub Copilot Chat Completion"
|
||||
from litellm import completion
|
||||
|
||||
response = completion(
|
||||
model="github_copilot/gpt-4",
|
||||
messages=[{"role": "user", "content": "Write a Python function to calculate fibonacci numbers"}],
|
||||
extra_headers={
|
||||
"editor-version": "vscode/1.85.1",
|
||||
"Copilot-Integration-Id": "vscode-chat"
|
||||
}
|
||||
)
|
||||
print(response)
|
||||
```
|
||||
|
||||
```python showLineNumbers title="GitHub Copilot Chat Completion - Streaming"
|
||||
from litellm import completion
|
||||
|
||||
stream = completion(
|
||||
model="github_copilot/gpt-4",
|
||||
messages=[{"role": "user", "content": "Explain async/await in Python"}],
|
||||
stream=True,
|
||||
extra_headers={
|
||||
"editor-version": "vscode/1.85.1",
|
||||
"Copilot-Integration-Id": "vscode-chat"
|
||||
}
|
||||
)
|
||||
|
||||
for chunk in stream:
|
||||
if chunk.choices[0].delta.content is not None:
|
||||
print(chunk.choices[0].delta.content, end="")
|
||||
```
|
||||
|
||||
## Usage - LiteLLM Proxy
|
||||
|
||||
Add the following to your LiteLLM Proxy configuration file:
|
||||
|
||||
```yaml showLineNumbers title="config.yaml"
|
||||
model_list:
|
||||
- model_name: github_copilot/gpt-4
|
||||
litellm_params:
|
||||
model: github_copilot/gpt-4
|
||||
```
|
||||
|
||||
Start your LiteLLM Proxy server:
|
||||
|
||||
```bash showLineNumbers title="Start LiteLLM Proxy"
|
||||
litellm --config config.yaml
|
||||
|
||||
# RUNNING on http://0.0.0.0:4000
|
||||
```
|
||||
|
||||
<Tabs>
|
||||
<TabItem value="openai-sdk" label="OpenAI SDK">
|
||||
|
||||
```python showLineNumbers title="GitHub Copilot via Proxy - Non-streaming"
|
||||
from openai import OpenAI
|
||||
|
||||
# Initialize client with your proxy URL
|
||||
client = OpenAI(
|
||||
base_url="http://localhost:4000", # Your proxy URL
|
||||
api_key="your-proxy-api-key" # Your proxy API key
|
||||
)
|
||||
|
||||
# Non-streaming response
|
||||
response = client.chat.completions.create(
|
||||
model="github_copilot/gpt-4",
|
||||
messages=[{"role": "user", "content": "How do I optimize this SQL query?"}],
|
||||
extra_headers={
|
||||
"editor-version": "vscode/1.85.1",
|
||||
"Copilot-Integration-Id": "vscode-chat"
|
||||
}
|
||||
)
|
||||
|
||||
print(response.choices[0].message.content)
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
|
||||
<TabItem value="litellm-sdk" label="LiteLLM SDK">
|
||||
|
||||
```python showLineNumbers title="GitHub Copilot via Proxy - LiteLLM SDK"
|
||||
import litellm
|
||||
|
||||
# Configure LiteLLM to use your proxy
|
||||
response = litellm.completion(
|
||||
model="litellm_proxy/github_copilot/gpt-4",
|
||||
messages=[{"role": "user", "content": "Review this code for bugs"}],
|
||||
api_base="http://localhost:4000",
|
||||
api_key="your-proxy-api-key",
|
||||
extra_headers={
|
||||
"editor-version": "vscode/1.85.1",
|
||||
"Copilot-Integration-Id": "vscode-chat"
|
||||
}
|
||||
)
|
||||
|
||||
print(response.choices[0].message.content)
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
|
||||
<TabItem value="curl" label="cURL">
|
||||
|
||||
```bash showLineNumbers title="GitHub Copilot via Proxy - cURL"
|
||||
curl http://localhost:4000/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer your-proxy-api-key" \
|
||||
-H "editor-version: vscode/1.85.1" \
|
||||
-H "Copilot-Integration-Id: vscode-chat" \
|
||||
-d '{
|
||||
"model": "github_copilot/gpt-4",
|
||||
"messages": [{"role": "user", "content": "Explain this error message"}]
|
||||
}'
|
||||
```
|
||||
|
||||
</TabItem>
|
||||
</Tabs>
|
||||
|
||||
## Getting Started
|
||||
|
||||
1. Ensure you have GitHub Copilot access (paid GitHub subscription required)
|
||||
2. Run your first LiteLLM request - you'll be prompted to authenticate
|
||||
3. Follow the device flow authentication process
|
||||
4. Start making requests to GitHub Copilot through LiteLLM
|
||||
|
||||
## Configuration
|
||||
|
||||
### Environment Variables
|
||||
|
||||
You can customize token storage locations:
|
||||
|
||||
```bash showLineNumbers title="Environment Variables"
|
||||
# Optional: Custom token directory
|
||||
export GITHUB_COPILOT_TOKEN_DIR="~/.config/litellm/github_copilot"
|
||||
|
||||
# Optional: Custom access token file name
|
||||
export GITHUB_COPILOT_ACCESS_TOKEN_FILE="access-token"
|
||||
|
||||
# Optional: Custom API key file name
|
||||
export GITHUB_COPILOT_API_KEY_FILE="api-key.json"
|
||||
```
|
||||
|
||||
### Headers
|
||||
|
||||
GitHub Copilot supports various editor-specific headers:
|
||||
|
||||
```python showLineNumbers title="Common Headers"
|
||||
extra_headers = {
|
||||
"editor-version": "vscode/1.85.1", # Editor version
|
||||
"editor-plugin-version": "copilot/1.155.0", # Plugin version
|
||||
"Copilot-Integration-Id": "vscode-chat", # Integration ID
|
||||
"user-agent": "GithubCopilot/1.155.0" # User agent
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -415,7 +415,6 @@ const sidebars = {
|
|||
"providers/galadriel",
|
||||
"providers/topaz",
|
||||
"providers/groq",
|
||||
"providers/github",
|
||||
"providers/deepseek",
|
||||
"providers/elevenlabs",
|
||||
"providers/fireworks_ai",
|
||||
|
|
@ -426,6 +425,8 @@ const sidebars = {
|
|||
"providers/xinference",
|
||||
"providers/cloudflare_workers",
|
||||
"providers/deepinfra",
|
||||
"providers/github",
|
||||
"providers/github_copilot",
|
||||
"providers/ai21",
|
||||
"providers/nlp_cloud",
|
||||
"providers/replicate",
|
||||
|
|
|
|||
|
|
@ -1119,6 +1119,7 @@ from .llms.azure.chat.o_series_transformation import AzureOpenAIO1Config
|
|||
from .llms.watsonx.completion.transformation import IBMWatsonXAIConfig
|
||||
from .llms.watsonx.chat.transformation import IBMWatsonXChatConfig
|
||||
from .llms.watsonx.embed.transformation import IBMWatsonXEmbeddingConfig
|
||||
from .llms.github_copilot.chat.transformation import GithubCopilotConfig
|
||||
from .llms.nebius.chat.transformation import NebiusConfig
|
||||
from .main import * # type: ignore
|
||||
from .integrations import *
|
||||
|
|
|
|||
|
|
@ -261,6 +261,7 @@ LITELLM_CHAT_PROVIDERS = [
|
|||
"llamafile",
|
||||
"lm_studio",
|
||||
"galadriel",
|
||||
"github_copilot", # GitHub Copilot Chat API
|
||||
"novita",
|
||||
"meta_llama",
|
||||
"featherless_ai",
|
||||
|
|
@ -421,6 +422,7 @@ openai_compatible_providers: List = [
|
|||
"llamafile",
|
||||
"lm_studio",
|
||||
"galadriel",
|
||||
"github_copilot", # GitHub Copilot Chat API
|
||||
"novita",
|
||||
"meta_llama",
|
||||
"featherless_ai",
|
||||
|
|
|
|||
|
|
@ -622,6 +622,14 @@ def _get_openai_compatible_provider_info( # noqa: PLR0915
|
|||
or "https://api.galadriel.com/v1"
|
||||
) # type: ignore
|
||||
dynamic_api_key = api_key or get_secret_str("GALADRIEL_API_KEY")
|
||||
elif custom_llm_provider == "github_copilot":
|
||||
(
|
||||
api_base,
|
||||
dynamic_api_key,
|
||||
custom_llm_provider,
|
||||
) = litellm.GithubCopilotConfig()._get_openai_compatible_provider_info(
|
||||
model, api_base, api_key, custom_llm_provider
|
||||
)
|
||||
elif custom_llm_provider == "novita":
|
||||
api_base = (
|
||||
api_base
|
||||
|
|
|
|||
345
litellm/llms/github_copilot/authenticator.py
Normal file
345
litellm/llms/github_copilot/authenticator.py
Normal file
|
|
@ -0,0 +1,345 @@
|
|||
import json
|
||||
import os
|
||||
import time
|
||||
from datetime import datetime
|
||||
from typing import Any, Dict, Optional
|
||||
|
||||
import httpx
|
||||
|
||||
from litellm._logging import verbose_logger
|
||||
from litellm.llms.custom_httpx.http_handler import _get_httpx_client
|
||||
|
||||
from .common_utils import (
|
||||
APIKeyExpiredError,
|
||||
GetAccessTokenError,
|
||||
GetAPIKeyError,
|
||||
GetDeviceCodeError,
|
||||
RefreshAPIKeyError,
|
||||
)
|
||||
|
||||
# Constants
|
||||
GITHUB_CLIENT_ID = "Iv1.b507a08c87ecfe98"
|
||||
GITHUB_DEVICE_CODE_URL = "https://github.com/login/device/code"
|
||||
GITHUB_ACCESS_TOKEN_URL = "https://github.com/login/oauth/access_token"
|
||||
GITHUB_API_KEY_URL = "https://api.github.com/copilot_internal/v2/token"
|
||||
|
||||
|
||||
class Authenticator:
|
||||
def __init__(self) -> None:
|
||||
"""Initialize the GitHub Copilot authenticator with configurable token paths."""
|
||||
# Token storage paths
|
||||
self.token_dir = os.getenv(
|
||||
"GITHUB_COPILOT_TOKEN_DIR",
|
||||
os.path.expanduser("~/.config/litellm/github_copilot"),
|
||||
)
|
||||
self.access_token_file = os.path.join(
|
||||
self.token_dir,
|
||||
os.getenv("GITHUB_COPILOT_ACCESS_TOKEN_FILE", "access-token"),
|
||||
)
|
||||
self.api_key_file = os.path.join(
|
||||
self.token_dir, os.getenv("GITHUB_COPILOT_API_KEY_FILE", "api-key.json")
|
||||
)
|
||||
self._ensure_token_dir()
|
||||
|
||||
def get_access_token(self) -> str:
|
||||
"""
|
||||
Login to Copilot with retry 3 times.
|
||||
|
||||
Returns:
|
||||
str: The GitHub access token.
|
||||
|
||||
Raises:
|
||||
GetAccessTokenError: If unable to obtain an access token after retries.
|
||||
"""
|
||||
try:
|
||||
with open(self.access_token_file, "r") as f:
|
||||
access_token = f.read().strip()
|
||||
if access_token:
|
||||
return access_token
|
||||
except IOError:
|
||||
verbose_logger.warning(
|
||||
"No existing access token found or error reading file"
|
||||
)
|
||||
|
||||
for attempt in range(3):
|
||||
verbose_logger.debug(f"Access token acquisition attempt {attempt + 1}/3")
|
||||
try:
|
||||
access_token = self._login()
|
||||
try:
|
||||
with open(self.access_token_file, "w") as f:
|
||||
f.write(access_token)
|
||||
except IOError:
|
||||
verbose_logger.error("Error saving access token to file")
|
||||
return access_token
|
||||
except (GetDeviceCodeError, GetAccessTokenError, RefreshAPIKeyError) as e:
|
||||
verbose_logger.warning(f"Failed attempt {attempt + 1}: {str(e)}")
|
||||
continue
|
||||
|
||||
raise GetAccessTokenError(
|
||||
message="Failed to get access token after 3 attempts",
|
||||
status_code=401,
|
||||
)
|
||||
|
||||
def get_api_key(self) -> str:
|
||||
"""
|
||||
Get the API key, refreshing if necessary.
|
||||
|
||||
Returns:
|
||||
str: The GitHub Copilot API key.
|
||||
|
||||
Raises:
|
||||
GetAPIKeyError: If unable to obtain an API key.
|
||||
"""
|
||||
try:
|
||||
with open(self.api_key_file, "r") as f:
|
||||
api_key_info = json.load(f)
|
||||
if api_key_info.get("expires_at", 0) > datetime.now().timestamp():
|
||||
return api_key_info.get("token")
|
||||
else:
|
||||
verbose_logger.warning("API key expired, refreshing")
|
||||
raise APIKeyExpiredError(
|
||||
message="API key expired",
|
||||
status_code=401,
|
||||
)
|
||||
except IOError:
|
||||
verbose_logger.warning("No API key file found or error opening file")
|
||||
except (json.JSONDecodeError, KeyError) as e:
|
||||
verbose_logger.warning(f"Error reading API key from file: {str(e)}")
|
||||
except APIKeyExpiredError:
|
||||
pass # Already logged in the try block
|
||||
|
||||
try:
|
||||
api_key_info = self._refresh_api_key()
|
||||
with open(self.api_key_file, "w") as f:
|
||||
json.dump(api_key_info, f)
|
||||
token = api_key_info.get("token")
|
||||
if token:
|
||||
return token
|
||||
else:
|
||||
raise GetAPIKeyError(
|
||||
message="API key response missing token",
|
||||
status_code=401,
|
||||
)
|
||||
except IOError as e:
|
||||
verbose_logger.error(f"Error saving API key to file: {str(e)}")
|
||||
raise GetAPIKeyError(
|
||||
message=f"Failed to save API key: {str(e)}",
|
||||
status_code=500,
|
||||
)
|
||||
except RefreshAPIKeyError as e:
|
||||
raise GetAPIKeyError(
|
||||
message=f"Failed to refresh API key: {str(e)}",
|
||||
status_code=401,
|
||||
)
|
||||
|
||||
def _refresh_api_key(self) -> Dict[str, Any]:
|
||||
"""
|
||||
Refresh the API key using the access token.
|
||||
|
||||
Returns:
|
||||
Dict[str, Any]: The API key information including token and expiration.
|
||||
|
||||
Raises:
|
||||
RefreshAPIKeyError: If unable to refresh the API key.
|
||||
"""
|
||||
access_token = self.get_access_token()
|
||||
headers = self._get_github_headers(access_token)
|
||||
|
||||
max_retries = 3
|
||||
for attempt in range(max_retries):
|
||||
try:
|
||||
sync_client = _get_httpx_client()
|
||||
response = sync_client.get(GITHUB_API_KEY_URL, headers=headers)
|
||||
response.raise_for_status()
|
||||
|
||||
response_json = response.json()
|
||||
|
||||
if "token" in response_json:
|
||||
return response_json
|
||||
else:
|
||||
verbose_logger.warning(
|
||||
f"API key response missing token: {response_json}"
|
||||
)
|
||||
except httpx.HTTPStatusError as e:
|
||||
verbose_logger.error(
|
||||
f"HTTP error refreshing API key (attempt {attempt+1}/{max_retries}): {str(e)}"
|
||||
)
|
||||
except Exception as e:
|
||||
verbose_logger.error(f"Unexpected error refreshing API key: {str(e)}")
|
||||
|
||||
raise RefreshAPIKeyError(
|
||||
message="Failed to refresh API key after maximum retries",
|
||||
status_code=401,
|
||||
)
|
||||
|
||||
def _ensure_token_dir(self) -> None:
|
||||
"""Ensure the token directory exists."""
|
||||
if not os.path.exists(self.token_dir):
|
||||
os.makedirs(self.token_dir, exist_ok=True)
|
||||
|
||||
def _get_github_headers(self, access_token: Optional[str] = None) -> Dict[str, str]:
|
||||
"""
|
||||
Generate standard GitHub headers for API requests.
|
||||
|
||||
Args:
|
||||
access_token: Optional access token to include in the headers.
|
||||
|
||||
Returns:
|
||||
Dict[str, str]: Headers for GitHub API requests.
|
||||
"""
|
||||
headers = {
|
||||
"accept": "application/json",
|
||||
"editor-version": "vscode/1.85.1",
|
||||
"editor-plugin-version": "copilot/1.155.0",
|
||||
"user-agent": "GithubCopilot/1.155.0",
|
||||
"accept-encoding": "gzip,deflate,br",
|
||||
}
|
||||
|
||||
if access_token:
|
||||
headers["authorization"] = f"token {access_token}"
|
||||
|
||||
if "content-type" not in headers:
|
||||
headers["content-type"] = "application/json"
|
||||
|
||||
return headers
|
||||
|
||||
def _get_device_code(self) -> Dict[str, str]:
|
||||
"""
|
||||
Get a device code for GitHub authentication.
|
||||
|
||||
Returns:
|
||||
Dict[str, str]: Device code information.
|
||||
|
||||
Raises:
|
||||
GetDeviceCodeError: If unable to get a device code.
|
||||
"""
|
||||
try:
|
||||
sync_client = _get_httpx_client()
|
||||
resp = sync_client.post(
|
||||
GITHUB_DEVICE_CODE_URL,
|
||||
headers=self._get_github_headers(),
|
||||
json={"client_id": GITHUB_CLIENT_ID, "scope": "read:user"},
|
||||
)
|
||||
resp.raise_for_status()
|
||||
resp_json = resp.json()
|
||||
|
||||
required_fields = ["device_code", "user_code", "verification_uri"]
|
||||
if not all(field in resp_json for field in required_fields):
|
||||
verbose_logger.error(f"Response missing required fields: {resp_json}")
|
||||
raise GetDeviceCodeError(
|
||||
message="Response missing required fields",
|
||||
status_code=400,
|
||||
)
|
||||
|
||||
return resp_json
|
||||
except httpx.HTTPStatusError as e:
|
||||
verbose_logger.error(f"HTTP error getting device code: {str(e)}")
|
||||
raise GetDeviceCodeError(
|
||||
message=f"Failed to get device code: {str(e)}",
|
||||
status_code=400,
|
||||
)
|
||||
except json.JSONDecodeError as e:
|
||||
verbose_logger.error(f"Error decoding JSON response: {str(e)}")
|
||||
raise GetDeviceCodeError(
|
||||
message=f"Failed to decode device code response: {str(e)}",
|
||||
status_code=400,
|
||||
)
|
||||
except Exception as e:
|
||||
verbose_logger.error(f"Unexpected error getting device code: {str(e)}")
|
||||
raise GetDeviceCodeError(
|
||||
message=f"Failed to get device code: {str(e)}",
|
||||
status_code=400,
|
||||
)
|
||||
|
||||
def _poll_for_access_token(self, device_code: str) -> str:
|
||||
"""
|
||||
Poll for an access token after user authentication.
|
||||
|
||||
Args:
|
||||
device_code: The device code to use for polling.
|
||||
|
||||
Returns:
|
||||
str: The access token.
|
||||
|
||||
Raises:
|
||||
GetAccessTokenError: If unable to get an access token.
|
||||
"""
|
||||
sync_client = _get_httpx_client()
|
||||
max_attempts = 12 # 1 minute (12 * 5 seconds)
|
||||
|
||||
for attempt in range(max_attempts):
|
||||
try:
|
||||
resp = sync_client.post(
|
||||
GITHUB_ACCESS_TOKEN_URL,
|
||||
headers=self._get_github_headers(),
|
||||
json={
|
||||
"client_id": GITHUB_CLIENT_ID,
|
||||
"device_code": device_code,
|
||||
"grant_type": "urn:ietf:params:oauth:grant-type:device_code",
|
||||
},
|
||||
)
|
||||
resp.raise_for_status()
|
||||
resp_json = resp.json()
|
||||
|
||||
if "access_token" in resp_json:
|
||||
verbose_logger.info("Authentication successful!")
|
||||
return resp_json["access_token"]
|
||||
elif (
|
||||
"error" in resp_json
|
||||
and resp_json.get("error") == "authorization_pending"
|
||||
):
|
||||
verbose_logger.debug(
|
||||
f"Authorization pending (attempt {attempt+1}/{max_attempts})"
|
||||
)
|
||||
else:
|
||||
verbose_logger.warning(f"Unexpected response: {resp_json}")
|
||||
except httpx.HTTPStatusError as e:
|
||||
verbose_logger.error(f"HTTP error polling for access token: {str(e)}")
|
||||
raise GetAccessTokenError(
|
||||
message=f"Failed to get access token: {str(e)}",
|
||||
status_code=400,
|
||||
)
|
||||
except json.JSONDecodeError as e:
|
||||
verbose_logger.error(f"Error decoding JSON response: {str(e)}")
|
||||
raise GetAccessTokenError(
|
||||
message=f"Failed to decode access token response: {str(e)}",
|
||||
status_code=400,
|
||||
)
|
||||
except Exception as e:
|
||||
verbose_logger.error(
|
||||
f"Unexpected error polling for access token: {str(e)}"
|
||||
)
|
||||
raise GetAccessTokenError(
|
||||
message=f"Failed to get access token: {str(e)}",
|
||||
status_code=400,
|
||||
)
|
||||
|
||||
time.sleep(5)
|
||||
|
||||
raise GetAccessTokenError(
|
||||
message="Timed out waiting for user to authorize the device",
|
||||
status_code=400,
|
||||
)
|
||||
|
||||
def _login(self) -> str:
|
||||
"""
|
||||
Login to GitHub Copilot using device code flow.
|
||||
|
||||
Returns:
|
||||
str: The GitHub access token.
|
||||
|
||||
Raises:
|
||||
GetDeviceCodeError: If unable to get a device code.
|
||||
GetAccessTokenError: If unable to get an access token.
|
||||
"""
|
||||
device_code_info = self._get_device_code()
|
||||
|
||||
device_code = device_code_info["device_code"]
|
||||
user_code = device_code_info["user_code"]
|
||||
verification_uri = device_code_info["verification_uri"]
|
||||
|
||||
print( # noqa: T201
|
||||
f"Please visit {verification_uri} and enter code {user_code} to authenticate."
|
||||
)
|
||||
|
||||
return self._poll_for_access_token(device_code)
|
||||
37
litellm/llms/github_copilot/chat/transformation.py
Normal file
37
litellm/llms/github_copilot/chat/transformation.py
Normal file
|
|
@ -0,0 +1,37 @@
|
|||
from typing import Optional, Tuple
|
||||
|
||||
from litellm.exceptions import AuthenticationError
|
||||
from litellm.llms.openai.openai import OpenAIConfig
|
||||
|
||||
from ..authenticator import Authenticator
|
||||
from ..common_utils import GetAPIKeyError
|
||||
|
||||
|
||||
class GithubCopilotConfig(OpenAIConfig):
|
||||
GITHUB_COPILOT_API_BASE = "https://api.github.com/copilot/v1"
|
||||
def __init__(
|
||||
self,
|
||||
api_key: Optional[str] = None,
|
||||
api_base: Optional[str] = None,
|
||||
custom_llm_provider: str = "openai",
|
||||
) -> None:
|
||||
super().__init__()
|
||||
self.authenticator = Authenticator()
|
||||
|
||||
def _get_openai_compatible_provider_info(
|
||||
self,
|
||||
model: str,
|
||||
api_base: Optional[str],
|
||||
api_key: Optional[str],
|
||||
custom_llm_provider: str,
|
||||
) -> Tuple[Optional[str], Optional[str], str]:
|
||||
api_base = self.GITHUB_COPILOT_API_BASE
|
||||
try:
|
||||
dynamic_api_key = self.authenticator.get_api_key()
|
||||
except GetAPIKeyError as e:
|
||||
raise AuthenticationError(
|
||||
model=model,
|
||||
llm_provider=custom_llm_provider,
|
||||
message=str(e),
|
||||
)
|
||||
return api_base, dynamic_api_key, custom_llm_provider
|
||||
49
litellm/llms/github_copilot/common_utils.py
Normal file
49
litellm/llms/github_copilot/common_utils.py
Normal file
|
|
@ -0,0 +1,49 @@
|
|||
"""
|
||||
Constants for Copilot integration
|
||||
"""
|
||||
from typing import Optional, Union
|
||||
|
||||
import httpx
|
||||
|
||||
from litellm.llms.base_llm.chat.transformation import BaseLLMException
|
||||
|
||||
|
||||
class GithubCopilotError(BaseLLMException):
|
||||
def __init__(
|
||||
self,
|
||||
status_code,
|
||||
message,
|
||||
request: Optional[httpx.Request] = None,
|
||||
response: Optional[httpx.Response] = None,
|
||||
headers: Optional[Union[httpx.Headers, dict]] = None,
|
||||
body: Optional[dict] = None,
|
||||
):
|
||||
super().__init__(
|
||||
status_code=status_code,
|
||||
message=message,
|
||||
request=request,
|
||||
response=response,
|
||||
headers=headers,
|
||||
body=body,
|
||||
)
|
||||
|
||||
|
||||
|
||||
class GetDeviceCodeError(GithubCopilotError):
|
||||
pass
|
||||
|
||||
|
||||
class GetAccessTokenError(GithubCopilotError):
|
||||
pass
|
||||
|
||||
|
||||
class APIKeyExpiredError(GithubCopilotError):
|
||||
pass
|
||||
|
||||
|
||||
class RefreshAPIKeyError(GithubCopilotError):
|
||||
pass
|
||||
|
||||
|
||||
class GetAPIKeyError(GithubCopilotError):
|
||||
pass
|
||||
|
|
@ -2310,6 +2310,7 @@ class LlmProviders(str, Enum):
|
|||
HUMANLOOP = "humanloop"
|
||||
TOPAZ = "topaz"
|
||||
ASSEMBLYAI = "assemblyai"
|
||||
GITHUB_COPILOT = "github_copilot"
|
||||
SNOWFLAKE = "snowflake"
|
||||
LLAMA = "meta_llama"
|
||||
NSCALE = "nscale"
|
||||
|
|
|
|||
|
|
@ -0,0 +1 @@
|
|||
# This file makes the tests directory a Python package
|
||||
|
|
@ -0,0 +1 @@
|
|||
# This file makes the tests/litellm directory a Python package
|
||||
|
|
@ -0,0 +1 @@
|
|||
# This file makes the tests/litellm/litellm_core_utils directory a Python package
|
||||
|
|
@ -0,0 +1,180 @@
|
|||
import json
|
||||
import os
|
||||
import time
|
||||
from datetime import datetime, timedelta
|
||||
from unittest.mock import MagicMock, mock_open, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from litellm.llms.github_copilot.authenticator import Authenticator
|
||||
from litellm.llms.github_copilot.common_utils import (
|
||||
APIKeyExpiredError,
|
||||
GetAccessTokenError,
|
||||
GetAPIKeyError,
|
||||
GetDeviceCodeError,
|
||||
RefreshAPIKeyError,
|
||||
)
|
||||
|
||||
|
||||
class TestGitHubCopilotAuthenticator:
|
||||
@pytest.fixture
|
||||
def authenticator(self):
|
||||
with patch("os.path.exists", return_value=False), patch("os.makedirs") as mock_makedirs:
|
||||
auth = Authenticator()
|
||||
mock_makedirs.assert_called_once()
|
||||
return auth
|
||||
|
||||
@pytest.fixture
|
||||
def mock_http_client(self):
|
||||
mock_client = MagicMock()
|
||||
mock_response = MagicMock()
|
||||
mock_client.get.return_value = mock_response
|
||||
mock_client.post.return_value = mock_response
|
||||
mock_response.raise_for_status.return_value = None
|
||||
return mock_client, mock_response
|
||||
|
||||
def test_init(self):
|
||||
"""Test the initialization of the authenticator."""
|
||||
with patch("os.path.exists", return_value=False), patch("os.makedirs") as mock_makedirs:
|
||||
auth = Authenticator()
|
||||
assert auth.token_dir.endswith("/github_copilot")
|
||||
assert auth.access_token_file.endswith("/access-token")
|
||||
assert auth.api_key_file.endswith("/api-key.json")
|
||||
mock_makedirs.assert_called_once()
|
||||
|
||||
def test_ensure_token_dir(self):
|
||||
"""Test that the token directory is created if it doesn't exist."""
|
||||
with patch("os.path.exists", return_value=False), patch("os.makedirs") as mock_makedirs:
|
||||
auth = Authenticator()
|
||||
mock_makedirs.assert_called_once_with(auth.token_dir, exist_ok=True)
|
||||
|
||||
def test_get_github_headers(self, authenticator):
|
||||
"""Test that GitHub headers are correctly generated."""
|
||||
headers = authenticator._get_github_headers()
|
||||
assert "accept" in headers
|
||||
assert "editor-version" in headers
|
||||
assert "user-agent" in headers
|
||||
assert "content-type" in headers
|
||||
|
||||
headers_with_token = authenticator._get_github_headers("test-token")
|
||||
assert headers_with_token["authorization"] == "token test-token"
|
||||
|
||||
def test_get_access_token_from_file(self, authenticator):
|
||||
"""Test retrieving an access token from a file."""
|
||||
mock_token = "mock-access-token"
|
||||
|
||||
with patch("builtins.open", mock_open(read_data=mock_token)):
|
||||
token = authenticator.get_access_token()
|
||||
assert token == mock_token
|
||||
|
||||
def test_get_access_token_login(self, authenticator):
|
||||
"""Test logging in to get an access token."""
|
||||
mock_token = "mock-access-token"
|
||||
|
||||
with patch.object(authenticator, "_login", return_value=mock_token), \
|
||||
patch("builtins.open", mock_open()), \
|
||||
patch("builtins.open", side_effect=IOError) as mock_read:
|
||||
token = authenticator.get_access_token()
|
||||
assert token == mock_token
|
||||
authenticator._login.assert_called_once()
|
||||
|
||||
def test_get_access_token_failure(self, authenticator):
|
||||
"""Test that an exception is raised after multiple login failures."""
|
||||
with patch.object(authenticator, "_login", side_effect=GetDeviceCodeError(message="Test error", status_code=400)), \
|
||||
patch("builtins.open", side_effect=IOError):
|
||||
with pytest.raises(GetAccessTokenError):
|
||||
authenticator.get_access_token()
|
||||
assert authenticator._login.call_count == 3
|
||||
|
||||
def test_get_api_key_from_file(self, authenticator):
|
||||
"""Test retrieving an API key from a file."""
|
||||
future_time = (datetime.now() + timedelta(hours=1)).timestamp()
|
||||
mock_api_key_data = json.dumps({"token": "mock-api-key", "expires_at": future_time})
|
||||
|
||||
with patch("builtins.open", mock_open(read_data=mock_api_key_data)):
|
||||
api_key = authenticator.get_api_key()
|
||||
assert api_key == "mock-api-key"
|
||||
|
||||
def test_get_api_key_expired(self, authenticator):
|
||||
"""Test refreshing an expired API key."""
|
||||
past_time = (datetime.now() - timedelta(hours=1)).timestamp()
|
||||
mock_expired_data = json.dumps({"token": "expired-api-key", "expires_at": past_time})
|
||||
mock_new_data = {"token": "new-api-key", "expires_at": (datetime.now() + timedelta(hours=1)).timestamp()}
|
||||
|
||||
with patch("builtins.open", mock_open(read_data=mock_expired_data)), \
|
||||
patch.object(authenticator, "_refresh_api_key", return_value=mock_new_data), \
|
||||
patch("json.dump") as mock_json_dump:
|
||||
api_key = authenticator.get_api_key()
|
||||
assert api_key == "new-api-key"
|
||||
authenticator._refresh_api_key.assert_called_once()
|
||||
|
||||
def test_refresh_api_key(self, authenticator, mock_http_client):
|
||||
"""Test refreshing an API key."""
|
||||
mock_client, mock_response = mock_http_client
|
||||
mock_token = "mock-access-token"
|
||||
mock_api_key_data = {"token": "new-api-key", "expires_at": 12345}
|
||||
|
||||
with patch.object(authenticator, "get_access_token", return_value=mock_token), \
|
||||
patch("litellm.llms.github_copilot.authenticator._get_httpx_client", return_value=mock_client), \
|
||||
patch.object(mock_response, "json", return_value=mock_api_key_data):
|
||||
result = authenticator._refresh_api_key()
|
||||
assert result == mock_api_key_data
|
||||
mock_client.get.assert_called_once()
|
||||
authenticator.get_access_token.assert_called_once()
|
||||
|
||||
def test_refresh_api_key_failure(self, authenticator, mock_http_client):
|
||||
"""Test failure to refresh an API key."""
|
||||
mock_client, mock_response = mock_http_client
|
||||
mock_token = "mock-access-token"
|
||||
|
||||
with patch.object(authenticator, "get_access_token", return_value=mock_token), \
|
||||
patch("litellm.llms.github_copilot.authenticator._get_httpx_client", return_value=mock_client), \
|
||||
patch.object(mock_response, "json", return_value={}):
|
||||
with pytest.raises(RefreshAPIKeyError):
|
||||
authenticator._refresh_api_key()
|
||||
assert mock_client.get.call_count == 3
|
||||
|
||||
def test_get_device_code(self, authenticator, mock_http_client):
|
||||
"""Test getting a device code."""
|
||||
mock_client, mock_response = mock_http_client
|
||||
mock_device_code_data = {
|
||||
"device_code": "mock-device-code",
|
||||
"user_code": "ABCD-EFGH",
|
||||
"verification_uri": "https://github.com/login/device"
|
||||
}
|
||||
|
||||
with patch("litellm.llms.github_copilot.authenticator._get_httpx_client", return_value=mock_client), \
|
||||
patch.object(mock_response, "json", return_value=mock_device_code_data):
|
||||
result = authenticator._get_device_code()
|
||||
assert result == mock_device_code_data
|
||||
mock_client.post.assert_called_once()
|
||||
|
||||
def test_poll_for_access_token(self, authenticator, mock_http_client):
|
||||
"""Test polling for an access token."""
|
||||
mock_client, mock_response = mock_http_client
|
||||
mock_token_data = {"access_token": "mock-access-token"}
|
||||
|
||||
with patch("litellm.llms.github_copilot.authenticator._get_httpx_client", return_value=mock_client), \
|
||||
patch.object(mock_response, "json", return_value=mock_token_data), \
|
||||
patch("time.sleep"):
|
||||
result = authenticator._poll_for_access_token("mock-device-code")
|
||||
assert result == "mock-access-token"
|
||||
mock_client.post.assert_called_once()
|
||||
|
||||
def test_login(self, authenticator):
|
||||
"""Test the login process."""
|
||||
mock_device_code_data = {
|
||||
"device_code": "mock-device-code",
|
||||
"user_code": "ABCD-EFGH",
|
||||
"verification_uri": "https://github.com/login/device"
|
||||
}
|
||||
mock_token = "mock-access-token"
|
||||
|
||||
with patch.object(authenticator, "_get_device_code", return_value=mock_device_code_data), \
|
||||
patch.object(authenticator, "_poll_for_access_token", return_value=mock_token), \
|
||||
patch("builtins.print") as mock_print:
|
||||
result = authenticator._login()
|
||||
assert result == mock_token
|
||||
authenticator._get_device_code.assert_called_once()
|
||||
authenticator._poll_for_access_token.assert_called_once_with("mock-device-code")
|
||||
mock_print.assert_called_once()
|
||||
|
|
@ -0,0 +1,128 @@
|
|||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from datetime import datetime, timedelta
|
||||
from typing import AsyncGenerator
|
||||
from unittest.mock import AsyncMock, MagicMock, mock_open, patch
|
||||
|
||||
import pytest
|
||||
|
||||
sys.path.insert(0, os.path.abspath("../.."))
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
from respx import MockRouter
|
||||
|
||||
import litellm
|
||||
|
||||
# Import at the top to make the patch work correctly
|
||||
import litellm.llms.github_copilot.chat.transformation
|
||||
from litellm import Choices, Message, ModelResponse, Usage, acompletion, completion
|
||||
from litellm.exceptions import AuthenticationError
|
||||
from litellm.llms.github_copilot.authenticator import Authenticator
|
||||
from litellm.llms.github_copilot.chat.transformation import GithubCopilotConfig
|
||||
from litellm.llms.github_copilot.common_utils import (
|
||||
APIKeyExpiredError,
|
||||
GetAccessTokenError,
|
||||
GetAPIKeyError,
|
||||
GetDeviceCodeError,
|
||||
RefreshAPIKeyError,
|
||||
)
|
||||
|
||||
|
||||
def test_github_copilot_config_get_openai_compatible_provider_info():
|
||||
"""Test the GitHub Copilot configuration provider info retrieval."""
|
||||
|
||||
config = GithubCopilotConfig()
|
||||
|
||||
# Mock the authenticator to avoid actual API calls
|
||||
mock_api_key = "gh.test-key-123456789"
|
||||
config.authenticator = MagicMock()
|
||||
config.authenticator.get_api_key.return_value = mock_api_key
|
||||
|
||||
# Test with default values
|
||||
model = "github_copilot/gpt-4"
|
||||
(
|
||||
api_base,
|
||||
dynamic_api_key,
|
||||
custom_llm_provider,
|
||||
) = config._get_openai_compatible_provider_info(
|
||||
model=model,
|
||||
api_base=None,
|
||||
api_key=None,
|
||||
custom_llm_provider="github_copilot",
|
||||
)
|
||||
|
||||
assert api_base == "https://api.github.com/copilot/v1"
|
||||
assert dynamic_api_key == mock_api_key
|
||||
assert custom_llm_provider == "github_copilot"
|
||||
|
||||
# Test with authentication failure
|
||||
config.authenticator.get_api_key.side_effect = GetAPIKeyError(
|
||||
message="Failed to get API key",
|
||||
status_code=401,
|
||||
)
|
||||
|
||||
with pytest.raises(AuthenticationError) as excinfo:
|
||||
config._get_openai_compatible_provider_info(
|
||||
model=model,
|
||||
api_base=None,
|
||||
api_key=None,
|
||||
custom_llm_provider="github_copilot",
|
||||
)
|
||||
|
||||
assert "Failed to get API key" in str(excinfo.value)
|
||||
|
||||
|
||||
@patch("litellm.llms.github_copilot.authenticator.Authenticator.get_api_key")
|
||||
@patch("litellm.llms.openai.openai.OpenAIChatCompletion.completion")
|
||||
def test_completion_github_copilot_mock_response(mock_completion, mock_get_api_key):
|
||||
"""Test the completion function with GitHub Copilot provider."""
|
||||
|
||||
# Mock the API key return value
|
||||
mock_api_key = "gh.test-key-123456789"
|
||||
mock_get_api_key.return_value = mock_api_key
|
||||
|
||||
# Mock completion response
|
||||
mock_response = MagicMock()
|
||||
mock_response.choices = [MagicMock()]
|
||||
mock_response.choices[0].message.content = "Hello, I'm GitHub Copilot!"
|
||||
mock_completion.return_value = mock_response
|
||||
|
||||
# Test non-streaming completion
|
||||
messages = [
|
||||
{"role": "system", "content": "You're GitHub Copilot, an AI assistant."},
|
||||
{"role": "user", "content": "Hello, who are you?"},
|
||||
]
|
||||
|
||||
# Create a properly formatted headers dictionary
|
||||
headers = {
|
||||
"editor-version": "Neovim/0.9.0",
|
||||
"Copilot-Integration-Id": "vscode-chat",
|
||||
}
|
||||
|
||||
response = completion(
|
||||
model="github_copilot/gpt-4",
|
||||
messages=messages,
|
||||
extra_headers=headers,
|
||||
)
|
||||
|
||||
assert response is not None
|
||||
|
||||
# Verify the get_api_key call was made (can be called multiple times)
|
||||
assert mock_get_api_key.call_count >= 1
|
||||
|
||||
# Verify the completion call was made with the expected params
|
||||
mock_completion.assert_called_once()
|
||||
args, kwargs = mock_completion.call_args
|
||||
|
||||
# Check that the proper authorization header is set
|
||||
assert "headers" in kwargs
|
||||
# Check that the model name is correctly formatted
|
||||
assert (
|
||||
kwargs.get("model") == "gpt-4"
|
||||
) # Model name should be without provider prefix
|
||||
assert kwargs.get("messages") == messages
|
||||
|
||||
|
||||
Loading…
Add table
Reference in a new issue