``get_llm_provider_logic.py`` instantiates ``ChatGPTConfig`` /
``GithubCopilotConfig`` and calls ``_get_openai_compatible_provider_info``
at every ``add_deployment`` cycle during proxy startup (30s tick when
``STORE_MODEL_IN_DB=True``). The old code eagerly called
``get_access_token()`` / ``get_api_key()`` through the filesystem
authenticator at that point. With no tokens on disk this falls through
to ``_login_device_code()``, which prints a device prompt to stdout and
polls the IdP for up to 15 minutes — blocking startup and (for OAuth
credentials stored in the DB) duplicating work that ``validate_environment``
is about to do correctly at request time.
Resolution is now a pure metadata pass: pick an ``api_base``, let the
``oauth:<name>`` marker pass through untouched. Actual token resolution
still happens at request time via ``resolve_authenticator`` inside
``validate_environment``, which is where we have the full
``litellm_params`` anyway.
Tests updated to assert ``get_access_token`` / ``get_api_key`` are NOT
called during resolution, and that the ``oauth:`` marker passes through.
Reported symptom from a working OAuth request to
``chatgpt.com/backend-api/codex/responses``:
APIConnectionError: ChatgptException - Unknown items in responses
API response: []
The ChatGPT / Codex backend ships the response body as SSE. Each
output item arrives in its own ``response.output_item.done`` event,
and the terminal ``response.completed`` frame carries an empty
``response.output`` array — it is effectively a "we're done" signal
rather than the carrier for the items.
The existing parser only read ``response.output`` off the completed
frame, so it handed the downstream chat-completions translator a
``ResponsesAPIResponse`` with ``output=[]``, which blew up on
``_convert_response_output_to_choices``.
Fix: accumulate items from ``response.output_item.done`` during the
loop and, when ``response.completed`` arrives with an empty output,
substitute the accumulated list. The canonical OpenAI shape (items
already populated on ``response.completed``) still works — we only
fill in when the completed frame itself is empty.
Extracted the SSE loop into ``_parse_codex_sse_response`` to keep
``transform_response_api_response`` under the PLR0915 threshold.
Added a regression test that reproduces the Codex wire shape: one
``output_item.done`` event followed by a ``response.completed`` with
``output=[]``. 174 tests pass; Black + Ruff clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reported symptom: Test Model fails immediately with
GetLLMProvider Exception - Failed to request device code:
Client error '403 Forbidden' for url
'https://auth.openai.com/api/accounts/deviceauth/usercode'
No 15-minute hang this time — it comes back instantly.
Root cause: the ChatGPT ``DBAuthenticator`` inherited
``get_access_token`` from the filesystem ``Authenticator``, whose last
line is:
tokens = self._login_device_code()
That's the right fall-through for the CLI (no tokens on disk → start a
new login). In the proxy's DB-backed context it is catastrophic: the
server tries to initiate an unattended device-code request, OpenAI
returns 403 on the ``usercode`` endpoint (no browser to walk through
consent), and the admin sees the cryptic error above.
Copilot's ``DBAuthenticator`` already overrides ``get_access_token``
and raises ``GetAccessTokenError`` cleanly — this commit mirrors that
in the ChatGPT side:
- If no credential is loaded in ``litellm.credential_list``, raise a
401 pointing the admin at STORE_MODEL_IN_DB and the UI sign-in.
- If the token is expired and refresh fails, raise a 401 quoting the
IdP's refresh error.
- Never call ``_login_device_code`` from the proxy path.
Added regression tests that mock ``_login_device_code`` and assert it
is never called across the three failure modes above (missing
credential, happy path, refresh-error). 173 tests pass; Black + Ruff
clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Audit of every ``self.authenticator.*`` call-site across both providers
turned up two more paths that would skip the ``oauth:`` dispatch:
- ``github_copilot/embedding/transformation.py`` — the embedding
``validate_environment`` and ``get_complete_url`` both call the
filesystem authenticator directly. Same failure mode as the chat
transformation before the previous fix: a request with
``api_key: oauth:<name>`` would hit the filesystem authenticator, find
no tokens on disk, and block the server thread on the 15-minute
device-code poll. Now goes through ``resolve_authenticator`` too.
- ``chatgpt/responses/transformation.py::get_complete_url`` — called
separately from ``validate_environment``. Lower severity (the env-
based ``get_api_base`` just returns the constant fallback when no
auth file is present, so it works in practice) but still a direct
call that ought to share the same dispatch.
After this commit, ``grep -rn 'self\.authenticator\.' litellm/llms/
{chatgpt,github_copilot}/`` returns zero matches — every auth access
goes through ``resolve_authenticator``.
170 tests pass (same count; the chat regression test already covers
this family of bug — no new test was worth adding here).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reported symptom: "Test model" in the UI hangs indefinitely, a new
device code appears in the proxy logs, and the whole UI is frozen.
Root cause: the ChatGPT **chat** transformation still used the
filesystem-only ``Authenticator`` unconditionally. When the UI's
``/health/test_connection`` runs in the default ``mode="chat"``:
UI → /health/test_connection (sync wait)
→ litellm.ahealth_check(mode="chat")
→ litellm.acompletion(...)
→ ChatGPTConfig._get_openai_compatible_provider_info(api_key="oauth:X")
→ self.authenticator.get_access_token() # filesystem authenticator
→ _login_device_code() # no tokens on disk → prints a new
# device code and polls for up to 15
# minutes, blocking the request
``ChatGPTResponsesAPIConfig`` already had the ``resolve_authenticator``
dispatch (the PR's original scope), so the responses-mode path was
fine. The chat path was overlooked.
Fix:
- Apply ``resolve_authenticator`` to both call-sites in
``ChatGPTConfig`` (``_get_openai_compatible_provider_info`` and
``validate_environment``). When ``api_key`` carries the ``oauth:``
prefix, both methods now reach for the DB-backed authenticator and
the stored credential — no device-code prompt, no 15-minute hang.
- Extend ``resolve_authenticator`` to the 3-arg shape
``(api_key, litellm_params, fallback)`` so it matches the Copilot
resolver and covers the chat call-site that has ``api_key`` but not
``litellm_params``. Updated the single existing call-site in
``responses/transformation.py`` and the existing unit tests.
- Add a regression test asserting that ``ChatGPTConfig`` never calls
the filesystem authenticator when ``api_key`` is ``oauth:<name>``
(fs_auth is a MagicMock that raises on any attribute access).
170 tests pass; Black + Ruff clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Without STORE_MODEL_IN_DB=True the proxy still writes OAuth credentials
to LiteLLM_CredentialsTable, but ``proxy_config.get_credentials`` — the
DB → ``litellm.credential_list`` reload run at startup — lives inside
``if store_model_in_db is True:`` in proxy_server.py. Result: silent
data loss on restart and request-time ``api_key: oauth:<name>`` failures
because the name is no longer in the in-memory cache.
Gate the /start endpoints (both ChatGPT and Copilot) behind
``get_secret_bool("STORE_MODEL_IN_DB", False)``. Admins get a clear 400
up front instead of sitting through the 15-minute device-code poll only
to discover the flow silently doesn't persist.
Tests: setenv STORE_MODEL_IN_DB=True in the autouse fixture so the
success-path tests don't each have to opt in, plus one new test per
provider asserting 400 + message when the env var is absent. 167 cases
pass locally.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The DB → litellm.credential_list reload that makes OAuth credentials
survive a proxy restart lives inside the same ``if store_model_in_db
is True:`` block in proxy_server.py as the model-deployment scheduler.
Without the env var the write succeeds but nothing reads it back on
startup — credentials appear to vanish, and any model with
``api_key: oauth:<name>`` fails at request time.
This was caught during deployment testing. Not a new requirement
introduced by this PR (the same block gates the standard Add Model UI
flow), but easy to miss if you're only thinking about OAuth. Added a
prominent note to the CLAUDE.md OAuth section.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previously the ChatGPT (OAuth) provider in the Add Model form rendered
a plain text input for ``api_key`` and expected the admin to type the
``oauth:<credential_name>`` marker by hand — easy to typo, and
discoverability depended on reading the tooltip.
Add a new ``field_type: "oauth_credential_select"`` that
``provider_specific_fields.tsx`` renders as an antd Select populated
from ``useCredentials()``, filtered to OAuth-backed credentials
(``credential_info.type`` in ``{chatgpt_oauth, copilot_oauth}``).
Picking a row stores ``oauth:<credential_name>`` as the form value, so
the downstream request path is unchanged.
Wire-up:
- ``ProviderCredentialField`` Literal (Python) and
``ProviderCredentialFieldMetadata`` union (TypeScript) both extended
with the new type.
- ``provider_create_fields.json`` for ``ChatGPT`` updated to use it.
- Empty-state: if no OAuth credentials exist yet, the dropdown shows a
"No OAuth credentials found" hint pointing at Credentials → Add
Credential.
No change to Copilot's entry — Copilot's existing ``api_key``/``api_base``
text fields stay, since that provider still supports plain PAT auth
alongside OAuth. Admins who want the OAuth flow for Copilot can still
type ``oauth:<name>`` into the ``api_key`` field manually; the dropdown
will cover that too in a follow-up if you want symmetry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Add Model form reads its provider list from
``/public/providers/fields`` (served from
``public_endpoints/provider_create_fields.json``), not from the
frontend ``Providers`` enum directly. Add a ChatGPT entry there with a
single required ``api_key`` field pre-filled with ``oauth:`` so the
admin's workflow is:
1. Credentials → Add Credential → ChatGPT (OAuth) → sign in → credential
named ``my-chatgpt``
2. Add Model → provider: ChatGPT (OAuth) → model name → api_key field
shows ``oauth:`` placeholder + tooltip explaining the convention →
admin types ``oauth:my-chatgpt`` → save
At request time the ChatGPT responses transformation recognises the
``oauth:`` prefix and routes through ``DBAuthenticator`` for the stored
credential. Copilot was already in the JSON so its Add Model flow
already works.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reported error after the cross-loop fix:
Tokens obtained but DB persist failed: Unable to match input value to
any allowed input type for the field. Parse errors: [...
``credential_values`` should be of any of the following types:
``JsonNullValueInput``, ``Json`` ...]
Prisma's Json columns ingest pre-serialized JSON *strings*, not raw
Python dicts. The ``/credentials`` endpoint wraps the payload with
``jsonify_object(...)`` (in ``litellm/proxy/utils.py``) which does
``json.dumps`` per nested-dict field. My persist helper was skipping
that step and passing raw dicts — Prisma's binding layer then couldn't
match them to the ``Json`` type.
Route the ``credential_values`` + ``credential_info`` dicts through
``jsonify_object`` before the upsert. Same fix in both ChatGPT and
Copilot ``db_authenticator.py``. Tests updated to assert the serialized
string form (``json.loads(kwargs[...]["credential_values"])``).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reported error during device-code login:
Tokens obtained but DB persist failed: <asyncio.locks.Event object at
0x... [unset]> is bound to a different event loop
Root cause: the background worker was a ``threading.Thread`` that ran
``asyncio.run(persist_credential_to_db(item))``. That creates a fresh
event loop in the worker thread, but ``prisma_client``'s internal
asyncio primitives (locks, futures) are bound to the proxy's main loop.
Awaiting a prisma call from the worker loop hits the cross-loop error.
Fix:
- Refactor the background flow from a thread to an asyncio task
(``asyncio.create_task``) scheduled on the proxy's main loop. Blocking
IO in the flow (device-code poll, token exchange) runs in
``loop.run_in_executor``. The DB persist step (``await
persist_credential_to_db(item)``) now naturally shares a loop with
prisma_client. Same treatment for both ChatGPT and Copilot endpoints.
- For the DBAuthenticator refresh path (called from sync
``validate_environment`` during a request, same cross-loop risk), add
``_register_proxy_main_loop`` + ``_schedule_db_persist`` now prefers
``asyncio.run_coroutine_threadsafe`` onto the registered loop.
``/chatgpt/oauth/start`` and ``/copilot/oauth/start`` register the loop
on invocation (guaranteed to be the proxy's main loop). Falls back to
the old thread + ``asyncio.run`` path only when no loop is registered
(CLI / tests, where prisma_client is ``None`` anyway).
Tests updated: ``TestBackgroundWorker`` now drives the async task
directly via ``await _run_device_code_flow_async(...)``. The
``test_creates_session_and_spawns_worker`` test stubs the task to a
no-op instead of patching ``threading.Thread``.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The enum entry ``Providers.ChatGPT = "ChatGPT (OAuth)"`` was added but
the corresponding ``provider_map["ChatGPT"]`` and
``providerLogoMap[Providers.ChatGPT]`` entries were missing. The
dropdown option still renders, but:
- without ``provider_map[key]``, ``custom_llm_provider`` resolves to
``undefined`` on submit (harmless for the OAuth path — we never hit
that submit branch — but still inconsistent with every other
provider), and
- without ``providerLogoMap`` the Option icon falls through to the
"first-letter div" fallback, which reads as a visual glitch.
Add both. Reuses ``openai_small.svg`` since ChatGPT is an OpenAI product.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CONTRIBUTING.md requires corresponding tests for new UI components.
Adds:
- OAuthDeviceLoginButton.test.tsx (6 cases) — Sign-in button disabled
state, startCall invocation, user-code + verification-URL rendering,
onSuccess callback on poll success, error-state rendering on poll
failure, and cancel flow.
- ChatGPTLoginButton.test.tsx (1 case) — renders with the ChatGPT label.
- CopilotLoginButton.test.tsx (1 case) — renders with the GitHub Copilot
label.
Real timers throughout; polling tests wait up to 10s for the 3s-interval
first poll to fire. 20/20 local tests pass across my new + adjacent
existing UI test files; ``npm run build`` succeeds.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Greptile flagged the inline proxy imports as an acknowledged P2 style
nit. Documenting the two real reasons they're inline so the next reader
doesn't 'fix' them into a module-level import and break things.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Drop the redundant decorator-level ``dependencies=[Depends(user_api_key_auth)]``
on all 8 OAuth endpoints (4 each in chatgpt/copilot). The same dependency
is already injected as a parameter, which runs it and binds the result —
FastAPI's dependency cache makes the decorator-level duplicate a no-op at
runtime, just visual noise.
- Finish the @tremor/react → antd migration in ``credentials.tsx``: rename
the ``AntdButton`` alias to ``Button`` and remove the ``Button`` entry
from the @tremor/react import, so the top-level "Add Credential" button
also goes through antd. No rendered changes (antd Button's children +
onClick API matches what that call site already used).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add "Sign in with ChatGPT" and "Sign in with GitHub Copilot" device-code
OAuth flows to the Add Credential modal. Tokens persist as encrypted JSON
in LiteLLM_CredentialsTable and are picked up at request time when a
model's api_key is set to "oauth:<credential_name>"; a DBAuthenticator
subclass reads from the in-memory credential cache (sync) and writes
refreshed tokens back via a fire-and-forget worker thread.
Per-row Refresh button rotates tokens on demand — ChatGPT via the IdP's
refresh_token grant, Copilot by re-deriving the short-lived API key from
the stored GitHub access token.
Also ships a litellm-chatgpt-login CLI with an optional PKCE+loopback
flow alongside the existing device-code path for local sign-in outside
the UI.
Provider-specific surface lives in new files under
litellm/llms/{chatgpt,github_copilot}/db_authenticator.py and
litellm/proxy/{chatgpt,copilot}_oauth_endpoints/. Shared-file delta is
~275 lines across 10 pre-existing files, the bulk being a pure append
at the end of networking.tsx.
Dispatch convention + operational caveats documented in the new
"ChatGPT / Copilot OAuth Credentials" section of CLAUDE.md.
Security:
- /chatgpt/oauth/* and /copilot/oauth/* endpoints require PROXY_ADMIN
(view-only admins excluded from write paths)
- session_id query params are Query(...) annotated
- session-cap check + slot reservation are atomic; reserved slot is
cleaned up on device-code failure
- PKCE loopback callback html.escape()s the IdP's error_description
before rendering
Tests: 165 cases covering DBAuthenticator read/write, config dispatch,
endpoint admin-only, 429 cap, slot cleanup, background-worker success /
auth-failure / DB-persist-failure paths, PKCE helpers + XSS regression,
CLI broad-exception + KeyboardInterrupt.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The data-testid attributes added to React components are not present
in the CI-built UI output. Switch to using getByRole and getByText
selectors which work with the rendered DOM regardless of build cache.
Add E2E tests covering:
- Test connection with bad credentials shows failure modal
- Adding a specific model and verifying it appears in All Models table
- Adding a wildcard route and verifying it appears in All Models table
- Verifying model dropdown shows provider-specific models (existing test updated)
Added data-testid attributes to UI components to support stable test selectors.
Tests verified passing 3/3 consecutive runs with zero flakiness.
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 30, 8) (push) Waiting to run
Reviewer flagged that cleanup failures were silently swallowed and
suggested asserting `delete.ok()`. While thinking through the fix, the
actual question turned out to be "does the cleanup matter at all?" —
and the answer is no.
The e2e runner (`run_e2e.sh`) spins up a fresh postgres container per
invocation and tears it down at the end, so every local and CI run
starts with an empty DB. Playwright retries share the same DB but each
attempt creates a new model with a unique `Date.now()` name and only
queries its own model, so orphans from failed attempts never collide
with later attempts or other tests. Nothing else in the suite reads
the all-models table.
Keeping the cleanup would also turn every write test into an implicit
delete test, coupling responsibilities and inflating runtime — which
is probably why `teams.spec.ts` (create a team), `keys.spec.ts`
(update key limits), etc. all leave their entities in place. Matching
that convention, drop the try/finally block and the `createdModelId`
tracking. 12 lines removed, no behavior change.
Covers the full write-path flow for team-scoped models on the Models +
Endpoints page: create via /model/new, click the row to open the detail
view, click Edit Settings, change TPM/RPM, click Save Changes, assert
the new values render back. Cleans up via /model/delete in finally so
reruns stay deterministic.
Requires store_model_in_db: true in the fixture general_settings so the
proxy accepts /model/new and /model/delete. No existing test in the
dashboard e2e suite reads the all-models table or hits the model CRUD
endpoints, so enabling the flag has no cross-test impact.
The suite was superseded by ui/litellm-dashboard/e2e_tests/ on 2026-04-08
and is no longer referenced by CircleCI, docs, or Makefile targets. Drop
the directory wholesale and remove the orphaned e2e:psql npm script that
pointed at its runner.
Updates the expected header text to "Guardrails Settings" to match
GuardrailSettingsView's rendering, and moves the mock guardrails
from team_info.guardrails (legacy top-level path that nothing
reads) to team_info.metadata.guardrails where the component
actually looks. Also tightens the assertion to verify the
individual guardrail names appear, not just the section header.
Previously these were silently dropped with a verbose warning, which
could break observability integrations without surfacing a clear error.
Now raises ValueError with remediation steps (configure server-side
or pass the resolved value) so callers get immediate, actionable feedback.
Converts GuardrailSettingsView from @tremor/react (Badge, Text) to
antd (Tag, plain spans) as part of the Tremor migration. Also
captures the "no new Tremor imports" rule in CLAUDE.md and expands
the existing note in AGENTS.md with the specific antd equivalents
and the yellow→gold gotcha.
Pulls the Global / Team-specific subsection rendering out of
TeamInfo.tsx into a shared GuardrailSettingsView component with
card and inline variants, used on both the team Overview tab
(inside the existing Tremor Card) and the Team Settings tab read
view. The Global subsection header now carries a GlobalOutlined
icon, and since the icon is load-bearing the edit-form chip
coloring is simplified to a single blue instead of green/blue.
