Reported symptom: Test Model fails immediately with
GetLLMProvider Exception - Failed to request device code:
Client error '403 Forbidden' for url
'https://auth.openai.com/api/accounts/deviceauth/usercode'
No 15-minute hang this time — it comes back instantly.
Root cause: the ChatGPT ``DBAuthenticator`` inherited
``get_access_token`` from the filesystem ``Authenticator``, whose last
line is:
tokens = self._login_device_code()
That's the right fall-through for the CLI (no tokens on disk → start a
new login). In the proxy's DB-backed context it is catastrophic: the
server tries to initiate an unattended device-code request, OpenAI
returns 403 on the ``usercode`` endpoint (no browser to walk through
consent), and the admin sees the cryptic error above.
Copilot's ``DBAuthenticator`` already overrides ``get_access_token``
and raises ``GetAccessTokenError`` cleanly — this commit mirrors that
in the ChatGPT side:
- If no credential is loaded in ``litellm.credential_list``, raise a
401 pointing the admin at STORE_MODEL_IN_DB and the UI sign-in.
- If the token is expired and refresh fails, raise a 401 quoting the
IdP's refresh error.
- Never call ``_login_device_code`` from the proxy path.
Added regression tests that mock ``_login_device_code`` and assert it
is never called across the three failure modes above (missing
credential, happy path, refresh-error). 173 tests pass; Black + Ruff
clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reported symptom: "Test model" in the UI hangs indefinitely, a new
device code appears in the proxy logs, and the whole UI is frozen.
Root cause: the ChatGPT **chat** transformation still used the
filesystem-only ``Authenticator`` unconditionally. When the UI's
``/health/test_connection`` runs in the default ``mode="chat"``:
UI → /health/test_connection (sync wait)
→ litellm.ahealth_check(mode="chat")
→ litellm.acompletion(...)
→ ChatGPTConfig._get_openai_compatible_provider_info(api_key="oauth:X")
→ self.authenticator.get_access_token() # filesystem authenticator
→ _login_device_code() # no tokens on disk → prints a new
# device code and polls for up to 15
# minutes, blocking the request
``ChatGPTResponsesAPIConfig`` already had the ``resolve_authenticator``
dispatch (the PR's original scope), so the responses-mode path was
fine. The chat path was overlooked.
Fix:
- Apply ``resolve_authenticator`` to both call-sites in
``ChatGPTConfig`` (``_get_openai_compatible_provider_info`` and
``validate_environment``). When ``api_key`` carries the ``oauth:``
prefix, both methods now reach for the DB-backed authenticator and
the stored credential — no device-code prompt, no 15-minute hang.
- Extend ``resolve_authenticator`` to the 3-arg shape
``(api_key, litellm_params, fallback)`` so it matches the Copilot
resolver and covers the chat call-site that has ``api_key`` but not
``litellm_params``. Updated the single existing call-site in
``responses/transformation.py`` and the existing unit tests.
- Add a regression test asserting that ``ChatGPTConfig`` never calls
the filesystem authenticator when ``api_key`` is ``oauth:<name>``
(fs_auth is a MagicMock that raises on any attribute access).
170 tests pass; Black + Ruff clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Without STORE_MODEL_IN_DB=True the proxy still writes OAuth credentials
to LiteLLM_CredentialsTable, but ``proxy_config.get_credentials`` — the
DB → ``litellm.credential_list`` reload run at startup — lives inside
``if store_model_in_db is True:`` in proxy_server.py. Result: silent
data loss on restart and request-time ``api_key: oauth:<name>`` failures
because the name is no longer in the in-memory cache.
Gate the /start endpoints (both ChatGPT and Copilot) behind
``get_secret_bool("STORE_MODEL_IN_DB", False)``. Admins get a clear 400
up front instead of sitting through the 15-minute device-code poll only
to discover the flow silently doesn't persist.
Tests: setenv STORE_MODEL_IN_DB=True in the autouse fixture so the
success-path tests don't each have to opt in, plus one new test per
provider asserting 400 + message when the env var is absent. 167 cases
pass locally.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reported error after the cross-loop fix:
Tokens obtained but DB persist failed: Unable to match input value to
any allowed input type for the field. Parse errors: [...
``credential_values`` should be of any of the following types:
``JsonNullValueInput``, ``Json`` ...]
Prisma's Json columns ingest pre-serialized JSON *strings*, not raw
Python dicts. The ``/credentials`` endpoint wraps the payload with
``jsonify_object(...)`` (in ``litellm/proxy/utils.py``) which does
``json.dumps`` per nested-dict field. My persist helper was skipping
that step and passing raw dicts — Prisma's binding layer then couldn't
match them to the ``Json`` type.
Route the ``credential_values`` + ``credential_info`` dicts through
``jsonify_object`` before the upsert. Same fix in both ChatGPT and
Copilot ``db_authenticator.py``. Tests updated to assert the serialized
string form (``json.loads(kwargs[...]["credential_values"])``).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reported error during device-code login:
Tokens obtained but DB persist failed: <asyncio.locks.Event object at
0x... [unset]> is bound to a different event loop
Root cause: the background worker was a ``threading.Thread`` that ran
``asyncio.run(persist_credential_to_db(item))``. That creates a fresh
event loop in the worker thread, but ``prisma_client``'s internal
asyncio primitives (locks, futures) are bound to the proxy's main loop.
Awaiting a prisma call from the worker loop hits the cross-loop error.
Fix:
- Refactor the background flow from a thread to an asyncio task
(``asyncio.create_task``) scheduled on the proxy's main loop. Blocking
IO in the flow (device-code poll, token exchange) runs in
``loop.run_in_executor``. The DB persist step (``await
persist_credential_to_db(item)``) now naturally shares a loop with
prisma_client. Same treatment for both ChatGPT and Copilot endpoints.
- For the DBAuthenticator refresh path (called from sync
``validate_environment`` during a request, same cross-loop risk), add
``_register_proxy_main_loop`` + ``_schedule_db_persist`` now prefers
``asyncio.run_coroutine_threadsafe`` onto the registered loop.
``/chatgpt/oauth/start`` and ``/copilot/oauth/start`` register the loop
on invocation (guaranteed to be the proxy's main loop). Falls back to
the old thread + ``asyncio.run`` path only when no loop is registered
(CLI / tests, where prisma_client is ``None`` anyway).
Tests updated: ``TestBackgroundWorker`` now drives the async task
directly via ``await _run_device_code_flow_async(...)``. The
``test_creates_session_and_spawns_worker`` test stubs the task to a
no-op instead of patching ``threading.Thread``.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add "Sign in with ChatGPT" and "Sign in with GitHub Copilot" device-code
OAuth flows to the Add Credential modal. Tokens persist as encrypted JSON
in LiteLLM_CredentialsTable and are picked up at request time when a
model's api_key is set to "oauth:<credential_name>"; a DBAuthenticator
subclass reads from the in-memory credential cache (sync) and writes
refreshed tokens back via a fire-and-forget worker thread.
Per-row Refresh button rotates tokens on demand — ChatGPT via the IdP's
refresh_token grant, Copilot by re-deriving the short-lived API key from
the stored GitHub access token.
Also ships a litellm-chatgpt-login CLI with an optional PKCE+loopback
flow alongside the existing device-code path for local sign-in outside
the UI.
Provider-specific surface lives in new files under
litellm/llms/{chatgpt,github_copilot}/db_authenticator.py and
litellm/proxy/{chatgpt,copilot}_oauth_endpoints/. Shared-file delta is
~275 lines across 10 pre-existing files, the bulk being a pure append
at the end of networking.tsx.
Dispatch convention + operational caveats documented in the new
"ChatGPT / Copilot OAuth Credentials" section of CLAUDE.md.
Security:
- /chatgpt/oauth/* and /copilot/oauth/* endpoints require PROXY_ADMIN
(view-only admins excluded from write paths)
- session_id query params are Query(...) annotated
- session-cap check + slot reservation are atomic; reserved slot is
cleaned up on device-code failure
- PKCE loopback callback html.escape()s the IdP's error_description
before rendering
Tests: 165 cases covering DBAuthenticator read/write, config dispatch,
endpoint admin-only, 429 cap, slot cleanup, background-worker success /
auth-failure / DB-persist-failure paths, PKCE helpers + XSS regression,
CLI broad-exception + KeyboardInterrupt.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 30, 8) (push) Waiting to run
The suite was superseded by ui/litellm-dashboard/e2e_tests/ on 2026-04-08
and is no longer referenced by CircleCI, docs, or Makefile targets. Drop
the directory wholesale and remove the orphaned e2e:psql npm script that
pointed at its runner.
Previously these were silently dropped with a verbose warning, which
could break observability integrations without surfacing a clear error.
Now raises ValueError with remediation steps (configure server-side
or pass the resolved value) so callers get immediate, actionable feedback.
Brings the date-range branch in line with the non-date-range branch which
already hashes sk- prefixed tokens before querying. Adds coverage for
filter-combination behavior in view_spend_logs.
- Log a warning when dropping callback params that carry os.environ/
references so operators notice the misconfiguration.
- Require absolute paths in oidc/file/ and correct the documented
example to use the leading-slash form.
- Drop the unused return value from _reject_os_environ_references.
- Reject os.environ/ references supplied via /health/test_connection
request params instead of resolving them; config-sourced values are
already resolved before reaching the endpoint.
- Skip os.environ/ references in dynamic callback params loaded from
per-request metadata.
- Constrain oidc/file/ to an allowed credential directory allowlist
(defaults to /var/run/secrets and /run/secrets, overridable via
LITELLM_OIDC_ALLOWED_CREDENTIAL_DIRS).
Budget table entries (team members, end-users) used duration_in_seconds()
for a sliding-window reset, while keys/users/teams used calendar-aligned
get_budget_reset_time(). This made "30d" and "1mo" mean different things
depending on entity type. Now both paths use get_budget_reset_time() for
consistent calendar-aligned resets (e.g. "30d" → 1st of next month).
Fixes#25432
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The model_max_budget limiter tracks spend in one code path
(async_log_success_event) and enforces budget limits in another
(is_key_within_model_budget via user_api_key_auth). These two paths
used different model name formats to build cache keys:
- Tracking used standard_logging_payload["model"], which is the
deployment-level model name (e.g. "vertex_ai/claude-opus-4-6@default")
- Enforcement used request_data["model"], which is the model group
alias (e.g. "claude-opus-4-6")
Because the cache keys never matched, the enforcement path always read
None for current spend, silently allowing all requests through even
after the budget was exceeded. This affected any provider that decorates
model names with provider prefixes or version suffixes (Vertex AI,
Bedrock, etc.).
Fix: use model_group (the user-facing alias) from StandardLoggingPayload
for spend tracking, falling back to model when model_group is None.
This aligns the tracking cache key with the enforcement cache key.
Fixes the same root cause reported in #15223 and #10052.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two code paths in key_management_endpoints.py call hash_token()
unconditionally when invalidating the user_api_key_cache after a key
update. When the caller passes a pre-hashed token ID (not an sk-
prefixed key), hash_token() double-hashes it, producing a cache key
that does not match the actual cached entry. Cache invalidation
silently fails.
This is compounded by update_cache() which writes the stale cached key
object back with a fresh 60s TTL after every successful request,
preventing natural TTL expiry. The stale entry (with outdated fields
like max_budget=None) persists indefinitely under load.
PR #24969 fixed this in update_key_fn but missed two other call sites:
- _process_single_key_update (bulk update path)
- _execute_virtual_key_regeneration (key rotation path)
Fix: replace hash_token() with _hash_token_if_needed() in both
locations, matching the pattern already used elsewhere in the file.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Renames the new per-guardrail opt-out field from `disabled_global_guardrails`
to `opted_out_global_guardrails` to eliminate the one-character collision with
the legacy `disable_global_guardrails` boolean kill switch. Adds a type guard
on the new gate so a misnamed bool can't crash the guardrail check. Filters
duplicates out of the team-edit guardrail display for legacy teams that have a
global name persisted in `metadata.guardrails` from before this PR. Drops the
unused `isGuardrailsLoading` and `guardrailsError` destructures left in
AddModelForm after the hook refactor.
Adds Python tests for the new gate behavior (root, litellm_metadata, metadata,
non-matching name, empty list, malformed bool value, opt-in coexistence) and
extends useGuardrails.test.ts to exercise the global / optional partition
logic that the rebuilt hook performs in its `select` transform.
Wires the legacy kill switch and the new opt-out list together in the team
edit form so they can never fall out of sync:
- Toggling the kill switch reactively updates the Guardrails Select via
`onValuesChange` — switch on strips all globals from the selection, switch
off re-adds them. Existing opt-in extras are preserved either way.
- When the switch is on, global options in the Select are individually
disabled (greyed out) so the user can still manage opt-in guardrails but
cannot accidentally re-enable a global the kill switch is bypassing.
- The save handler writes both fields together: `disable_global_guardrails`
reflects the switch, and `opted_out_global_guardrails` is set to either
every global (when the switch is on) or the user's explicit opt-outs.
- `effectiveGuardrails` for the form's initialValues honors the kill switch
on legacy teams so the form opens in a state that matches what the runtime
gate is actually doing — fixes the visual lie where chips appeared active
while the switch was bypassing them.
The backend gate already reads the list as the primary path with the bool
as a fallback, so untouched legacy teams keep working until they get edited,
at which point they migrate naturally.
Rename disable_global_guardrail → disable_global_guardrails to match
the key name used by litellm_pre_call_utils.py, the API endpoints,
and the UI when propagating key/team metadata.
The singular form was introduced in PR #16983 and has never matched
the plural form written by the rest of the codebase, so the feature
silently did nothing.
Re-applies fix originally from #25488. Original commit could not be
merged due to missing signature.
Co-Authored-By: Remi Mabon <remi.mabon@redcare-pharmacy.com>
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 30, 8) (push) Has been cancelled
The test expected "stream": false in the serialized request body, but
stream is only included in optional_params when explicitly passed by
the caller. litellm.completion() defaults stream=None which is excluded
from non_default_params. Assert individual fields instead of the full
serialized JSON to avoid brittleness around optional field inclusion.
usage-based-routing tracks RPM in log_success_event which runs in a
background ThreadPoolExecutor. The cache update races with the next
call's routing check in both sync (tight loop) and async (concurrent
gather) modes, making over-limit detection non-deterministic.
Skip the over-limit parametrization (num_try_send > num_allowed_send).
The under-limit cases (2 sent, 3 allowed) still verify the routing
strategy works. Rate-limit enforcement is properly tested with mocks
in tests/test_litellm/test_router/test_enforce_model_rate_limits.py.
- Add _verify_org_access to deprecated POST /organization/info endpoint
- Move get_user_object to module-level import in organization_endpoints.py
- Add tests for _verify_team_access 403 denial path
