Reported symptom: Test Model fails immediately with
GetLLMProvider Exception - Failed to request device code:
Client error '403 Forbidden' for url
'https://auth.openai.com/api/accounts/deviceauth/usercode'
No 15-minute hang this time — it comes back instantly.
Root cause: the ChatGPT ``DBAuthenticator`` inherited
``get_access_token`` from the filesystem ``Authenticator``, whose last
line is:
tokens = self._login_device_code()
That's the right fall-through for the CLI (no tokens on disk → start a
new login). In the proxy's DB-backed context it is catastrophic: the
server tries to initiate an unattended device-code request, OpenAI
returns 403 on the ``usercode`` endpoint (no browser to walk through
consent), and the admin sees the cryptic error above.
Copilot's ``DBAuthenticator`` already overrides ``get_access_token``
and raises ``GetAccessTokenError`` cleanly — this commit mirrors that
in the ChatGPT side:
- If no credential is loaded in ``litellm.credential_list``, raise a
401 pointing the admin at STORE_MODEL_IN_DB and the UI sign-in.
- If the token is expired and refresh fails, raise a 401 quoting the
IdP's refresh error.
- Never call ``_login_device_code`` from the proxy path.
Added regression tests that mock ``_login_device_code`` and assert it
is never called across the three failure modes above (missing
credential, happy path, refresh-error). 173 tests pass; Black + Ruff
clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reported symptom: "Test model" in the UI hangs indefinitely, a new
device code appears in the proxy logs, and the whole UI is frozen.
Root cause: the ChatGPT **chat** transformation still used the
filesystem-only ``Authenticator`` unconditionally. When the UI's
``/health/test_connection`` runs in the default ``mode="chat"``:
UI → /health/test_connection (sync wait)
→ litellm.ahealth_check(mode="chat")
→ litellm.acompletion(...)
→ ChatGPTConfig._get_openai_compatible_provider_info(api_key="oauth:X")
→ self.authenticator.get_access_token() # filesystem authenticator
→ _login_device_code() # no tokens on disk → prints a new
# device code and polls for up to 15
# minutes, blocking the request
``ChatGPTResponsesAPIConfig`` already had the ``resolve_authenticator``
dispatch (the PR's original scope), so the responses-mode path was
fine. The chat path was overlooked.
Fix:
- Apply ``resolve_authenticator`` to both call-sites in
``ChatGPTConfig`` (``_get_openai_compatible_provider_info`` and
``validate_environment``). When ``api_key`` carries the ``oauth:``
prefix, both methods now reach for the DB-backed authenticator and
the stored credential — no device-code prompt, no 15-minute hang.
- Extend ``resolve_authenticator`` to the 3-arg shape
``(api_key, litellm_params, fallback)`` so it matches the Copilot
resolver and covers the chat call-site that has ``api_key`` but not
``litellm_params``. Updated the single existing call-site in
``responses/transformation.py`` and the existing unit tests.
- Add a regression test asserting that ``ChatGPTConfig`` never calls
the filesystem authenticator when ``api_key`` is ``oauth:<name>``
(fs_auth is a MagicMock that raises on any attribute access).
170 tests pass; Black + Ruff clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Without STORE_MODEL_IN_DB=True the proxy still writes OAuth credentials
to LiteLLM_CredentialsTable, but ``proxy_config.get_credentials`` — the
DB → ``litellm.credential_list`` reload run at startup — lives inside
``if store_model_in_db is True:`` in proxy_server.py. Result: silent
data loss on restart and request-time ``api_key: oauth:<name>`` failures
because the name is no longer in the in-memory cache.
Gate the /start endpoints (both ChatGPT and Copilot) behind
``get_secret_bool("STORE_MODEL_IN_DB", False)``. Admins get a clear 400
up front instead of sitting through the 15-minute device-code poll only
to discover the flow silently doesn't persist.
Tests: setenv STORE_MODEL_IN_DB=True in the autouse fixture so the
success-path tests don't each have to opt in, plus one new test per
provider asserting 400 + message when the env var is absent. 167 cases
pass locally.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reported error after the cross-loop fix:
Tokens obtained but DB persist failed: Unable to match input value to
any allowed input type for the field. Parse errors: [...
``credential_values`` should be of any of the following types:
``JsonNullValueInput``, ``Json`` ...]
Prisma's Json columns ingest pre-serialized JSON *strings*, not raw
Python dicts. The ``/credentials`` endpoint wraps the payload with
``jsonify_object(...)`` (in ``litellm/proxy/utils.py``) which does
``json.dumps`` per nested-dict field. My persist helper was skipping
that step and passing raw dicts — Prisma's binding layer then couldn't
match them to the ``Json`` type.
Route the ``credential_values`` + ``credential_info`` dicts through
``jsonify_object`` before the upsert. Same fix in both ChatGPT and
Copilot ``db_authenticator.py``. Tests updated to assert the serialized
string form (``json.loads(kwargs[...]["credential_values"])``).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Reported error during device-code login:
Tokens obtained but DB persist failed: <asyncio.locks.Event object at
0x... [unset]> is bound to a different event loop
Root cause: the background worker was a ``threading.Thread`` that ran
``asyncio.run(persist_credential_to_db(item))``. That creates a fresh
event loop in the worker thread, but ``prisma_client``'s internal
asyncio primitives (locks, futures) are bound to the proxy's main loop.
Awaiting a prisma call from the worker loop hits the cross-loop error.
Fix:
- Refactor the background flow from a thread to an asyncio task
(``asyncio.create_task``) scheduled on the proxy's main loop. Blocking
IO in the flow (device-code poll, token exchange) runs in
``loop.run_in_executor``. The DB persist step (``await
persist_credential_to_db(item)``) now naturally shares a loop with
prisma_client. Same treatment for both ChatGPT and Copilot endpoints.
- For the DBAuthenticator refresh path (called from sync
``validate_environment`` during a request, same cross-loop risk), add
``_register_proxy_main_loop`` + ``_schedule_db_persist`` now prefers
``asyncio.run_coroutine_threadsafe`` onto the registered loop.
``/chatgpt/oauth/start`` and ``/copilot/oauth/start`` register the loop
on invocation (guaranteed to be the proxy's main loop). Falls back to
the old thread + ``asyncio.run`` path only when no loop is registered
(CLI / tests, where prisma_client is ``None`` anyway).
Tests updated: ``TestBackgroundWorker`` now drives the async task
directly via ``await _run_device_code_flow_async(...)``. The
``test_creates_session_and_spawns_worker`` test stubs the task to a
no-op instead of patching ``threading.Thread``.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add "Sign in with ChatGPT" and "Sign in with GitHub Copilot" device-code
OAuth flows to the Add Credential modal. Tokens persist as encrypted JSON
in LiteLLM_CredentialsTable and are picked up at request time when a
model's api_key is set to "oauth:<credential_name>"; a DBAuthenticator
subclass reads from the in-memory credential cache (sync) and writes
refreshed tokens back via a fire-and-forget worker thread.
Per-row Refresh button rotates tokens on demand — ChatGPT via the IdP's
refresh_token grant, Copilot by re-deriving the short-lived API key from
the stored GitHub access token.
Also ships a litellm-chatgpt-login CLI with an optional PKCE+loopback
flow alongside the existing device-code path for local sign-in outside
the UI.
Provider-specific surface lives in new files under
litellm/llms/{chatgpt,github_copilot}/db_authenticator.py and
litellm/proxy/{chatgpt,copilot}_oauth_endpoints/. Shared-file delta is
~275 lines across 10 pre-existing files, the bulk being a pure append
at the end of networking.tsx.
Dispatch convention + operational caveats documented in the new
"ChatGPT / Copilot OAuth Credentials" section of CLAUDE.md.
Security:
- /chatgpt/oauth/* and /copilot/oauth/* endpoints require PROXY_ADMIN
(view-only admins excluded from write paths)
- session_id query params are Query(...) annotated
- session-cap check + slot reservation are atomic; reserved slot is
cleaned up on device-code failure
- PKCE loopback callback html.escape()s the IdP's error_description
before rendering
Tests: 165 cases covering DBAuthenticator read/write, config dispatch,
endpoint admin-only, 429 cap, slot cleanup, background-worker success /
auth-failure / DB-persist-failure paths, PKCE helpers + XSS regression,
CLI broad-exception + KeyboardInterrupt.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Waiting to run
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 30, 8) (push) Waiting to run
Previously these were silently dropped with a verbose warning, which
could break observability integrations without surfacing a clear error.
Now raises ValueError with remediation steps (configure server-side
or pass the resolved value) so callers get immediate, actionable feedback.
Brings the date-range branch in line with the non-date-range branch which
already hashes sk- prefixed tokens before querying. Adds coverage for
filter-combination behavior in view_spend_logs.
- Log a warning when dropping callback params that carry os.environ/
references so operators notice the misconfiguration.
- Require absolute paths in oidc/file/ and correct the documented
example to use the leading-slash form.
- Drop the unused return value from _reject_os_environ_references.
- Reject os.environ/ references supplied via /health/test_connection
request params instead of resolving them; config-sourced values are
already resolved before reaching the endpoint.
- Skip os.environ/ references in dynamic callback params loaded from
per-request metadata.
- Constrain oidc/file/ to an allowed credential directory allowlist
(defaults to /var/run/secrets and /run/secrets, overridable via
LITELLM_OIDC_ALLOWED_CREDENTIAL_DIRS).
Budget table entries (team members, end-users) used duration_in_seconds()
for a sliding-window reset, while keys/users/teams used calendar-aligned
get_budget_reset_time(). This made "30d" and "1mo" mean different things
depending on entity type. Now both paths use get_budget_reset_time() for
consistent calendar-aligned resets (e.g. "30d" → 1st of next month).
Fixes#25432
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two code paths in key_management_endpoints.py call hash_token()
unconditionally when invalidating the user_api_key_cache after a key
update. When the caller passes a pre-hashed token ID (not an sk-
prefixed key), hash_token() double-hashes it, producing a cache key
that does not match the actual cached entry. Cache invalidation
silently fails.
This is compounded by update_cache() which writes the stale cached key
object back with a fresh 60s TTL after every successful request,
preventing natural TTL expiry. The stale entry (with outdated fields
like max_budget=None) persists indefinitely under load.
PR #24969 fixed this in update_key_fn but missed two other call sites:
- _process_single_key_update (bulk update path)
- _execute_virtual_key_regeneration (key rotation path)
Fix: replace hash_token() with _hash_token_if_needed() in both
locations, matching the pattern already used elsewhere in the file.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Renames the new per-guardrail opt-out field from `disabled_global_guardrails`
to `opted_out_global_guardrails` to eliminate the one-character collision with
the legacy `disable_global_guardrails` boolean kill switch. Adds a type guard
on the new gate so a misnamed bool can't crash the guardrail check. Filters
duplicates out of the team-edit guardrail display for legacy teams that have a
global name persisted in `metadata.guardrails` from before this PR. Drops the
unused `isGuardrailsLoading` and `guardrailsError` destructures left in
AddModelForm after the hook refactor.
Adds Python tests for the new gate behavior (root, litellm_metadata, metadata,
non-matching name, empty list, malformed bool value, opt-in coexistence) and
extends useGuardrails.test.ts to exercise the global / optional partition
logic that the rebuilt hook performs in its `select` transform.
Wires the legacy kill switch and the new opt-out list together in the team
edit form so they can never fall out of sync:
- Toggling the kill switch reactively updates the Guardrails Select via
`onValuesChange` — switch on strips all globals from the selection, switch
off re-adds them. Existing opt-in extras are preserved either way.
- When the switch is on, global options in the Select are individually
disabled (greyed out) so the user can still manage opt-in guardrails but
cannot accidentally re-enable a global the kill switch is bypassing.
- The save handler writes both fields together: `disable_global_guardrails`
reflects the switch, and `opted_out_global_guardrails` is set to either
every global (when the switch is on) or the user's explicit opt-outs.
- `effectiveGuardrails` for the form's initialValues honors the kill switch
on legacy teams so the form opens in a state that matches what the runtime
gate is actually doing — fixes the visual lie where chips appeared active
while the switch was bypassing them.
The backend gate already reads the list as the primary path with the bool
as a fallback, so untouched legacy teams keep working until they get edited,
at which point they migrate naturally.
Rename disable_global_guardrail → disable_global_guardrails to match
the key name used by litellm_pre_call_utils.py, the API endpoints,
and the UI when propagating key/team metadata.
The singular form was introduced in PR #16983 and has never matched
the plural form written by the rest of the codebase, so the feature
silently did nothing.
Re-applies fix originally from #25488. Original commit could not be
merged due to missing signature.
Co-Authored-By: Remi Mabon <remi.mabon@redcare-pharmacy.com>
- Add _verify_org_access to deprecated POST /organization/info endpoint
- Move get_user_object to module-level import in organization_endpoints.py
- Add tests for _verify_team_access 403 denial path
- Introduced a new method in `FileContentStreamingHandler` to resolve streaming request parameters, enhancing the routing logic based on credentials.
- Updated the `should_stream_file_content` method to check against supported providers.
- Cleaned up type hints and imports across multiple files for better organization and clarity.
- Added comprehensive tests to validate the new routing behavior and ensure original data integrity during streaming requests.
* fix(s3): add retry with exponential backoff for transient S3 503/500 errors
S3 occasionally returns 503 "Slow Down" during PUT operations when
request rates spike above partition limits. The current code makes a
single upload attempt via httpx — unlike boto3, httpx has no built-in
retry for transient S3 errors. Failed uploads permanently lose the
request's audit/logging data.
Add exponential backoff retry (3 attempts, 1s/2s delays) for S3
500/503 responses in both async_upload_data_to_s3 and
upload_data_to_s3. Logs a warning on each retry with the S3 object
key for observability.
In production we observed ~18 permanent S3 upload failures per day
(124 over 7 days) — all transient 503s that would have succeeded on
a single retry.
* test(s3): add unit tests for S3 upload retry logic
Tests cover:
- Async retry on 503 (succeeds on second attempt)
- Async retry on 500
- Exhausted retries on persistent 503 (calls handle_callback_failure)
- No retry on 4xx errors (403)
- Sync retry on 503
* style(s3): move time import to module level
Address review feedback: move `import time` from inside
upload_data_to_s3 to the top-level imports per project style guide.
