mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-28 06:23:07 +00:00
8 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Yassin Kortam
|
db2039b11b
|
feat(github_copilot): support a GHE.com host on per-user Copilot credentials (#47610)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
Mateo Wang
|
6c4c134f23
|
fix(secret_managers)!: resolve oidc/file and oidc/env refs in admin-configured api_key per request (#47341)
* fix(secret_managers): resolve oidc/file and oidc/env refs in admin-configured api_key per request An api_key set to oidc/file/<path> or oidc/env/<VAR> in a credential_list entry or a deployment's litellm_params was sent upstream as literal text, so every call answered 401. The SDK client wrapper now reads the referenced file or env var on each call and never writes the token back, so a rotated token is used on the next call. The existing server-owned federation gates treat an oidc/ api_key as server-owned by value, so a request body, a non-admin credential or model write, user_config, fallback targets and routing tier params can never carry or name one. Other oidc/ kinds answer 400 * fix(secret_managers): strip only the leading oidc/ from a secret reference A token file under a directory named oidc resolved to the wrong path, because every oidc/ in the reference was removed * fix(auth): refuse oidc/ api_key refs in advisor tools and scope the route gates to them The advisor tool's own api_key now answers a 400 when it holds an oidc/ reference, since its sub-call runs through the client wrapper that resolves one. Stored fallback targets, auto-router tiers and user_config refuse only an oidc/ api_key, inline or through the credential they name, so their workload identity federation handling stays as it was. Provider model discovery for a wildcard deployment resolves the reference before calling the provider. * fix(auth): allow admin oidc api_key on deployment writes, resolve refs on discovery, batch probe and realtime health, refuse client api_base on oidc deployments * fix(utils): resolve oidc api_key refs only from admin-configured sources Mark an oidc/ api_key read from a proxy-level deployment or a stored credential, and resolve only that marked value. Team deployments, prompt templates, nested configs and a direct SDK api_key send the text as written. An unsupported reference kind raises a 401 so the router fails over to the group's next deployment. * fix(utils): send a credential's oidc token only to its own api_base host A stored credential's oidc/ api_key now resolves only for the scheme, host and port of the credential's own api_base or base_url, so a team model or vector store that names the credential with another api_base gets a 401 and that host gets nothing. Saving a team model with an inline oidc/ api_key answers 400, since team rows never resolve one. * fix(proxy): send an admin oidc api_key only to the hosts configured beside it Every admin oidc/ reference now carries the origins of the host params set beside it, checked for api_base, base_url and the Bedrock, S3, SageMaker and deployment_url params. Token counting reads the reference only when it calls the provider, answering 401 when it cannot, and pass-through 401s redact internal paths. * fix(proxy): redact the token path from token count auth errors * test(security): cover token count, Bedrock endpoint, api_base update and pass-through path redaction for oidc api_key refs * fix(proxy): redact the token path from secret reference errors where they are raised * fix(proxy): keep the secret reference's path out of the error a caller sees The 401 for an unreadable oidc api_key or pass-through credential now names only the reference kind and a fixed reason, so a token path under /run/secrets or a custom credential dir never reaches the caller, while the server log keeps the full reference for the operator * fix(secret_managers): strip line breaks from the logged secret reference A reference holding a newline can no longer forge a server log line * fix(secret_managers): label an unknown oidc reference kind instead of echoing it An api_key such as oidc/env_MY_VAR (a typo for oidc/env/MY_VAR) came back in full in the caller's 401 and in the spend log, since the kind was whatever text followed oidc/. A kind get_secret does not know now reads oidc/<unknown kind>. The relative-path hint uses oidc/file//<absolute path>, which the client-message redaction on chat routes leaves whole. * test(health): run the real model auth check in the oidc test connection test * test(secret_managers): read an oidc/file token stored under a directory named oidc * fix(secret_managers): count a URL in the model name as an upstream host for oidc api_key references A huggingface/* wildcard call could name its own endpoint URL as the model and receive the resolved token. The origin check now treats a URL in the model as a host the reference must allow. The integration contract also reads the proxy's own database instead of the pytest worker's. * fix(secret_managers): treat a default port as explicit when matching an oidc api_key origin A credential on https://api.mistral.ai refused a call to https://api.mistral.ai:443, though both reach the same server. The origin check now compares scheme, host and effective port, as assert_same_origin does, and an unparseable api_base is refused instead of raising. The advisor test sets the real proxy flag instead of replacing our own helpers. * test: type the oidc request-body helpers and drive the tier test through mock_response * fix(secret_managers): refuse an oidc api_key reference whose call drops the configured host |
||
|
yuneng-jiang
|
b2cc5f25f6
|
fix(credentials): keep credential labels and edits consistent across UI, API, and replicas (#45835)
* fix(credentials): keep credential labels and edits consistent across UI, API, and replicas The credential modal flattened stored credential values into the same form object as the credential's own name, label, and provider, so a stored value keyed credential_name or display_name replaced the real one and the edit went to the wrong credential or relabeled it. The modal now submits the top-level fields and credential_values separately PATCH /credentials now reads the row it merges onto from the writer, so an edit inside the read replica's lag window no longer writes back an older label or key POST /credentials answers 409 for a name defined in config instead of creating a DB copy that shadows it display_name rules live on the server only. It rejects control, formatting, and line separator characters, and the UI stops enforcing its own length check and shows the server's error The models table, model detail view, and model edit picker show the credential's technical name under its label whenever the two differ * fix(ui): show credential source in its own column like models The config marker sat under the credential name. The credentials table now has a Source column with a Config or DB status badge, matching the Source column on the models table * test(credentials): unit-test label and prefill helpers and keep the full credential name on hover Moves displayNameChange and initialFormValues out of the modal so their rules are unit tested, keeps only the wiring cases in the modal's integration test, makes the label acceptance test write through the real credentials repository, and adds the full credential name as a hover title where the models pages truncate it |
||
|
Yassin Kortam
|
d36b66e19e
|
feat(github_copilot): per-user GitHub OAuth connections for Copilot credentials (#45241)
* feat(github_copilot): per-user GitHub OAuth connections for Copilot credentials Related to LIT-9306 * fix(ui): make per-user GitHub Copilot OAuth credential-only in Add Model * fix(proxy): render LiteLLM_UserProviderCredentials prisma spans * style(ui): prettier-format add_model tests * fix(lint): annotate connection endpoints and narrow per-user credential excepts * fix(lint): satisfy type-discipline gate for per-user copilot code * fix(types): clear basedpyright and strict-lint gate regressions Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(types): keep runtime guards on untyped responses input Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style: ruff format per-user Copilot files * fix(lint): keep type-discipline suppressions on flagged lines * fix(github_copilot): resolve team and deployment-id per-user routes, share token cache invalidation * test(github_copilot): scope response-cache mutation via monkeypatch, assert route allowlist * fix(github_copilot): fall back to the database on Redis errors and skip worker-local token caching * fix(lint): suppress mutable return annotation after base merge * style: shorten rebind-ok reason so the dispatch line stays formatted * fix(github_copilot): harden per-user token cache, overrides, fallbacks, headers and device flow * fix(lint): suppress MutableMapping param on session auth pinning * test(github_copilot): narrow pytest.raises to HTTPException for service-key override * fix(github_copilot): fail closed on revoke errors, strip caller auth headers, widen fallback discovery * fix(github_copilot): verify revoke tombstone, bound fallback discovery, cover aliased fallbacks * fix(github_copilot): make fallback discovery a router superset and guard credential override at deployment * style(router): drop explanatory comment on per-user credential guard * fix(github_copilot): gate fallback discovery on per-user credentials and bound aggregate targets * fix(github_copilot): count every fallback mapping toward the discovery limit * fix(github_copilot): suppress banned typing cast imports with per-line reasons * test(github_copilot): cover per-user 401 cooldown skip through the router failure callback * test(github_copilot): cover per-user 401 cooldown skip through router and fallback paths * fix(github_copilot): cover auto-router discovery, per-user fallback cooldown and connect cache race * fix(github_copilot): cover every strategy router kind and verify the connect cache write * fix merge conflict marker remnant in credential endpoint tests * fix(github_copilot): do not purge user connections on label-only credential updates * fix(github_copilot): read user provider credentials from the writer engine * test(github_copilot): move per-user auth type form test to the integration tier * test(github_copilot): annotate new credential regression test locals as Final * fix(github_copilot): type per-user OAuth JSON and DB boundaries * fix(github_copilot): validate per-user OAuth boundaries and restore truncated comments * style(github_copilot): complete cut-off suppression reasons on new casts * refactor(github_copilot): annotate new boundary-typing locals as Final --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
Yassin Kortam
|
fbf4af301c
|
feat(microsoft_365_copilot): add Microsoft 365 Copilot chat provider with OAuth token exchange (#45158)
* feat(microsoft_365_copilot): add Microsoft 365 Copilot chat provider with OAuth token exchange * fix(ui): create a credential for credential-only auth types in Add Model * fix(microsoft_365_copilot): accept max_tokens and list the chat model * fix(microsoft_365_copilot): register provider model set * fix(ui): hide other auth types' fields when credential-only types are filtered * fix(health): do not inherit stored auth settings when a connection test brings its own api_key * docs(health): restore connection-test inheritance docstrings * fix(lint): suppress justified provider boundary casts * fix(lint): satisfy M365 type-discipline gate * fix(types): eliminate type-check gate regressions * style: shorten suppression reasons so ruff format leaves them on one line * refactor(types): drop shared-file type widening unrelated to the Copilot provider * fix(health): pass caller headers as secret_fields to connection-test probes * fix(types): type connection-test probe and token counter locally * fix(lint): drop cast import from connection-test header getter * chore: remove explanatory comments flagged by review * fix(proxy): gate OAuth client credentials behind proxy admin * fix(router): skip cooldown on caller-scoped OAuth auth failures * fix(router): skip fallback cooldown on caller-scoped OAuth auth failures * fix(router): type caller-scoped cooldown lookups * fix(ci): include Microsoft 365 Copilot tests in unit shard * fix(proxy): keep saved endpoint when test_connection retests a deployment by id * fix(m365): collapse doubled Graph replies * fix(m365): preserve trailing system messages --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
0d17f954c0
|
feat(credentials): add display_name and make credential_name immutable on PATCH (#43148)
* feat(credentials): add credential_alias and make credential_name immutable on PATCH Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(ui): query credential rows by role to stay under the no-node-access budget Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): expect credential_alias in load_credential_list dump Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(ui): drive the credential SearchSelect by placeholder and option roles Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): pass the decrypted CredentialItem to update_db_credential during master key rotation Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(ui): trim credential_alias in CredentialModal so whitespace-only input clears the alias Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(credentials): replace credential_alias with display_name and reject edits to config credentials credential_name stays the immutable reference key. display_name is a nullable, trimmed, max 255 character label set on POST or PATCH (omit keeps it, null clears it, blank is a 400). Reads return display_name plus an in-memory source tag (db or config), and PATCH or DELETE on a config-defined credential answers 400 instead of a misleading 404 * feat(ui): show credential display names everywhere and lock config credentials The credentials table shows the display name with the credential name beneath it, badges config credentials and disables their edit and delete actions. The edit modal adds an editable display name next to the read-only credential name and only sends it when it changed. Every credential picker and reference (add model, model info, models table, vector store form and info) shows the label while still submitting credential_name * feat(cli): set display names on credentials and show them in the list * refactor(credentials): keep the new credential types inside the lint gate budgets * refactor(cli): print credential command JSON through one helper * fix(credentials): treat an empty credential_name on PATCH as omitted and pin the 404 for vanished rows A blank credential_name never renamed anything, so PATCH accepts it again instead of answering 400. New tests pin a trimmed display_name on PATCH, the repository carrying display_name, and a 404 (not the config-owned 400) for a DB credential another worker already deleted. The hydration helper no longer copies display_name, since none of its callers read it, and the CLI update passes display_name straight through because the exactly-one check already makes it None when clearing. * fix(ui): keep a model's credential when its picker text is emptied and skip the admin-only list for other roles Clearing the search text in the model edit form's credential picker used to submit null, silently detaching the model's credential on save. None is now the only way to clear it. The models table fetches /credentials only for proxy admins, since everyone else got a 403 on each page load and falls back to the raw name anyway. Also fixes a type error in the re-use credential dialog and adds tests for the display name surviving a provider switch, the 255 character limit, the request payload, and the vector store None choice. * fix(client): percent-encode the credential name when updating its display name A name with ? or # was cut short in the URL, so the PATCH landed on a different credential or 404'd. * fix(credentials): answer 405 with Allow: GET for PATCH and DELETE on config credentials A config-defined credential exists (GET returns it) but is read-only through the API, which is what 405 Method Not Allowed means. 400 described a malformed request, and 404 would claim the credential does not exist. * fix(proxy): ignore a display_name set on config credential_list entries so non-string values cannot fail boot --------- Co-authored-by: yuneng <yuneng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
0ed1c08f02
|
feat(anthropic): workload identity federation and pluggable identity sources (#44448)
* feat(anthropic): workload identity federation and pluggable identity sources Backend half of #38818 (internal copy of the fork PR #38013), rebuilt as one commit on top of litellm_internal_staging without the dashboard changes. Deployments on anthropic/ without a static api_key can exchange an OIDC workload assertion for a short-lived sk-ant-oat01 token through a shared RFC 7523 JWT-bearer engine. The assertion comes from a mounted token file, an env token, a LiteLLM-signed issuer, or Keycloak, chosen per deployment, per named credential, or through ANTHROPIC_IDENTITY_SOURCE. The federation fields are server-owned: refused inline in request bodies and on POST /model/new, proxy-admin only on credentials, and the token exchange is pinned to api.anthropic.com unless LITELLM_ANTHROPIC_WIF_ALLOWED_HOSTS adds a host. GET /credentials/{name}/jwks exports the public key set of a LiteLLM-signed credential for the Claude Console. The OpenAI federation trio from #39613 rides along on the backend side with the same server-owned handling. Fixes #28607 Resolves LIT-6107 Co-authored-by: derhornspieler <15236687+derhornspieler@users.noreply.github.com> * fix(anthropic): let batch-result downloads mint from deployment params and accept host:port allowlist entries The files handler enabled workload identity on batch-result downloads but never received the deployment's litellm_params, so a deployment authenticating through a named credential could only mint from process-wide env vars. It now threads litellm_params through to the auth header the way the batch retrieve path already does. LITELLM_ANTHROPIC_WIF_ALLOWED_HOSTS entries written as host:port were read by urlsplit as a scheme, so the allowlist kept the raw entry while the exchange compared bare hostnames and refused the gateway. Entries are now parsed as network locations whether or not they carry a scheme. * fix(types): move the WIF kwargs key sets to a leaf module so the kwargs funnel imports without a cycle * test(anthropic): pin case-insensitive matching of WIF exchange-host allowlist entries * fix(anthropic): end workload identity federation errors without a period so the router suffix reads cleanly * fix(proxy): decrypt stored litellm_params before the WIF write gate * fix(proxy): hide WIF secret references from /health output * fix(proxy): keep the proxy error shape on credential endpoint refusals * fix(proxy): hide identity token file paths from /health output * fix(anthropic): rename the federation workspace param so Bedrock's anthropic_workspace_id keeps working The Bedrock Claude Platform route already reads anthropic_workspace_id from optional_params, so banning that spelling as a server-owned federation parameter broke a pre-existing client capability. The federation field is now anthropic_federation_workspace_id (env ANTHROPIC_FEDERATION_WORKSPACE_ID), which restores the base branch's behavior for Bedrock callers, drops the Bedrock-specific hint from the refusal message, and deletes the unconditional ban constant that no longer had a reader * fix(auth): share one exchanged token across workers reading the same assertion Anthropic accepts each identity assertion exactly once, so two uvicorn workers reading the same token file both minting from it means the second exchange is denied with jti_reused. Minted tokens now land in a per-user 0700 cache directory guarded by a file lock, so workers on the same host reuse one exchange until the token expires or the assertion rotates. A 401 is only retried when the re-read assertion actually differs, and the denial hint explains jti_reused. LITELLM_TOKEN_EXCHANGE_CACHE_DIR moves the cache and an empty value disables it * fix: keep anthropic federation from being shadowed or leaked An empty or whitespace-only ANTHROPIC_API_KEY counted as set, so a federated deployment sent an empty x-api-key on every call instead of minting a token. Blank values now read as unset, and a real static key on a federated deployment logs once that it outranks federation and nothing is being federated. The exchange-host allowlist matched hostnames only, so a second process on another port of an allowed host was trusted with the workload's identity token. An entry that names a port now trusts that port alone, while a bare host still trusts every port. The shared token store exists so the workers reading one projected token file do not each spend its single-use jti. A source that mints its own assertion per exchange shares nothing with another worker, so it no longer writes a live token to disk for a lookup that can never hit. * fix: unlink a staged token file a failed write leaves behind The 401 denial hint now also says federation ignores ANTHROPIC_WORKSPACE_ID, which the Bedrock Claude platform provider already reads. * refactor: move anthropic jwks derivation behind a provider-owned tagged union * fix: unlink the staged token file when its write fails at close A buffered write only reaches the disk when the handle closes, so a full disk surfaces at close and left the staging file behind holding a usable token. * fix(anthropic): close the staging descriptor before writing the shared token file * fix(wif): judge federation writes by what they set, not what is stored The admin gate read the stored deployment, so a team admin lost edit, delete and Test Connection on any deployment carrying federation params. It now returns early unless the submitted fields touch the federation surface, and a Test Connection probe that points the deployment at its own api_base is still refused, with the 403 no longer wrapped into a 500 The rest of the same review pass: POST /model/new refuses only a blocking value of `blocked`, so a client that always sends `blocked: false` is not turned away; a request body can no longer pick which federated identity to mint as by naming a stored credential; an advisory refresh the executor refuses disarms the entry instead of wedging the identity until the follower timeout; the static-key shadow warning resolves its env fallback inside the cache instead of once per request; credential writes drop nulls before storing them; the token exchange validates the endpoint URL before reading an assertion and keeps refusing redirects across a client heal; /health hides every server-owned federation field from non-admins; and the async create_file and create_batch paths say which setting is missing when the provider resolves no URL * fix(proxy): let a deployment write name a federated credential reject_federated_credential_reference runs from is_request_body_safe, which pre_db_read_auth_checks calls on every route, so it also fired on POST /model/new, /model/update, /model/{id}/update and /health/test_connection. A proxy admin could no longer attach a federated credential to a deployment over the API or the Admin UI, leaving a static config.yaml entry as the only way to configure the feature the rejection told the caller to go configure, and _reject_non_admin_wif_write never got to make the call it exists to make. is_request_body_safe now takes the route and skips only the credential-reference check on the routes that reach can_user_make_model_call. Federation fields typed inline into a body stay refused everywhere, and a call naming a federated credential still cannot pick the identity it mints as. * refactor(proxy): derive health display policy from the federation key sets The health check module hand-copied the five workload identity fields whose value is a credential, so a shared proxy surface named provider-specific parameters and a newly added secret-bearing field would have gone on being displayed until someone remembered both places WIF_SECRET_BEARING_KEYS now sits beside the key sets it splits out of, types/utils derives secret_bearing_wif_litellm_params from it, and the health layer splats that tuple the same way it already splats the admin-only one * fix(anthropic_wif): treat blank identity-source fields as unset * test(proxy): classify the federation params in the credential slot registry main's registry test (#43298) now fails the build for any credential-named deployment param without a classification. The five federation fields that carry a token, a token file path, or a signing or client secret reference are Unplanted, matching WIF_SECRET_BEARING_KEYS; the four remaining Keycloak settings name a URL, a client id, an auth method, or a scope and are NotSecret * fix(anthropic_wif): declare federation params as owned connection leaves and chart their metrics Register the 18 Anthropic and 3 OpenAI federation params as frozen ConnectionSettings leaves so the owned-kwarg registry, the kwargs funnel and the request-body ban list read one declaration. Pass the deployment api_base through to the count-tokens handler instead of a pre-suffixed URL, which doubled the /count_tokens path on main's prompt-cache predictor. Add the five litellm_anthropic_wif_* families to the all-metrics Grafana dashboard. * fix(credentials): gate PATCH on WIF fields resolved from model_id The credential PATCH handler checked server-owned workload identity federation fields only on the values the caller sent, while a body that named a deployment through model_id had its credential values resolved after that check. A non-admin could therefore copy a federated deployment's WIF fields onto an ordinary credential. Resolve the incoming values first and run the non-admin gate on them, matching the POST path * fix(anthropic): count tokens with ANTHROPIC_AUTH_TOKEN through the shared auth header Count-tokens walked its own credential ladder: a static key, else skip minting when ANTHROPIC_AUTH_TOKEN is set, else mint a federated token. With only the auth token set it forwarded nothing and the proxy silently fell back to its local tokenizer while chat on the same deployment authenticated with that token. The handler now takes the auth header that AnthropicModelInfo.aget_auth_header resolves, the same ladder chat, files, batches and skills use, and merges the oauth beta a minted or consumer token carries with the token-counting beta --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> Co-authored-by: derhornspieler <15236687+derhornspieler@users.noreply.github.com> Co-authored-by: mateo-berri <happymvw@gmail.com> |
||
|
devin-ai-integration[bot]
|
25109a523b
|
test(proxy): move utils, agent_endpoints and endpoint tests into tests/unit/proxy (#44006)
* test(proxy): move utils, agent_endpoints and endpoint tests into tests/unit/proxy Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): package moved unit test directories Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): exclude proxy-db-owned files from the misc target Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): drop the redundant fixture docstrings in the proxy conftest Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: yuneng <yuneng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |