* test(proxy): verify NO_DOCS/NO_REDOC/NO_OPENAPI gate every doc surface
Confirms /docs, /redoc and /openapi.json are reachable by default and
each 404 with no schema in the body when NO_DOCS/NO_REDOC/NO_OPENAPI are
set, and that disabling them does not affect other routes.
* test(proxy): pin the production docs_url construction to the real helpers
Addresses Greptile 4/5 feedback: the disabled-path tests built a fresh
FastAPI() app, which could miss drift in the actual
FastAPI(docs_url=..., ...) call in proxy_server.py. This pins that
exact call site against _get_docs_url/_get_redoc_url/_get_openapi_url.
Coordination Redis (spend counters, budget-window enforcement, and the
reset_spend cache-eviction broadcast) previously only attached when a
deployment set general_settings.coordination_redis or
litellm_settings.cache. Bare REDIS_HOST/REDIS_PORT env vars alone did
nothing, so a multi-replica proxy with no cache block got no cross-pod
coordination at all: a key reset on one pod never cleared another
pod's stale budget enforcement.
The inferred Redis is pinged before being adopted, and a malformed
REDIS_CLUSTER_NODES/REDIS_SENTINEL_NODES value is tolerated too: env
vars can be set for an unrelated reason with nothing reachable there,
and guessing wrong must not turn a previously harmless in-memory-only
proxy into one that fails to boot or raises on its next cache write.
* fix(logging): redact credential query params from the uvicorn access log
Raw virtual keys reached container stdout two ways:
- `GET /key/info?key=sk-...`, `/global/spend/report?api_key=sk-...`,
`/key/spend/report`, `/spend/logs`, `/user/daily/activity` and the Gemini
passthrough routes all put the credential in the request target, and
`uvicorn.access` had no redaction filter (only `uvicorn.error` did).
- the key budget error interpolates `LiteLLM_VerificationToken.key_name`,
a column with no enforced shape, into a message that is both logged and
returned to the caller.
`SecretRedactionFilter` cannot be reused on an access logger: it collapses the
record into `record.msg` and clears `record.args`, and uvicorn's AccessFormatter
unpacks those args at emit time, so every access line would raise TypeError.
`AccessLogRedactionFilter` scrubs the positional args in place instead.
An access line is the one input to the secret regex an unauthenticated caller
controls end to end, so two bounds go with it. The request target is cut back to
a whole query parameter under 512 characters before it is scanned, since a half
parameter is too short to match its own pattern and would be logged raw, and the
dropped tail is not logged at all. The connection-string pattern is bounded too,
because its user half could previously re-scan the rest of the string from every
`://`: a 16 KB URL of `a://` pairs took 314s and now takes 0.12s, with the caps
set high enough that an RDS IAM auth token used as a DSN password still redacts.
Credential query params are terminated by `&` like the existing `key=` and
`sig=` patterns, so redacting one param no longer swallows the rest of the
request line, and a second credential in the same query string is now redacted
on its own instead of surviving once the first one stops the span. `key_name` is
echoed into the budget error only when it still has the masked `sk-...abcd`
shape `abbreviate_api_key` writes, so a value put there by a direct DB write or
a migration falls back to the key alias.
Also point the `/key/info` and spend-report examples at the sha256 hash both
endpoints already accept, so callers stop putting raw keys in URLs that
third-party access logs record.
Resolves LIT-5909
* test(logging): assert on emitted access lines instead of filter registration
The two registration tests checked that an AccessLogRedactionFilter instance
sits in uvicorn.access.filters, which is the shape of the code rather than its
behavior. Handing the logger a real access record and reading what a handler
wrote covers the same wiring and still fails when the registration is removed.
* fix(logging): redact percent-encoded credentials from access logs
?k%65y=sk%2D... is a working credential once the request parser decodes it,
but the redaction patterns match literal text and never see it. Decode the
request target as a detector and drop the query when decoding reveals a
secret. The decoded text is never logged back, so a %0A cannot forge a
following log line
Also accept any four non-space characters in the masked key_name check, since
abbreviate_api_key copies the last four characters of a custom key verbatim
and those can be punctuation or non-ASCII
* fix(auth): keep control codes out of the masked key label
/key/generate accepts a custom key ending in an escape sequence, and
abbreviate_api_key copies those four characters into key_name verbatim, so
the over-budget message carried them to a terminal and a log viewer. Bar
whitespace and C0/C1 control codes from the four, and keep everything else
The mcp_tool_search virtual tool only did substring token matching, so a native MCP client asking for "FX" could not find a tool described as "foreign exchange rates" even though the same catalog is ranked by embeddings on /responses and /chat/completions.
Adds litellm_settings.mcp_tool_search (embedding_model, top_k, similarity_threshold, core_tools). With an embedding model the caller's authorized catalog from _list_mcp_tools is ranked by cosine similarity of name plus description; configured core tools the caller can reach come first and do not consume top_k. Without an embedding model the keyword fallback keeps the old behavior. Settings are hot-reloadable from the DB, exposed on /get and /update mcp_tool_search_settings, and editable from the Admin UI under MCP Servers > Tool Search. The embedding index is shared with agent_search via a new SemanticTextIndex.
Resolves LIT-6751
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
get_api_key had no callers. main.py imported it without using it, and
because main.py declares no __all__, the star import in __init__.py
published it as litellm.get_api_key. It duplicated key resolution that
get_llm_provider_logic already performs, which is how a misspelled env
var survived in it unnoticed until #35985. Drop the definition, the
unused import, the test that pinned the ai21 branch, and ratchet the
lint budgets down by the violations it carried.
Resolves LIT-5245
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Permanent Prisma/query-engine faults keep the 503 status and no_db_connection type but stop claiming the database is temporarily unreachable. A permanent fault anywhere in the exception chain outranks the transport error that surfaced it. MCP bridge and DCR flows gain a faulted resolution state with matching wording. Resolves LIT-5208
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(agents): redact secret litellm_params fields from all /v1/agents responses
Secret-bearing litellm_params fields (aws_secret_access_key, api_key, and
similar) are now write-only: list, get, create, update, and patch
responses always replace them with a fixed marker, regardless of caller
role. Editing an agent no longer requires resending a real credential --
an update that omits a sensitive field, or echoes the marker back,
preserves the stored value; a real value still rotates it.
* fix(agents): redact secrets nested inside dicts/lists in litellm_params too
Greptile found that a secret nested one level down under a
non-sensitively-named key, or inside a list of per-provider configs, was
neither redacted on read nor restored symmetrically on write (the marker
string could get persisted as the real value). Recurse into lists on the
read side, and mirror that recursion on the write side so restoration
isn't limited to top-level keys. Also fixes a regression the redact
rewrite introduced (a plain string leaf like a model name was being
misinterpreted as a JSON blob and redacted), and suppresses 3 new
test-quality-gate findings on an established repo-wide mocking pattern
this PR's new tests also use.
* fix(agents): guard list-position credential restore against misassignment
Two more real gaps Greptile/veria found in the recursive redact/restore
mechanism, verified directly against the exact reported shape
(litellm_params.model_list, each entry carrying its own nested
litellm_params.api_key/aws_secret_access_key) before fixing:
- Positional restoration inside a list could attach one entry's stored
credential to a different entry if the list were reordered or resized
between GET and PUT/PATCH. Restoration by index now only fires when the
incoming and existing entries match on every non-secret field; otherwise
the caller's own value is used (never a guessed cross-entry secret).
- A subtree collapsed to the flat REDACTED_BY_LITELM marker by the
read-side recursion depth cap couldn't be recovered on write (the marker
string itself would get persisted). Restore now recognizes that shape and
recovers the whole existing subtree.
Both covered by regression tests mirroring the exact model_list shape
reported, mutation-verified.
* fix(agents): simplify list-entry credential restore to positional matching
The content-match guard from the previous commit fixed one Greptile
finding (cross-entry misassignment on reorder) but introduced a worse one:
it also rejected restoration whenever an entry's own non-secret fields
changed, which is the common case (rename a model_list entry while
leaving its own secret masked) -- silently dropping the stored credential
on an ordinary edit.
There is no stable per-element identity in a plain dict[str, object]
schema, so no rule can satisfy both 'restore whenever the entry itself
only had its secret masked' and 'never restore across a reorder' at once.
Positional correspondence is what every other part of this restore (and
the endpoints' full-replace-on-PUT semantics) already assumes, so drop
the content-match gate and rely on it here too: this fixes the common
case correctly and accepts cross-entry misassignment on a simultaneous
reorder-plus-masked-echo as a known, narrow, documented limitation (not a
leak between different agents or tenants, since it only reshuffles one
agent's own stored values). Tests updated to pin the accepted trade-off
explicitly rather than asserting it away, and to cover the previously
broken ordinary-edit case.
Adds a configurable password-strength policy (default: min 12 chars,
upper/lower/number/special, all individually toggleable, floored at 8
so a misconfigured minimum cannot disable the length check, and
unicode-aware so an accented letter cannot satisfy the special-
character requirement) enforced on every path that sets a local
user's password: /user/update, /user/bulk_update, and the invitation
onboarding claim flow.
Adds general_settings.disable_password_login_when_sso_enabled, which
rejects username/password login on /login, /v2/login and /v3/login
(including the UI_USERNAME/UI_PASSWORD admin fallback) once ANY
configured SSO provider is FULLY ready: every companion secret/
endpoint an OAuth provider needs, checked independently per provider
so a stray leftover client id for an unused provider can't mask a
different, fully configured one; and for SAML, the optional
python3-saml runtime being importable, checked without letting a
fully-missing package's ModuleNotFoundError take down password login
itself. SSO becomes the enforced boundary for interactive UI access
without an incomplete, mixed, or half-installed SSO setup locking
every admin out or breaking login outright. Master-key API access is
untouched, and unsetting the setting plus a restart restores password
login as the documented recovery path.
* fix(security): restrict and validate file uploads at /v1/files and /upload/logo
Extends fast-fail upload validation to every purpose at POST /v1/files,
not just purpose=batch: a configurable max_file_size_mb size cap and a
blocked_file_extensions denylist, plus rejection of filenames carrying a
directory-traversal component before anything is read, stored, or
forwarded to a provider.
Also fixes two concrete gaps found while auditing every upload surface:
the Azure Blob Storage backend derived a blob path's extension with
filename.split(".")[-1], which does not parse path structure and let a
crafted filename embed a directory traversal sequence into the stored
blob path; and POST /upload/logo (the admin UI logo upload) had no
role check at all, so any authenticated API key, not just a proxy
admin, could write a file to the server's disk.
* fix(lint): drop cast()/mutation from settings coercion, sync blocked_file_extensions on reload
Replaces the TypeAdapter+cast() reads of max_file_size_mb and
blocked_file_extensions with small isinstance-based validators, since the
codebase's cast() budget (LIT006) had no headroom left. Also adds the
blocked_file_extensions reload block that was missing from
_update_general_settings: it was registered as an editable setting but
never re-synced into runtime state, so a value set through the DB-backed
settings editor would silently never take effect (Greptile finding).
* fix(security): declare max_file_size_mb and blocked_file_extensions on ConfigGeneralSettings
The DB-backed general-settings update endpoints validate every field
through ConfigGeneralSettings.model_fields before persisting it, so
without these declarations an operator could never actually set either
setting through that path even though both were registered for the
Admin UI's settings editor and reloaded on config refresh (Greptile
finding). blocked_file_extensions is typed as a tuple, not a list, to
stay out of the immutable-collections lint budget; the stored JSON
value is unaffected since the raw request payload, not the validated
model, is what gets persisted.
* chore: regenerate schema.d.ts for the new ConfigGeneralSettings fields
* fix(security): normalize configured blocked_file_extensions casing
check_blocked_extension lowercased the uploaded filename's extension
before comparing but compared it against blocked_extensions verbatim,
so an admin-configured blocked_file_extensions: ['.EXE'] would never
match an uploaded payload.exe (Greptile finding). Normalizes the
configured values the same way at comparison time, and adds the
missing case (mismatched-case config, lowercase upload) as a
regression test, mutation-checked against the unfixed comparison.
* fix(security): restore caller-owned stream position after size inspection
_file_size_bytes unconditionally seeked back to 0 after measuring a
BinaryIO's length, discarding wherever the caller had actually
positioned it (Greptile finding). Saves and restores the original
position instead. Rewrites the existing test that had encoded the
old "always resets to 0" behavior as its expectation, and adds a
sibling case for the under-cap path; both are mutation-checked
against the unfixed always-reset-to-0 behavior.
Carries a mutable-ok suppression on the router session rewrite for the
tightened LIT002 budget, since the realtime callees deep-copy and
JSON-dump the session, and captures the realtime session kwargs through
an async mock in the router tests instead of an untyped dict.
parseDynamicAgentForForm recovered a credential field's value from a
stored model string by splitting both the model_template and the model
on "/" and matching by array index. That breaks for any placeholder
value that itself contains "/", such as a Bedrock AgentCore runtime ARN
resource path (runtime/<runtime-id>), silently dropping everything
after the first slash when populating the edit form. Saving without
touching the field then persisted the truncated ARN.
Replace the index-matching split with a non-mutating template parse
(split on the placeholder pattern, escape and rejoin the literal
segments into a regex) so a placeholder captures everything it needs
regardless of embedded slashes. Also add a lightweight ARN-shape
validator for the AgentCore runtime ARN field, guarded against a
malformed pattern string, so a truncated value is rejected client-side
before it reaches the backend.
Resolves LIT-6737
* fix(proxy): share per-model budget counters across replicas through the spend counter cache
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): keep the shared fake Redis store immutable
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Native AgentCore A2A always sent either a fresh generated runtime session id
or the single configured runtimeSessionId, so related turns lost context and
unrelated callers shared one AgentCore microVM. The runtime session id is now
params.message.contextId scoped to the calling key hash, then runtimeSessionId,
then generated, and is length-validated (33-256) before the header is signed.
Invalid ids surface as JSON-RPC -32602 / HTTP 400 instead of a 500.
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
When the headroom_retrieve tool is exposed to a client that runs its own
tool-execution loop (the LiteLLM MCP gateway path), the client executes the
retrieve call and sends the recovered original content back as a tool result
on the next turn. The guardrail then compressed that row again, and because
CCR is content-addressed it collapsed back to the exact same hash it was just
retrieved from. The model never saw the expansion and the agent looped.
Hold tool-result rows that carry headroom_retrieve output back from the
compression service, the same way the live turn and trailing tool exchange are
already protected, so the expansion survives. Retrieve calls are matched by the
direct headroom_retrieve name and the mcp__<server>__headroom_retrieve gateway
name. Because a long gateway name is truncated past 64 chars in the
OpenAI-translated view the guardrail scans, the pairing also falls back to the
tool-call id read from the request's own untranslated messages, which is never
truncated.
Fixes#38558
