Authorization and pricing read the first model and language query value, but the raw query was forwarded, so Deepgram (which honours the last repeated value) could be sent a model the key was never allowed. Later duplicates of those two keys are now dropped before the upstream URL is built
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A caller could pick a model with only a pre-recorded registry row, or no row at all, and the session would be billed at the pre-recorded rate or logged at zero cost, so budgets did not apply. The route now closes the WebSocket with 1008 before dialing Deepgram unless deepgram/streaming/<model> (or the -multilingual row for language=multi) is an exact registry hit, and the logging handler applies the same check so a registry change under a live session records the duration with no cost instead of a substitute rate
Regression tests cover the route refusal, an operator-supplied streaming row for another model being accepted, and the handler never substituting the pre-recorded rate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): make SettingsStore.clear() terminate when the config file owns a key
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): compare the cleared settings store against an unmutated refill mapping
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep the resolved values of config-owned keys across SettingsStore.clear()
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): limit repeated failed Admin UI sign-in attempts
The Admin UI sign-in endpoints accept an unbounded number of password attempts.
All three call authenticate_user, and none of them keeps any record of how many
times a given caller has already been refused, so a misbehaving or misconfigured
client can retry indefinitely at full speed.
A LoginThrottle is now a required argument to authenticate_user, so the accounting
lives at the one function all three endpoints share and a fourth endpoint cannot be
added without deciding what to pass. Failures are counted per username and source
address over a fixed window and further attempts are refused with 429 and a
Retry-After header. The check runs before the database lookup and before the
password comparison, so a refused caller does no further work.
Only genuine credential rejections count. Configuration errors do not, a refused
attempt does not extend the window, and a successful sign-in clears the bucket.
The username is case folded because the user lookup is case insensitive, so casing
cannot multiply the allowance. Both credential rejections now return one identical
message. SSO is unaffected; it never calls this function.
max_failed_login_attempts (10) and failed_login_window_seconds (900) are read from
config.yaml, with LITELLM_DISABLE_LOGIN_RATE_LIMIT to turn the accounting off. They
are deliberately not database backed, so editing YAML always wins and an operator
refused by a bad value can recover.
* test(proxy): clear failed-login TTLs when resetting the throttle between tests
* feat(proxy): harden Admin UI login throttling
* docs(proxy): clarify login throttle configuration
* fix(proxy): keep the login throttle inside the type budget and fail safe on secret errors
* fix(proxy): warn about per-worker sign-in counters without a module global
* fix: honor environment login throttle settings
* fix: satisfy login setting type checks
* refactor(proxy): keep authenticate_user within the C901 budget after merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): write failed-login counters and their expiry in one Redis call
Use RedisCache.async_increment_with_floor (a single Lua INCRBY + EXPIRE) for the
shared login counters instead of the two-step INCRBYFLOAT then EXPIRE, so a
counter can never be committed to Redis without its expiry. The repair in
_remaining_window now only covers expiries stripped out of band (PERSIST, a
restore) and uses the same atomic call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): resolve the login rate limit kill switch once per process
Reading LITELLM_DISABLE_LOGIN_RATE_LIMIT through get_secret_bool on every
unauthenticated sign-in attempt meant a hosted secret manager in read mode
was queried once per password guess, before any counter was checked
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): count failed sign-ins in Redis alone while it answers
Every worker spends one shared budget and a successful sign-in clears it for all
of them. This worker's own counter is only consulted while Redis raises, so an
outage degrades to per-worker accounting instead of switching the control off
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep counting the failed sign-ins Redis missed once it answers again
A guess is recorded in exactly one place, Redis or this worker's own store when Redis
refused it, so the count is the sum of the two. Redis is read through
async_batch_get_counts, which raises on failure, instead of async_get_cache, which
swallows it into None and read as an empty counter
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): read the shared sign-in counter through a tuple of keys
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(caching): spell out the key collections batch_get_counts accepts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): warn about per-worker login counters even without general_settings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): regenerate schema.d.ts after merging main
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): inject settings and Redis cache into LoginThrottle.from_request
Removes the runtime import of proxy_server from login_throttle so the
throttle module no longer participates in the import cycle CodeQL
flagged (py/cyclic-import). Callers pass general_settings and
redis_usage_cache explicitly; behavior is unchanged.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): drop the section banner comment from the login tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): assert separate login counter stores in the spray regression test instead of a comment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): throttle failed Admin UI sign-ins per source and source/username
Replace the username-global lockout with counters keyed by source address and by
source/username pair. Each has a fixed counting window (60s) and a separate
block TTL (300s). Blocks are soft: a correct password still signs in, wrong
passwords from a blocked key take one of 5 held slots per worker and are held
30s before a 429. Once a pair is blocked its failures stop counting against the
source. The source scope runs only when trusted_proxy_ranges is set, IPv6 is
grouped by /64, and per-source limits accept IP and CIDR overrides with
longest-prefix matching. Redis is authoritative through one Lua script per
failure, with bounded per-worker fallback when Redis raises.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): explain the internal patches in the login throttle tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): key held sign-in attempts on the source while the source is blocked
An active source block now takes precedence over a pair block, so every blocked username behind one blocked source shares the source's five held slots instead of getting five each
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): apply source overrides to IPv4-mapped IPv6 sign-in sources
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): catch only Redis failures when falling back to local login counters
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): type the login throttle's local store and pass frozen Redis script arguments
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep the sign-in hold pool from refusing a correct password
The held-attempt cap ran before the password check, so five parked wrong
guesses from a blocked source turned the soft block into a lockout for the
real user. The slot is now taken only after a wrong password, and the
pool-full refusal carries the block's remaining time as Retry-After
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): stub DATABASE_URL in the hold-pool regression test so it passes off the dev box
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(proxy): drop a comment that restated the NUM_WORKERS assignment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): match IPv4-mapped IPv6 peers against IPv4 trusted proxy ranges
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep mapped-notation trusted proxy ranges matching mapped peers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): hard-block throttled Admin UI sign-ins with no credential bypass
A blocked source, or source and username pair, is now refused with 429 before the database lookup and password check, in place of the soft block that held wrong guesses for 30 seconds and let a correct password through. The env admin credentials and the master key typed into the login form are refused like any other credential while blocked; recovery is the master key as an API bearer token, which never goes through the sign-in path
trusted_proxy_ranges: [] now means clients connect directly, so the peer address is the source and the per-source limit stays on. Only an unset or malformed value leaves the topology unknown, warns at startup and turns the per-source limit off
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): move login throttle sentinels into constants
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): derive the per-username sign-in allowance from the address limit
The per-address-and-username allowance is now half the effective address allowance, rounded up, instead of a separate max_failed_login_attempts_per_user setting. A per-address override therefore raises or effectively removes both limits for that address, and no second override table is needed
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): raise the sign-in block explicitly and type the empty settings mapping
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): round the per-username sign-in allowance down and exempt an address with an override of 0
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): import LoginThrottle under TYPE_CHECKING for the throttle helper annotation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): break ties between equivalent login limit overrides deterministically
Two spellings of one network share a prefix length, so the exemption wins the tie, then the higher limit, regardless of mapping order
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): treat a malformed trusted_proxy_ranges entry as an undeclared topology
A list with an entry that is not an address or CIDR range no longer switches
the per-source Admin UI sign-in limit on against the direct peer address, so a
typo cannot make a shared ingress address the bucket for every user behind it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): reject blank trusted_proxy_ranges entries before they are dropped
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yucheng Zhu <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A merge on this branch dropped the ^ anchor and MULTILINE flag from the
doc_key_pattern, so the unanchored match started swallowing key names
into captured fields whenever a row's description cell itself contains
a pipe (Literal unions and similar). The check then reported 27
long-documented keys as undocumented. Restores the exact pattern used
on main, verified against a live litellm-docs checkout: all 58 Router
init params resolve as documented.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
MutableMapping.clear pops items until the mapping is empty, but __delitem__
keeps config-owned keys, so clear spun forever on any store that had loaded a
config file. unittest.mock.patch.dict calls clear on exit, which is why the
proxy-infra and proxy-endpoints shards hung at 99 percent until the 20 minute
job timeout on every run since the store started refusing config-owned writes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Addresses GHSA-5p39-cfhj-2xmp and GHSA-82r6-8w77-94w6 flagged by the
osv-scan job. Hand-edited the single anyio block so the lockfile stays
byte-stable for CI's older uv version; dependency set is unchanged.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The two pass-through reload tests register real FastAPI routes on the
shared proxy app and only clean up the internal registry, so any test
running after them in the same worker sees stray /v1/kept-* and
/v1/deleted-* routes. test_component_allowlists counts those as
uncovered and fails. Snapshot app.routes and the registry up front and
restore both in a finally block.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
