* ci: run the unit_selection.sh shard files on every event instead of only fork pull requests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: rename fork-flag to unit-flag now that it applies on every event
* test: move tests/test_litellm root and small trees into tests/unit
Pure renames, no content changes. Follow-up commits in this PR fix
references, merge the three files that already existed in tests/unit,
keep live-provider tests in tests/test_litellm and wire CI.
* test: carry tests/test_litellm conftest isolation into tests/unit
Callback lists, routing fallbacks, cached HTTP clients, logger state, AWS,
proxy-URL and keychain env, and session-end client cleanup now reset for
unit tests too. The environment isolation owns its MonkeyPatch so a test's
own monkeypatch is undone before the model-cost teardown runs.
* test: merge, split and prune the moved root and small-tree tests
Merge batches/test_batch_utils.py and the chat_completions and messages
dispatch tests into the files that already existed in tests/unit. Keep
the live Gemini interactions tests, the async image-fetch format test and
the OpenAI embedding scorer test in tests/test_litellm since they need
real network or keys. Put test_router.py under tests/unit/test_router so
the existing package no longer shadows it. Delete eight tests the audit
found superseded by stronger ones kept in this move.
* ci: run the moved root and small-tree tests under their legacy flags
Add the misc and responses-caching-types flags to unit_selection.sh and
CircleCI, extend enterprise-routing and mcp-integration, and point the
legacy GHA shards, Makefile, redis-compat workflow, merge smoke manifest
and change classifier at the new paths.
* test: make the new tests/unit directories packages
tests/unit/test_package_layout.py requires every directory to carry an
__init__.py, and without one the moved and retained
test_litellm_responses_bridge.py modules collide on import.
* test: scope the unit socket block to tests/unit in shared sessions
The GHA shards collect the legacy test-path and the unit selection in one
pytest session. The unit conftest's loopback-only block leaked into legacy
modules that reach the network at import. The legacy conftest now lifts the
restriction at collect and setup time, and the unit conftest re-applies it
when collecting its own modules.
* test: move tests/test_litellm/llms into tests/unit/llms
Rename-only. Moves the provider tests and the fine-tuning fixtures they
load, mirroring the old paths. Follow-up commits merge, split and wire them.
* test: merge, split and prune the moved llms tests
Merges the Databricks chat transformation tests into the existing unit
file, keeps the tests that need real keys or the network in
tests/test_litellm, deletes the audited tests a stronger unit test
already covers, and points imports at tests.unit.llms.
* ci: run the moved llms tests under their legacy flags
The Vertex AI and All Other Providers shards keep their legacy test-path
for the retained files and add the llm-vertex-ai and llm-other-providers
unit selections. CircleCI gets matching unit jobs.
* test: make the tests/unit/llms directories packages
Adds __init__.py to the moved dirs and drops the legacy ones whose
directories no longer hold tests.
* test: drop script runners and path hacks the llms split left dangling
The __main__ runners in the split openai_like files and the Databricks e2e
runner called tests that now live in the other half of the split or were
deleted. The retained legacy halves also no longer need sys.path edits.
* test: give the shard-script tests their own GITHUB_OUTPUT
They only passed where the runner set it. The CircleCI unit job's env
allowlist drops it, so the script's redirect failed there.
* test: point the router and module-deletion checks at tests/unit
router_code_coverage and code_qa_check_tests only searched tests/test_litellm,
so the moved router tests no longer counted. The two silent-experiment tests
the audit deleted were the only direct callers of those methods; they are
replaced with tests that assert the forwarded shadow request and the
recursion guard.
* test: move tests/test_litellm integrations and secret_managers into tests/unit
Rename-only. Mirrors the old paths, including the directory conftests
and the prompt and JSON fixtures. Follow-up commits prune and wire them.
* test: prune and repoint the moved integrations tests
Deletes the 7 audited tests a stronger test in the same tree already
covers, imports the TLS sink helpers from their new conftest path, and
restores os.environ after each integrations test. Some presets write
OTEL_EXPORTER_OTLP_HEADERS straight into os.environ, and without the
legacy tree's test ordering that header leaked into the AgentOps tests.
* ci: run the moved integrations tests under their legacy flag
The integrations GHA shard and a new CircleCI job run the integrations
unit selection. secret_managers joins the misc selection.
* docs: point integrations and secret_managers references at tests/unit
* test: make the moved integrations directories packages
* test: keep the Databricks manual e2e runner and fix the SageMaker Nova run path
The Databricks e2e file is a manual script whose main() calls the tests
that were pruned, so pruning them broke the documented run. It is back to
its main version. The SageMaker Nova docstring now points at the file's
real location in tests/local_testing.
* test: move tests/test_litellm core utils, routing, responses, caching and rust_bridge into tests/unit
Rename-only. Mirrors the old paths, including fixtures, the stubtest config
and the native-route wheel script. Two files that collide with existing unit
files are merged in a follow-up commit.
* test: merge, prune and repoint the moved core, routing, responses, caching and rust_bridge tests
Merges the two files that collided with existing unit files, folding the
legacy extra case into test_is_chat_completion_cached_dict, and deletes the
9 audited tests a stronger test in the same file already covers.
Keeps what needs the network in tests/test_litellm: test_tokenizers pulls a
tokenizer from the Hugging Face hub, and the gpt2 and r50k_base tokenizer
cases download their BPE files. The unit core_utils conftest points
TIKTOKEN_CACHE_DIR at litellm's bundled encodings so the rest never depend on
import order to stay offline, and FakeSecretVault moves to a shared module
so both trees can build it.
* ci: run the moved core, routing, responses, caching and rust_bridge tests under their flags
core_utils gets a core-utils flag and CircleCI job, and its GHA shard keeps
the legacy path for the retained network tests. router_utils and
router_strategy join enterprise-routing, responses joins
responses-caching-types (minus responses/mcp, which mcp-integration owns),
caching joins caching-local and rust_bridge joins misc. The redis-compat,
test-rust, stubtest and merge-smoke paths follow the move.
* docs: point the Rust crate references at tests/unit
* test: make the moved core, routing and rust_bridge directories packages
* test: keep the no-loop DualCache batch_get_cache regression test
It runs the sync path outside any event loop, which the inside-loop test
cannot, so a change that picks the Redis client by loop state would only
show up there.
* test: keep the job's UNIT_FLAG out of the shard-script tests
* fix(url_utils): block 192.0.0.0/24 on every Python patch release
* test: move the new budget limiter tests into tests/unit/router_strategy
* test: move the new sentry scrubbing tests into tests/unit/litellm_core_utils
* test: move the new zerobus tests into tests/unit/integrations
* test: make tests/unit/integrations/zerobus a package
* test: load litellm's own tiktoken cache setup once instead of resetting it per test
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(spend): capture-rate check of LiteLLM spend against the OpenAI bill
* fix(spend): claim the alert lock after the check, NaN gauge on no rate, 180-day range cap, live settings, OpenAI adapter under llms
* fix(spend): chart the capture-rate gauge in the all-metrics dashboard and clear it when the check is removed
* fix(prometheus): record the capture-rate gauge when api_provider is an excluded label
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(proxy): honor model_info.discoverable on the model listing endpoints
A model_list entry marked model_info: {discoverable: false} is left out of GET /v1/models (OpenAI and Anthropic shapes, scope=expand and wildcard routes included), the list path of GET /v1/model/info and GET /model_group/info for every caller without the admin view, while direct requests naming the model keep routing to it. The field defaults to None so an absent flag reads as discoverable and nothing is persisted or echoed for configs that never set it.
* fix(proxy): hide flagged team models under their public name and cover the scope=expand filter
The discoverability lookup now resolves a listed name with the caller's team context, so a team-scoped deployment marked discoverable: false drops out for that team's keys under its public name instead of failing open. The scope=expand branch is now exercised by a team admin caller, and the OCI secrets test builds a real UserAPIKeyAuth instead of a spec mock that has no pydantic fields.
* perf(proxy): resolve only candidate names in the discoverable filter
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(proxy): configurable key_alias_pattern for key generate, update, and regenerate
Adds litellm_settings.key_alias_pattern, a regex every key_alias sent to
/key/generate, /key/service-account/generate, /key/update, and
/key/{key}/regenerate has to fully match. A non-matching alias gets a 400
that names the setting and the pattern. When set, it replaces the built-in
rule enable_key_alias_format_validation turns on, and the baseline
unsafe-name check still runs first. An invalid regex fails config load.
* fix(proxy): cap key_alias length under key_alias_pattern and type the test fixtures
* style(proxy): declare key_alias_pattern with a PEP 604 union
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(gateway): expose /api/event_logging/batch on the gateway allowlist
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(gateway): route /api/event_logging to gateway pods in helm and terraform
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): stop max_budget leaking between proxy_server and native_compaction tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: benchmark and gate an installed release wheel
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: simplify installed-wheel benchmark check
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(rust): add native tokenizer codec
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(tokenizer): route Python tokenization through the Rust extension
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(lint): format tokenizer call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(packaging): restore runtime dependencies and native images
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(tokenizer): preserve Python SDK behavior with Rust tokenizers
* fix(tokenizer): restore compatibility paths
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(tokenizer): count custom tokenizers directly
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(tokenizer): preserve caller-supplied Python tokenizer counts
* fix(tokenizer): reuse packaged vocabularies in the native wheel
* refactor(rust_bridge): route token counting through the catalog as RUST_OPT_IN
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(spend_tracking): compare tokenizer groups by value
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(deps): re-resolve filelock under the <4.0 pin
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(llms): align transformation override signatures with base configs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* build(rust): use fat LTO to keep the native wheel under the 35 MB limit
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(tokenizer): preserve Python defaults with opt-in Rust dispatch
* test(proxy): tolerate missing litellm.utils.Tokenizer when patching it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): patch the tokenizer dispatch function instead of the removed alias
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(tokenizer): give the Rust wrappers the tiktoken and tokenizers surface
Callers of litellm.encoding and litellm.create_tokenizer must see the same
read-only API whichever backend the catalog selects.
- OpenAIEncoding mirrors tiktoken.Encoding: n_vocab, max_token_value,
token_byte_values, encode_single_token, encode_with_unstable,
encode_to_numpy, decode_with_offsets, is_special_token, repr; the Rust
tiktoken crate keeps a Vocabulary beside each CoreBPE and reports the
requested encoding name (gpt2 stays gpt2).
- HuggingFaceTokenizer mirrors the read-only tokenizers.Tokenizer surface
(token_to_id, id_to_token, get_vocab, get_vocab_size,
get_added_tokens_decoder, num_special_tokens_to_add, padding, truncation,
encode_special_tokens, from_buffer); HuggingFaceEncoding gains the
char/word/token lookups, pad, truncate, set_sequence_id and merge.
Mutators stay on the Python tokenizer.
- from_json/from_pretrained claim the fork gate only when the huggingface
feature is compiled in; the surrogate fallback matches on the Codec.
- Tokenizer caching is keyed on the same catalog Context the dispatch runs
on; rust_tokenizer reads the encoding name without loading an encoding;
LITELLM_RUST parsing is cached.
- Drop the unused tiktoken_encoding_for_model export and Error::Download.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(tokenizer): close the exhaustive matches with assert_never
CodeQL reads a `match` over a Literal with no default arm as an implicit
`None` return. `assert_never` makes the exhaustiveness explicit for both the
HuggingFace tokenizer loader and the Rust token-counter factory.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* feat(tokenizer): derive the fast counter from the shared tokenizer
The count-only counter (`fast` feature) and the codec each parsed the same
artifact: TokenCounter took the Anthropic JSON and the tiktoken rank files
from Python while Tokenizer loaded them again. One parse now serves both.
- FastTokenizer builds from a model another loader holds: `from_shared`
takes the Arc<tokenizers::Tokenizer> the HF codec keeps, and
`from_*_pairs` take the ranks the tiktoken vocabulary already parsed.
- `FastCounter::fast_counter` in the core crate derives it from either codec;
encodings the fast scanner does not reproduce are refused.
- Native `Tokenizer.count(text, fast=False)` opts into that counter, built
once per tokenizer on first use; `TokenCounter.from_tokenizer(tokenizer,
fast=False)` replaces the JSON and rank-file constructors.
- The Python route counts over the native tokenizers the codec path shares
(`native_encoding`, `native_anthropic`) and no longer reads rank files;
the packaged Anthropic tokenizer has one loader, `tokenizer_dispatch.anthropic`.
- Public wrappers gain `count(text, fast=False)`.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Rotating through POST /key/regenerate needs a running proxy, which a refused boot does not have. The refusal now counts the stored values that decrypt under the unsafe key. When there are none it only asks for a new key. When there are some it also asks for LITELLM_MIGRATE_FROM_MASTER_KEY, and the next boot with a safe key re-encrypts them and logs that the variable can be deleted. Leaving the variable set afterwards is a no-op with one notice.
Use contextlib.suppress for the cancelled task in stop_view_setup_task, make the legacy prisma setup test inject a plain mock for the synchronous start_view_setup_task and assert it is called, and drop the docstrings the branch added to tests
The proxy used to boot with no master key (every request accepted without
authentication) and with sk-1234, the key every example used. It now stops at
startup, before it connects to the database, and prints how to fix it: where the
bad key came from, a copy-pastable command that generates a secure key, and,
when the public key is also encrypting a database, a link to the rotation guide
general_settings.dangerously_allow_unsafe_proxy: true or
LITELLM_DANGEROUSLY_ALLOW_UNSAFE_PROXY=true starts the proxy anyway, for local
development. CI and test boots that rely on sk-1234 or on no key set it
BREAKING CHANGE: deployments with no master key, an empty one, or sk-1234 no
longer start until they set a real key or opt in to the override
- apply_db_row only clears runtime values for keys the row actually changed, so an env-resolved DB-owned setting survives a reload
- DELETE /config/field/delete refuses a key the config file owns instead of silently rewriting the row
- GET /config/field/info reports the declared value of a config-owned key, not the env-resolved secret
- SettingsStore gains a short-circuiting __bool__ so truthiness checks stop at the first key
- _initialize_jwt_auth resolves os.environ refs into a local mapping instead of mutating the shared general_settings dict
- rejected_writes compares against the resolved value, matching what __setitem__ accepts
- a stored value identical to the config template is no longer reported as shadowed
- the enterprise email-settings and coordination-redis writers go through reject_config_owned_writes
The config file winning over the database was silent. An admin who had set
a value through the UI and later pinned the same key in the file saw their
stored value quietly stop applying, with nothing said at boot and nothing
said when a later write was refused.
Startup now warns once per key whose stored value differs from the file's,
naming the key and what to do about it. The refusal raised on a write to a
config-owned key carries the same sentence, so the log and the 400 read
identically, and both call out that a stored value exists and will never be
applied. The /config/update refusal gained the same detail.
Keys the file does not declare are untouched: the database still owns them,
and a stored value equal to the file's is not worth a warning.
On a fresh database where the migrations run in a separate job while the
proxy boots with DISABLE_SCHEMA_UPDATE=true, the startup view check ran
as a fire-and-forget task, used up its three 10 second retries before
LiteLLM_SpendLogs existed, and died with an unretrieved exception. The
spend views were never created, so the /global/spend routes returned 500
until the pod was restarted
PrismaClient now holds a view setup task. It polls to_regclass for the
spend-log table every 5 seconds, creates the views and loads the spend
log row count once the table is there, keeps polling if an attempt raises
while the schema is still settling, and logs an ERROR with the last
failure if nothing worked after 15 minutes. Proxy shutdown cancels the
task
The spend route e2e tests for the five view-backed routes are no longer
skipped and wait for the views through the harness convergence helper
* fix(proxy): dispatch llm_api_check moderation through during_call_hook
ProxyLogging.during_call_hook only ran async_moderation_hook for CustomGuardrail callbacks, so a
CustomLogger such as the prompt injection detector with llm_api_check enabled never called the
configured moderation model. Dispatch any CustomLogger that overrides async_moderation_hook and hand
the proxy router to every registered prompt injection detector at startup so that call can route
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(enterprise): resolve openai_moderations model at call time and default to omni-moderation-latest
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep queued moderation running past a V1 pre_call guardrail
A V1 CustomGuardrail with moderation_check pre_call returned out of
during_call_hook before asyncio.gather, abandoning already-queued
CustomLogger moderation coroutines and skipping every later callback.
Skip only that guardrail instead.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(utils): skip null tool_calls when formatting prompts for moderation hooks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Applying a database row dropped the runtime layer for every key the row
carried, including keys the config file owns. Those runtime entries hold
the env-resolved config values, so after a reload a key written as
os.environ/<NAME> read back as that literal string. The store now keeps
the runtime entry for a key the config owns and clears only the rest.
Visible as store_model_in_db silently turning itself off: the reload read
the raw reference, coerced it to False, and overwrote the resolved global.
A write into a settings store for a key the config file declares used to
land in the runtime layer and then lose to the config on every read, so
the caller saw success while nothing changed. It now raises
ConfigOwnedKeyError, and the allowed-IP routes turn that into a 400
naming the key instead of reporting success on a list they never changed.
Both allowed-IP routes now build a new list rather than mutating the one
the config layer holds, and the os.environ resolver rebuilds the config
it is given instead of writing back into it, so a reader can no longer
corrupt the raw values the store keeps for provenance.
The database reload leaves a config-owned key alone rather than writing a
normalized copy back over it, which would now raise and abort the rest of
the reconcile pass.
load_config used to return a local general_settings dict that it had
normalized in place, turning the configured role_permissions entries into
RoleBasedPermissions objects. It now returns the SettingsStore, which never
saw that write, so JWT auth received raw dicts and every request failed with
"'dict' object has no attribute 'role'" whenever role_permissions was set.
Convert the entries in the consumer instead, with a TypeAdapter, so the value
is parsed wherever it comes from. load_config keeps validating at boot, so a
malformed entry still fails startup rather than the first request.
mcp_allowed_clients entries become {alias, value} objects: the value is what the JWT claim or header must equal, the alias is the name the dashboard and logs show. The Network Settings section is renamed Allowed Clients with one alias/value row per client
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): limit repeated failed Admin UI sign-in attempts
The Admin UI sign-in endpoints accept an unbounded number of password attempts.
All three call authenticate_user, and none of them keeps any record of how many
times a given caller has already been refused, so a misbehaving or misconfigured
client can retry indefinitely at full speed.
A LoginThrottle is now a required argument to authenticate_user, so the accounting
lives at the one function all three endpoints share and a fourth endpoint cannot be
added without deciding what to pass. Failures are counted per username and source
address over a fixed window and further attempts are refused with 429 and a
Retry-After header. The check runs before the database lookup and before the
password comparison, so a refused caller does no further work.
Only genuine credential rejections count. Configuration errors do not, a refused
attempt does not extend the window, and a successful sign-in clears the bucket.
The username is case folded because the user lookup is case insensitive, so casing
cannot multiply the allowance. Both credential rejections now return one identical
message. SSO is unaffected; it never calls this function.
max_failed_login_attempts (10) and failed_login_window_seconds (900) are read from
config.yaml, with LITELLM_DISABLE_LOGIN_RATE_LIMIT to turn the accounting off. They
are deliberately not database backed, so editing YAML always wins and an operator
refused by a bad value can recover.
* test(proxy): clear failed-login TTLs when resetting the throttle between tests
* feat(proxy): harden Admin UI login throttling
* docs(proxy): clarify login throttle configuration
* fix(proxy): keep the login throttle inside the type budget and fail safe on secret errors
* fix(proxy): warn about per-worker sign-in counters without a module global
* fix: honor environment login throttle settings
* fix: satisfy login setting type checks
* refactor(proxy): keep authenticate_user within the C901 budget after merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): write failed-login counters and their expiry in one Redis call
Use RedisCache.async_increment_with_floor (a single Lua INCRBY + EXPIRE) for the
shared login counters instead of the two-step INCRBYFLOAT then EXPIRE, so a
counter can never be committed to Redis without its expiry. The repair in
_remaining_window now only covers expiries stripped out of band (PERSIST, a
restore) and uses the same atomic call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): resolve the login rate limit kill switch once per process
Reading LITELLM_DISABLE_LOGIN_RATE_LIMIT through get_secret_bool on every
unauthenticated sign-in attempt meant a hosted secret manager in read mode
was queried once per password guess, before any counter was checked
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): count failed sign-ins in Redis alone while it answers
Every worker spends one shared budget and a successful sign-in clears it for all
of them. This worker's own counter is only consulted while Redis raises, so an
outage degrades to per-worker accounting instead of switching the control off
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep counting the failed sign-ins Redis missed once it answers again
A guess is recorded in exactly one place, Redis or this worker's own store when Redis
refused it, so the count is the sum of the two. Redis is read through
async_batch_get_counts, which raises on failure, instead of async_get_cache, which
swallows it into None and read as an empty counter
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): read the shared sign-in counter through a tuple of keys
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(caching): spell out the key collections batch_get_counts accepts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): warn about per-worker login counters even without general_settings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): regenerate schema.d.ts after merging main
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): inject settings and Redis cache into LoginThrottle.from_request
Removes the runtime import of proxy_server from login_throttle so the
throttle module no longer participates in the import cycle CodeQL
flagged (py/cyclic-import). Callers pass general_settings and
redis_usage_cache explicitly; behavior is unchanged.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): drop the section banner comment from the login tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): assert separate login counter stores in the spray regression test instead of a comment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): throttle failed Admin UI sign-ins per source and source/username
Replace the username-global lockout with counters keyed by source address and by
source/username pair. Each has a fixed counting window (60s) and a separate
block TTL (300s). Blocks are soft: a correct password still signs in, wrong
passwords from a blocked key take one of 5 held slots per worker and are held
30s before a 429. Once a pair is blocked its failures stop counting against the
source. The source scope runs only when trusted_proxy_ranges is set, IPv6 is
grouped by /64, and per-source limits accept IP and CIDR overrides with
longest-prefix matching. Redis is authoritative through one Lua script per
failure, with bounded per-worker fallback when Redis raises.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): explain the internal patches in the login throttle tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): key held sign-in attempts on the source while the source is blocked
An active source block now takes precedence over a pair block, so every blocked username behind one blocked source shares the source's five held slots instead of getting five each
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): apply source overrides to IPv4-mapped IPv6 sign-in sources
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): catch only Redis failures when falling back to local login counters
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): type the login throttle's local store and pass frozen Redis script arguments
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep the sign-in hold pool from refusing a correct password
The held-attempt cap ran before the password check, so five parked wrong
guesses from a blocked source turned the soft block into a lockout for the
real user. The slot is now taken only after a wrong password, and the
pool-full refusal carries the block's remaining time as Retry-After
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): stub DATABASE_URL in the hold-pool regression test so it passes off the dev box
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(proxy): drop a comment that restated the NUM_WORKERS assignment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): match IPv4-mapped IPv6 peers against IPv4 trusted proxy ranges
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep mapped-notation trusted proxy ranges matching mapped peers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): hard-block throttled Admin UI sign-ins with no credential bypass
A blocked source, or source and username pair, is now refused with 429 before the database lookup and password check, in place of the soft block that held wrong guesses for 30 seconds and let a correct password through. The env admin credentials and the master key typed into the login form are refused like any other credential while blocked; recovery is the master key as an API bearer token, which never goes through the sign-in path
trusted_proxy_ranges: [] now means clients connect directly, so the peer address is the source and the per-source limit stays on. Only an unset or malformed value leaves the topology unknown, warns at startup and turns the per-source limit off
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): move login throttle sentinels into constants
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): derive the per-username sign-in allowance from the address limit
The per-address-and-username allowance is now half the effective address allowance, rounded up, instead of a separate max_failed_login_attempts_per_user setting. A per-address override therefore raises or effectively removes both limits for that address, and no second override table is needed
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): raise the sign-in block explicitly and type the empty settings mapping
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): round the per-username sign-in allowance down and exempt an address with an override of 0
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): import LoginThrottle under TYPE_CHECKING for the throttle helper annotation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): break ties between equivalent login limit overrides deterministically
Two spellings of one network share a prefix length, so the exemption wins the tie, then the higher limit, regardless of mapping order
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): treat a malformed trusted_proxy_ranges entry as an undeclared topology
A list with an entry that is not an address or CIDR range no longer switches
the per-source Admin UI sign-in limit on against the direct peer address, so a
typo cannot make a shared ingress address the bucket for every user behind it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): reject blank trusted_proxy_ranges entries before they are dropped
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yucheng Zhu <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The two pass-through reload tests register routes on the shared app and
never take them off, so a later allowlist test in the same xdist worker
finds routes that no component exposes. Restore the app's route list
when each of those tests finishes.
The two pass-through reload tests register real FastAPI routes on the
shared proxy app and only clean up the internal registry, so any test
running after them in the same worker sees stray /v1/kept-* and
/v1/deleted-* routes. test_component_allowlists counts those as
uncovered and fails. Snapshot app.routes and the registry up front and
restore both in a finally block.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
POST /config/update stored keys the config file owns and answered 200
while the file silently kept winning. Run the config-owned check for
general_settings, litellm_settings, and router_settings before the first
database read, the way /config/field/update already does, so a refused
request stores nothing.
The config reload re-loaded the merged settings as yaml settings, which
turned every saved router setting read-only after one tick. Read the
saved router settings row instead so database-owned values stay writable.
The integration harness seeds num_retries through /config/update instead
of the config file, which is what the effective-settings and observed
routing tests need to keep exercising a database-owned value.
