The proxy used to boot with no master key (every request accepted without
authentication) and with sk-1234, the key every example used. It now stops at
startup, before it connects to the database, and prints how to fix it: where the
bad key came from, a copy-pastable command that generates a secure key, and,
when the public key is also encrypting a database, a link to the rotation guide
general_settings.dangerously_allow_unsafe_proxy: true or
LITELLM_DANGEROUSLY_ALLOW_UNSAFE_PROXY=true starts the proxy anyway, for local
development. CI and test boots that rely on sk-1234 or on no key set it
BREAKING CHANGE: deployments with no master key, an empty one, or sk-1234 no
longer start until they set a real key or opt in to the override
Shipped proxy configs now read general_settings.master_key from
os.environ/LITELLM_MASTER_KEY, the .env examples ship a blank value with
the openssl generate command above it, and READMEs, the missing env vars
page and Admin UI code snippets show a generate command or the
<your-master-key> placeholder instead of the literal sk-1234
The two CircleCI docker runs that mount proxy_server_config.yaml and
oai_misc_config.yaml now pass LITELLM_MASTER_KEY so their runtime key is
unchanged
POST /config/update stored keys the config file owns and answered 200
while the file silently kept winning. Run the config-owned check for
general_settings, litellm_settings, and router_settings before the first
database read, the way /config/field/update already does, so a refused
request stores nothing.
The config reload re-loaded the merged settings as yaml settings, which
turned every saved router setting read-only after one tick. Read the
saved router settings row instead so database-owned values stay writable.
The integration harness seeds num_retries through /config/update instead
of the config file, which is what the effective-settings and observed
routing tests need to keep exercising a database-owned value.
The check spins up a hypercorn TLS peer and drives the SDK's own httpx
handlers at it, so it needs litellm importable, hypercorn installed and a
loopback socket. It lived under tests/e2e, whose Buildkite runner image
installs neither litellm nor hypercorn by design (the suite drives a
remote proxy over HTTP), so every scheduled e2e build since #230 failed
to import the module and pytest reported it as a collection error. The
unit tree bans sockets, so it does not belong there either
tests/integration is the CircleCI tier built for real TCP against local
protocol peers. This adds an sdk shard to it for cases that exercise the
SDK's clients with no gateway in the path, registers the two HTTP/2
nodes in the contracts manifest, and adds the shard to the CircleCI
matrix. The test now flips the feature through LITELLM_HTTP2 (the user
surface) instead of patching module attributes, and asserts the version
the peer observed on the wire next to the one the client reports
Adds integration contracts for MCP lifecycle, protocol errors and OAuth
configuration, A2A wire versions, the OpenAI consumer path, persisted
toolsets, callback delivery, guardrail effects, configured prices, the
filtered spend ledger, and a CircleCI-owned browser flow for project
detachment, with the ASGI, browser-state, client and MCP helpers they use.
Consolidates the eleven commits previously stacked on
litellm_integration_providers onto its rebased tip
Both readiness loops broke out on success and fell through on timeout, so a
mock Presidio server that failed to bind left the run going with nothing
serving /analyze. The guardrail then errored at request time and the failure
surfaced as an unrelated Playwright assertion in presidioUserStory.spec.ts
rather than as the missing fixture it actually was.
Fail the local runner with the port in the message, and add the matching wait
step to both CircleCI UI jobs, which had no readiness check at all.
The guardrail section of the release checklist is done by hand every cut:
create a Presidio guardrail through the wizard, send a sentence with PII from
the playground, then open Logs and check the guardrail caught it. Nothing
covered that path, so a break anywhere along it surfaced only when someone
happened to repeat the steps.
Adds a spec that walks it once and turns the eyeball checks into assertions.
The strongest of them is the leak check: it reads the request back and fails if
the stored prompt still carries the raw address or number, which is what the
manual step is really looking for.
The stack gains a Presidio stand-in that answers the two routes the guardrail
calls, detecting a fixed regex set with a Luhn check on card numbers. Real
Presidio's detection quality is Presidio's business, and pinning the UI lane to
it would mean two heavy containers with spaCy models on every CI run for a test
that is about LiteLLM's integration. The real analyzer stays covered in the
Python lane. The stand-in returns the same entities at the same spans as the
real one for the checklist's sentence, and driving the real guardrail against it
produces the same record shape, so a test written against it is written against
the product's real behavior.
Making the stand-in return the text unmasked turns the spec red on the raw
address reaching the spend log, so the leak assertion reads live data.
run_e2e.sh and both CircleCI UI jobs start the stand-in alongside the mock LLM.
Its port is overridable like the others so two checkouts can run at once.
Two routers in one process shared a single handler, because the callback
manager dedupes on the class name plus the handler's public attributes and the
handler had none. The second router's requests were never counted. The handler
now carries the id of the cache it was built on, so routers with different
caches both register while the two selectors one router builds for its routing
groups still collapse into one.
Clamping a negative count back to zero used SET, which drops the key's TTL, so
the next write started the hour over. It uses INCRBY by the negative amount now,
which leaves the expiry alone.
The Lua script had no test that ran it, so tests/local_testing covers both the
sync and async paths against a real Redis, and the file is wired into the
CircleCI job that provides one.
Drops `-x` from all 24 pytest invocations in .circleci/config.yml. With
`-x`, a job stops at its first failure, so a second broken test in the
same suite stays invisible until the first is fixed and CI is re-run.
That turns one round trip into N when a job has several broken tests.
This is exactly what happened in #39770: fixing
test_missing_model_parameter_curl in
tests/store_model_in_db_tests/test_openai_error_handling.py immediately
unmasked test_chat_completion_bad_model_with_spend_logs in the same
file, which had been failing for a long time without ever being
reported.
Only `-x` is removed; -v/-vv/-s/-n/--reruns and every other flag are
untouched.
This reverts merge commit 2b1bd20834 (#31125)
Two CircleCI jobs on the staging-to-main promotion went red the moment
that PR landed. proxy_multi_instance_tests boots two proxies against one
database, and both now race the same migration:
Error: P3018 A migration failed to apply
Database error code: 40P01, deadlock detected
Process 73 waits for ShareLock on virtual transaction 4/11;
blocked by process 75. Process 75 waits for ExclusiveLock on
advisory lock [16384,0,72707369,1]; blocked by process 73
Neither proxy comes up, so the job times out after 300s waiting on
localhost:4000. The same wait took 36.5s on the last green run
Timeline: #31125 merged at 18:46:14Z and the failing run started at
18:49:59Z. The merge commit is not an ancestor of the last green
revision (194a3cc) and is an ancestor of the first failing one
(01de2837)
The v2 resolver was meant to avoid exactly this class of contention, so
the deadlock looks like a bug in it rather than a reason to abandon it.
Putting the default back to v1 buys time to fix it without holding up
the release
The v2 resolver skips the diff-and-force recovery that caused schema
thrashing when two LiteLLM versions contend for one database during a
rolling deploy. The standalone migration Job already defaulted to v2; this
aligns the proxy-server path.
v1 stays reachable two ways: --use_legacy_migration_resolver on the CLI, and
USE_V2_MIGRATION_RESOLVER=false for containerised deploys, where
prisma_migration.py calls run_server with a fixed argv and the env var is the
only route in. --use_v2_migration_resolver still parses, so existing commands
do not die on an unknown option.
Because v2 fails fast where v1 retried every failed deploy, a database that is
not accepting connections yet, or another instance holding the migration
advisory lock, would now kill a boot that used to ride it out. Those two
failures are retried, with Prisma's stderr logged each round, and still raise
once the attempts are spent.
Moves the resolver tests from litellm-proxy-extras/tests, which no CI job
runs, into tests/litellm-proxy-extras, and repoints the dedicated Postgres
CircleCI job at the legacy path so v1 keeps real-DB and proxy-boot coverage.
