The managed files hook's content read looped the file's model mappings and asked each deployment for the file. A file LiteLLM stored itself maps every model to its storage url, so the read sent that internal id to the upstream server, failed, and the batch rate limiter failed open: a key's TPM limit did not apply to a LiteLLM-executed batch. The hook now returns the stored bytes from the file's storage backend before it consults any deployment
The proxy used to boot with no master key (every request accepted without
authentication) and with sk-1234, the key every example used. It now stops at
startup, before it connects to the database, and prints how to fix it: where the
bad key came from, a copy-pastable command that generates a secure key, and,
when the public key is also encrypting a database, a link to the rotation guide
general_settings.dangerously_allow_unsafe_proxy: true or
LITELLM_DANGEROUSLY_ALLOW_UNSAFE_PROXY=true starts the proxy anyway, for local
development. CI and test boots that rely on sk-1234 or on no key set it
BREAKING CHANGE: deployments with no master key, an empty one, or sk-1234 no
longer start until they set a real key or opt in to the override
The resource and data source links on the provider's registry docs
overview page 404 when clicked. They are written as relative paths like
./resources/team, and the registry serves the overview at
.../latest/docs with no trailing slash and passes hrefs through
unrewritten, so the browser resolves them to .../latest/resources/team.
Drops the link markup and keeps both lists and their descriptions. No
relative form works in both places: only a docs/-prefixed target
resolves correctly on the registry, and that same path is wrong when
reading the file on GitHub. The registry sidebar already links every
resource and data source for the version being read.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The last-known org copy was written only on get_org_object's DB-read path. The
virtual-key auth prefetch fills the same 5s org entry directly, so with keys and
JWTs of one org on the same worker the JWT lookup always hit the cache, never
wrote the copy, and a DB outage turned that JWT traffic into 503s again.
get_org_object_for_request now writes the copy itself whenever this worker holds
none, under the management-object TTL, and get_org_object is back to its shape
on main.
An explicit target_storage=litellm_db upload was accepted for any model, so an OpenAI model's litellm_db://<uuid> id was sent to OpenAI as input_file_id and a model-less upload left a content row nothing can read; it now answers 400 on target_storage. An explicit target_storage skips the files api probe and the purpose and single-target gates, which only decide whether LiteLLM keeps the file itself, so an azure_storage user_data upload for a vLLM model reaches the storage path again as it did before this branch. cancel_batch authorizes the model of every LiteLLM-managed batch id before it branches, the way retrieve_batch already does, so the LiteLLM-executed branch gets the check its provider sibling had. Restores the test_afile_delete_passes_trusted_model_credentials_to_router definition line an earlier commit dropped
Peer pods gate the settings poll on general_settings.supported_db_objects,
which validates against SupportedDBObjectType. Without a member for this
name an operator could not opt in, so a configured allowlist left every
pod but the one that served the write on stale settings.
Also types the dashboard's settings payload off the generated schema
instead of Record<string, any>.
Shipped proxy configs now read general_settings.master_key from
os.environ/LITELLM_MASTER_KEY, the .env examples ship a blank value with
the openssl generate command above it, and READMEs, the missing env vars
page and Admin UI code snippets show a generate command or the
<your-master-key> placeholder instead of the literal sk-1234
The two CircleCI docker runs that mount proxy_server_config.yaml and
oai_misc_config.yaml now pass LITELLM_MASTER_KEY so their runtime key is
unchanged
Greptile flagged the unannotated list and append against the repository's
immutable-state and Final-local rules (LIT001/LIT010). Recording the call on an
AsyncMock removes the accumulator entirely and matches how the neighbouring
audit-log tests in this file read their captured arguments.
A JWT whose team sits in an org resolves the org on every request, and the
org row is cached for only DEFAULT_IN_MEMORY_TTL seconds while the team and
user rows ride the 60s management-object TTL. A few seconds into a database
outage the org lookup failed closed and that traffic got 503s while the same
request through a virtual key kept succeeding on its cached team.
get_org_object now also keeps a last-known copy of the org row under the
management-object TTL, and get_org_object_for_request serves that copy when
the database is unreachable, so JWT traffic degrades the same way the team
lookup does. A missing copy keeps the previous behaviour: fail closed unless
allow_requests_on_db_unavailable is set.
Two tests in the config/misc management suite were failing every run against
the Buildkite e2e stack, and one of them took the rest of the build with it.
test_add_allowed_ip_does_not_store_unrelated_config_value posted 127.0.0.1 to
/add/allowed_ip. That route sets the live general_settings["allowed_ips"] that
auth_utils._check_valid_ip reads before it persists anything, and the check is
exact string membership with no CIDR support, so from the moment the POST
returns only 127.0.0.1 can reach the proxy. The runner 403s on its very next
call, and the deferred /delete/allowed_ip sits behind the same auth dependency,
so the cleanup is locked out too and every later test in the build 403s. Build
254's first attempt lost 459 of its 465 failures to that one cascade.
There is no safe way to exercise the route against a shared proxy: nothing
reports the caller's address as the proxy sees it, so a test cannot allowlist
itself first. Move the claim to the route's own TestClient suite, where the
auth dependency is overridden and general_settings is per-test, and record the
route in the module docstring beside /cache/settings and the Vault override so
it is not re-added. save_config's end of the contract was already covered by
test_ProxyConfig_save_config_merges_changed_keys_without_copying_file_settings;
the new test covers the route's end, that what it hands save_config differs
from the loaded config in allowed_ips and nothing else.
The unrelated-key probe also only ever worked on one lane: max_parallel_requests
was added to tests/e2e/gateway/stage_mirror_ci_config.yml and never to the
Buildkite stack's config, where resolve() reports it as "unset" rather than
"config". That key is now unused, so drop it again.
test_config_update_persists_router_setting_to_get wrote router_settings.
num_retries, which both lanes declare in their config file, so the config-
ownership work correctly refuses it with a 400. Switch to retry_after, which is
declared by neither lane, is accepted by /config/update, and is reported back by
GET /router/settings. Verified against a live proxy: max_fallbacks also takes
the write but never reads back, so the read-back poll is what picks the key.
A batch upload naming a model on a LiteLLM-executed provider now checks
that the key may call that model before the upstream server is probed for
a Files API, matching the order batch create already uses. Only targets on
an executed provider are checked here, so provider-model uploads keep
their existing behavior.
File content reads and writes move out of the storage backend into
ManagedFileContentRepository, so the backend no longer queries Prisma
directly.
