The managed files hook's content read looped the file's model mappings and asked each deployment for the file. A file LiteLLM stored itself maps every model to its storage url, so the read sent that internal id to the upstream server, failed, and the batch rate limiter failed open: a key's TPM limit did not apply to a LiteLLM-executed batch. The hook now returns the stored bytes from the file's storage backend before it consults any deployment
- apply_db_row only clears runtime values for keys the row actually changed, so an env-resolved DB-owned setting survives a reload
- DELETE /config/field/delete refuses a key the config file owns instead of silently rewriting the row
- GET /config/field/info reports the declared value of a config-owned key, not the env-resolved secret
- SettingsStore gains a short-circuiting __bool__ so truthiness checks stop at the first key
- _initialize_jwt_auth resolves os.environ refs into a local mapping instead of mutating the shared general_settings dict
- rejected_writes compares against the resolved value, matching what __setitem__ accepts
- a stored value identical to the config template is no longer reported as shadowed
- the enterprise email-settings and coordination-redis writers go through reject_config_owned_writes
vLLM serves no /v1/files or /v1/batches, so a hosted_vllm deployment can never
host a batch. Batch inputs for such a deployment now land in a LiteLLM-owned
storage backend, the batch is executed line by line through the deployment's
own chat, completion, embedding, or responses route, and the batch plus its
output and error files are served back from the database under the creating key
Adds a projects entry to the team_admin_editable_team_fields setting. When set, team admins (legacy admins list or members_with_roles role admin) can call /project/new and /project/update for the teams they administer. The two routes join self_managed_routes so the endpoint check runs instead of the route gate's blanket 401. /project/delete stays proxy admin only
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Two callers only want the provider's own id behind an advertised one, and both
reached past the class to get it out of the decrypt tuple. The poller also typed
its rows as the pydantic projection, which has no `id`, while it is handed a
Prisma row and reads `job.id` six times.
Claude-Session: https://claude.ai/code/session_01RHAjRxNhXTpKHeGMZ1nDKi
The poller narrated its own straightforward behavior in seven multi-paragraph
docstrings, which the repo's comment policy rules out. Each is now the claim a
reader needs to avoid a wrong edit and nothing more. Also drops the deprecated
`Dict` and `Optional` aliases the file still used.
Claude-Session: https://claude.ai/code/session_01RHAjRxNhXTpKHeGMZ1nDKi
model_object_id is documented as "the id returned by the backend API
provider", and that is what batches and fine-tuning jobs store there. The
background responses create stored the advertised id in it instead, which
is encrypted with a fresh nonce on every call, so the row had no stable
handle on the generation it describes.
The cost poller now reads the provider id straight off the row. Rows
written before this still carry the advertised id there, and decrypting is
a no-op on an id that is already the provider's, so both shapes resolve
through the same call.
Claude-Session: https://claude.ai/code/session_01RHAjRxNhXTpKHeGMZ1nDKi
The previous commit stored the finished ResponsesAPIResponse in file_object. That
duplicates content the provider still serves from its own copy, and the usage and
spend it was meant to preserve already land in LiteLLM_SpendLogs on every billed
call regardless of store_prompts_in_spend_logs, which gates only the messages and
response body columns.
The poller now writes status alone, as it did before. The write stays per job
rather than one bulk update so a single failure cannot strand the rest of the
cycle.
Claude-Session: https://claude.ai/code/session_01Hn5E8Jz1LjGLFyiYxBRcBW
Every pod and uvicorn worker schedules its own CheckResponsesCost against the
shared LiteLLM_ManagedObjectTable. The poller selected eligible rows, performed
the billed retrieval, and only then marked them completed in one bulk write, so
two pollers could select the same terminal response and both record a charge
before either completion update landed.
Each row is now claimed with a compare-and-swap on batch_processed before the
read, because the read is what prices the job: aget_responses stamped with the
poll origin writes the spend log itself, so there is no later point at which to
serialize. A row whose read raised, or whose provider status is still
non-terminal, releases its claim so a later cycle retries it rather than
retiring it unbilled. That is the failure #37050 fixed on the batch side.
A pod that dies between winning the claim and billing would otherwise strand the
row: it holds a claim nobody will release and its status never reaches terminal,
so every later cycle re-selects it and loses. The updated_at arm of the claim
takes such a row back after three poll cycles, and since updated_at is @updatedAt
a healthy in-flight claim written moments ago is never stolen.
The poller now also persists the finished response onto its managed row instead
of writing status alone, so the row carries the generation's usage rather than
the stale queued copy stored at create time.
Reuses the existing batch_processed column, so no migration. It already sits on
the shared table defaulted to false and was unused by response rows.
Claude-Session: https://claude.ai/code/session_01Hn5E8Jz1LjGLFyiYxBRcBW
Narrow or remove reportAny / reportExplicitAny sites in provider
transformations, caching, guardrails, proxy endpoints and enterprise
batch-cost polling. Public parameters widen to Mapping/Sequence rather
than dict/list so no caller signature breaks, and runtime behavior is
unchanged.
* fix(hide-secrets): restore credential coverage lost to the 4.5 entropy limit
Shannon entropy is bounded by log2(length), so the 4.5 limit #39879 shipped
cannot score any value shorter than 23 characters, and it catches a random
32-character base64 credential only about two thirds of the time. A line like
REDIS_PASSWORD=aB3dE6gH9jK2mN5p therefore reaches the provider in the clear.
Add a keyword plugin that yields the credential-shaped value assigned to a
credential-named key, reusing detect_secrets' own maintained denylist so
camelCase, snake_case and SCREAMING_CASE all work with no local word list, and
re-run the assignment-quoting transform detect_secrets skips once its first
pass has matched.
The entropy limits are untouched, so #39879's false-positive fix still holds.
* fix(hide-secrets): read the assignments in a prompt that is mostly prose
configparser aborts the whole parse on the first line it cannot read, so a
message like "Here is my config, can you review it?" followed by
REDIS_PASSWORD=... lost every assignment to that one prose line. Hand the
parser only the lines it can read, dedent the assignments inside a pasted
config, and keep each key distinct by line number so a config naming api_key
once per model keeps every value instead of only the last.
* fix(hide-secrets): drop the plugin docstrings and pin the block-scalar shapes
* fix(hide-secrets): keep a comment or an indented header from closing an open value
* fix(hide-secrets): drop the explanatory comments from the new scan helpers
* fix(hide-secrets): accept punctuation in a credential value
The value filter only allowed the URL-safe Base64 alphabet, so a password
such as hunter2!brahms or p@ssw0rd!2026 passed through unredacted while
the upstream keyword plugin had already matched it. The filter now rejects
only whitespace and brackets, which keeps function calls, subscripts and
sentences out while letting symbol-heavy passwords through.
* fix(hide-secrets): redact every credential on a line and skip timestamps and plain urls
replaces the inherited first-match scan with finditer over every keyword
match, drops iso 8601 timestamps and userinfo-free urls from credential
values, and threads the parser's open-option state through
itertools.accumulate instead of rebinding it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): scan the first token of an assignment and ignore surrounding punctuation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(hide-secrets): drop the unreachable configparser error fallback
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): keep prose after a credential key out of the keyword detector
A bare value followed by ordinary words (secret_sauce: Worcestershire sauce)
is prose, so the synthetic assignment is only built when the value stands
alone or is followed by a shell operator, comment, or another assignment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): scan the first token of shell-style assignments regardless of what follows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): keep spaced assignments in scope when shell text follows the value
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(hide-secrets): drop docstrings that restate the test names
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): stop reading a comparison operator as a trailing assignment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): keep dashed flags as assignment trailers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Converge on staging's delete plumbing (_S3DeleteContext read from the logging call's additional_args, _sign_s3_request_without_body, the credential-stripping delete_data in the managed-files hook) and keep this PR's listing support, the 400 mapping for out-of-bucket file ids, the proxy-admin-only raw cloud id rule, and the OpenAI FileDeleted delete response.
Two staging tests move to this PR's contract: an out-of-bucket delete raises BedrockError 400 instead of ValueError, and deleting a stored provider output returns FileDeleted rather than the stored file object.
GET /v1/files through a provider config now returns the OpenAI page shape
(object list, data, first_id, last_id, has_more) instead of a bare array, and
DELETE /v1/files/{id} on a managed id answers the OpenAI FileDeleted shape with
deleted true instead of an empty body
Bedrock listing asks S3 for max-keys=0 when the purpose is one Bedrock never
stores under LiteLLM's prefixes, and batch_output listing no longer requires an
input bucket when only s3_output_bucket_name is configured. The mock request
behind the 400 for a foreign file id uses the same https://litellm.ai URL the
exception module uses
* fix(hide-secrets): stop redacting benign identifiers and make redaction deterministic
The OpenAI key detector matched `sk-` anywhere inside a word, so `<task-notification>`
became `<ta[REDACTED]>`, and the Base64 entropy limit of 3.0 flagged ordinary quoted
identifiers such as `"application/json"` and model ids. Redaction also iterated a
hash-seeded set, so the same request produced different bytes on different workers and
broke prompt caching.
- require a standalone `sk-`/`sk_` token with a digit (still catches sk-proj-/sk-ant-)
- raise Base64HighEntropyString limit from 3.0 to the detect-secrets default 4.5
- redact overlapping matches longest-first in a stable order
Resolves LIT-7049
* fix(hide-secrets): treat separators as key boundaries and defer sk_live_ to the stripe detector
The standalone-token boundary also rejected keys glued to a preceding `_`, `-`
or percent-encoded delimiter (`openai_sk-…`, `key-sk-…`, `Bearer%20sk-…`), which
the old pattern redacted, and `sk_live_…` was counted by both the OpenAI and the
Stripe detector.
* fix(hide-secrets): keep the openai key scan linear on repeated sk separators
The digit requirement was a lookahead, so every `sk` inside a long
`[a-zA-Z0-9_-]` run re-scanned the rest of that run looking for a digit.
100 KB of `-sk-` took over 5s in the worker's event loop and the proxy
closed the connection without a response. The check now runs once per
match in `analyze_string` instead.
* chore(hide-secrets): remove redundant performance test comment
* fix(hide-secrets): consume complete openai key tokens
* chore(hide-secrets): remove redundant fixture comment
* chore(hide-secrets): remove redundant test docstrings
* fix(hide-secrets): redact whole stripe live keys
* style(hide-secrets): wrap secret sorting key
* fix(batches): register ownership for every batch create path
Since the team isolation change, the managed files hook decided whether a
response came from a create by looking for the managed input file id on it,
which only the unified input path sets. Batches created from a model-encoded
input file id, a model param, or a raw provider id with ?provider= never got
an ownership row, so they vanished from GET /v1/batches for the key that
created them.
The create endpoint now stamps a create marker on the response before the
hooks run, and the hook keys ownership registration and the batch-created
metric on that marker instead of on the input id format.
* test(batches): assert ownership registration through the managed files hook
The endpoint tests asserted the private create marker, which is wiring, not
behaviour. They now run the create through the real managed files hook and
assert the ownership row is written for the creating key on every create
path, with the unified path driven by a genuine encoded input file id
instead of patched decoders.
