* fix(hide-secrets): restore credential coverage lost to the 4.5 entropy limit
Shannon entropy is bounded by log2(length), so the 4.5 limit #39879 shipped
cannot score any value shorter than 23 characters, and it catches a random
32-character base64 credential only about two thirds of the time. A line like
REDIS_PASSWORD=aB3dE6gH9jK2mN5p therefore reaches the provider in the clear.
Add a keyword plugin that yields the credential-shaped value assigned to a
credential-named key, reusing detect_secrets' own maintained denylist so
camelCase, snake_case and SCREAMING_CASE all work with no local word list, and
re-run the assignment-quoting transform detect_secrets skips once its first
pass has matched.
The entropy limits are untouched, so #39879's false-positive fix still holds.
* fix(hide-secrets): read the assignments in a prompt that is mostly prose
configparser aborts the whole parse on the first line it cannot read, so a
message like "Here is my config, can you review it?" followed by
REDIS_PASSWORD=... lost every assignment to that one prose line. Hand the
parser only the lines it can read, dedent the assignments inside a pasted
config, and keep each key distinct by line number so a config naming api_key
once per model keeps every value instead of only the last.
* fix(hide-secrets): drop the plugin docstrings and pin the block-scalar shapes
* fix(hide-secrets): keep a comment or an indented header from closing an open value
* fix(hide-secrets): drop the explanatory comments from the new scan helpers
* fix(hide-secrets): accept punctuation in a credential value
The value filter only allowed the URL-safe Base64 alphabet, so a password
such as hunter2!brahms or p@ssw0rd!2026 passed through unredacted while
the upstream keyword plugin had already matched it. The filter now rejects
only whitespace and brackets, which keeps function calls, subscripts and
sentences out while letting symbol-heavy passwords through.
* fix(hide-secrets): redact every credential on a line and skip timestamps and plain urls
replaces the inherited first-match scan with finditer over every keyword
match, drops iso 8601 timestamps and userinfo-free urls from credential
values, and threads the parser's open-option state through
itertools.accumulate instead of rebinding it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): scan the first token of an assignment and ignore surrounding punctuation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(hide-secrets): drop the unreachable configparser error fallback
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): keep prose after a credential key out of the keyword detector
A bare value followed by ordinary words (secret_sauce: Worcestershire sauce)
is prose, so the synthetic assignment is only built when the value stands
alone or is followed by a shell operator, comment, or another assignment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): scan the first token of shell-style assignments regardless of what follows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): keep spaced assignments in scope when shell text follows the value
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(hide-secrets): drop docstrings that restate the test names
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): stop reading a comparison operator as a trailing assignment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): keep dashed flags as assignment trailers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Converge on staging's delete plumbing (_S3DeleteContext read from the logging call's additional_args, _sign_s3_request_without_body, the credential-stripping delete_data in the managed-files hook) and keep this PR's listing support, the 400 mapping for out-of-bucket file ids, the proxy-admin-only raw cloud id rule, and the OpenAI FileDeleted delete response.
Two staging tests move to this PR's contract: an out-of-bucket delete raises BedrockError 400 instead of ValueError, and deleting a stored provider output returns FileDeleted rather than the stored file object.
GET /v1/files through a provider config now returns the OpenAI page shape
(object list, data, first_id, last_id, has_more) instead of a bare array, and
DELETE /v1/files/{id} on a managed id answers the OpenAI FileDeleted shape with
deleted true instead of an empty body
Bedrock listing asks S3 for max-keys=0 when the purpose is one Bedrock never
stores under LiteLLM's prefixes, and batch_output listing no longer requires an
input bucket when only s3_output_bucket_name is configured. The mock request
behind the 400 for a foreign file id uses the same https://litellm.ai URL the
exception module uses
* fix(hide-secrets): stop redacting benign identifiers and make redaction deterministic
The OpenAI key detector matched `sk-` anywhere inside a word, so `<task-notification>`
became `<ta[REDACTED]>`, and the Base64 entropy limit of 3.0 flagged ordinary quoted
identifiers such as `"application/json"` and model ids. Redaction also iterated a
hash-seeded set, so the same request produced different bytes on different workers and
broke prompt caching.
- require a standalone `sk-`/`sk_` token with a digit (still catches sk-proj-/sk-ant-)
- raise Base64HighEntropyString limit from 3.0 to the detect-secrets default 4.5
- redact overlapping matches longest-first in a stable order
Resolves LIT-7049
* fix(hide-secrets): treat separators as key boundaries and defer sk_live_ to the stripe detector
The standalone-token boundary also rejected keys glued to a preceding `_`, `-`
or percent-encoded delimiter (`openai_sk-…`, `key-sk-…`, `Bearer%20sk-…`), which
the old pattern redacted, and `sk_live_…` was counted by both the OpenAI and the
Stripe detector.
* fix(hide-secrets): keep the openai key scan linear on repeated sk separators
The digit requirement was a lookahead, so every `sk` inside a long
`[a-zA-Z0-9_-]` run re-scanned the rest of that run looking for a digit.
100 KB of `-sk-` took over 5s in the worker's event loop and the proxy
closed the connection without a response. The check now runs once per
match in `analyze_string` instead.
* chore(hide-secrets): remove redundant performance test comment
* fix(hide-secrets): consume complete openai key tokens
* chore(hide-secrets): remove redundant fixture comment
* chore(hide-secrets): remove redundant test docstrings
* fix(hide-secrets): redact whole stripe live keys
* style(hide-secrets): wrap secret sorting key
* fix(batches): register ownership for every batch create path
Since the team isolation change, the managed files hook decided whether a
response came from a create by looking for the managed input file id on it,
which only the unified input path sets. Batches created from a model-encoded
input file id, a model param, or a raw provider id with ?provider= never got
an ownership row, so they vanished from GET /v1/batches for the key that
created them.
The create endpoint now stamps a create marker on the response before the
hooks run, and the hook keys ownership registration and the batch-created
metric on that marker instead of on the input id format.
* test(batches): assert ownership registration through the managed files hook
The endpoint tests asserted the private create marker, which is wiring, not
behaviour. They now run the create through the real managed files hook and
assert the ownership row is written for the creating key on every create
path, with the unified path driven by a genuine encoded input file id
instead of patched decoders.
Batch creation snapshotted the team's organization with a direct
litellm_teamtable query on every create. Go through get_team_object
instead, which serves the team auth already cached and only falls back
to the database when the team was never cached.
* Fix hide-secrets guardrail: playground redaction, UI dropdown entry, spend-log telemetry
The hide-secrets guardrail never implemented apply_guardrail, so the UI test
playground echoed secrets verbatim; it was missing from the Add Guardrail
dropdown; and it recorded no guardrail_information, so Spend Logs could not
distinguish a redacted request from a clean one.
- implement apply_guardrail (unified interface) with use_native_lifecycle_hooks
so proxied traffic stays on async_pre_call_hook (per-key opt-out and
data["prompt"] handling live only there)
- record standard_logging_guardrail_information (allow/mask + masked_entity_count)
via _process_response/_process_error; opted-out keys and legacy nameless
callback instances record nothing
- advertise hide-secrets in /guardrails/ui/add_guardrail_settings (pre_call only)
and /guardrails/ui/provider_specific_params with a config model
Resolves LIT-3548
* Fix hide-secrets passthrough telemetry and JSON config input
* fix(guardrails): validate hide-secrets object config before submit
- apply_guardrail treats empty-string-only texts as no input, so no
false allow is recorded
- the UI object field keeps raw text while editing and blocks submission
until it parses to a JSON object, instead of posting a string to an
object-only API
- supported_modes_by_provider keeps its dict[str, list[str]] value type
* fix(guardrails): record no hide-secrets telemetry when nothing was inspected
walk_user_text and the prompt redaction now report how many non-empty
strings they visited; when neither inspected anything (image-only
content, empty strings), the run records no guardrail entry instead of
an 'allow' row that counts a check which never saw any text.
OpenAIFilesPurpose was missing evals, which OpenAI documents. The upload
route validates against that set, so POST /v1/files with purpose=evals was
already being rejected, and the new listing validator extended the same
rejection to GET /v1/files?purpose=evals, turning a purpose OpenAI accepts
into a hard 400. Nothing branches exhaustively on the type, so widening it
changes no routing.
The managed-file listing test fake only understood a created_by filter. The
OR filter a key carrying both a user_id and a team_id produces, the team_id
filter a service-account key produces, and the empty filter a proxy admin
produces all fell through it and returned every row, so the shapes most real
keys send went uncovered. The fake now applies the filter it is handed, and
the listing is tested against all three, including paging an OR filter
across a cursor.
Two docstrings claimed the continuation chunk bounds what a filtered page
costs. It bounds queries per row scanned; the walk is still linear in the
rows the caller owns.
* test: drop the cwd-relative sys.path.insert calls from the test suite
TQ003 stands at 1,077 across 1,058 files, and 1,015 of them are the same shape:
sys.path.insert(0, os.path.abspath("../..")) and its deeper siblings. The
argument resolves against the working directory rather than the file, so from
the repo root, where every job runs pytest, it inserts the directory two levels
above the checkout. It has never pointed at litellm. The package is installed
into the environment anyway, which is what actually makes the import work, and
what the rule's message has said all along.
Removing them leaves 1,634 imports of sys and os with no remaining reference,
and those go too, except where another test module imports the name back out of
the file. The rest of TQ003 is 62 call sites that resolve against __file__ or a
variable, which are a different question and are left alone.
Collection is identical either way: 45,871 tests and the same 51 pre-existing
collection errors before and after, and ruff reports no new undefined name.
* test: drop the duplicate imports the sys.path sweep exposed to F811
* test(pre-call-utils): restore the os import the new bedrock tests need
The managed hook returned the plain dict build_list_page builds, while
every other GET /v1/files path returns an SDK page object. A post-call
success hook or a logging callback that reads response.data off the
listing raised AttributeError as soon as a request took the managed path
FileListPage is a pydantic model over the same five fields, so hooks read
.data again and the response body does not move: jsonable_encoder gives
the same keys in the same order for the model and for the dict. It sits
in litellm.types.llms.openai because base_llm/files/transformation.py
already imports from there and cannot import proxy modules. It is
deliberately not subscriptable, since the provider-backed path returns a
page object that is not either, and dict access would be a third contract
to keep alive
Also reject a purpose the Files API never accepts. An unknown purpose
matches no row, so the listing answered an empty page for what is really
a bad request, while the upload route in this same file already refuses
those values against get_args(OpenAIFilesPurpose). The check runs before
the first query, and only in the managed hook, so providers that define
their own purposes keep them
Also put back the route's original except tail. Sending every error
through handle_exception_on_proxy changed error.type on a bad
target_model_names from "None" to the exception class name, which a
caller matching on the body would read as a break. create_file in this
file already pairs base's tail with a ProxyException passthrough, so
list_files does the same and the handle_exception_on_proxy import is gone
* test: use monkeypatch.setenv for env writes in tests/test_litellm
`os.environ["X"] = v` inside a test leaks the value into every test that runs
after it in the same worker, so ordering decides the result. 262 of those
writes across 40 files now go through pytest's `monkeypatch` fixture, which
restores the previous value at teardown.
The rewrite skips any test that a mock.patch-family decorator wraps, any test
with defaulted positional parameters, any test whose own name is called
directly elsewhere, and rebinds nothing inside nested defs, because in each of
those cases appending a fixture parameter changes what pytest or mock binds.
Ratchets the TQ004 ceiling from 768 to 506.
* fix(test): delete the key through monkeypatch instead of popping it first
Five tests popped a key straight out of `os.environ`, ran, then restored it with
`monkeypatch.setenv`. By the time monkeypatch saw the name it was already gone,
so it recorded "absent" as the value to go back to and deleted the key at
teardown. On a worker that inherited a real `RESEND_API_KEY`, `SENDGRID_API_KEY`,
`UI_PASSWORD`, `LITELLM_SALT_KEY` or `OPENAI_API_KEY`, every test after the first
one ran without it.
`monkeypatch.delenv(..., raising=False)` removes the key and restores whatever
was there, so the try/finally the manual restore needed goes with it.
* chore(test): leave the two cost-calc files to the PR that rewrites them fully
Both files are also in #37815, which converts the module-global writes as well
as the env writes and folds them into one fixture. Two PRs rewriting the same
lines differently is a conflict nobody benefits from resolving, so this one
drops back to staging on those two and keeps the other 39.
TQ004 clears 200 here instead of 275; the rest moves with #37815.
The chunk loop read `limit + 1` rows at a time, so a small limit whose
matches sit far behind the newest rows advanced a couple of rows per
query. A `purpose` that matches only the last of 10000 owned rows at
`limit=1` cost 5001 sequential find_many calls for one HTTP request,
which any authenticated caller could ask for on purpose.
Once a scan has to continue past its first chunk, widen the chunk to
FILE_LIST_CONTINUATION_CHUNK_SIZE. That same case now costs 21 queries.
The first chunk keeps its `limit + 1` size, so a page the newest rows
already fill still costs exactly one query and reads nothing extra.
Rows whose blob will not parse drop out of a page the way a filter does,
so they get the bound too, not just the purpose filter.
The floor only changes how many round trips a page costs, never what it
returns: chunk boundaries do not affect a keyset scan, so the page is
still `matches[:page_size]`, `has_more` is still `len(matches) >
page_size`, and empty data still implies `has_more` false.
The managed file listing cut the page to `limit` first and applied the
purpose filter in Python afterwards, so a page whose rows all failed the
filter came back as `data: []` with `has_more: true`. openai-python stops
paging the moment `data` is empty, so `files.list(purpose="batch", limit=1)`
returned nothing at all instead of every batch file.
Read successive keyset chunks until the page holds `limit + 1` matches or
the caller's rows run out, then return at most `limit` of them. `data` is
now non-empty whenever matching files remain, its last id is always a
usable cursor, and `has_more: false` only ever means the caller has seen
everything. Rows whose stored blob will not parse drop out in the same
loop, so they cannot empty a page either.
That also makes the `next_cursor_id` escape hatch on `build_list_page`
dead, so it goes back to what it was for the batch and vector-store
listings that share it.
Also move `validate_file_list_limit` up into the list_files route, so the
target_model_names and provider branches reject an out-of-range limit the
same way the managed file store already did.
The unscoped GET /v1/files limit check accepted 0, which OpenAI's minimum
of 1 does not allow, and the route's except block rebuilt every error with
getattr(e, "status_code", 500). ProxyException has no status_code, so the
400 it raises went out as a 500 and the OpenAI SDK retried it three times.
Errors now go through handle_exception_on_proxy, the helper the sibling
batches route already uses, and the unknown-cursor error is a ProxyException
so it carries type invalid_request_error and param after instead of the
literal "None". The cursor still 400s whether the file belongs to someone
else or does not exist at all
A page whose rows are all dropped by the purpose filter, or by a row
that does not parse, used to come back with an empty data list, has_more
true and last_id null, so the caller had no cursor to advance with and
stopped one page short of files it owns. last_id now falls back to the
last row the page read.
Also drops the OpenAIFilesPurpose import that the widened purpose
annotation left unused.
The owner-scoped listing read every row the caller owns in one query, so an
admin key that owns every file on the proxy pulled the whole table into one
response. Page it with a keyset cursor on unified_file_id instead, and accept
limit and after on GET /v1/files so a client can walk the pages. limit follows
what OpenAI documents for that route: 1 to 10000, default 10000.
An after cursor is resolved inside the caller's own scope, so an id they do not
own gets a 400 rather than a page, and has_more now reflects whether another
row exists instead of always being false.
Refs #37714
`pytest.raises(Exception)` with no `match=` passes on any error that broad. A
TypeError from a refactor, a botched fixture, an import that moved: all of them
read as the rejection the test claims to police, so the test goes green for the
wrong reason and stays green after the behaviour it guards is gone.
PT011 closes that gap for the 317 sites B017 could not reach, because B017 only
fires on a single-statement body with no `as e` binding. Each pattern here is the
message the code actually raised, recorded by running the sites under a plugin
that logged the concrete type and text per call site, so the assertions describe
observed behaviour rather than a guess. Where a site raises more than one message
across its parametrize cases, the pattern is an alternation of what was seen;
where the exception carries an empty `str()` and puts the text on `.message`, the
site keeps a narrow `noqa` with the reason.
PT014 removes four parametrize cases that were listed twice. The duplicate re-runs
an assertion that already passed, and it usually marks a case someone meant to
vary and forgot to edit.
Every pod and uvicorn worker schedules its own CheckBatchCost poller against the
shared managed-object table, so two of them can select the same completed batch in
one polling window and both write an aretrieve_batch spend log for it, counting
that batch's cost twice.
Claim the row with a compare-and-swap on batch_processed, and skip the batch when
another pod already holds it. The claim sits immediately before the spend log is
written rather than before the results fetch, because batch_processed is also what
blocks deletion of the files the fetch reads and what keeps an unbilled row
selectable by later poll cycles, so claiming up front would strand the spend of any
worker that died mid-fetch. A failed spend log write hands the row back.
Co-authored-by: Yassin Kortam <yassin@berri.ai>
The retrieve path now defers a managed batch's accounting to CheckBatchCost, which
bills the key, team, and tags stored on the managed object row. The /v1/batches
create hook never persisted api_key or request_tags there (only the passthrough
creates did), so the poller attributed the cost to the user alone and the creating
key's spend stayed at zero.
The Anthropic batch create never persisted the creating key's hashed token or its
request tags on the managed object, so when CheckBatchCost billed the batch hours
later there was nothing to attribute it to. Key spend, key budgets and tag spend
never moved for batch usage.
Persist both from the create, the way the Vertex passthrough already does, and
register the batch only from the collection route. An id-scoped route cannot
rebuild the unified object id, because it embeds the model and the model comes
from the create's request body, so it could only claim a row it did not create or
fail the model_object_id unique constraint.
The shared metadata helpers, the route predicate and the registration-result
logging now live in batch_attribution instead of being copied per provider. The
Anthropic write previously logged success unconditionally, before the
fire-and-forget task had run.
Resolves LIT-5288
A poll of a Vertex passthrough batch wrote nothing to the managed-object row,
so status and file_object stayed frozen at the create-time snapshot and
GET /v1/batches served a stale status and an empty output file id for the life
of the batch. Only the create may claim a batch, but every observation of one
may refresh its state.
store_unified_object_id takes create_if_missing, which the poll clears: it
refreshes status and file_object through update_many, and leaves a row that is
absent absent rather than creating one owned by the observer, since created_by
and team_id are written by whoever reaches the create branch. The update payload
is now shared with the upsert so it cannot drift into writing api_key,
request_tags, created_by or team_id.
The passthrough identity re-assertion that was previously part of this PR ships
separately in #36121, so this PR keeps only the batch attribution work.
The creating key owns user_api_key_alias only when it actually has one. Guarding
the overwrite on the presence of a key rather than on a resolved alias nulled the
field out for every key generated without key_alias, and for any key rotated or
deleted before its batch finished, losing the creating user's alias that the spend
row previously carried. The guard now matches the team-alias line below it.
update_batch_in_database now fetches the batch row by unified_object_id
when the caller omits db_batch_object, so the cancel endpoint attributes
newly registered output and error files to the batch owner and returns
unified managed ids instead of raw provider ids. Idempotent cancels that
do not change the stored status also skip the redundant DB write now.
Repair two pre-existing mock tests in test_openai_batches_endpoint.py
that asserted values inside lazy percent-format log strings, and give
the cancel test's prisma mock an awaitable find_first.
The skip warning interpolated the full pydantic ValidationError, whose
string embeds input_value with the rejected row's contents. Managed-file
rows carry a caller-supplied filename, so a malformed row copied that
into operational logs.
Log the error locations, types, and messages via errors() with input,
url, and context excluded, keeping the field-level diagnostics without
the values. Non-validation failures fall back to the exception type.
get_user_created_file_ids validated every row's file_object without a
guard, so a single row failing OpenAIFileObject validation raised
ValidationError and turned the whole GET /v1/files response into a 500.
#35365 covered the null case only, leaving malformed or partial rows
able to take the entire listing down.
Rows now parse through a helper that returns None on failure and logs a
warning, matching how list_user_batches already tolerates rows it cannot
parse, so one bad row costs its own entry instead of the caller's whole
listing. Null rows stay silent since the batch cost poller registers
those legitimately.
Refs #35361
Terminal batch retrieve could return the raw provider output_file_id, which
skips managed-file ownership checks on /v1/files/{id}/content and lets any key
on the proxy download another user's batch output.
Retrieve now registers the missing managed-file row before responding, and
attributes ownership to the durable batch owner rather than the retrieving
caller, so output and error ids always come back as unified managed ids.
Fixes#33989
