* fix(batches): register ownership for every batch create path
Since the team isolation change, the managed files hook decided whether a
response came from a create by looking for the managed input file id on it,
which only the unified input path sets. Batches created from a model-encoded
input file id, a model param, or a raw provider id with ?provider= never got
an ownership row, so they vanished from GET /v1/batches for the key that
created them.
The create endpoint now stamps a create marker on the response before the
hooks run, and the hook keys ownership registration and the batch-created
metric on that marker instead of on the input id format.
* test(batches): assert ownership registration through the managed files hook
The endpoint tests asserted the private create marker, which is wiring, not
behaviour. They now run the create through the real managed files hook and
assert the ownership row is written for the creating key on every create
path, with the unified path driven by a genuine encoded input file id
instead of patched decoders.
Batch creation snapshotted the team's organization with a direct
litellm_teamtable query on every create. Go through get_team_object
instead, which serves the team auth already cached and only falls back
to the database when the team was never cached.
The search= param on /key/list, /audit, and /spend/logs/ui, plus key_hash= on /key/list, now compare the pasted value verbatim. Only a copied key ID (the hash) matches, so a raw virtual key never needs to travel in a GET query string
Claude-Session: https://claude.ai/code/session_01Q5sbiogJzPcCRmYSbaHxZf
GET /key/list?search= matches the key hash (a raw sk- key is hashed
first) or a case-insensitive alias substring, and key_hash= now hashes a
raw sk- value too. GET /v1/memory?search= matches a key prefix or an
exact memory_id. GET /audit?search= matches id, object_id, changed_by,
or changed_by_api_key. GET /spend/logs/ui?search= matches request_id
across all time and api_key, team_id, user, end_user, session_id, or
model_id inside the date window; session grouping is skipped while a
search is active.
Claude-Session: https://claude.ai/code/session_01Q5sbiogJzPcCRmYSbaHxZf
Batch cost attribution and the legacy queue endpoint already store the
VerificationToken hash in user_api_key, but omitted user_api_key_hash.
Since v1.99 the spend-log writer re-hashes any key without that provenance
flag, so DailyUserSpend.api_key no longer joins VerificationToken and Usage
shows key-hash-... rows with null api_key_alias / user_email.
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Adds a configurable password-strength policy (default: min 12 chars,
upper/lower/number/special, all individually toggleable, floored at 8
so a misconfigured minimum cannot disable the length check, and
unicode-aware so an accented letter cannot satisfy the special-
character requirement) enforced on every path that sets a local
user's password: /user/update, /user/bulk_update, and the invitation
onboarding claim flow.
Adds general_settings.disable_password_login_when_sso_enabled, which
rejects username/password login on /login, /v2/login and /v3/login
(including the UI_USERNAME/UI_PASSWORD admin fallback) once ANY
configured SSO provider is FULLY ready: every companion secret/
endpoint an OAuth provider needs, checked independently per provider
so a stray leftover client id for an unused provider can't mask a
different, fully configured one; and for SAML, the optional
python3-saml runtime being importable, checked without letting a
fully-missing package's ModuleNotFoundError take down password login
itself. SSO becomes the enforced boundary for interactive UI access
without an incomplete, mixed, or half-installed SSO setup locking
every admin out or breaking login outright. Master-key API access is
untouched, and unsetting the setting plus a restart restores password
login as the documented recovery path.
Second pass over the highest-Any-density modules that the first pass left
untouched: guardrail hooks, the gemini and anthropic transformation layers,
the proxy spend-tracking and pass-through endpoints, and the caching clients.
Untyped `response.json()` bodies and `dict[str, Any]` request payloads are
described once at their boundary with a TypedDict or Protocol, so the fields
read downstream resolve to real types instead of Any. No cast, no type: ignore,
no noqa, and no new Any annotations.
The managed batch listing fetched one page of rows, derived has_more from
that raw fetch, then dropped every row whose stored blob would not parse.
last_id came from the survivors, so a page of corrupt or legacy rows came
back as data [], last_id null, has_more true, and a client following
last_id could not advance. The OpenAI SDK's auto-paginator, which cursors
off the last item in data, stopped silently and returned a truncated list.
Read chunks until page_size + 1 batches survive parsing and file-id
resolution or the caller's rows run out, the way the managed file listing
already does, so a page carries data and a usable cursor while parseable
rows remain and has_more only says true when another one exists. The first
chunk keeps the old page_size + 1 size so a healthy page still costs one
query; a scan that has to continue widens to the file listing's
continuation chunk and stops resolving rows once the page is full.
Project auth expands all-proxy-models, * patterns, and access-group names
to many concrete models, but the rate limiter looks quotas up by the exact
requested model name, so a quota keyed on one of those entries is never
applied. Fail loudly with a 400 instead of storing an unenforceable quota
Types provider request and response bodies at their boundaries with
TypedDicts and Protocols instead of dict[str, Any], so the untyped-to-typed
crossing is paid once per boundary rather than once per field read.
Removes 1,700 reportAny/reportExplicitAny errors and 1,953 basedpyright
errors overall, plus 310 ruff strict-rule and 106 LIT-rule violations.
No cast, type: ignore, noqa, or new Any annotations anywhere in the diff.
Ratchets the basedpyright, ruff-strict, and type-discipline budgets to the
new counts so the cleared headroom cannot silently grow back.
* fix(logging): stop billing and logging response reads as LLM calls
Retrieving, deleting or cancelling a stored response, and vector store management calls, run through the same logging lifecycle as inference. A retrieved response replays the usage of the call that created it, so every read priced it again and wrote a second spend log row for the same tokens. Non-inference calls now cost 0, report no usage, log no placeholder chat message, and get a litellm.responses_management operation name instead of reading as chat.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(responses): keep billing background response jobs after the poll
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(logging): use an empty list for read-call messages
A tuple matches no branch in the loggers that walk this value, so lunary's
parse_messages falls through to clean_message and raises AttributeError on the
success hook. An empty list reads as no messages everywhere: it satisfies the
isinstance(list) checks in newrelic, mlflow and datadog, iterates zero times in
traceloop and helicone, and is what StandardLoggingPayload.messages is typed to
hold. None would be type-legal too but is not iterable, so it trades one crash
for another in mlflow and traceloop.
* fix(otel): stop the legacy emitter reporting replayed tokens on response reads
The zeroing so far lands in the standard logging payload, which the legacy
OpenTelemetry emitter does not read for usage: it takes prompt, completion and
total tokens straight off the response object, so a retrieval span still carried
the token counts of the call that produced the response, and the token usage
histogram still recorded them. That emitter is the default, so the spend row said
zero while the trace said otherwise. The background cost poller keeps its counts,
the same exemption the pricing path already makes.
* fix(logging): keep billing a background response when its retrieval is read
A response created with background=true comes back queued and carries no usage, so
its create bills nothing. The retrieval that first sees the finished job is the only
place that job's tokens are ever visible, and pricing every read at zero therefore
loses the spend outright rather than deduplicating it. On a proxy without the
enterprise cost poller a background job ended up costing $0 end to end.
is_unbilled_non_inference_call now takes the response it is deciding about and treats
a background response the same way it already treats the poller's own read, which is
the same exemption seen from the other side. The legacy OpenTelemetry emitter's time
per output token metric picks up the read gate it was missing, so it stops dividing a
read's latency by the replayed completion token count.
* test(proxy): pass the read response to the non-inference predicate
The poller test called is_unbilled_non_inference_call with the pre-background signature, so it broke when the predicate gained the response it classifies. It now hands the predicate a foreground read, and asserts that the same read is free without the origin stamp, so the stamp is what the test proves.
* fix(otel): stop the v2 metrics recorder reporting replayed tokens on response reads
The v2 span builder sources usage from the standard logging payload, so the
earlier fix already zeroes it there. The metrics recorder reads response_obj
directly, so a responses-management read still recorded the original
generation's tokens into gen_ai.client.token.usage and divided generation time
by them for gen_ai.server.time_per_output_token.
The read still records operation and response duration, under the
litellm.responses_management operation, so it stays observable.
* fix(proxy): keep the response-cost headers on calls priced at zero
Pricing responses reads and vector-store management routes at zero dropped the whole
x-litellm-response-cost family off those replies. The header build reads a falsy zero as
a cost this response never recorded and filters it out, and a call that returns before
pricing stores no cost breakdown for the component headers to read, so a client parsing
the cost off a read got a KeyError where it had previously been handed a number.
Those calls now advertise the family at zero. Retrieving a background response, and the
cost poller's read of one, still report their real cost.
The params-taking form of the predicate moves from opentelemetry into
internal_call_metadata so the proxy header build and the OTEL recorders share one copy.
* fix(proxy): report a zero cost split only under a zero cost total
The component headers were filled from call-type membership alone, while the
total they sit beside keeps its real value when the read priced normally, so a
breakdown that had not landed by the time headers were built could advertise a
real total next to an all-zero split. The split is now reported as zero only
when the total agrees with it, and is otherwise left absent.
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Yucheng Zhu <yucheng@berri.ai>
Every repository handed its `.table` back untyped, so a dozen modules had
each grown a private `_PrismaTableActions` Protocol to paper over it. They
had drifted: some declared `update` as returning the row, others the row or
None, and none agreed on whether `find_many` was covariant
Replace all of them with a single `TableActions[RowT_co]` in
`litellm/repositories/prisma_protocols.py`, keyed to the prisma row each
repository is bound to. Query inputs stay `Mapping[str, object]` so callers
keep passing plain dicts, and `find_many` returns `Sequence` so the row type
stays covariant
Typing the nullable returns honestly surfaced paths that were already
crashing. A team admin could never edit or delete a memory entry owned by
their team: the write-auth check fed a raw prisma row to a helper that
expects the domain model, so `members_with_roles` arrived as plain dicts and
the request died as a 500 instead of applying the edit. Non-admin members hit
the same 500 in place of the 403 they were owed, so refusal and breakage were
indistinguishable. `/v2/model/info?user_models_only=true` dereferenced a
missing user row rather than returning the 400 the route already had, three
team routes dereferenced a team deleted between the read and the write, and
the agent registry dereferenced a missing agent instead of naming it
basedpyright drops 2,132 errors, 1,454 of them reportAny and 73
reportExplicitAny. The dashboard's generated types pick up `string[]` where
they had `unknown[]` for a team's members, admins and models
Staging split batch output-line costing into _safe_output_line_stats /
_compute_output_line_stats / _output_line_cost so one uncostable line can no
longer zero a whole batch, and added _provider_output_file_id so model-encoded
output file ids decode before the fetch. This branch's pass/fail counting was
written against the pre-split shape, where every None line meant a provider
failure.
Keep staging's structure and layer the counts on a three-way classification: a
provider-reported failure yields PROVIDER_FAILED, a provider-successful line
litellm cannot price yields UNCOSTABLE and stays in successful_requests billed
at $0. Without that split a litellm-side pricing gap would be reported to the
customer as a failed request and the counts would stop reconciling with the
provider's own request_counts.
Route the error-file fetch through _provider_output_file_id too, and carry the
new dataclass return through the callers staging added after this branch
forked.
