Multi-window budgets (budget_limits on keys/teams) currently keep window
spend only in cache. Every cold or expired counter recomputes the window
by aggregating LiteLLM_SpendLogs, which has no usable index for that
query and saturates the DB on large tables (#35766).
This adds a LiteLLM_BudgetWindowSpend table holding one row per
configured window, keyed (entity_type, entity_id, window_duration),
with window_start identifying the period the spend belongs to.
Follow-up PRs maintain these rows from the spend update writer and move
window budget enforcement reads onto them.
A model access group could gate which models a caller reaches but never how
much that group of callers could spend in total. Capping a shared pool meant
setting a per-entity budget on every key by hand, which caps each key
separately and still leaves no way to read what the group cost.
Spend is attributed to a group only when the group's name appears on an
allowlist the caller was granted (key, team, team-member scope, project or
org) and that group serves the requested model. Asking for a model that
merely belongs to a group attributes nothing, because nothing about the
caller named the group. Levels are unioned rather than ranked, so a team
granted "*" whose member is scoped to one group still counts as gated by
that group.
Enforcement runs on both paths tags already use: a reservation counter on
the pre-call path and a read-time max_budget check inside the existing
concurrent budget gather, so the ceiling still holds under
disable_budget_reservation.
Adds LiteLLM_ModelAccessGroupBudgetTable, which is the only place a group is
ever a row: the groups themselves stay free-text strings in
model_info.access_groups, so a row exists only once someone gives that group
a budget. GET, PUT and DELETE /access_group/{name}/budget manage it, and
/access_group/{name}/info now carries the spend and budget alongside the
models.
Adds the durable row that a model access group budget hangs off. Model
access groups live only as free-text strings inside
model_info.access_groups, so unlike tags there is no existing row to
carry a budget_id.
Foundation only: schema, migration, repository, entity type, spend
transaction bucket, auth carrier field and registry cache keys. Nothing
reads or writes these yet.
The omitted-format path deliberately no longer dispatches through
embeddings.create, so four legacy tests now intercept at the transport or
client.post instead. Also adds a bypass error-path unit test, rewords a stale
comment and a README scope note, and ratchets the lint budgets down.
Second pass over the highest-Any-density modules that the first pass left
untouched: guardrail hooks, the gemini and anthropic transformation layers,
the proxy spend-tracking and pass-through endpoints, and the caching clients.
Untyped `response.json()` bodies and `dict[str, Any]` request payloads are
described once at their boundary with a TypedDict or Protocol, so the fields
read downstream resolve to real types instead of Any. No cast, no type: ignore,
no noqa, and no new Any annotations.
Open, the trace sidebar is byte-identical to before: the toggle sits over its
header exactly where it did, and the header keeps the padding that makes room
for it. Collapsed, that button has nowhere to live, so the drawer header shows
one instead, on the model row or the request id row when the log names no
model.
Both come from SidebarToggle, so they cannot drift in design. The chevrons now
point the way the sidebar will move: right while it is open, left while it is
collapsed.
Putting the toggle in the drawer header unconditionally stranded it on its own
line: the model row renders empty for a log that names no model or provider, so
the chevron sat alone above the request id.
The sidebar keeps the toggle whenever it is open, in its own header. Collapsed,
the toggle moves into the drawer header and joins the model row, or the request
id row when there is no model to join. Shared between both through
SidebarToggle so the two call sites cannot drift.
The collapsed rail kept the toggle in flow but left a 40px stub of empty
sidebar on screen. The toggle now leads the drawer header's first row, ahead of
the provider logo and the model name, so it reads as part of the header and the
sidebar goes back to unmounting when collapsed.
Still no absolute positioning and no stacking level: the button is a normal
in-flow child of the header row it sits in. DrawerHeader takes the collapsed
state and the toggle handler as props rather than reaching for the drawer's
state.
Consolidate the new regression tests into
test_openai_embedding_encoding_format_default.py, replacing mocks that
pinned the old float default with respx captures of the request body,
and update the stale local_testing default-float test to assert
omission
When no encoding_format is set on the call, the model config, or
LITELLM_DEFAULT_EMBEDDING_ENCODING_FORMAT, leave the field out of the
upstream request instead of defaulting to float, and bypass the OpenAI
SDK's own base64 default so nothing re-adds it on the wire. Downstreams
that reject encoding_format, such as a second LiteLLM proxy fronting
Bedrock Titan embeddings, now work when the client omits the field.
Fixes#38661
request_id can be chosen by the client through x-litellm-call-id, so an
unbounded NOT (request_id = ANY(batch)) let a replayed old id drop that id's
historical LiteLLM_SpendLogs row from the one-time seed while its increment
still landed. The increment now carries the request start, the batch keeps
the earliest one, and the seed only excludes ids whose startTime is at or
after it.
