Reprice ten more retired xAI slugs (grok-3 and grok-3-mini families,
grok-4-1-fast) to the grok-4.3 rates they now bill at, with family-correct
deprecation dates. Restore cache_read_input_token_cost on the Bedrock Grok 4.6
entries so implicit cache hits bill at the cache-read rate while explicit
cachePoint stays unsupported. Drop the unsourced 1080p video rate and the
gemini/ live native-audio entry the Gemini API 404s on. Add Groq qwen3.8-27b
tool-use flags per Groq docs. Extend the xai and gemini tests to lock all of
this in
OpenAI's function-calling validator rejects tool parameters carrying
oneOf/anyOf/allOf/enum/const/not at the top level, while the ChatGPT
backend Codex talks to natively accepts them, so an MCP tool declaring a
top-level union 400s through the proxy. Merge the branches into the
object schema for OpenAI itself only, walking the namespace-nested tools
current Codex builds send, on both /v1/responses and /v1/responses/compact
Two problems, both caught in review.
The new table only landed in the root schema.prisma. Client generation reads
litellm/proxy/schema.prisma and packaging reads the copy under
litellm-proxy-extras, so the generated client had no
litellm_modelaccessgroupbudgettable and every budget read and write against
it would have failed at runtime. The root is the source of truth; both
copies are now byte-identical to it.
Nothing incremented spend:model_access_group:{group} after a call. Only the
reservation path ever wrote it, so with disable_budget_reservation the
read-time check was reading a counter nobody maintained and falling back to
the row's spend, which is cached for up to DEFAULT_MODEL_ACCESS_GROUP_CACHE_TTL.
A caller could run well past the pool inside that window, which is precisely
the case the read-time check exists to cover.
increment_spend_counters now takes the matched groups and charges them
through _init_and_increment_unreserved_spend_counter, so a group already
covered by a reservation is skipped rather than counted twice. The cost
callback sources the names with get_request_model_access_groups, the same
reader the spend writer uses.
Regenerates the lazy OpenAPI snapshot and dashboard schema.d.ts from the merged tree.
Passes list/dict copies into NewMCPServerRequest from the connector conversion and switches the
server-managed NewMCPServerRequest fields to keyword defaults so the basedpyright budget gate stays flat.
Multi-window budgets (budget_limits on keys/teams) currently keep window
spend only in cache. Every cold or expired counter recomputes the window
by aggregating LiteLLM_SpendLogs, which has no usable index for that
query and saturates the DB on large tables (#35766).
This adds a LiteLLM_BudgetWindowSpend table holding one row per
configured window, keyed (entity_type, entity_id, window_duration),
with window_start identifying the period the spend belongs to.
Follow-up PRs maintain these rows from the spend update writer and move
window budget enforcement reads onto them.
A model access group could gate which models a caller reaches but never how
much that group of callers could spend in total. Capping a shared pool meant
setting a per-entity budget on every key by hand, which caps each key
separately and still leaves no way to read what the group cost.
Spend is attributed to a group only when the group's name appears on an
allowlist the caller was granted (key, team, team-member scope, project or
org) and that group serves the requested model. Asking for a model that
merely belongs to a group attributes nothing, because nothing about the
caller named the group. Levels are unioned rather than ranked, so a team
granted "*" whose member is scoped to one group still counts as gated by
that group.
Enforcement runs on both paths tags already use: a reservation counter on
the pre-call path and a read-time max_budget check inside the existing
concurrent budget gather, so the ceiling still holds under
disable_budget_reservation.
Adds LiteLLM_ModelAccessGroupBudgetTable, which is the only place a group is
ever a row: the groups themselves stay free-text strings in
model_info.access_groups, so a row exists only once someone gives that group
a budget. GET, PUT and DELETE /access_group/{name}/budget manage it, and
/access_group/{name}/info now carries the spend and budget alongside the
models.
Adds the durable row that a model access group budget hangs off. Model
access groups live only as free-text strings inside
model_info.access_groups, so unlike tags there is no existing row to
carry a budget_id.
Foundation only: schema, migration, repository, entity type, spend
transaction bucket, auth carrier field and registry cache keys. Nothing
reads or writes these yet.
The omitted-format path deliberately no longer dispatches through
embeddings.create, so four legacy tests now intercept at the transport or
client.post instead. Also adds a bypass error-path unit test, rewords a stale
comment and a README scope note, and ratchets the lint budgets down.
Second pass over the highest-Any-density modules that the first pass left
untouched: guardrail hooks, the gemini and anthropic transformation layers,
the proxy spend-tracking and pass-through endpoints, and the caching clients.
Untyped `response.json()` bodies and `dict[str, Any]` request payloads are
described once at their boundary with a TypedDict or Protocol, so the fields
read downstream resolve to real types instead of Any. No cast, no type: ignore,
no noqa, and no new Any annotations.
