Direct access on /model/info was read from the user record alone, so an
unrestricted user calling with a key limited to a few models saw every non-team
deployment, including ones the key gets a 403 on. Resolve the key's grant the
same way and intersect the two.
Resolving a grant now also expands access groups, which the key path needs and
the user path was missing.
Claude-Session: https://claude.ai/code/session_01XL7LBFEew4wi8gphVCDq6n
An Admin Viewer and a group whose name contains a slash both reached an
enabled Set budget action that could only ever come back 403 or 404. Gate
the row actions on proxy admin and on the name being addressable, with the
reason in the tooltip.
Model access group budgets shipped API-only, so the only way to give a group a
budget was a curl. Adds an Access Group Budgets tab under Models & Endpoints
that lists every group with the spend drawn against its shared pool, and a
modal to set, edit or clear the budget.
/access_group/list now carries each group's budget and spend inline, so the
table renders from one read instead of one follow-up request per row.
The typed rewrite made the properties recursion call .get on every child,
so a malformed schema with a string or list property value raised
AttributeError where it previously passed through untouched.
The first pass accepted any empty completion whose finish_reason was "length",
which also swallowed a fallback that produced nothing at all. Require the
response to have billed completion tokens as well, so empty content is
accepted only when the budget was demonstrably spent on non-visible reasoning.
Asserts on completion_tokens rather than reasoning_tokens because the latter
is provider-optional; with empty content and a refusal of null, consumed
completion tokens are reasoning by elimination, since visible text would be
content. Both counts are reported in the failure message.
Folds the three body accessors onto one _parsed helper instead of re-parsing
per call, and adds completion_tokens_of / reasoning_tokens_of alongside.
Multi-window budgets (budget_limits on keys/teams) currently keep window
spend only in cache. Every cold or expired counter recomputes the window
by aggregating LiteLLM_SpendLogs, which has no usable index for that
query and saturates the DB on large tables (#35766).
This adds a LiteLLM_BudgetWindowSpend table holding one row per
configured window, keyed (entity_type, entity_id, window_duration),
with window_start identifying the period the spend belongs to.
Follow-up PRs maintain these rows from the spend update writer and move
window budget enforcement reads onto them.
The LLM classifier's cost was recorded on the routing decision but never
reached any savings surface: per-request autorouter_savings stayed gross
and the session rollup recorded only the served request's spend, so
/auto_router/benchmarks overstated savings and understated routed spend.
Net the classifier cost into the savings figure at its one computation
owner and fold it into the rollup turn's spend, keeping
baseline_spend = spend + saved_spend. The response header's numeric
guard now shares the same reader.
Fixes#38816
max_tokens=64 caps reasoning plus visible output on gpt-5.5, so the fallback
target can legitimately return finish_reason="length" with empty content.
litellm-e2e build 90 hit exactly that: the response cost of $0.002005 backs
out to 64 completion tokens at gpt-5.5's $3e-05/token, i.e. the whole budget
spent reasoning about "say hi" with none left to answer. The fallback itself
worked -- 200, served by gpt-5.5-2026-04-23, x-litellm-attempted-fallbacks
present -- so the only thing that failed was an assertion about OpenAI's token
budgeting rather than about routing.
Raise the reliability helper's budget to 512 and accept empty content only
when finish_reason is "length". Empty content under any other finish_reason
still fails, so the tests keep catching a fallback that returns nothing for a
reason we do control.
The relaxed assertion lives in the helper both reliability fallback tests
share, so test_timeout_routes_to_fallback is covered too; it has the same
shape and had not tripped yet.
* fix(proxy): run SMTP send_email off the event loop with a connection timeout
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): format utils.py and update _create_smtp_connection tests for timeout
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep malformed SMTP_TIMEOUT inside the email error boundary
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: retrigger ci
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: exclude misaligned circleci coverage flag from merged codecov report
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: retrigger ci for codecov and benchmarks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: disable carryforward for the circleci codecov flag
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: exclude carried-forward coverage from the codecov patch status
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: stop carrying forward the dead circleci codecov flag
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Multi-window budgets (budget_limits on keys/teams) currently keep window
spend only in cache. Every cold or expired counter recomputes the window
by aggregating LiteLLM_SpendLogs, which has no usable index for that
query and saturates the DB on large tables (#35766).
This adds a LiteLLM_BudgetWindowSpend table holding one row per
configured window, keyed (entity_type, entity_id, window_duration),
with window_start identifying the period the spend belongs to.
Follow-up PRs maintain these rows from the spend update writer and move
window budget enforcement reads onto them.
The paginated search select diffs the input against the selected option's
label to recover what the user just typed, so a query started from a picked
value searches for the new text rather than the label plus the new text.
That diff can only express an insertion: it walks a common prefix and a
common suffix and returns what sits between them. A deletion leaves nothing
between them, so every deletion-only edit returned the empty string.
Backspacing once in a field showing a selected label therefore threw the
edit away. The empty result was stored as the query, the controlled input
re-rendered blank, and the server was asked for the unfiltered page instead
of the text the user left in the box.
An edit that yields no insertion but did change the value is a deletion, and
there the remaining text is the query the user means. Insertions and
whole-selection replacements are untouched.
Four cases in tests/e2e/quota_management/budgets, driving real OpenAI calls
through a group whose shared pool is drained to exhaustion: the spender key
stays blocked, a key that spent nothing of its own is blocked by the same
pool, a sibling group with no budget keeps serving, and the budget read
reports the spend drawn against the group.
Adds set/get/delete access group budget to BudgetClient and the four
matching rows to the coverage registry.
The Together catalog exposes only context_length, so the sync was recording
every chat model's context window as its output ceiling. New entries now carry
max_input_tokens and the legacy max_tokens from the catalog and get an output
ceiling only from a reviewed capability rule. GLM-5.2 and GLM-5.3-Flash rules
carry the documented 128K ceiling, and the 26 other inflated together_ai chat
entries drop max_output_tokens in both registry copies.
