Two problems, both caught in review.
The new table only landed in the root schema.prisma. Client generation reads
litellm/proxy/schema.prisma and packaging reads the copy under
litellm-proxy-extras, so the generated client had no
litellm_modelaccessgroupbudgettable and every budget read and write against
it would have failed at runtime. The root is the source of truth; both
copies are now byte-identical to it.
Nothing incremented spend:model_access_group:{group} after a call. Only the
reservation path ever wrote it, so with disable_budget_reservation the
read-time check was reading a counter nobody maintained and falling back to
the row's spend, which is cached for up to DEFAULT_MODEL_ACCESS_GROUP_CACHE_TTL.
A caller could run well past the pool inside that window, which is precisely
the case the read-time check exists to cover.
increment_spend_counters now takes the matched groups and charges them
through _init_and_increment_unreserved_spend_counter, so a group already
covered by a reservation is skipped rather than counted twice. The cost
callback sources the names with get_request_model_access_groups, the same
reader the spend writer uses.
A model access group could gate which models a caller reaches but never how
much that group of callers could spend in total. Capping a shared pool meant
setting a per-entity budget on every key by hand, which caps each key
separately and still leaves no way to read what the group cost.
Spend is attributed to a group only when the group's name appears on an
allowlist the caller was granted (key, team, team-member scope, project or
org) and that group serves the requested model. Asking for a model that
merely belongs to a group attributes nothing, because nothing about the
caller named the group. Levels are unioned rather than ranked, so a team
granted "*" whose member is scoped to one group still counts as gated by
that group.
Enforcement runs on both paths tags already use: a reservation counter on
the pre-call path and a read-time max_budget check inside the existing
concurrent budget gather, so the ceiling still holds under
disable_budget_reservation.
Adds LiteLLM_ModelAccessGroupBudgetTable, which is the only place a group is
ever a row: the groups themselves stay free-text strings in
model_info.access_groups, so a row exists only once someone gives that group
a budget. GET, PUT and DELETE /access_group/{name}/budget manage it, and
/access_group/{name}/info now carries the spend and budget alongside the
models.
Adds the durable row that a model access group budget hangs off. Model
access groups live only as free-text strings inside
model_info.access_groups, so unlike tags there is no existing row to
carry a budget_id.
Foundation only: schema, migration, repository, entity type, spend
transaction bucket, auth carrier field and registry cache keys. Nothing
reads or writes these yet.
The suggester pins temperature=0.2 for tool-selection determinism and passed no
drop_params, so an operator-supplied reasoning model whose only accepted temperature is 1
made litellm raise UnsupportedParamsError and the whole suggestion fail. The default
gpt-4o-mini is unaffected; the failure needs the caller to name a model.
Every other internal LLM call the proxy makes on a user's behalf already opts in through
judge_acompletion, which sets drop_params=True on both dispatch paths. This was the one
caller outside that contract, so the sampling preference is now advisory here too and the
call degrades instead of dying.
Resolves LIT-6352
The request and response tree passed the library's light palette in every
theme, so in dark mode the string values rendered dark green and the
punctuation rendered black on a near black surface. Pick the palette from
the resolved theme instead, and let the tree inherit the themed surface
rather than painting the library's own background.
Team-scoped deployments keep the internal model_name_{team_id}_{uuid} routing key and expose the public name in model_info.team_public_model_name. The dashboard links team model chips with the public name, so the exact filter now matches either name via the existing helper.
The where clause now uses the exact name string directly and skips the DB query when the typed search cannot occur in that name, so no new mutable literals are added (LIT002 gate).
With model=<group>&search=<term>, the router list was narrowed to the group but the DB query only matched the substring, so other groups' rows leaked into the page and total_count.
* feat(mcp): support RS256 signing for MCP gateway session tokens
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style: ruff format session token modules
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): enforce key strength on rotated public keys and unique kids
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Pass the selected group as the exact model= param on /v2/model/info rather than as the substring search, so a group like gpt-4 no longer pulls gpt-4o rows into the page and count. Drop two comments that restated helper behavior.
* fix(health): honor allow_requests_on_db_unavailable in readiness probe
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(health): bound readiness DB check and pass reconnect timeout
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(health): bound whole readiness DB check with one deadline
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(health): keep readiness deadline fallback within lint budgets
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(health): suppress TQ008 for proxy-global readiness patches
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(db): release reconnect lock when a waiting reconnect is cancelled
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: milan <milan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
The customStyle background only reaches the <pre>, so the prism theme's own
background stayed on the inner <code>. In dark mode that left a white card
with a dark box hugging every line. Let the block inherit the themed surface
from its wrapper instead.
* test: add logging e2e coverage (s3_v2, gcs_bucket, team langfuse callback, datadog failure)
Five new live e2e scenarios raising Logging & Guardrails registry coverage:
s3_v2 success and failure objects read back from the real S3 bucket,
gcs_bucket success record read back through the GCS JSON API (with
nextPageToken pagination and per-request bearer minting), team-scoped
Langfuse callback delivery with non-team isolation, and DataDog failure
event delivery queried by indexed model_group. datadog_reader gains
query-based variants of the marker search; the langfuse cell is a new
registry row. Bucket readers settle past a full flush interval so a
late duplicate cannot hide from the exactly-one assertions
* test: cover clock-skew day prefix in gcs read-back and retry team callback propagation
* test: key the s3 failure read-back on the provider error, not payload absence
* chore: rerun ci
* chore: rerun ci after config sync
* chore: rerun ci with pr lane env
* chore: rerun ci
* chore: rerun ci
* chore: rerun ci
* chore: rerun ci
* chore: rerun ci
* test: add guardrail e2e coverage (presidio masking, bedrock post and during call, moderation on messages) (#38553)
* test: add guardrail e2e coverage (presidio masking, bedrock post/during, moderation on messages)
* test: require the phone placeholder positively in the presidio masking predicate
* test: count only the 400 verdict body as a bedrock post_call block
* test(e2e): exempt the guardrail config echo from the post_call leak assertion
* test(e2e): pin the fail-closed contract for an unknown guardrail name (skipped, product gap)
* test(e2e): tolerate the readiness 503 from a transient db blip in the callback-config probes
