Budget enforcement trusts a current LiteLLM_BudgetWindowSpend row without
reconciling it against LiteLLM_SpendLogs, so an increment dropped after a
failed commit let the entity spend past its window limit after the next
counter reseed. Failed increments now go back on the in-memory queue, or
back to the Redis buffer, and retry on the next scheduler tick like every
other spend category.
Multi-window budgets (budget_limits on keys/teams) currently keep window
spend only in cache. Every cold or expired counter recomputes the window
by aggregating LiteLLM_SpendLogs, which has no usable index for that
query and saturates the DB on large tables (#35766).
This adds a LiteLLM_BudgetWindowSpend table holding one row per
configured window, keyed (entity_type, entity_id, window_duration),
with window_start identifying the period the spend belongs to.
Follow-up PRs maintain these rows from the spend update writer and move
window budget enforcement reads onto them.
request_id can be chosen by the client through x-litellm-call-id, so an
unbounded NOT (request_id = ANY(batch)) let a replayed old id drop that id's
historical LiteLLM_SpendLogs row from the one-time seed while its increment
still landed. The increment now carries the request start, the batch keeps
the earliest one, and the seed only excludes ids whose startTime is at or
after it.
The suggester pins temperature=0.2 for tool-selection determinism and passed no
drop_params, so an operator-supplied reasoning model whose only accepted temperature is 1
made litellm raise UnsupportedParamsError and the whole suggestion fail. The default
gpt-4o-mini is unaffected; the failure needs the caller to name a model.
Every other internal LLM call the proxy makes on a user's behalf already opts in through
judge_acompletion, which sets drop_params=True on both dispatch paths. This was the one
caller outside that contract, so the sampling preference is now advisory here too and the
call degrades instead of dying.
Resolves LIT-6352
The request and response tree passed the library's light palette in every
theme, so in dark mode the string values rendered dark green and the
punctuation rendered black on a near black surface. Pick the palette from
the resolved theme instead, and let the tree inherit the themed surface
rather than painting the library's own background.
Team-scoped deployments keep the internal model_name_{team_id}_{uuid} routing key and expose the public name in model_info.team_public_model_name. The dashboard links team model chips with the public name, so the exact filter now matches either name via the existing helper.
The where clause now uses the exact name string directly and skips the DB query when the typed search cannot occur in that name, so no new mutable literals are added (LIT002 gate).
With model=<group>&search=<term>, the router list was narrowed to the group but the DB query only matched the substring, so other groups' rows leaked into the page and total_count.
* feat(mcp): support RS256 signing for MCP gateway session tokens
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style: ruff format session token modules
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): enforce key strength on rotated public keys and unique kids
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Pass the selected group as the exact model= param on /v2/model/info rather than as the substring search, so a group like gpt-4 no longer pulls gpt-4o rows into the page and count. Drop two comments that restated helper behavior.
* fix(health): honor allow_requests_on_db_unavailable in readiness probe
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(health): bound readiness DB check and pass reconnect timeout
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(health): bound whole readiness DB check with one deadline
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(health): keep readiness deadline fallback within lint budgets
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(health): suppress TQ008 for proxy-global readiness patches
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(db): release reconnect lock when a waiting reconnect is cancelled
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: milan <milan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
The customStyle background only reaches the <pre>, so the prism theme's own
background stayed on the inner <code>. In dark mode that left a white card
with a dark box hugging every line. Let the block inherit the themed surface
from its wrapper instead.
* test: add logging e2e coverage (s3_v2, gcs_bucket, team langfuse callback, datadog failure)
Five new live e2e scenarios raising Logging & Guardrails registry coverage:
s3_v2 success and failure objects read back from the real S3 bucket,
gcs_bucket success record read back through the GCS JSON API (with
nextPageToken pagination and per-request bearer minting), team-scoped
Langfuse callback delivery with non-team isolation, and DataDog failure
event delivery queried by indexed model_group. datadog_reader gains
query-based variants of the marker search; the langfuse cell is a new
registry row. Bucket readers settle past a full flush interval so a
late duplicate cannot hide from the exactly-one assertions
* test: cover clock-skew day prefix in gcs read-back and retry team callback propagation
* test: key the s3 failure read-back on the provider error, not payload absence
* chore: rerun ci
* chore: rerun ci after config sync
* chore: rerun ci with pr lane env
* chore: rerun ci
* chore: rerun ci
* chore: rerun ci
* chore: rerun ci
* chore: rerun ci
* test: add guardrail e2e coverage (presidio masking, bedrock post and during call, moderation on messages) (#38553)
* test: add guardrail e2e coverage (presidio masking, bedrock post/during, moderation on messages)
* test: require the phone placeholder positively in the presidio masking predicate
* test: count only the 400 verdict body as a bedrock post_call block
* test(e2e): exempt the guardrail config echo from the post_call leak assertion
* test(e2e): pin the fail-closed contract for an unknown guardrail name (skipped, product gap)
* test(e2e): tolerate the readiness 503 from a transient db blip in the callback-config probes
