The sequential version sent one request at a time, so a Redis outage never
reached the concurrency where the failed-tracking alert body actually grows.
This drives the proxy with locust against one model group of three mock
deployments, two failing at order 1 and one serving at order 2, so every
request spends its retries on the failing pair and lands on the serving
deployment through the order-based fallback. Two phases, a healthy baseline
and a CLIENT PAUSE WRITE window, and every request must succeed in both.
Latency, RSS and CPU are reported as p50/p90/p99 per phase rather than
asserted on: RSS and CPU come from psutil on the proxy's process tree, since
a multi-worker proxy serves /metrics from the prometheus multiprocess
collector and that drops the process collector's series. Thresholds stay open
until weekly runs give real baselines.
Co-Authored-By: Claude Code <noreply@anthropic.com>
The recursion detector flags any unignored recursive function, so the
timeout classification now walks the explicit cause chain with a
bounded generator instead of calling itself
redis-py's blocking pool reports a saturated pool as ConnectionError chained
from asyncio.TimeoutError. The circuit breaker classified that as a hard
connectivity failure and opened at once while Redis was healthy. Follow the
explicit cause chain so it counts as a timeout and stays behind the
timeout_min_duration gate
Each checkpoint now samples until no new worker has answered for the settle
window, and the growth assertion refuses a worker set that changed between the
warm and after checkpoints instead of comparing only the intersection, so a
leaking worker reached by one checkpoint alone cannot drop out of the gate
The provider budget push runs inside the request success callback, so
awaiting the Redis pipeline there made every request wait for the round
trip. Hand it back to a task whose failure is logged through the breaker
aware logger, so an open breaker stays a debug line and a real Redis error
is one error line instead of an unretrieved task traceback
The release image installs only the proxy extras, and psutil is a locust and mirakuru dev
dependency, so /debug/memory/summary answered with an error and no ram_usage_mb on the e2e
gate. Fall back to /proc/self/statm and /proc/meminfo on Linux when psutil cannot be imported
The componentized chart rendered litellm.connectionPoolEnv into the gateway
container only, so with database.connectionPool.enabled and
gateway.collector.enabled the collector's Prisma client opened its own pool
straight to Postgres instead of going through the pod-local PgBouncer.
Render the same include in the collector container, drop the gateway.extraEnv
workaround from the collector test, and add enabled/disabled regression
assertions for the collector in both helm charts and the terraform aws and gcp
modules, which already pass the pool env to their collector containers. The
module READMEs note the IAM token-auth exception, where the collector keeps a
direct connection by design.
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
On the release gate the e2e tests only see the nginx router, and the chart's
ingress sent /debug/memory/summary to the backend catch-all, so the RSS check
measured the backend pod instead of the gateway workers that serve the failing
requests. Render it as an Exact gateway path next to /test, name the host in the
summary response so workers behind one origin never collide on pid alone, and
key the harness readings by (origin, hostname, pid)
The shared user popover copied alias, email and ID through three copy
buttons that all announced themselves as "Copy ID", so a screen reader
could not tell them apart. IdCell now takes the label, defaulting to the
old text everywhere else.
Also drops the closest("tr") the new link tests used, which put the
testing-library/no-node-access budget over its ceiling, and asserts the
sentinel row leaves User Email and the admin badge unlinked too.
Claude-Session: https://claude.ai/code/session_01NfwfQhamRNnSqgXMUjf3h4
The team detail page's Virtual Keys table showed Organization ID, User
Email, User ID and Created By as dead text, so getting from a key to the
org or user behind it meant copying an id and searching for it.
Those four cells now render as links, reusing the sentinel-aware href
helpers, so default_user_id and the litellm-dashboard team stay plain
text instead of pointing at pages that do not exist.
The Created By cell was a verbatim copy of the Virtual Keys page's user
popover, so that moved into the shared table_cells kit and both tables
now use the one implementation.
Claude-Session: https://claude.ai/code/session_01NfwfQhamRNnSqgXMUjf3h4
Rebuild the shared member table on DataTable so admins can search members by name, email or user id, sort by name, email, role, budget and spend, and filter by role. /team/info now returns each member's user_alias so the table can show a human-readable name
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(streaming): keep usage-only chunks from crashing streams with empty stream_options
The usage-only chunk branch in CustomStreamWrapper.chunk_creator indexed stream_options["include_usage"] directly, so a caller passing stream_options={} hit a KeyError that surfaced as MidStreamFallbackError. Streaming mock_response with an admission input_tokens count (#40637) now always emits such a chunk, which made the crash reachable. Reuse the send_stream_usage policy computed at init instead. Also annotate the #40637 test bindings with Final.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(streaming): report admitted zero prompt tokens instead of recounting in mock streams
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
