The connect-time 401 always named /.well-known/oauth-protected-resource/mcp/{server}, so a client on the
/{server}/mcp URL fetched a document whose resource was a different URL and RFC 9728 strict clients
(newer Claude Code) stopped before opening the browser. Reuse get_passthrough_resource_metadata_url so
the challenge is absolute and matches the inbound path, on both the connect challenge and OBO preflight
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
create-release.yml is back to calling the chaos test through no mechanism at
all; it never called it. Also drops the E2E_REDIS_CHAOS opt-in gate itself:
the redis_chaos marker still exists for -m selection and is still excluded
from the per-PR selector by path (tests/e2e/(ui|claude_code|load)/), but the
test no longer needs an env var to run once its file is targeted.
Co-Authored-By: Claude Code <noreply@anthropic.com>
The session-token grant refresh reads get_team_membership on every request. A
member with no LiteLLM_TeamMembership row (the common lite-login case) returned
None without caching, so every request re-queried the DB and defeated the auth
cache. Cache the miss under a plain-string sentinel with the management-object
TTL, mirroring the MCP no-permission sentinel. All three readers of the key
already treat a non-model value as no row, and team_member_update already evicts
it, so a newly-created per-member budget is picked up on the next request.
Now that create-release.yml gates stable and RC releases on this test directly,
the weekly schedule is redundant: every release gets a run against its own
commit instead of whatever happened to be on the default branch that Saturday.
workflow_dispatch stays for manual runs.
Co-Authored-By: Claude Code <noreply@anthropic.com>
An out-of-range cursorless page returns nothing, and reading its total off the
offset reported more sessions than exist (page 4 of 100 sessions at page size 50
claimed 150). Only a page that holds rows, or the first page, ends the list;
anything past it falls back to the bounded count.
Claude-Session: https://claude.ai/code/session_01ESi9JwaXDww1vP3Qsrr4Mz
A cursorless page that comes back without its lookahead row is the end of the
list, so the total is offset + len(page) and the bounded grouped COUNT over the
whole spend-log table is skipped. First pages on small deployments and every
offset last page now cost one query less.
Moves the count into _count_grouped_sessions and reworks the query-optimization
test that asserted the count always runs second onto a full page, where it does.
Claude-Session: https://claude.ai/code/session_01ESi9JwaXDww1vP3Qsrr4Mz
The chaos test only ran on a weekly cron, so a release could be cut from a
commit it had never covered. Making it callable lets create-release.yml run it
against the exact commit being tagged and refuse to tag if it fails.
Dev, nightly, alpha and beta tags skip the gate: they are cut far more often
than stable and RC tags, and the weekly schedule already covers the default
branch. Input validation moves into the gate job so a malformed tag or SHA
fails before spending a multi-minute chaos run.
Co-Authored-By: Claude Code <noreply@anthropic.com>
The streaming iterator hook timed the whole provider stream and logged that as the
guardrail duration, so PrometheusLogger added LLM generation time to
litellm_overhead_with_guardrails_latency_metric. The hook now accumulates the time
spent inside _filter_single_text per chunk and logs that sum, keeping start_time and
end_time as the wall-clock window.
Resolves LIT-7589
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A page size that does not divide SPEND_LOGS_PAGINATION_COUNT_CAP left the last
page starting inside the capped window and reading past it, so the rows
disagreed with the total reported next to them. The page limit now stops at the
end of that window, and has_more plus next_session_cursor still hand back a
cursor for walking further.
Claude-Session: https://claude.ai/code/session_01ESi9JwaXDww1vP3Qsrr4Mz
A page starting at or past SPEND_LOGS_PAGINATION_COUNT_CAP lies outside the
total the client is given, so it now returns no rows without running the page
query and the grouped top-N sort bound stays capped.
Rewrites the offset test to page a fake session store instead of asserting on
the generated SQL, and covers the last page inside the cap next to the first
page past it.
Claude-Session: https://claude.ai/code/session_01ESi9JwaXDww1vP3Qsrr4Mz
LiteLLM sends the Bedrock Mantle GPT rows through Bedrock's Responses endpoint, which refuses minimal on gpt-5.4 and gpt-5.5 like every other Bedrock GPT row. The earlier commit measured the raw chat endpoint, which accepts it, and dropped the flag by mistake. The ladder test now matches what the proxy path returns
Local runs measured 3.5 KB per request, so 10 KB keeps close to 3x headroom
while tightening from the earlier 12 KB.
Co-Authored-By: Claude Code <noreply@anthropic.com>
Local runs measured p50 0.19s, p90 0.23s, p99 0.69s, so 2s/3s/5s left several
times that as slack. 1s/2s/3s keeps a comfortable margin while catching a
smaller regression than the looser ceilings would have.
Co-Authored-By: Claude Code <noreply@anthropic.com>
Three local runs measured 1.33x-1.36x, so 2x is the tightest bound the data
supports and still catches a regression far smaller than 4x would. Noted in the
comment that this is the ceiling to loosen first if a weekly run trips it,
since core count shifts how much of baseline CPU is fixed per-request work.
Co-Authored-By: Claude Code <noreply@anthropic.com>
RSS moved 0.91x-1.40x across three identical local runs, so it stays loose at
2x rather than the arbitrary 1.5x carried over from the pre-padding-payload
calibration. CPU per request held steady at 1.33x-1.36x across the same runs,
so 4x replaces the looser 6x it inherited from stale numbers.
Co-Authored-By: Claude Code <noreply@anthropic.com>
A ratio against the healthy phase cannot bound either metric. Once the Redis
circuit breaker opens, a request skips Redis instead of waiting on its socket
timeout, so the chaos phase can measure cheaper than the baseline it is compared
against: local runs came in at 0.61x baseline p90 while a log-bytes ratio read
724x. Splitting Budget into RatioBudget and AbsoluteBudget lets RSS and CPU keep
the ratio they need, since both are machine-shaped, while latency and log volume
get the wall-clock ceiling a user actually cares about.
Co-Authored-By: Claude Code <noreply@anthropic.com>
Live calls to Bedrock Mantle and Converse on 2026-09-11: the gpt-5.6 luna, sol, and terra rows and gpt-6-astra return 200 on reasoning_effort=max, gpt-6-astra returns 400 on none, and Mantle gpt-5.4 and gpt-5.5 return 200 on minimal. The commercial Bedrock rows now carry exactly those flags, and the schema test asserts the measured ladder per row instead of a blanket mirror of the direct OpenAI rows
* fix(datadog_llm_obs): keep tool call and result structure under redaction and emit tool output tokens
Under datadog_llm_observability_params.turn_off_message_logging the span kept only one role plus "redacted-by-litellm" per message, so Datadog showed Tool Call 0, Tool Result 0 and no tool output token data. The shared CustomLogger hook collapsed the messages before the callback ran, and the Datadog redaction then dropped tool_calls and tool_results.
The Datadog callback now opts out of the shared message collapse (redacts_messages_itself) and redacts its own normalized messages, keeping roles, tool names, ids and types while replacing content, arguments and results. Tool result tokens are counted with litellm.token_counter before redaction and shipped as the tool_output_tokens metric. Other callbacks keep the inherited behavior.
Resolves LIT-7545
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(datadog_llm_obs): drop explanatory docstrings from the redaction change
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): regenerate schema.d.ts for the classifier descriptions changed in #40655
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): write failure spend log for guardrail-blocked /mcp-rest/tools/call
call_tool_rest_api only translated exceptions to HTTP responses, so a pre_mcp_call
guardrail block never reached failure_handler / async_failure_handler /
post_call_failure_hook and no LiteLLM_SpendLogs failure row was written. Extract
the failure logging from call_mcp_tool into _fire_mcp_tool_call_failure_logging
and run it in the REST route for anything raised between
common_processing_pre_call_logic and execute_mcp_tool
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): keep the original REST tool error when failure logging raises
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): log virtual mcp_tool_call failures and keep REST success latency scoped to tool execution
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Pad the locust payload to tens of KB so per-request bookkeeping cost scales with
body size instead of hiding behind a 40-byte prompt. Turn on
use_redis_transaction_buffer in the chaos config and JSON_LOGS in the workflow so
the spend buffer, pod lock, and JSON-encoded breaker tracebacks are all part of
the measured chaos cost. Add a log-bytes-per-request budget alongside latency,
RSS, and CPU, reading the proxy's log file size at each phase split; its ceiling
is uncalibrated since no chaos run has measured it yet.
Co-Authored-By: Claude Code <noreply@anthropic.com>
The Anthropic Messages route reaches the same Redis touchpoints and cost-tracking
callback through its own request path, so a failure-path regression there would not
surface from chat completions alone. Each simulated user now picks one endpoint round
robin and stays on it, and the per-endpoint split is asserted and reported so a run
that silently drove only one route fails instead of passing.
Co-Authored-By: Claude Code <noreply@anthropic.com>
