* fix(hide-secrets): restore credential coverage lost to the 4.5 entropy limit
Shannon entropy is bounded by log2(length), so the 4.5 limit #39879 shipped
cannot score any value shorter than 23 characters, and it catches a random
32-character base64 credential only about two thirds of the time. A line like
REDIS_PASSWORD=aB3dE6gH9jK2mN5p therefore reaches the provider in the clear.
Add a keyword plugin that yields the credential-shaped value assigned to a
credential-named key, reusing detect_secrets' own maintained denylist so
camelCase, snake_case and SCREAMING_CASE all work with no local word list, and
re-run the assignment-quoting transform detect_secrets skips once its first
pass has matched.
The entropy limits are untouched, so #39879's false-positive fix still holds.
* fix(hide-secrets): read the assignments in a prompt that is mostly prose
configparser aborts the whole parse on the first line it cannot read, so a
message like "Here is my config, can you review it?" followed by
REDIS_PASSWORD=... lost every assignment to that one prose line. Hand the
parser only the lines it can read, dedent the assignments inside a pasted
config, and keep each key distinct by line number so a config naming api_key
once per model keeps every value instead of only the last.
* fix(hide-secrets): drop the plugin docstrings and pin the block-scalar shapes
* fix(hide-secrets): keep a comment or an indented header from closing an open value
* fix(hide-secrets): drop the explanatory comments from the new scan helpers
* fix(hide-secrets): accept punctuation in a credential value
The value filter only allowed the URL-safe Base64 alphabet, so a password
such as hunter2!brahms or p@ssw0rd!2026 passed through unredacted while
the upstream keyword plugin had already matched it. The filter now rejects
only whitespace and brackets, which keeps function calls, subscripts and
sentences out while letting symbol-heavy passwords through.
* fix(hide-secrets): redact every credential on a line and skip timestamps and plain urls
replaces the inherited first-match scan with finditer over every keyword
match, drops iso 8601 timestamps and userinfo-free urls from credential
values, and threads the parser's open-option state through
itertools.accumulate instead of rebinding it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): scan the first token of an assignment and ignore surrounding punctuation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(hide-secrets): drop the unreachable configparser error fallback
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): keep prose after a credential key out of the keyword detector
A bare value followed by ordinary words (secret_sauce: Worcestershire sauce)
is prose, so the synthetic assignment is only built when the value stands
alone or is followed by a shell operator, comment, or another assignment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): scan the first token of shell-style assignments regardless of what follows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): keep spaced assignments in scope when shell text follows the value
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(hide-secrets): drop docstrings that restate the test names
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): stop reading a comparison operator as a trailing assignment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): keep dashed flags as assignment trailers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(rust): count tiktoken cl100k_base admission tokens in Rust
The Rust admission token counter only had the Anthropic tokenizer, so every
other model (OpenAI gpt-4 family, Azure, Gemini, Bedrock non-Claude, Mistral)
tokenized with tiktoken on the Python inference worker.
Add an exact cl100k_base counter to litellm-token-counter: the vendored rank
file (base64 token / rank lines, the bytes Python's tiktoken uses) is parsed
into a byte-level BPE model and the cl100k split pattern is a handwritten
scanner over the shared Unicode classes, so no regex engine runs per request.
Both tokenizers share the message, tool and reply-priming accounting.
The PyO3 TokenCounter gains a from_cl100k_ranks constructor; Python reads the
rank file and passes it in, the way claude_json_str already works. The bridge
selects the counter through the same predicates litellm.token_counter uses
(huggingface_tokenizer_kind, openai_tokenizer_encoding), declines o200k_base,
downloaded HuggingFace and custom tokenizers to Python, and budget reservation
counts once per distinct tokenizer a request names.
The legacy gpt-3.5-turbo-0301 message accounting (4 per message, -1 per name)
stays in Python: the selector declines it through the predicate token_counter
itself uses.
* feat(rust): count tiktoken o200k_base admission tokens in Rust (#40794)
Add a handwritten o200k_base split scanner and TokenCounter::from_o200k_ranks
next to the cl100k_base counter, sharing MergeRanks and the request
accounting. The Python bridge selects it when openai_tokenizer_encoding
names o200k_base, so gpt-4o, gpt-4.1, gpt-5, o1/o3/o4 and chatgpt-4o
requests stop tokenizing on the Python worker under LITELLM_RUST=true
Co-authored-by: yassin <yassin@berri.ai>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): stop logging the request payload as guardrail_response on pre_call hooks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): snapshot pre_call request before the hook so in-place edits log as mask
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): log non-mapping pre_call hook results as mask instead of raising
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): treat legacy functions and tool_choice edits as mask in pre_call logging
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): log a pre_call rejection string as is instead of "mask"
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: shivam <shivam@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng <yucheng@berri.ai>
The get_team_membership negative cache stores a NO_TEAM_MEMBERSHIP_SENTINEL
for a session-token member with no LiteLLM_TeamMembership row. The two
create paths that add a row with a per-member budget, /team/member_add and
the /team/update budget backfill, did not evict that sentinel, so the new
per-member budget stayed unenforced until the membership cache TTL expired.
Add _evict_created_membership_caches and call it from both sites so the
budget applies on the next request.
Mantle's /openai/v1/responses rejects reasoning.summary values other than "auto" with 400 unsupported_parameter. Drop it with a warning under drop_params, otherwise raise UnsupportedParamsError naming the remedy.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A value length limit clips the input.value blob, so the per-index keys are the
only untruncated copy of a message. Indexing the leading prompt messages dropped
the live user turn from every span attribute on long conversations. Keep message
0 and the most recent turns under the same span-wide budget, original indices
preserved, reply reservation unchanged
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
create-release.yml is back to calling the chaos test through no mechanism at
all; it never called it. Also drops the E2E_REDIS_CHAOS opt-in gate itself:
the redis_chaos marker still exists for -m selection and is still excluded
from the per-PR selector by path (tests/e2e/(ui|claude_code|load)/), but the
test no longer needs an env var to run once its file is targeted.
Co-Authored-By: Claude Code <noreply@anthropic.com>
The session-token grant refresh reads get_team_membership on every request. A
member with no LiteLLM_TeamMembership row (the common lite-login case) returned
None without caching, so every request re-queried the DB and defeated the auth
cache. Cache the miss under a plain-string sentinel with the management-object
TTL, mirroring the MCP no-permission sentinel. All three readers of the key
already treat a non-model value as no row, and team_member_update already evicts
it, so a newly-created per-member budget is picked up on the next request.
Now that create-release.yml gates stable and RC releases on this test directly,
the weekly schedule is redundant: every release gets a run against its own
commit instead of whatever happened to be on the default branch that Saturday.
workflow_dispatch stays for manual runs.
Co-Authored-By: Claude Code <noreply@anthropic.com>
An out-of-range cursorless page returns nothing, and reading its total off the
offset reported more sessions than exist (page 4 of 100 sessions at page size 50
claimed 150). Only a page that holds rows, or the first page, ends the list;
anything past it falls back to the bounded count.
Claude-Session: https://claude.ai/code/session_01ESi9JwaXDww1vP3Qsrr4Mz
A cursorless page that comes back without its lookahead row is the end of the
list, so the total is offset + len(page) and the bounded grouped COUNT over the
whole spend-log table is skipped. First pages on small deployments and every
offset last page now cost one query less.
Moves the count into _count_grouped_sessions and reworks the query-optimization
test that asserted the count always runs second onto a full page, where it does.
Claude-Session: https://claude.ai/code/session_01ESi9JwaXDww1vP3Qsrr4Mz
The streaming iterator hook timed the whole provider stream and logged that as the
guardrail duration, so PrometheusLogger added LLM generation time to
litellm_overhead_with_guardrails_latency_metric. The hook now accumulates the time
spent inside _filter_single_text per chunk and logs that sum, keeping start_time and
end_time as the wall-clock window.
Resolves LIT-7589
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A page size that does not divide SPEND_LOGS_PAGINATION_COUNT_CAP left the last
page starting inside the capped window and reading past it, so the rows
disagreed with the total reported next to them. The page limit now stops at the
end of that window, and has_more plus next_session_cursor still hand back a
cursor for walking further.
Claude-Session: https://claude.ai/code/session_01ESi9JwaXDww1vP3Qsrr4Mz
A page starting at or past SPEND_LOGS_PAGINATION_COUNT_CAP lies outside the
total the client is given, so it now returns no rows without running the page
query and the grouped top-N sort bound stays capped.
Rewrites the offset test to page a fake session store instead of asserting on
the generated SQL, and covers the last page inside the cap next to the first
page past it.
Claude-Session: https://claude.ai/code/session_01ESi9JwaXDww1vP3Qsrr4Mz
LiteLLM sends the Bedrock Mantle GPT rows through Bedrock's Responses endpoint, which refuses minimal on gpt-5.4 and gpt-5.5 like every other Bedrock GPT row. The earlier commit measured the raw chat endpoint, which accepts it, and dropped the flag by mistake. The ladder test now matches what the proxy path returns
Local runs measured 3.5 KB per request, so 10 KB keeps close to 3x headroom
while tightening from the earlier 12 KB.
Co-Authored-By: Claude Code <noreply@anthropic.com>
Local runs measured p50 0.19s, p90 0.23s, p99 0.69s, so 2s/3s/5s left several
times that as slack. 1s/2s/3s keeps a comfortable margin while catching a
smaller regression than the looser ceilings would have.
Co-Authored-By: Claude Code <noreply@anthropic.com>
Three local runs measured 1.33x-1.36x, so 2x is the tightest bound the data
supports and still catches a regression far smaller than 4x would. Noted in the
comment that this is the ceiling to loosen first if a weekly run trips it,
since core count shifts how much of baseline CPU is fixed per-request work.
Co-Authored-By: Claude Code <noreply@anthropic.com>
RSS moved 0.91x-1.40x across three identical local runs, so it stays loose at
2x rather than the arbitrary 1.5x carried over from the pre-padding-payload
calibration. CPU per request held steady at 1.33x-1.36x across the same runs,
so 4x replaces the looser 6x it inherited from stale numbers.
Co-Authored-By: Claude Code <noreply@anthropic.com>
A ratio against the healthy phase cannot bound either metric. Once the Redis
circuit breaker opens, a request skips Redis instead of waiting on its socket
timeout, so the chaos phase can measure cheaper than the baseline it is compared
against: local runs came in at 0.61x baseline p90 while a log-bytes ratio read
724x. Splitting Budget into RatioBudget and AbsoluteBudget lets RSS and CPU keep
the ratio they need, since both are machine-shaped, while latency and log volume
get the wall-clock ceiling a user actually cares about.
Co-Authored-By: Claude Code <noreply@anthropic.com>
