* fix(db): carry DATABASE_SSLMODE/DATABASE_SSLROOTCERT into the assembled writer and reader URLs
The componentized gateway supervisor starts the in-container PgBouncer from the
DATABASE_URL assembled out of the discrete DATABASE_* vars before config.yaml is
read, so an IAM URL had no way to request verified TLS: PgBouncer dialed the
server with server_tls_sslmode = prefer (no SNI, no verification) and public
RDS endpoints rejected the handshake. Two new env vars, exposed by the chart as
database.writer.sslMode / sslRootCert, are appended as libpq sslmode/sslrootcert
to every writer and reader URL the settings assemble (never to a pinned URL),
then translated for Prisma as before. Token refresh now also carries Prisma's
sslmode/sslcert/sslaccept over into the re-minted URL
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(db): keep TLS params on the CLI password URL and the initial IAM reader mint
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(db): treat DATABASE_SSLROOTCERT on its own as verify-full and cover collector and migrations TLS env
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(db): type the reader mint TLS test double and drop its mutable capture
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The cost tracking callback logged its own ERROR with a traceback for every request whose spend counter increment timed out, on top of the cache layer's throttled line. Timeouts now take the same path as breaker-open refusals: invalidate the counters and return. Also exposes is_redis_timeout_failure publicly for that caller and drops the comment on the new constant
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): sync logging_obj guardrail info on every record so post_call entries survive streamed chat completions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(guardrails): hoist regression test imports to module scope
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
/health/services?service=webhook fired a budget_crossed alert for the
caller's own user_id with any authenticated key. That alert writes the
same dedup cache entry the auth-time user budget alert uses, so a
non-admin could pre-populate it and suppress their real budget alert
for the cache TTL. Match the newrelic and pointfive branches and reject
non-admin callers with a 403 before the alert fires.
* fix(otel v2): name Langfuse traces from the langfuse_trace_name header or metadata.trace_name
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(otel v2): type the named-request helper in the Langfuse logger tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): restore credential coverage lost to the 4.5 entropy limit
Shannon entropy is bounded by log2(length), so the 4.5 limit #39879 shipped
cannot score any value shorter than 23 characters, and it catches a random
32-character base64 credential only about two thirds of the time. A line like
REDIS_PASSWORD=aB3dE6gH9jK2mN5p therefore reaches the provider in the clear.
Add a keyword plugin that yields the credential-shaped value assigned to a
credential-named key, reusing detect_secrets' own maintained denylist so
camelCase, snake_case and SCREAMING_CASE all work with no local word list, and
re-run the assignment-quoting transform detect_secrets skips once its first
pass has matched.
The entropy limits are untouched, so #39879's false-positive fix still holds.
* fix(hide-secrets): read the assignments in a prompt that is mostly prose
configparser aborts the whole parse on the first line it cannot read, so a
message like "Here is my config, can you review it?" followed by
REDIS_PASSWORD=... lost every assignment to that one prose line. Hand the
parser only the lines it can read, dedent the assignments inside a pasted
config, and keep each key distinct by line number so a config naming api_key
once per model keeps every value instead of only the last.
* fix(hide-secrets): drop the plugin docstrings and pin the block-scalar shapes
* fix(hide-secrets): keep a comment or an indented header from closing an open value
* fix(hide-secrets): drop the explanatory comments from the new scan helpers
* fix(hide-secrets): accept punctuation in a credential value
The value filter only allowed the URL-safe Base64 alphabet, so a password
such as hunter2!brahms or p@ssw0rd!2026 passed through unredacted while
the upstream keyword plugin had already matched it. The filter now rejects
only whitespace and brackets, which keeps function calls, subscripts and
sentences out while letting symbol-heavy passwords through.
* fix(hide-secrets): redact every credential on a line and skip timestamps and plain urls
replaces the inherited first-match scan with finditer over every keyword
match, drops iso 8601 timestamps and userinfo-free urls from credential
values, and threads the parser's open-option state through
itertools.accumulate instead of rebinding it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): scan the first token of an assignment and ignore surrounding punctuation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(hide-secrets): drop the unreachable configparser error fallback
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): keep prose after a credential key out of the keyword detector
A bare value followed by ordinary words (secret_sauce: Worcestershire sauce)
is prose, so the synthetic assignment is only built when the value stands
alone or is followed by a shell operator, comment, or another assignment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): scan the first token of shell-style assignments regardless of what follows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): keep spaced assignments in scope when shell text follows the value
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(hide-secrets): drop docstrings that restate the test names
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): stop reading a comparison operator as a trailing assignment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): keep dashed flags as assignment trailers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(rust): count tiktoken cl100k_base admission tokens in Rust
The Rust admission token counter only had the Anthropic tokenizer, so every
other model (OpenAI gpt-4 family, Azure, Gemini, Bedrock non-Claude, Mistral)
tokenized with tiktoken on the Python inference worker.
Add an exact cl100k_base counter to litellm-token-counter: the vendored rank
file (base64 token / rank lines, the bytes Python's tiktoken uses) is parsed
into a byte-level BPE model and the cl100k split pattern is a handwritten
scanner over the shared Unicode classes, so no regex engine runs per request.
Both tokenizers share the message, tool and reply-priming accounting.
The PyO3 TokenCounter gains a from_cl100k_ranks constructor; Python reads the
rank file and passes it in, the way claude_json_str already works. The bridge
selects the counter through the same predicates litellm.token_counter uses
(huggingface_tokenizer_kind, openai_tokenizer_encoding), declines o200k_base,
downloaded HuggingFace and custom tokenizers to Python, and budget reservation
counts once per distinct tokenizer a request names.
The legacy gpt-3.5-turbo-0301 message accounting (4 per message, -1 per name)
stays in Python: the selector declines it through the predicate token_counter
itself uses.
* feat(rust): count tiktoken o200k_base admission tokens in Rust (#40794)
Add a handwritten o200k_base split scanner and TokenCounter::from_o200k_ranks
next to the cl100k_base counter, sharing MergeRanks and the request
accounting. The Python bridge selects it when openai_tokenizer_encoding
names o200k_base, so gpt-4o, gpt-4.1, gpt-5, o1/o3/o4 and chatgpt-4o
requests stop tokenizing on the Python worker under LITELLM_RUST=true
Co-authored-by: yassin <yassin@berri.ai>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): stop logging the request payload as guardrail_response on pre_call hooks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): snapshot pre_call request before the hook so in-place edits log as mask
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): log non-mapping pre_call hook results as mask instead of raising
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): treat legacy functions and tool_choice edits as mask in pre_call logging
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(guardrails): log a pre_call rejection string as is instead of "mask"
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: shivam <shivam@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng <yucheng@berri.ai>
