Commit graph

49820 commits

Author SHA1 Message Date
Joshua Valluru
05d2c316f5 refactor(mcp): reuse in-memory discovery storage 2026-09-11 17:53:51 -07:00
mateo
115535c3ad test(model_prices): cover Fireworks DeepSeek V4.1 costs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 00:49:26 +00:00
ryan-crabbe-berri
b27d2cce77
feat(ui): link the Organization and Deleted By cells on Deleted Teams (#40751)
* feat(ui): link the Organization and Deleted By cells on Deleted Teams

Both columns rendered as plain text, so tracing a deleted team back to its
org or to whoever removed it meant copying an id into another page's search
box. Route them through IdentityCell with orgDetailHref and userDetailHref.
Team ID stays unlinked because the team itself is gone.

Claude-Session: https://claude.ai/code/session_01NfwfQhamRNnSqgXMUjf3h4

* test(ui): mount a router mock for the Deleted Teams page test

The page test renders the table, and the newly linked cells call useRouter,
which throws without an App Router mounted. Matches how the other 35 test
files in the suite stub next/navigation.

Claude-Session: https://claude.ai/code/session_01NfwfQhamRNnSqgXMUjf3h4
2026-09-11 17:49:07 -07:00
ryan-crabbe-berri
c53f72c764
feat(ui): link the Created By cell on the Prompts page (#40753)
The column was plain muted text, so finding out who owns a prompt meant
copying the id into the Users page search box. Route it through
IdentityCell with userDetailHref, which keeps the proxy admin placeholder
unlinked.

Claude-Session: https://claude.ai/code/session_01NfwfQhamRNnSqgXMUjf3h4
2026-09-11 17:49:01 -07:00
ryan-crabbe-berri
06964e5603
feat(ui): link the User ID, Created By and Deleted By cells on Deleted Keys (#40750)
* feat(ui): link the User ID, Created By and Deleted By cells on Deleted Keys

All three columns rendered as plain text, so auditing a deleted key meant
copying an id into the Users page search box. Route them through
IdentityCell with userDetailHref, which keeps the proxy admin placeholder
unlinked. User Email and Team Alias stay as they are: the deleted key table
has no column for either, so the API never populates them.

Claude-Session: https://claude.ai/code/session_01NfwfQhamRNnSqgXMUjf3h4

* test(ui): mount a router mock for the Deleted Keys page test

The page test renders the table, and the newly linked cells call useRouter,
which throws without an App Router mounted. Matches how the other 35 test
files in the suite stub next/navigation.

Claude-Session: https://claude.ai/code/session_01NfwfQhamRNnSqgXMUjf3h4
2026-09-11 17:48:57 -07:00
ryan-crabbe-berri
1be930664f
feat(ui): link the User ID and Team ID cells on the Memory page (#40752)
Both columns rendered as dead pills, so tracing a memory row back to its
owner meant copying an id into another page's search box. IdCell grows an
href prop that turns the pill into a client-routed link, and the Memory
columns pass the shared entityLinks helpers so the proxy admin and
dashboard sentinels stay unlinked.

Claude-Session: https://claude.ai/code/session_01NfwfQhamRNnSqgXMUjf3h4
2026-09-11 17:48:44 -07:00
kerry-berri
b957c25241
Merge pull request #40811 from BerriAI/litellm_pr_template_assumptions
docs(pr-template): note untested assumptions under Caveats
2026-09-11 17:47:54 -07:00
ryan-crabbe-berri
4a2edf1702
feat(ui): link the Organization cell on the Teams page (#40749)
The Teams table showed a team's organization as plain text, so getting
from a team to the org that owns it meant copying the alias and searching
the Organizations page by hand.

It now uses the same link helper the key tables use, so the cell points
at the org detail page.

Claude-Session: https://claude.ai/code/session_01NfwfQhamRNnSqgXMUjf3h4
2026-09-11 17:47:35 -07:00
devin-ai-integration[bot]
bf146e2cac
fix(otel v2): name Langfuse traces from the langfuse_trace_name header or metadata.trace_name (#40793)
* fix(otel v2): name Langfuse traces from the langfuse_trace_name header or metadata.trace_name

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(otel v2): type the named-request helper in the Langfuse logger tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-11 17:37:35 -07:00
mateo
6cffb31e5c feat(model_prices): add DeepSeek V4.1 Flash on Fireworks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 00:36:20 +00:00
jesus
96bf276ab9 fix(cli): remove the temp catalog when the atomic replace fails
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 00:34:06 +00:00
jesus
d1653fa40d fix(cli): replace Codex catalog atomically and skip sync on unreadable instructions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 00:34:06 +00:00
jesus
003b53abbb feat(cli): sync Codex /model picker from proxy /v1/models in lite codex
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-12 00:34:06 +00:00
Kerry Lu
55c5babbc5 docs(pr-template): fold assumptions guidance into Caveats instructions
Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-11 17:33:53 -07:00
Kerry Lu
dac491dc3f docs(pr-template): fold Assumptions Made into Caveats
Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-11 17:30:57 -07:00
Kerry Lu
6882811cf6 docs(pr-template): move Assumptions Made above User Flow
Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-11 17:28:55 -07:00
Kerry Lu
f77c8feb84 docs(pr-template): rename Assumptions to Assumptions Made
Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-11 17:28:02 -07:00
Joshua Valluru
0690520080 test(mcp): isolate static-prefix discovery UI assets 2026-09-11 17:26:35 -07:00
Yuneng Jiang
90bf2911dc
test(e2e): verify IdP readiness through real HTTP 2026-09-11 17:25:38 -07:00
Kerry Lu
b6b5e27d7b docs(pr-template): add an Assumptions section
Action item from the v1.100.0 OOM RCA. LIT-6780 recorded "running without
--detailed_debug was not tried" as unverified; the fix PR closing it stated
"Only happens with --detailed_debug on" as fact without running that test, and
the untested half is where the customer-facing OOM lived. Nothing in the
template asked for the hedge, so it disappeared between the ticket and review.

The section asks for each untested claim plus what breaks if it is wrong, and
for any hedge on the linked ticket to be carried forward or explicitly closed
out, so reviewers and coding agents have something concrete to attack.

Co-Authored-By: Claude Code <noreply@anthropic.com>
2026-09-11 17:23:31 -07:00
Yuneng Jiang
ec64005ff7
ci: run the JWT stack startup checks 2026-09-11 17:17:03 -07:00
devin-ai-integration[bot]
dab7f6a86a
feat(proxy): expose complexity routing headers (#40792)
(cherry picked from commit c817faec7a)

Co-authored-by: Tin <tin@berri.ai>
Co-authored-by: Claude Code <noreply@anthropic.com>
2026-09-11 17:14:40 -07:00
Yuneng Jiang
a706dbbb7d
chore: merge current staging into JWT E2E infrastructure 2026-09-11 17:13:52 -07:00
Joshua Valluru
f66239a50f test(mcp): cover static-prefix issuer discovery 2026-09-11 17:13:47 -07:00
Joshua Valluru
da61fa3564 fix(mcp): preserve existing relay discovery identifiers 2026-09-11 17:11:42 -07:00
Yuneng Jiang
77f406dc00
test(e2e): start persistent Keycloak in the changed-test runner 2026-09-11 17:09:35 -07:00
Joshua Valluru
17863fa5cf fix(mcp): match per-server OAuth metadata issuers 2026-09-11 17:05:35 -07:00
kerry-berri
8e4f2abb40
Merge pull request #40482 from BerriAI/litellm_e2e_redis_timeout
test(load): add a Redis timeout chaos load test
2026-09-11 16:58:52 -07:00
yucheng-berri
e4706fa409
fix(hide-secrets): restore credential coverage lost to the 4.5 entropy limit (#40190)
* fix(hide-secrets): restore credential coverage lost to the 4.5 entropy limit

Shannon entropy is bounded by log2(length), so the 4.5 limit #39879 shipped
cannot score any value shorter than 23 characters, and it catches a random
32-character base64 credential only about two thirds of the time. A line like
REDIS_PASSWORD=aB3dE6gH9jK2mN5p therefore reaches the provider in the clear.

Add a keyword plugin that yields the credential-shaped value assigned to a
credential-named key, reusing detect_secrets' own maintained denylist so
camelCase, snake_case and SCREAMING_CASE all work with no local word list, and
re-run the assignment-quoting transform detect_secrets skips once its first
pass has matched.

The entropy limits are untouched, so #39879's false-positive fix still holds.

* fix(hide-secrets): read the assignments in a prompt that is mostly prose

configparser aborts the whole parse on the first line it cannot read, so a
message like "Here is my config, can you review it?" followed by
REDIS_PASSWORD=... lost every assignment to that one prose line. Hand the
parser only the lines it can read, dedent the assignments inside a pasted
config, and keep each key distinct by line number so a config naming api_key
once per model keeps every value instead of only the last.

* fix(hide-secrets): drop the plugin docstrings and pin the block-scalar shapes

* fix(hide-secrets): keep a comment or an indented header from closing an open value

* fix(hide-secrets): drop the explanatory comments from the new scan helpers

* fix(hide-secrets): accept punctuation in a credential value

The value filter only allowed the URL-safe Base64 alphabet, so a password
such as hunter2!brahms or p@ssw0rd!2026 passed through unredacted while
the upstream keyword plugin had already matched it. The filter now rejects
only whitespace and brackets, which keeps function calls, subscripts and
sentences out while letting symbol-heavy passwords through.

* fix(hide-secrets): redact every credential on a line and skip timestamps and plain urls

replaces the inherited first-match scan with finditer over every keyword
match, drops iso 8601 timestamps and userinfo-free urls from credential
values, and threads the parser's open-option state through
itertools.accumulate instead of rebinding it

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(hide-secrets): scan the first token of an assignment and ignore surrounding punctuation

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(hide-secrets): drop the unreachable configparser error fallback

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(hide-secrets): keep prose after a credential key out of the keyword detector

A bare value followed by ordinary words (secret_sauce: Worcestershire sauce)
is prose, so the synthetic assignment is only built when the value stands
alone or is followed by a shell operator, comment, or another assignment

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(hide-secrets): scan the first token of shell-style assignments regardless of what follows

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(hide-secrets): keep spaced assignments in scope when shell text follows the value

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(hide-secrets): drop docstrings that restate the test names

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(hide-secrets): stop reading a comparison operator as a trailing assignment

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(hide-secrets): keep dashed flags as assignment trailers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-11 16:49:48 -07:00
yucheng
7a083353a8 Merge branch 'litellm_internal_staging' into litellm_model_armor_logging_only_streaming
Some checks are pending
ai-gateway image / ai-gateway release image (push) Waiting to run
LiteLLM Rust / rust-lint (push) Waiting to run
LiteLLM Rust / rust-test (push) Waiting to run
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-11 23:48:35 +00:00
devin-ai-integration[bot]
359b7a8489
feat(rust): count tiktoken cl100k_base admission tokens in Rust (#40777)
* feat(rust): count tiktoken cl100k_base admission tokens in Rust

The Rust admission token counter only had the Anthropic tokenizer, so every
other model (OpenAI gpt-4 family, Azure, Gemini, Bedrock non-Claude, Mistral)
tokenized with tiktoken on the Python inference worker.

Add an exact cl100k_base counter to litellm-token-counter: the vendored rank
file (base64 token / rank lines, the bytes Python's tiktoken uses) is parsed
into a byte-level BPE model and the cl100k split pattern is a handwritten
scanner over the shared Unicode classes, so no regex engine runs per request.
Both tokenizers share the message, tool and reply-priming accounting.

The PyO3 TokenCounter gains a from_cl100k_ranks constructor; Python reads the
rank file and passes it in, the way claude_json_str already works. The bridge
selects the counter through the same predicates litellm.token_counter uses
(huggingface_tokenizer_kind, openai_tokenizer_encoding), declines o200k_base,
downloaded HuggingFace and custom tokenizers to Python, and budget reservation
counts once per distinct tokenizer a request names.

The legacy gpt-3.5-turbo-0301 message accounting (4 per message, -1 per name)
stays in Python: the selector declines it through the predicate token_counter
itself uses.

* feat(rust): count tiktoken o200k_base admission tokens in Rust (#40794)

Add a handwritten o200k_base split scanner and TokenCounter::from_o200k_ranks
next to the cl100k_base counter, sharing MergeRanks and the request
accounting. The Python bridge selects it when openai_tokenizer_encoding
names o200k_base, so gpt-4o, gpt-4.1, gpt-5, o1/o3/o4 and chatgpt-4o
requests stop tokenizing on the Python worker under LITELLM_RUST=true

Co-authored-by: yassin <yassin@berri.ai>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-11 23:47:05 +00:00
shivam
f5c298a816 fix(bedrock_mantle): satisfy LIT002 gate for reasoning summary handler
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-11 23:47:00 +00:00
Yuneng Jiang
4dd6af5722
chore: merge latest staging into JWT E2E foundation 2026-09-11 16:46:20 -07:00
shivam
eb2ce87cb3 refactor(bedrock_mantle): drop unneeded mutable-ok suppressions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-11 23:45:43 +00:00
devin-ai-integration[bot]
d78861bb29
fix(guardrails): stop logging the request payload as guardrail_response on pre_call hooks (#39699)
* fix(guardrails): stop logging the request payload as guardrail_response on pre_call hooks

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(guardrails): snapshot pre_call request before the hook so in-place edits log as mask

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(guardrails): log non-mapping pre_call hook results as mask instead of raising

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(guardrails): treat legacy functions and tool_choice edits as mask in pre_call logging

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(guardrails): log a pre_call rejection string as is instead of "mask"

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: shivam <shivam@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng <yucheng@berri.ai>
2026-09-11 16:44:56 -07:00
Yuneng Jiang
f7b1fdc39c
test(e2e): check cleanup warnings through one teardown action 2026-09-11 16:43:31 -07:00
Yuneng Jiang
9a80bf2ad4
test(e2e): harden JWT fixtures and cover management lifecycles 2026-09-11 16:36:12 -07:00
shivam
37a1b859d7 fix(bedrock_mantle): harden reasoning summary validation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-11 23:34:14 +00:00
yucheng
a15309dfe8 refactor(otel): trim the message cap docstrings to one line each
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-11 23:32:44 +00:00
ryan-crabbe-berri
0f10c06241 fix(proxy): evict negative membership cache when a member row is created
The get_team_membership negative cache stores a NO_TEAM_MEMBERSHIP_SENTINEL
for a session-token member with no LiteLLM_TeamMembership row. The two
create paths that add a row with a per-member budget, /team/member_add and
the /team/update budget backfill, did not evict that sentinel, so the new
per-member budget stayed unenforced until the membership cache TTL expired.

Add _evict_created_membership_caches and call it from both sites so the
budget applies on the next request.
2026-09-11 16:31:32 -07:00
ryan-crabbe-berri
4e9d414603
Merge pull request #40748 from BerriAI/litellm_fix_gemini_reasoning_effort_400
fix(vertex_ai): return 400 for invalid reasoning_effort instead of 500
2026-09-11 16:31:16 -07:00
yujonglee
83ab0113f0
feat(ocr): add Vertex DeepSeek adapter and remove legacy OCR pipeline (#40509)
* feat(ocr): add Vertex DeepSeek adapter

* fix(ocr): restore stacked CI coverage

* style(ocr): apply workspace rustfmt

* fix(ocr): deduplicate stacked gateway error mapping

* test(ocr): keep response format checks at dispatch

* fix(ocr): initialize gateway input provenance

* refactor(ocr): preserve DeepSeek extra params

* refactor(ocr): align Vertex DeepSeek preparation

* fix(gateway): drop removed OCR credential error variant

* fix(ocr): fail closed for deferred hooks and Vertex destinations

* fix(ocr): preserve DeepSeek credential provenance
2026-09-11 16:22:57 -07:00
yujonglee
b8928170e9
feat(ocr): add Vertex Mistral adapter (#40507)
* feat(ocr): add Vertex Mistral adapter

* test(ocr): validate Vertex credentials at adapter boundary

* refactor(ocr): preserve Vertex Mistral extra params

* refactor(ocr): align Vertex authentication lifecycle

* refactor(ocr): keep Vertex preparation behind bridge

* fix(ocr): protect Vertex credential destinations

* fix(auth): restrict request Vertex token endpoints
2026-09-11 16:22:56 -07:00
yujonglee
0dd5e6e289
feat(ocr): add Reducto legacy and v3 adapters (#40535)
* feat(ocr): add Reducto adapters

* fix(ocr): decline missing Reducto credentials

* fix(ocr): map Reducto credentials in gateway errors

* test(ocr): keep Reducto coverage at SDK boundary

* test(ocr): remove stale gateway Reducto cases

* fix(ocr): stop retaining Reducto responses by default

* refactor(ocr): preserve Reducto extra params

* refactor(ocr): adopt request preparation contract

* fix(ocr): preserve provider model passthrough

* fix(ocr): reject unknown Reducto models

* fix(ocr): preserve Reducto provider options
2026-09-11 16:22:56 -07:00
yujonglee
b544f2244b
feat(ocr): add Azure Document Intelligence adapter (#40534)
* feat(ocr): add Azure Document Intelligence

* fix(ocr): decline missing Document Intelligence credentials

* fix(ocr): map Document Intelligence credentials

* test(ocr): expose Azure transport to adapter tests

* fix(ocr): preserve native responses through Rust bridge

* feat(core): add URL query pair completion

* fix(ocr): declare Document Intelligence native responses

* fix(auth): preserve Azure credential provenance in OCR adapters

* refactor(ocr): use shared native response handling

* refactor(ocr): preserve Document Intelligence extra params

* refactor(ocr): adopt request preparation contract

* refactor(ocr): keep native response handling behind bridge

* fix(ocr): prevent credential-bearing polling redirects

* fix(ocr): update Azure auth imports

* fix(ocr): bound Document Intelligence polling rate

* fix(ocr): preserve proxy credential provenance

* test(ocr): assert native bridge format support
2026-09-11 16:22:55 -07:00
yujonglee
5e23db8e03
feat(ocr): add Azure Mistral adapter with native authentication (#40502)
* feat(ocr): move Azure credential resolution to Rust

* fix(auth): keep shared primitives warning-free

* fix(auth): preserve missing key provider errors

* fix(auth): enforce Azure input provenance

* fix(ocr): preserve proxy credential provenance
2026-09-11 16:22:55 -07:00
shivam
bf9e8ea9ef fix(bedrock_mantle): gate reasoning.summary on the OpenAI Responses path
Mantle's /openai/v1/responses rejects reasoning.summary values other than "auto" with 400 unsupported_parameter. Drop it with a warning under drop_params, otherwise raise UnsupportedParamsError naming the remedy.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-11 23:19:17 +00:00
ryan-crabbe-berri
d70e64d973
Merge pull request #40644 from BerriAI/litellm_logs_last_page_jump
fix(ui): jump straight to the last Request Logs page instead of advancing one page
2026-09-11 16:17:56 -07:00
yucheng
a067557dae fix(otel): index the opener and the latest prompt turns, not the oldest
A value length limit clips the input.value blob, so the per-index keys are the
only untruncated copy of a message. Indexing the leading prompt messages dropped
the live user turn from every span attribute on long conversations. Keep message
0 and the most recent turns under the same span-wide budget, original indices
preserved, reply reservation unchanged

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-11 23:16:53 +00:00
kerry
ffc16a4b0e test(e2e): restore the E2E_REDIS_CHAOS opt-in for the redis chaos test
Some checks failed
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-11 22:46:31 +00:00