Commit graph

41258 commits

Author SHA1 Message Date
mateo-berri
1255094de3 fix(e2e): retry transient turn failures and move the weekly anomaly run to Saturday before the stable release cut 2026-07-21 16:55:44 -07:00
mateo-berri
48295df0e9 test(e2e): raise warm cache read floor to 0.65 from measured healthy and regression baselines 2026-07-21 16:21:45 -07:00
mateo-berri
1692170264 fix(e2e): count aborted-session turns as failures and require a spend stability window 2026-07-21 15:34:56 -07:00
mateo-berri
b572cb80d1 test(e2e): add weekly session-anomaly load test against real providers 2026-07-21 14:54:02 -07:00
ryan-crabbe-berri
e1afc64baa
refactor(ui): migrate inline provider logo lookups to the shared Logo component (#34141)
* fix(ui): bundle provider logos as static imports and unify fallback in Logo component

providerLogoMap values are now content-hashed bundle URLs emitted by
static imports instead of /ui/assets/logos/ path strings, so any
deployment that serves the app JS also serves the logos: dev server,
proxy /ui mount, server_root_path sub-paths, and the split-chart nginx
image where the old route 404d in production. A missing file is now a
build error instead of a silent runtime 404.

resolveLogoSrc passes /_next/ URLs through untouched so bundled values
never get double-prefixed with the server root path. The new Logo
molecule owns resolution and the letter-avatar fallback and warns with
the failing URL on load error; ProviderLogo delegates to it. The three
bare img sites in the agents wizard render through Logo, fixing their
broken-image bug.

Dashscope now uses qwen.png, RunwayML the on-disk runway.png, and the
GradientAI entry is removed (no plausible asset exists). soniox.svg and
ai21.svg drop a single mismatched intrinsic dimension attribute that
Turbopack's import-time image parser rejects. Dead logoSrc lookup in
AddModelForm deleted. Vitest resolves image imports to Next's
StaticImageData shape via a config plugin so tests exercise the same
/_next/ URLs as production.

* fix(ui): retry logo load when src changes after an error

Track which src errored instead of a boolean so a Logo instance whose
source changes in place (agents modal title) attempts the new URL
rather than staying on the letter-avatar until remount.

* refactor(ui): migrate inline provider logo lookups to the shared Logo component

Patterns B, C, and E from the logo consolidation: every inline
providerLogoMap lookup feeding a bare img with a hand-rolled DOM
fallback now renders through Logo (credential modal, vector store
create/info views, cost tracking margin and discount forms and tables).

getProviderDisplayInfo, handleImageError, and ProviderDisplayInfo are
deleted; getProviderLogoAndName is a strict superset of the exact-match
helper. The vector store logo map no longer duplicates provider logo
paths: shared entries reference providerLogoMap and the three
vector-store-only logos become static imports. The map itself stays
because milvus and s3_vectors have no Providers enum equivalent.

Sites that rendered nothing for an unmapped provider now render the
letter avatar. Representative tests per pattern assert the rendered img
src against providerLogoMap so a wrong provider-to-enum mapping fails,
plus letter-avatar fallbacks for unmapped providers.

* fix(ui): resolve vector store slugs through the vector store logo map

The vector store info provider badge fed backend slugs like pg_vector,
milvus, and s3_vectors to getProviderLogoAndName, which only knows LLM
providers, so those stores showed a letter avatar and a raw slug. The
pre-existing inline lookup had the same wrong-domain bug via
provider_map. Reinstate getVectorStoreProviderLogoAndName resolving
through vectorStoreProviderMap first with a fallback to the LLM
resolver, so vector-store-only providers get their own logo and display
name for the first time.
2026-07-21 14:28:05 -07:00
ryan-crabbe-berri
2e8403a073
fix(ui): bundle provider logos as static imports and unify fallback in Logo component (#34125)
* fix(ui): bundle provider logos as static imports and unify fallback in Logo component

providerLogoMap values are now content-hashed bundle URLs emitted by
static imports instead of /ui/assets/logos/ path strings, so any
deployment that serves the app JS also serves the logos: dev server,
proxy /ui mount, server_root_path sub-paths, and the split-chart nginx
image where the old route 404d in production. A missing file is now a
build error instead of a silent runtime 404.

resolveLogoSrc passes /_next/ URLs through untouched so bundled values
never get double-prefixed with the server root path. The new Logo
molecule owns resolution and the letter-avatar fallback and warns with
the failing URL on load error; ProviderLogo delegates to it. The three
bare img sites in the agents wizard render through Logo, fixing their
broken-image bug.

Dashscope now uses qwen.png, RunwayML the on-disk runway.png, and the
GradientAI entry is removed (no plausible asset exists). soniox.svg and
ai21.svg drop a single mismatched intrinsic dimension attribute that
Turbopack's import-time image parser rejects. Dead logoSrc lookup in
AddModelForm deleted. Vitest resolves image imports to Next's
StaticImageData shape via a config plugin so tests exercise the same
/_next/ URLs as production.

* fix(ui): retry logo load when src changes after an error

Track which src errored instead of a boolean so a Logo instance whose
source changes in place (agents modal title) attempts the new URL
rather than staying on the letter-avatar until remount.
2026-07-21 14:27:36 -07:00
ryan-crabbe-berri
1cc70f84c0
fix(ui): distinguish response cache from provider prompt caching (#34138)
* fix(ui): distinguish response cache from provider prompt caching

The log detail drawer labeled LiteLLM's response cache result as
"Cache Hit" and rendered a red "false" tag next to provider prompt
cache token counts, which read as prompt caching being broken. The
row is now labeled "Response Cache" with an explanatory tooltip,
shows a neutral "Miss" tag instead of a red one, and the prompt
cache token rows are prefixed with "Prompt Cache" and get their own
tooltips. Cost breakdown line items get the same prefix.

The Caching dashboard only reports response cache analytics but never
said so; it is renamed to "Response Cache" in the sidebar, gains a
scope description pointing to the Usage page and Logs for prompt
caching, and the ambiguous "Cached Tokens" stat card is renamed to
"Cached Completion Tokens".

* test(ui): cover renamed Response Cache sidebar item in e2e

The sidebar navigation spec now clicks the renamed "Response Cache"
item and asserts it routes to /ui/caching, and the menu label fixture
maps the new label while keeping "Caching" as a legacy alias.
Verified by running sidebar.spec.ts through run_e2e.sh (full harness:
built UI served by the proxy, seeded postgres); both tests pass.

* feat(ui): link cache tooltips and dashboard description to docs

The Response Cache tooltip links to the proxy caching docs and the
two prompt cache token tooltips link to the prompt caching docs, so
users can jump straight to the explanation of whichever mechanism
they are looking at. The Response Cache dashboard description links
both docs pages the same way.
2026-07-21 14:26:26 -07:00
ryan-crabbe-berri
b47fe730a4
fix(ui): stop cloning body-carrying requests into stream uploads in fetchClient middleware (#34122)
* fix(ui): stop cloning body-carrying requests into stream uploads in fetchClient middleware

The openapi-fetch middleware rebuilt every outgoing request with
new Request(url, request), which converts a string JSON body into a
ReadableStream with duplex=half. Chromium only allows streaming uploads
over HTTP/2 or HTTP/3, so against any HTTP/1.1 hop (uvicorn serves
HTTP/1.1 only) the fetch dies at the network layer with
net::ERR_ALPN_NEGOTIATION_FAILED, surfaced as "Failed to fetch".

GET callers were unaffected (null body); the first body-carrying caller
arrived with the MCP BYOK credential modal, breaking that flow on plain
http deployments in the v1.94.0 RCs.

The middleware now mutates headers on the original request when no
runtime base is registered, and when rebasing onto a runtime base it
rebuilds the request with the body materialized as bytes via
arrayBuffer(), which fetch sends with Content-Length instead of a
streaming upload

* Update ui/litellm-dashboard/src/lib/http/api.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-07-21 14:26:02 -07:00
Yassin Kortam
e9ac84dc8b
fix(proxy): stop save_config from snapshotting environment_variables into the DB (#34119)
save_config wrote the entire merged config to the DB config table on every call. Because get_config() resolves os.environ/ placeholders to plaintext and merges the environment_variables section, any endpoint that does get_config() then mutates one section then calls save_config() (/add/allowed_ip, delete_callback, model and cost-tracking settings, and others) accidentally persisted an environment_variables row holding all YAML/OS-sourced env vars. Once that row existed the DB overlay shadowed YAML and container env on every subsequent startup, so config/env changes were silently ignored

save_config now pops environment_variables from the DB write unless the caller passes include_env_vars=True. The dedicated /config/update path already writes env vars per-section via _upsert_section, so no current caller needs to opt in

Resolves LIT-2009
2026-07-21 21:23:28 +00:00
ryan-crabbe-berri
f9300d4312
fix(e2e): pin harness Python and surface proxy boot crash output (#34157)
run_e2e.sh let uv resolve any system Python; on machines where that is
3.14 the locked uvloop 0.21.0 fails to import (BaseDefaultEventLoopPolicy
was removed from asyncio.events) and the proxy dies at boot, which the
harness reported only as a misleading 180s health timeout. Pin the
interpreter to 3.13 (overridable via UV_PYTHON) and redirect proxy output
to a log file whose tail is printed when the proxy exits early or never
becomes healthy.
2026-07-21 21:17:43 +00:00
Yassin Kortam
fcaf673418
fix(router): stop per-deployment num_retries from double-counting as provider max_retries (#34129)
* fix(router): stop per-deployment num_retries from double-counting as provider max_retries

A model group with one deployment and num_retries set in the deployment's
litellm_params sent (1 + num_retries) ** 2 requests upstream instead of
1 + num_retries. The deployment's num_retries reached litellm.completion, which
copied it onto max_retries and set it on the provider client, so the provider SDK
retried num_retries times inside each of the Router's 1 + num_retries attempts.

The Router is the sole retry owner for routed calls, so completion() now forces the
provider-SDK max_retries to 0 whenever the call originates from the Router/proxy
(detected via model_group in the request metadata) and only keeps the num_retries
to max_retries alias for direct, non-routed litellm calls (the instructor use case).
This also stops a request- or deployment-level max_retries from nesting on top of
the Router's retries.

Resolves LIT-4385

* fix(router): make router-origin check robust and close test clients

Address review: detect the router marker in both metadata and litellm_metadata
independently (a non-empty metadata without model_group no longer hides a
model_group in litellm_metadata), and close the injected async clients in the
test fixture.
2026-07-21 21:13:32 +00:00
tin-berri
914079a68c
Merge pull request #34065 from BerriAI/litellm_lit4629_issuer_discovery
fix(mcp): let an admin-pinned issuer drive OAuth discovery for url-less servers
2026-07-21 14:06:55 -07:00
mubashir1osmani
ac5b51253a
test(e2e): add Other suite and Guardrails coverage incl. an MCP tool-call guardrail (#34149)
* test(e2e): add other suite covering master-key auth and health lifecycle

Covers the other.* holding-pen cells that were uncovered: master-key
valid_allows/invalid_denied on the admin /user/list gate, and the
lifecycle probes liveness.ping, readiness.public_probe,
readiness.reports_db_status, and readiness_details.authenticated_diagnostics.

New tests/e2e/other/ suite on the shared ProxyClient; the health probes
send no auth header to prove the public routes need no credential, and the
details route is asserted to reject an anonymous caller while exposing
version/db diagnostics to the master key.

* test(e2e): cover block_code_execution and openai_moderation guardrails

Extends the guardrails suite with two built-in guardrails registered per
request (default_on=False, opted in via the chat body's guardrails selector)
so neither intercepts unrelated traffic on the shared proxy.

block_code_execution.pre_call.blocks: a python code block plus a run-this
request is intercepted with the canned content-blocked message and the model
never runs, while the same code block asked about with don't-run-it reaches
the model. Verified live.

openai_moderations.pre_call.blocks: a flagged prompt is rejected 400 naming
the moderation policy while a benign prompt passes. The guardrail calls
OpenAI's moderation API; verifying it needs an OpenAI key with moderation
quota (this account currently 429s the moderation endpoint).

Adds a shared create_backend_model helper and a generic register() plus
per-request guardrails/max_tokens on the client so more built-ins can reuse
the same path.

* test(e2e): cover presidio PII masking (pre_call + post_call)

Registers a presidio guardrail per request (default_on=False) with the
analyzer/anonymizer bases supplied in the registration params, so the test
controls its own dependency and needs no proxy restart.

presidio.pre_call.masks: a repeat-verbatim request comes back with the
<EMAIL_ADDRESS> placeholder and never the raw email, proving the prompt was
anonymized before the model saw it.

presidio.post_call.masks: with apply_to_output the model's own emitted email
is masked on the way out, so the caller never receives the raw value.

Both verified live against real presidio analyzer + anonymizer containers.
logging_only is intentionally not covered: /spend/logs exposes no prompt
messages to read back the masked log, and a logging_only run also masked the
response, contradicting its contract; noted in the module docstring for a
follow-up.

* test(e2e): cover presidio logging_only masking via OTEL read-back

Adds the third presidio cell, guardrail.presidio.logging_only.masks. The
logging_only contract (mask what is logged, do not block) is verified by
reading the request's gen-AI span back from the real OTEL destination: the
span's gen_ai.input.messages attribute carries the <EMAIL_ADDRESS> placeholder,
never the raw email, and the call itself is not blocked.

Reads the trace via the shared OtelReader, promoted from logging/ to the suite
root so both suites use it. The masked prompt is polled to a deadline because
logging_only masks the payload asynchronously and the span can briefly export
before the mask lands. Drops the throwaway chat_send in favor of the existing
transport.send for the call-id capture.

* fix(e2e): tolerate cross-pod guardrail sync delay in team-opt-out test

Stage runs multiple gateway pods behind the shared key. POST /guardrails
registers a new default-on guardrail in-process immediately only on the
pod that served the create call; every other pod picks it up on its next
periodic DB sync (proxy_server.py, every 30s), so the very next chat call
can race a pod that has not synced yet. Poll to a 40s deadline instead of
asserting on the first response, matching the existing pattern in
test_budget_reset_advances_e2e.py.

* test(e2e): cover a guardrail on the MCP tool-call path (content_filter pre_mcp_call)

Adds guardrail.litellm_content_filter.pre_mcp_call.blocks: against the real
Datadog MCP server, a content_filter guardrail configured mode=pre_mcp_call
blocks a banned keyword in an MCP tool call's arguments with HTTP 400 attributed
to the pre_mcp_call hook, and lets a clean argument reach the upstream server.

The guardrail attaches with default_on because per-key/request guardrail
selection is dropped from the synthetic MCP request the hook sees; the banned
keyword is unique per run so default_on only intercepts this test's own call.
mode must be pre_mcp_call - a pre_call config silently no-ops on tools/call
because the event type is rewritten for call_mcp_tool.

Drives the tool directly via /mcp-rest/tools/call for a deterministic check of
the same pre_mcp_call enforcement the OpenAI-SDK chat path hits when a model
invokes an MCP tool.

* fix(e2e): mid-conversation messages test uses client.proxy not client.gateway

EndpointsClient exposes .proxy after the Gateway->ProxyClient rename; the
mid-conversation system test still referenced .gateway, which fails the e2e
basedpyright gate. Aligns it with the rest of the harness.

* test(e2e): address review on the guardrail coverage

MCP tool-call guardrail: poll the banned call until the guardrail is enforced
instead of asserting on the first call, so the control-plane -> data-plane
guardrail sync cannot race the check into a false pass-through; add a repeat
banned call after enforcement to guard against a partial-propagation state.

OpenAI moderation: distinguish a moderation-endpoint 429 (rate limit / no
moderation quota) from a guardrail failure, so an account-capability gap reads
as such rather than as "did not block". Runs green with a moderation-capable key.

* test(e2e): close partial-propagation false-pass in MCP guardrail block test

The single post-block repeat call could be load-balanced back to the same
already-synced data-plane pod, so the test could pass while another pod still
lacked the guardrail and let the banned MCP call reach Datadog. Anchor a wait to
the guardrail create time (every pod is guaranteed to have DB-synced only after a
full ~30s sync interval), then require the banned call to stay blocked across
several attempts; a pass-through after that window is a real leak, not a race.

* test(e2e): drop xfail-style rate-limit branch from openai_moderation test

OpenAI's /v1/moderations is free and returns 200 with the env key (verified
directly), so the RateLimitedError branch mislabeled the failure: a 429 there is
insufficient_quota (no account billing), not throttling. The branch also only
printed a softer message before failing anyway, an xfail-in-disguise the e2e rules
forbid. A 429 now falls through and fails loudly with the full result.
2026-07-21 14:06:29 -07:00
ryan-crabbe-berri
76c9eca25d
refactor(auth): derive temp budget increase without mutating the token (#34121)
* refactor(auth): derive temp budget bump without mutation, tz-aware auth datetimes

_update_key_budget_with_temp_budget_increase mutated max_budget in place, so correctness depended on every resolution path handing it a fresh copy of the cached token; one future re-cache of a live token would compound the bump per request. Return a model_copy instead so no caller can leak an increased budget into shared state.

Also fixes the three remaining DTZ005 naive datetime.now() calls in user_api_key_auth.py (auth span start, builder start_time, service-log end_time; all consumers convert to epoch or subtract same-pair datetimes) and ratchets the DTZ005 strict budget 244 -> 241.

* test: pin non-mutation of the temp budget helper input

Adversarial mutation-testing showed reverting the helper to in-place mutation still passed every test: the cache's copy-on-read layer masks the mutation in the integration test and the direct unit test only inspected the return value. Assert the input object is left untouched and the result is a distinct object so the purity guarantee itself is load-bearing.
2026-07-21 21:02:40 +00:00
Yassin Kortam
30ed840ff5
fix(e2e): use EndpointsClient.proxy after Gateway to ProxyClient rename (#34127)
The mid-conversation-system messages test still referenced the removed
client.gateway attribute, so the tests/e2e basedpyright gate reported 9
errors and went red on every e2e PR. EndpointsClient exposes .proxy, so
point the /v1/messages post helper at client.proxy.transport.
2026-07-21 20:53:40 +00:00
Tin Chi Lo
6959d9de69 test(mcp): pin the token and register wall messages for url-less servers 2026-07-21 13:53:14 -07:00
Tin Chi Lo
930676a9bf fix(mcp): let an admin-pinned issuer drive OAuth discovery for url-less servers 2026-07-21 13:53:14 -07:00
tin-berri
1d36290551
Merge pull request #34063 from BerriAI/litellm_lit4629_openapi_oauth_egress
fix(mcp): attach resolved OAuth credentials to OpenAPI spec_path tool calls
2026-07-21 13:51:17 -07:00
ryan-crabbe-berri
d2819baf0a
feat(ui): add block/unblock key action to key info page (#34116)
Adds a Block Key / Unblock Key action to the key info page, wired to the
existing /key/block and /key/unblock endpoints which previously had no UI.
The Reset Spend and Delete Key buttons move together with it into a new
overflow dropdown next to Regenerate Key, and a red Blocked tag shows next
to the key alias while the key is blocked.
2026-07-21 13:41:10 -07:00
Mateo Wang
a7e4c83013
Merge pull request #34154 from BerriAI/litellm_a2a_protocol_version_semver
fix(a2a): accept semver protocolVersion values like 0.3.0 in agent cards
2026-07-21 13:37:05 -07:00
ryan-crabbe-berri
ee0028a841
feat(ui): surface key budget_reset_at in key info and keys table (#34113)
* feat(budgets): add configurable budget_reset_time of day

Budgets reset at midnight in the configured timezone with no way to control
the time of day, so a drained daily budget surfaces as an overnight incident.
Add a litellm_settings.budget_reset_time option (e.g. "12:00") that shifts
day/week/month resets to a configurable wall-clock time in the existing
timezone, so the end of the budget window lands during business hours.

The reset time is parsed once into an immutable BudgetResetSettings and
injected into the reset job (constructor) and computation, rather than read
from a module-level global at call time. A malformed value fails fast at
startup. Sub-day durations ignore the offset. Unset preserves midnight resets.

* feat(ui): surface key budget_reset_at in key info and keys table
2026-07-21 13:35:14 -07:00
ryan-crabbe-berri
efa997dfe0
feat(budgets): add configurable budget_reset_time of day (#31007)
Budgets reset at midnight in the configured timezone with no way to control
the time of day, so a drained daily budget surfaces as an overnight incident.
Add a litellm_settings.budget_reset_time option (e.g. "12:00") that shifts
day/week/month resets to a configurable wall-clock time in the existing
timezone, so the end of the budget window lands during business hours.

The reset time is parsed once into an immutable BudgetResetSettings and
injected into the reset job (constructor) and computation, rather than read
from a module-level global at call time. A malformed value fails fast at
startup. Sub-day durations ignore the offset. Unset preserves midnight resets.
2026-07-21 13:35:01 -07:00
mateo-berri
2310211531 fix(a2a): reject malformed protocolVersion suffixes while keeping semver prereleases 2026-07-21 13:24:58 -07:00
mateo-berri
062e58fb1d fix(a2a): accept semver protocolVersion values like 0.3.0 in agent cards 2026-07-21 13:03:47 -07:00
yuneng-jiang
257ada88cc
chore(deps): bump pypdf to 6.14.2 and pyasn1 to 0.6.4 (#34148)
Both are lock-only moves. pypdf stays inside the existing
>=6.12.0,<7.0 constraint and pyasn1 is transitive, so pyproject.toml
is unchanged.

pypdf 6.13.3 carries CVE-2026-59935 / 59936 / 59937 / 59938, resolved
across 6.14.0 through 6.14.2. pyasn1 0.6.3 carries CVE-2026-59884 /
59885 / 59886, resolved in 0.6.4. All seven are resource-exhaustion
issues reachable through parsing untrusted input; pypdf is used for
page text extraction in the RAG ingestion file parser.

Scanning the lock before and after with CPE matching enabled takes the
count for these two packages from seven to zero.
2026-07-21 13:01:18 -07:00
yuneng-jiang
ae2f276d19
ci(image-scan): match Python packages against CPE data (#34136)
grype defaults match.python.using-cpes to false, so PyPI packages are
matched only against the GitHub Advisory Database. When a CVE is
published to NVD but its GHSA has not propagated to the global advisory
database, the scan reports clean even though grype's own database
already carries the NVD record with the correct version ranges.

The pypdf CVEs (CVE-2026-59935 / 59936 / 59937 / 59938, analyzed in NVD
since 2026-07-08) are the case that exposed this; their GHSA IDs are
still repo-level and return 404 from the global advisory API, so the
ecosystem matcher has nothing to match on.

Enabling CPE matching for Python closes that gap. Measured against a
v1.91.1 build the finding count goes from 28 to 38; the additions are
mostly actionable, and the few cross-product CPE collisions cannot fail
the build because --only-fixed drops the ones carrying no fix version
and the remainder land below the --fail-on high threshold.
2026-07-21 12:56:56 -07:00
yuneng-jiang
fcd236097e
fix(interactions): add queued to the Interaction status enum (#34135)
Google added a queued value to Interaction.status in the live Interactions
OpenAPI spec, so the compliance canary test_status_enum_values started
failing on every open PR. The exact-match assertion is deliberate; it is
how we find out the spec moved, so this adds the new value rather than
loosening the check, and mirrors it into the generated Status enums so
InteractionStatus stays truthful.
2026-07-21 12:36:21 -07:00
ryan-crabbe-berri
01d624e860
fix(ui): add tooltip to the Active key status badge (#34109) 2026-07-21 12:01:17 -07:00
yuneng-jiang
212a9213c4
refactor(ui): migrate agents table onto the shared DataTable (#34089)
* refactor(ui): migrate agents table onto the shared DataTable

Replace the hand-rolled tremor table inside AgentsPanel with the shared
DataTable, splitting the surface into a data-owning panel, a thin
AgentsTable consumer, and a getAgentsTableColumns definition composed
from the shared cell library.

Row delete moves from an inline icon button into the per-row overflow
menu, and the health-check toggle moves into the table toolbar since it
controls which rows the server returns. The loading skeleton is now
initial-load-only, so refetches keep the current rows on screen.

Drops the last @tremor/react import from AgentsPanel, so its
grandfathered eslint suppressions are pruned from the baseline.

* fix(ui): keep agents ordering and token changes correct in the migrated table

Sorting by created_at went through a raw accessor, and TanStack places
undefined ahead of real values, so an agent with no created_at jumped to
the top of the newest-first list. The pre-migration sort coerced a
missing date to epoch 0 and sorted it last; restore that by sorting on a
derived timestamp.

Reload the list when the access token changes rather than leaving the
previous token's rows on screen: show the skeleton for the new token,
drop the rows if that load fails, and ignore a superseded response so a
slow earlier request cannot overwrite newer rows. Refetches triggered by
delete or the health-check toggle still keep their rows.

Tests also reset the networking mocks between cases so an unconsumed
mockResolvedValueOnce queue cannot leak into the next test.
2026-07-21 10:28:49 -07:00
yucheng-berri
049c6836d2
fix(model_armor): sanitize error details by default (#33908)
* fix(model_armor): sanitize error details by default

Generated with AI

Co-Authored-By: Claude Code

* fix(model_armor): sanitize handler-raised HTTP errors and redact scanned content in guardrail logging

The async HTTP handler raises MaskedHTTPStatusError on any non-2xx via
raise_for_status, so the non-200 branch in make_model_armor_request never ran
against a live API and the raw upstream body reached callers and logs. Catch
the raised error and build the sanitized detail from the response status

Replace the empty-dict guardrail logging payload with field-level redaction of
the keys that echo scanned content (text, sanitizedText, findings) so guardrail
traces keep filter states and block reasons while scanned content stays out

Restore the upstream status code in the sanitized error detail, read guardrail
metadata from the same key the hooks write, and keep guardrail_status within
its typed literal values

* fix(model_armor): bound redactor recursion depth and allowlist it in the recursion detector

_redact_scanned_content walks provider JSON bounded by _REDACT_MAX_DEPTH=20 and
fails closed by returning the redaction sentinel at the cap

* fix(model_armor): honor fail_on_error for upstream API failures

API failures now raise a dedicated ModelArmorAPIError so hooks can tell them
apart from content-block HTTPExceptions; fail_on_error=False lets the request
proceed on a Model Armor outage again while fail-closed configs get the same
sanitized 400 as before

Also addresses review notes: sanitize_error_detail constructor annotation
matches the nullable config field, redaction is owned by the metadata write
sites so _process_response no longer re-applies it, and the request and
response debug log branches move into helpers

* test(model_armor): cover fail_on_error routing on during-call, post-call, streaming, and file-scan paths

* chore: remove accidentally committed pytest cache files

* fix(model_armor): keep sanitize_error_detail coerced across in-memory config reloads

update_in_memory_litellm_params assigns raw LitellmParams fields, so a hot
reloaded config carrying an explicit null would silently disable sanitization;
re-apply the only-explicit-False-opts-out coercion after the update

* fix(model_armor): redact matched malicious URIs and reuse the shared recursion depth constant

maliciousUriMatchedItems echoes the caller-supplied URL including path and
query, so it joins the scanned-content key set; the redactor depth cap now
comes from DEFAULT_MAX_RECURSE_DEPTH in litellm constants instead of a local
literal

* fix(model_armor): keep API failures out of the intervention trace status

Fail-closed upstream failures re-raise ModelArmorAPIError instead of
converting to HTTPException(400), so the shared guardrail logging keeps
recording them as guardrail_failed_to_respond while content blocks stay
guardrail_intervened. Callers see the same 500 shape as before this PR,
with the sanitized message

* chore(model_armor): drop explanatory comment per repository comment policy

---------

Co-authored-by: eugene-yao-zocdoc <eugene.yao@zocdoc.com>
2026-07-21 10:28:24 -07:00
yuneng-jiang
4647f85958
Merge pull request #34078 from BerriAI/litellm_/elated-thompson-4a0c84
refactor(ui): migrate access groups table to shared DataTable
2026-07-21 10:28:21 -07:00
yuneng-jiang
5beb0a735d
Merge pull request #34079 from BerriAI/litellm_/wizardly-bardeen-a47430
refactor(ui): migrate memory table onto shared DataTable
2026-07-21 10:28:10 -07:00
yuneng-jiang
0b4851dd81
Merge pull request #34081 from BerriAI/litellm_/gallant-kapitsa-a809e0
refactor(ui): migrate organizations table onto shared DataTable
2026-07-21 10:27:59 -07:00
yuneng-jiang
925beda06d
Merge pull request #34080 from BerriAI/litellm_/brave-bell-58da35
refactor(ui): migrate audit logs table onto shared DataTable
2026-07-21 10:27:48 -07:00
Mateo Wang
59ebe043c2
Merge pull request #34106 from BerriAI/litellm_gemini_36_flash_35_lite_day0
feat(gemini): day-0 pricing for gemini-3.6-flash and gemini-3.5-flash-lite
2026-07-21 10:23:14 -07:00
ryan-crabbe-berri
e20d3d4ecc
fix(ui): serve /ui/assets from the nginx image instead of SPA fallback (#34066) 2026-07-21 09:22:02 -07:00
mateo-berri
e411d637b3 feat(gemini): day-0 pricing for gemini-3.6-flash and gemini-3.5-flash-lite 2026-07-21 08:50:40 -07:00
Yuneng Jiang
ff8d8797dd
test(ui): pin memory table page-size behavior on the last page
Changing rows-per-page while on the last page recomputes the page
index from the top visible row, so the table lands on the new last
page instead of an out-of-range one. Pin that, since it depends on
the parent holding the full PaginationState rather than just the
page index.
2026-07-20 23:26:27 -07:00
Yuneng Jiang
879a287ba9
refactor(ui): migrate organizations table onto shared DataTable
The organizations admin table was a hand-rolled tremor/antd table in a single
snake_case file. This moves it onto the shared DataTable and cell library the other
migrated tables use, splitting it into a data-owning OrganizationsPanel, a thin
OrganizationsTable consumer, and a getOrganizationsTableColumns module

The models column no longer uses a per-row accordion whose expand state lived in the
parent; it renders the shared ModelsCell with truncation and a "+N more" tooltip,
matching every other table with a models column. Row actions (Edit, Delete) move into
a per-row overflow menu gated to proxy admins, while the detail view, create modal,
and delete modal stay in the panel. The server-side org id / org alias search stays
wired to the useOrganizations hook, and the table gains an initial-load skeleton plus
a search-aware empty state. The dead sort_by / sort_order filter fields, the misnamed
"Info" column that only ever showed a member count, and an unused refresh affordance
are dropped; the default created_at descending sort is preserved
2026-07-20 22:24:28 -07:00
Yuneng Jiang
e3b453fd0a
refactor(ui): migrate audit logs table onto shared DataTable
Move the Audit Logs table off the hand-rolled antd Table/Pagination onto the
shared DataTable and cell library, matching the other migrated admin tables
(Teams, Virtual Keys, Guardrails)

The single audit_logs.tsx is split into three PascalCase files: AuditLogsPanel
owns the data (server useQuery, pagination and filter state, the row-detail
drawer, and the enterprise preview gate), AuditLogsTable is a thin DataTable
consumer, and AuditLogsTableColumns exposes getAuditLogsTableColumns. The
AuditLogEntry type moves out of the request-logs columns.tsx into the audit
columns file, and AuditLogDrawer stays in the parent unchanged

Server pagination is wired through paginationMode="server" with the shared
footer replacing the standalone antd Pagination, keeping keepPreviousData
semantics so page flips keep rows visible and only the initial load shows the
skeleton. The six filters (Object ID, Changed By, Team ID, Key Hash, Action,
Table) move into a DataTableFilterDrawer plus toolbar with active-filter chips,
each resetting the page to the first. The Object ID cell is the clickable
identity cell that opens the drawer; there is no whole-row navigation, no
selection, and no per-row actions since the table is read-only

The enterprise query is now also gated on premiumUser so the preview path no
longer fires a doomed request for non-premium users
2026-07-20 22:23:54 -07:00
Yuneng Jiang
3a55dda7ec
refactor(ui): migrate memory table onto shared DataTable
Move the admin dashboard Memory table off the hand-rolled antd
<Table> onto the shared DataTable and cell library, matching the
pattern already used by Teams, Virtual Keys, and Guardrails.

MemoryView keeps the data (server useQuery, mutations) and owns the
detail drawer, edit modal, and delete modal; it now renders a thin
MemoryTable consumer plus a getMemoryTableColumns columns file. The
server pagination moves the full PaginationState up to the parent so
the shared footer's rows-per-page selector works, the key-prefix
search runs through the shared toolbar and resets the page on change,
and per-row view/edit/delete collapse into a single overflow menu.
Sorting stays off since the backend returns updated_at DESC.

The old page-reset effect is gone (the page now resets inside the
search handler), so its react-hooks/set-state-in-effect suppression
is pruned. The detail drawer moves into its own MemoryDetailDrawer
component to keep the parent under the complexity budget.
2026-07-20 22:22:28 -07:00
yuneng-jiang
c1b6c4062e
Merge pull request #34070 from BerriAI/litellm_/remaining-table-complexity-053477
refactor(ui): migrate available teams table onto shared DataTable
2026-07-20 21:52:34 -07:00
Mateo Wang
7e66f00fca
Merge pull request #34068 from BerriAI/litellm_codex_auto_drop_params 2026-07-20 23:45:29 -04:00
Mateo Wang
e3f7019fee
Merge pull request #34058 from BerriAI/litellm_mantle_codex_additional_tools
fix(bedrock_mantle): gate unsupported service_tier on drop_params for the Responses API
2026-07-20 23:41:53 -04:00
Yuneng Jiang
484cc12ab6
fix(ui): ignore stale available-teams fetch on unmount or token change 2026-07-20 20:35:51 -07:00
mateo-berri
fadec17a0f Merge origin/litellm_internal_staging into litellm_mantle_codex_additional_tools (resolve overlap with #33228 hoist) 2026-07-20 20:30:34 -07:00
Mateo Wang
07a355e867
Merge pull request #33228 from lyb0307/litellm_bedrock_mantle_codex_additional_tools
fix(bedrock_mantle): hoist Codex additional_tools input items to top-level tools
2026-07-20 23:14:47 -04:00
Yuneng Jiang
fe57eedb5c
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_/remaining-table-complexity-053477 2026-07-20 20:08:50 -07:00
Yuneng Jiang
99f215df68
refactor(ui): migrate available teams table onto shared DataTable 2026-07-20 20:08:42 -07:00
tin-berri
c9b50120ae
Merge pull request #34059 from BerriAI/litellm_lit4629_google_registry
fix(mcp): add Google Sheets, Drive, Calendar, and Docs to the OpenAPI registry
2026-07-20 20:08:21 -07:00