Okta commonly sends SCIM membership removals as a filtered path with no request
body value, e.g. Groups PATCH members[value eq "uid"] and Users PATCH
groups[value eq "tid"]. The patch handlers pulled ids only from op.value, so
these removes were a silent no-op and the member or team was never dropped
Add a linear-time filter parser reused by both the Groups members path and the
Users groups path so the id is taken from the [value eq "..."] filter when
op.value is absent, for add and remove ops. The eq operator is matched
case-insensitively, both quote styles are accepted, and the quoted value is
unescaped. The path-filter fallback only fires when the request body value is
omitted, so an explicit empty value no longer resurrects the filter id, and the
compared value must be quoted per the SCIM filter grammar
The SSO and Email Server settings pages read only stored config, so a gateway
configured entirely through environment variables rendered every field blank
even though both features were live. Rather than add per-endpoint env fallback,
resolve each setting through one typed config object.
A FieldDescriptor names, for one setting, where it lives in the stored row
(db_key), which process env var carries it (env_var), whether it is a secret,
and its effective default. A pure resolve_fields reconciles a descriptor table
against the stored row and the process environment with a fixed precedence and
reports per-field provenance (db, env, default, or unset). The SSO descriptor
table single-sources the field-to-env mapping that the read and write paths
previously duplicated, so they can no longer drift.
get_sso_settings and the /get/config/callbacks alerting block read through the
resolver instead of their own inline fallbacks. get_sso_settings no longer
decrypts stored values into os.environ; decryption happens once inside the
resolver via the pure helper, so a GET stops mutating the process environment.
The SSO response carries provenance so the UI can distinguish an env-sourced
value from a stored one, and secrets are masked at the endpoint (the resolver
returns them unmasked so the login path could consume them). os.environ remains
the runtime carrier; the SSO login and mail-send paths are unchanged.
The settings pages also submit only fields an admin actually edited, so a
rendered mask or env-sourced value is never written back over a working
secret, and generic_scope is a real SSO form field. Omitting a field from
/update/sso_settings clears it, which provider switching relies on; the deeper
write-path concern that behaviour points at is tracked in LIT-4498.
_add_team_members_to_team reconciled membership by reading the complete_team_data
snapshot captured at the start of team_member_add, appending in memory, and
writing the whole members_with_roles array back. Two concurrent /team/member_add
calls for the same team read the same snapshot, so the last write wins and one
member is silently lost. This affects every concurrent team member add, including
the SCIM group PATCH op:add path that routes through team_member_add
Reconcile members_with_roles inside a transaction that locks the team row with
SELECT ... FOR UPDATE before re-reading the current membership, so concurrent
writers serialize on the row lock and each appends onto the other's committed
result. The interactive transaction is exposed through a thin PrismaClient.tx()
passthrough and the locked read is encapsulated in
TeamRepository.get_members_with_roles_locked, and the SCIM group PATCH applies
membership as deltas so concurrent adds are not clobbered
SCIM delete_user removed the user from the legacy team.members column and deleted
their team membership rows, but never pruned members_with_roles, which is the
source of truth ScimTransformations reads for GET /Groups/{id}. A deleted user
therefore lingered as a dangling member reference on every team they belonged to
Prune each of the user's teams directly via team_member_delete before deleting
the user row, and only for teams whose members_with_roles actually contain the
user, so a real DB failure surfaces (the endpoint fails loudly and the user is
kept; SCIM DELETE is idempotent, so the IdP retries) while a user who was never
in a team's members_with_roles stays a no-op. patch_team_membership is left
unchanged for its other callers
* fix(scim): sync team roster and dedup teams for existing-user email upsert
When POST /scim/v2/Users matched an already-existing user by email,
handle_existing_user_by_email raw-wrote the user's teams array but never
touched the team roster, so the user appeared in the group on their profile
yet was absent from the team directly (members_with_roles stayed empty). It
also did not dedup the teams built from repeated SCIM groups.
Route the existing-user team assignment through the same
_handle_team_membership_changes / team_member_add path the PUT update_user
handler uses, so members_with_roles, LiteLLM_TeamMembership, and the user's
teams array stay in sync, and dedup the teams derived from user.groups. The
user_id rewrite to the new userName is preserved and sequenced before the
roster sync so the roster never references a stale primary key.
* fix(scim): surface roster add failures on existing-user email upsert
Route the existing-email upsert's roster sync through patch_team_membership
with a new opt-in raise_on_error flag so a genuine team_member_add failure
propagates instead of being swallowed, and the deduped teams array is only
persisted after the roster sync succeeds. Without this, a failed add left the
endpoint reporting success while user.teams listed a team members_with_roles
never received.
The benign already-a-member case stays a no-op even under the strict path, and
the flag defaults to False so the PUT update_user, PATCH patch_user, and group
callers keep their existing best-effort behavior. SCIM POST is idempotent, so
surfacing the error lets the IdP retry and converge.
* fix(scim): surface roster removal failures symmetrically with adds
Make team_member_delete failures fail loud under the strict roster sync used by
the existing-email upsert, mirroring the add path, so a swallowed removal can no
longer let the user's teams array drop a team the roster still holds. The
idempotent case where the user is already absent from the team stays a no-op,
matching how an add treats the user already being in the team. Best-effort
behavior is preserved for the default raise_on_error=False callers.
* fix(scim): use members_with_roles as the source of truth for group membership
SCIM group provisioning tracked membership inconsistently. Team creation and
the real team endpoints persist membership in members_with_roles (and each
member's user.teams), but the SCIM group PATCH handler and the GET /Groups
listing read the legacy team.members String[] column, which team creation never
populates. Seeding a PATCH result from that empty column made an Okta "add
member" operation recompute the member set from scratch and silently drop
everyone already in the team, so users ended up missing from the groups they
were provisioned into. Reading the same empty column on GET /Groups reported an
empty member list back to the IdP, which drove repeated re-provisioning.
Separately, add_new_member appended the team id to user.teams with an
unconditional array push. Under the concurrent group PATCHes an IdP sends during
a reconcile, each request passed the members_with_roles duplicate check and
pushed, so user.teams accumulated duplicate ids for the same team. A duplicate
also breaks auth logic that keys off the number of teams a user belongs to.
Read current membership from members_with_roles in the SCIM group PATCH seed and
the GET /Groups listing, and make the user.teams append idempotent via a
filtered update that no-ops once the team is present.
Resolves LIT-4283
* fix(scim): address review; atomic user-creation and stop writing legacy members
Keep the concurrent-safe team append but create the user via an atomic upsert
(create-or-update) instead of a check-then-create, so provisioning the same new
user concurrently cannot race into a duplicate-key failure; the team is still
appended idempotently by a filtered update so an existing user gets no duplicate
team id. Stop writing the legacy team.members column in the group PATCH apply so
the only membership record is the source of truth (members_with_roles plus each
member's user.teams), reconciled by team_member_add/team_member_delete.
Tests: existing add_new_member and team-creation mocks updated to the upsert
plus filtered-append shape, and new tests cover atomic creation and that the
PATCH apply does not write the legacy members column.
Relocates ui/litellm-dashboard/e2e_tests to tests/e2e/ui so all end to end
suites live under tests/e2e. The suite stays in TypeScript and becomes a
self-contained npm package with its own package.json, lockfile and tsconfig
instead of leaning on the dashboard's toolchain; the dashboard drops its
@playwright/test dependency, e2e scripts and knip/vitest/tsconfig carve-outs.
CI paths follow the move: both CircleCI jobs (main e2e and the
SERVER_ROOT_PATH migration smoke) and the test_server_root_path workflow now
install and run Playwright from tests/e2e/ui, with the node cache keyed on
both lockfiles. classify_changes.sh treats tests/e2e/ui as client so spec
edits keep skipping backend jobs. The suite's mock LLM fixture is excluded
from the e2e basedpyright zero-error gate in pyrightconfig.json since it
belongs to the TS suite, not the typed Python harness.
* fix(proxy): share CLI SSO login sessions across workers without enable_redis_auth_cache
* fix(proxy): make CLI SSO flow state redis-authoritative across workers
The CLI SSO flow is stored in a DualCache whose get_cache is memory-first, so
the worker that served /sso/cli/start keeps serving its stale in-memory flow and
never observes the sso_complete/session_data update another worker writes during
the OAuth callback. Attaching Redis alone is not enough; poll on the original
worker returns pending forever.
Read and write the flow directly through the attached Redis backend when present
so every worker sees the same authoritative state, falling back to the in-memory
DualCache only when no Redis is configured.
* fix(proxy): serialize CLI SSO flow as JSON for the redis round trip
RedisCache stores values via str(value) and parses reads with
json.loads then ast.literal_eval. The completed flow contains a
LitellmUserRoles enum in session_data.user_role, whose repr is not a
parseable literal, so any worker reading the completed flow from redis
raised SyntaxError and returned 400 "CLI login session not found".
Writing the flow as json.dumps makes the round trip lossless (the enum
is a str subclass) and fails loudly at write time if a non-serializable
value is ever added to the flow.
* fix(proxy): point CLI SSO session-not-found hint at configuring Redis
The error message and warning still told users to set enable_redis_auth_cache,
but the CLI SSO session cache now gets Redis unconditionally whenever one is
configured, so that flag no longer affects CLI login
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
* feat(guardrails): add only_scan_new_messages for per-session incremental scanning
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(guardrails): use fixed TTL constant and revert unrelated test formatting
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(guardrails): run only_scan_new_messages in the unified apply_guardrail path
The initial wiring lived in BedrockGuardrail.async_pre_call_hook, but the proxy
routes Bedrock through the unified apply_guardrail interface, so the flag had no
effect live. Move incremental selection into apply_guardrail: filter the flat
texts list against per-session scanned hashes, skip the Bedrock call when nothing
is new, and mark hashes only after a successful (non-blocked) scan. Full-context
fallback is preserved when there is no session id, the cache is unavailable, or a
masking guardrail is configured.
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* test(guardrails): cover session-id fallbacks and mark_texts_scanned guards
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* fix(guardrails): fall back to full scan when incremental guardrail masks content, use shared cache
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* test(guardrails): cover generic agent multi-turn incremental scan
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* test(guardrails): cover incremental scan cache resolver fallbacks
Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
* test(guardrails): cover flag interactions and /v1/messages incremental scan semantics
* feat(guardrails): make GUARDRAIL_SCANNED_MESSAGES_CACHE_TTL_SECONDS env configurable
* test(guardrails): prove skip_system/skip_tool are enforced upstream of incremental scan
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
Co-authored-by: Yucheng Zhu <yucheng@berri.ai>
Every dashboard login mints a 24h session key whose max_budget comes from
litellm.max_ui_session_budget, and all dashboard LLM traffic (playground,
auto router per-tier Test Connection probes) spends against and is gated
by that one key. The $0.25 default locked sessions out mid-testing with
"Budget has been exceeded ... Max budget: 0.25" and the setting appeared
in no docs, no UI, and no error text, so it read as a hardcoded cap.
Raise the default to $1. Give the setting an explicit typed arm in the
config loader (float coercion for env-var strings, null disables the
cap). Surface it on the Admin UI General settings tab through the
existing litellm_settings bridge as a new Dollar field type (positive
USD, unbounded above; the existing Float type is validated to (0, 1] for
fractions), with a spec-level default so clearing the field restores $1
instead of silently removing the cap, and enroll it in
LITELLM_SETTINGS_SAFE_DB_OVERRIDES so UI edits propagate to peer workers.
The UI theme and logging-callback read endpoints reported only stored
config while the features resolve their values from the process
environment, so a gateway configured purely through env vars showed
blank settings pages even though branding rendered and callbacks fired.
/get/ui_theme_settings read only litellm_settings.ui_theme_config;
logo_url and favicon_url now fall back to UI_LOGO_PATH and
LITELLM_FAVICON_URL when the stored config leaves them blank.
process_callback (the logging-callbacks block of /get/config/callbacks)
reported every callback env var as unset unless it lived in the config
environment_variables overlay; it now falls back to os.getenv, matching
the slack block. Secret values stay redacted for non-admins via the
existing callback role gate.
Stored values keep winning over the environment, so the UI-driven flow
is unchanged.
Resolves LIT-4667
The Cache Settings page read only the database row, so a response cache
pointed at Redis purely through REDIS_* env vars showed a blank page
while the cache worked. It also masked credentials on read with a
partial-reveal string and re-persisted whatever the form submitted, so
an admin who edited an unrelated field and pressed Save wrote the mask
string over the real Redis password, breaking auth.
GET /cache/settings now overlays the same REDIS_* kwargs the runtime
resolves from when the stored config leaves a field unset, and redacts
credentials with a fixed marker. POST /cache/settings restores the
stored secret behind any credential echoed back as the marker or omitted,
and drops an env-sourced marker rather than persisting it; the response
no longer echoes plaintext credentials. The connection test resolves a
redacted credential back to the stored value the same way. The dashboard
never prefills a credential and drops the marker from the save payload,
mirroring the Coordination Redis tab.
Resolves LIT-4315
JWT auth built UserAPIKeyAuth without user_email even though the resolved
user row and the JWT email claim were both available, so the user_email
label on Prometheus metrics and user_api_key_user_email in
StandardLogging/SpendLogs metadata were always None for JWT traffic.
Plumb user_email through JWTAuthBuilderResult: auth_builder returns the
user row email when set, falling back to the user_email_jwt_field claim
(covers the scope-based proxy-admin path where no user row is loaded).
The JWT branch now stamps it on the proxy-admin return, the standard
valid_token, and the auto-registered virtual key object.
Resolves LIT-4238
The add_deployment and get_credentials background jobs that keep a multi-pod
deployment in sync with config-in-DB objects (models, credentials, guardrails,
general settings, etc.) polled the database on a hardcoded 30s interval, with
no way to trade convergence latency against DB load.
Expose it as the general_setting proxy_config_reload_interval_seconds (env
PROXY_CONFIG_RELOAD_INTERVAL_SECONDS parsed via get_env_int, default 30),
threaded like the existing proxy_batch_polling_interval knob, and surface it on
the admin general-settings page so it is reachable from the dashboard and
persists to the DB for all pods. Non-positive values are rejected at the UI
(gt=0) and fall back to 30s with a warning on the env/config/DB paths.
save_config wrote the entire merged config to the DB config table on every call. Because get_config() resolves os.environ/ placeholders to plaintext and merges the environment_variables section, any endpoint that does get_config() then mutates one section then calls save_config() (/add/allowed_ip, delete_callback, model and cost-tracking settings, and others) accidentally persisted an environment_variables row holding all YAML/OS-sourced env vars. Once that row existed the DB overlay shadowed YAML and container env on every subsequent startup, so config/env changes were silently ignored
save_config now pops environment_variables from the DB write unless the caller passes include_env_vars=True. The dedicated /config/update path already writes env vars per-section via _upsert_section, so no current caller needs to opt in
Resolves LIT-2009
* refactor(auth): derive temp budget bump without mutation, tz-aware auth datetimes
_update_key_budget_with_temp_budget_increase mutated max_budget in place, so correctness depended on every resolution path handing it a fresh copy of the cached token; one future re-cache of a live token would compound the bump per request. Return a model_copy instead so no caller can leak an increased budget into shared state.
Also fixes the three remaining DTZ005 naive datetime.now() calls in user_api_key_auth.py (auth span start, builder start_time, service-log end_time; all consumers convert to epoch or subtract same-pair datetimes) and ratchets the DTZ005 strict budget 244 -> 241.
* test: pin non-mutation of the temp budget helper input
Adversarial mutation-testing showed reverting the helper to in-place mutation still passed every test: the cache's copy-on-read layer masks the mutation in the integration test and the direct unit test only inspected the return value. Assert the input object is left untouched and the result is a distinct object so the purity guarantee itself is load-bearing.
Budgets reset at midnight in the configured timezone with no way to control
the time of day, so a drained daily budget surfaces as an overnight incident.
Add a litellm_settings.budget_reset_time option (e.g. "12:00") that shifts
day/week/month resets to a configurable wall-clock time in the existing
timezone, so the end of the budget window lands during business hours.
The reset time is parsed once into an immutable BudgetResetSettings and
injected into the reset job (constructor) and computation, rather than read
from a module-level global at call time. A malformed value fails fast at
startup. Sub-day durations ignore the offset. Unset preserves midnight resets.
* fix(model_armor): sanitize error details by default
Generated with AI
Co-Authored-By: Claude Code
* fix(model_armor): sanitize handler-raised HTTP errors and redact scanned content in guardrail logging
The async HTTP handler raises MaskedHTTPStatusError on any non-2xx via
raise_for_status, so the non-200 branch in make_model_armor_request never ran
against a live API and the raw upstream body reached callers and logs. Catch
the raised error and build the sanitized detail from the response status
Replace the empty-dict guardrail logging payload with field-level redaction of
the keys that echo scanned content (text, sanitizedText, findings) so guardrail
traces keep filter states and block reasons while scanned content stays out
Restore the upstream status code in the sanitized error detail, read guardrail
metadata from the same key the hooks write, and keep guardrail_status within
its typed literal values
* fix(model_armor): bound redactor recursion depth and allowlist it in the recursion detector
_redact_scanned_content walks provider JSON bounded by _REDACT_MAX_DEPTH=20 and
fails closed by returning the redaction sentinel at the cap
* fix(model_armor): honor fail_on_error for upstream API failures
API failures now raise a dedicated ModelArmorAPIError so hooks can tell them
apart from content-block HTTPExceptions; fail_on_error=False lets the request
proceed on a Model Armor outage again while fail-closed configs get the same
sanitized 400 as before
Also addresses review notes: sanitize_error_detail constructor annotation
matches the nullable config field, redaction is owned by the metadata write
sites so _process_response no longer re-applies it, and the request and
response debug log branches move into helpers
* test(model_armor): cover fail_on_error routing on during-call, post-call, streaming, and file-scan paths
* chore: remove accidentally committed pytest cache files
* fix(model_armor): keep sanitize_error_detail coerced across in-memory config reloads
update_in_memory_litellm_params assigns raw LitellmParams fields, so a hot
reloaded config carrying an explicit null would silently disable sanitization;
re-apply the only-explicit-False-opts-out coercion after the update
* fix(model_armor): redact matched malicious URIs and reuse the shared recursion depth constant
maliciousUriMatchedItems echoes the caller-supplied URL including path and
query, so it joins the scanned-content key set; the redactor depth cap now
comes from DEFAULT_MAX_RECURSE_DEPTH in litellm constants instead of a local
literal
* fix(model_armor): keep API failures out of the intervention trace status
Fail-closed upstream failures re-raise ModelArmorAPIError instead of
converting to HTTPException(400), so the shared guardrail logging keeps
recording them as guardrail_failed_to_respond while content blocks stay
guardrail_intervened. Callers see the same 500 shape as before this PR,
with the sanitized message
* chore(model_armor): drop explanatory comment per repository comment policy
---------
Co-authored-by: eugene-yao-zocdoc <eugene.yao@zocdoc.com>
* adding deepkeep as custom guardrail
* adding deepkeep as a custom guardrail
* adding deepkeep as a custom guardrail (hooks)
* adding litellm/proxy/_experimental/out/ to .gitignore
* adding deepkeep as custom guardrail in litellm
* removing sentinel_fortress
* comparing schema.prisma files
* fix(deepkeep): address greptile review comments
- extra_headers: fix type annotation (list -> Dict[str, str]) and actually
merge them into _build_request_headers() so user-configured headers
reach the DeepKeep API
- user_api_key_hash: only fall back to user_api_key_token when no
explicit hash is already set, avoiding silent overwrite
- apply_guardrail: preserve tool_calls and structured_messages in the
return value so downstream callers don't lose that content
Adds tests for all four fixes.
* fix(deepkeep): address greptile review comments
- extra_headers: fix type annotation (list -> Dict[str, str]) and actually
merge them into _build_request_headers() so user-configured headers
reach the DeepKeep API
- user_api_key_hash: only fall back to user_api_key_token when no
explicit hash is already set, avoiding silent overwrite
- apply_guardrail: preserve tool_calls and structured_messages in the
return value so downstream callers don't lose that content
Adds tests for all four fixes.
* fix: add missing __init__.py and allowlist entries for upstream merge
- tests/test_litellm/proxy/client/__init__.py: fixes pytest collection
collision with tests/test_litellm/models/test_models.py (same basename)
- tests/test_litellm/models/__init__.py: same fix
- backend/routes/allowlist.py: add /config_overrides/ and /v1/unified_access_group
prefixes for new routes added by upstream
* fix(ui/tests): resolve frontend-lint failures in new test files
- useLogDetails.test.ts: add Wrapper.displayName, replace 'null as any'
with null, type resolveCall promise resolver properly
- usePaginatedDailyActivity.test.ts: remove unused waitFor import,
add Wrapper.displayName, change Record<string,any> to Record<string,unknown>
- UsageViewSelect.adminFiltering.test.tsx: replace all props:any with
explicit SelectProps/BadgeProps/SelectOption types, replace (X as any).displayName
with direct X.displayName assignment
no-explicit-any count: 2034 (budget: 2040). Prettier check: clean.
* fix(ui): sync proxy/_experimental/out/ exactly to upstream
245 stale JS chunk files from earlier merges were left in the out/
directory but had been deleted in upstream. The Docker image in CI is
built by copying this directory verbatim, so the stale artifacts caused
the SERVER_ROOT_PATH redirect E2E to fail.
Synced by: git checkout upstream/litellm_internal_staging -- out/ (adds
new files) + git rm on every file present in HEAD but absent from
upstream.
* Update litellm/proxy/guardrails/guardrail_hooks/deepkeep/deepkeep.py
Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>
* fix(makefile): fall back to upstream/litellm_internal_staging for strict-budget gate
origin/litellm_internal_staging exists on BerriAI's CI but not on forks
that use a different remote name (e.g. Azure DevOps as origin). Fall
back to upstream/litellm_internal_staging when the origin ref is absent.
* linter reformat
* fix(deepkeep): apply guardrail tool/tool_call redactions from API response
When DeepKeep returns GUARDRAIL_INTERVENED with redacted tools or
tool_calls, the previous code ignored those redactions and forwarded
the original (potentially sensitive) values to the model — a guardrail
bypass for content embedded in tool schemas or function arguments.
Fix: prefer response_json["tools"] / response_json["tool_calls"] when
present, falling back to the originals only when the guardrail did not
return replacements — consistent with the existing pattern for texts and
images.
Refactor _build_return_inputs() into a private static helper to keep
apply_guardrail() under the PLR0915 statement limit (50).
Adds test_apply_guardrail_applies_tool_redactions_from_response to
assert that redacted tool payloads from the API response are used.
* Update litellm/proxy/guardrails/guardrail_hooks/deepkeep/deepkeep.py
Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>
* fix(lint): move base-ref fallback into ruff_strict_gate.py; revert Makefile
The previous Makefile fix had a shell bug: 'git rev-parse --verify'
writes the resolved SHA to stdout, so the $$(...) substitution captured
both the SHA and the echo output, handing '--base <sha>\norigin/...' as
two tokens to the Python script, causing exit code 1 in CI.
Fix: revert Makefile to its original single-line invocation and add
_resolve_base() to ruff_strict_gate.py. The function checks whether the
requested ref resolves; if not, it tries the 'upstream/' equivalent
before falling back to the original ref (letting git emit a clear error).
Behaviour in BerriAI CI: origin/litellm_internal_staging resolves → used
as before, no change.
Behaviour on forks with a different 'origin': falls back to
upstream/litellm_internal_staging transparently.
* fix(lint): fix UP006/UP045/F401 in changed files; add depth guard to check_any_discipline
- Replace Dict/List/Optional/Tuple typing imports with built-in equivalents
(UP006, UP045) across files touched in this PR diff, then clean up
the now-unused typing imports (F401).
- Add _MAX_CONTAINS_ANY_DEPTH guard to check_any_discipline.contains_any()
to prevent RecursionError on deeply-nested mypy types.
* fix(lint): resolve all three CI lint job failures
1. lint (ruff_strict_gate) — UP006/UP045/F401 violations introduced on
changed lines. Fixed Dict/List/Optional/Tuple → built-in equivalents
across every file in the PR diff; cleaned up now-unused typing imports.
2. any-discipline — RecursionError in check_any_discipline.contains_any()
on deeply-nested mypy types. Upstream fixed this by converting to an
iterative stack-based algorithm (merged). Also added deepkeep.py to
any-discipline-budget.json via 'make lint-any-budget-update' so the
new file's Any count is baselined instead of failing against the
zero-baseline default.
3. basedpyright reportMissingParameterType — **kwargs in DeepKeepGuardrail
__init__ lacked a type annotation. Added **kwargs: Any.
* Update litellm/deepkeep_tilt_config.yaml
Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>
* fix(lint): black reformat after merge
* fix(deepkeep): honour empty-list replacements in _build_return_inputs
When DeepKeep returns GUARDRAIL_INTERVENED with an intentional empty
replacement (e.g. texts:[], tool_calls:[]) the previous truthiness check
treated [] as absent and forwarded the original content downstream —
a guardrail bypass for any case where the firewall wants to fully clear
a field.
Fix: replace all response_json.get(field) truthiness checks with
'is not None' comparisons so that an empty list is respected as a
deliberate replacement. Applies to texts, images, tools, tool_calls,
and the original-input fallback guards.
Adds test_apply_guardrail_honours_empty_list_replacements.
* fix(test): replace live httpbin.org call with mocked transport in test_pass_through_with_httpbin_redirect
Root cause of OOM: the test made a real HTTP request to https://httpbin.org
inside a pytest-xdist worker. Under memory pressure the worker's httpx client
and redirect-following logic allocated enough virtual memory to trip the OOM
killer (confirmed by ulimit -v 16GB reproducing the crash with 'node down: Not
properly terminated' on this exact test).
Fix: replace the real network call with a custom httpx.AsyncBaseTransport that
returns a pre-built 302 -> 200 response sequence in-memory. The test now runs
hermetically with no network dependency and no excess memory allocation.
ulimit -v 16GB: 24,284 passed (0 crashes) after this fix.
* fix: merge upstream/litellm_internal_staging (197 commits), resolve conflicts
7 conflicts resolved:
- 6 Python files: upstream added new code with old-style typing (Optional,
Dict, List) on lines where we had ruff-fixed modern syntax (str | None,
dict, list). Took upstream's version then re-ran ruff UP006/UP045/F401
--fix to keep both the new content and ruff compliance.
- test_openapi_compliance.py: upstream replaced 'role' with 'steps' in
output_fields and updated the spec comment. Took upstream's version.
Also: added _resolve_base() fallback to type_check_gate.py and removed
the hard 'git fetch origin litellm_internal_staging' from the Makefile's
lint-basedpyright target (same pattern as ruff_strict_gate.py fix).
* fix: merge upstream (41 commits), resolve .gitignore conflict, fix BLE001
- .gitignore: upstream removed package.json/out/ ignore entries; took theirs
- deepkeep.py: added '# noqa: BLE001' on catch-all Exception handler
(BLE001 rule newly enforced in ruff-strict-budget)
- type_check_gate.py: added _resolve_base() fallback for basedpyright gate
- Makefile: removed hard 'git fetch origin' from lint-basedpyright target
* fix: merge upstream (57 commits), resolve conflicts
- Makefile: upstream added lint-fetch-base target; made it tolerant of
missing origin/litellm_internal_staging (git fetch || true)
- test_websearch_chat_completion.py: took upstream's new assertions and
skipif marker
- anthropic_cache_control_hook.py: upstream added new code using List/Dict/Tuple
which were undefined after our earlier UP006 cleanup; replaced with
built-in list/dict/tuple
* fix(coverage): revert ruff UP006/UP045 changes on upstream files
The previous ruff fixes (Dict→dict, Optional→X|None) on 7 upstream files
added ~500 changed lines of pure type-annotation no-ops to our PR diff.
codecov/patch penalised these uncovered lines, dropping patch coverage
to 51.35% (target 61.83%).
Fix: revert these files to exactly match upstream/litellm_internal_staging.
The ruff_strict_gate still passes because the violations exist equally in
both the base and HEAD (total == base_count → no breach).
* fix: merge upstream (130 commits), resolve Makefile + base_email conflicts
- Makefile: upstream changed lint deps to $(LINT_DEP_INSTALL)/$(LINT_DEP_BASE);
kept our --base removal (handled by _resolve_base in Python scripts)
- base_email.py: took upstream's dedup cache addition
- deepkeep.py: ruff format after merge
* chore: remove lint/format-only changes and non-feature files
Revert all lint-infra and black/ruff-reformat-only changes back to
upstream/litellm_internal_staging so the PR diff shows only the DeepKeep
guardrail feature:
- Makefile, scripts/ruff_strict_gate.py, scripts/type_check_gate.py
(lint-gate infra)
- credential_migration.py + enterprise/* + assorted test files
(black-reformat / xdist test-isolation drift)
- backend/routes/allowlist.py (merge glue)
Remove non-feature local artifacts: build-and-push.sh,
deepkeep_tilt_config.yaml, stray __init__.py collision shims, and
unrelated UI test files.
* fix(lint): add reason to BLE001 noqa to satisfy type-discipline gate (LIT003)
The type-discipline budget ratcheted LIT003's ceiling to 292 as upstream
fixed reasonless suppressions, so our '# noqa: BLE001' (code but no
reason) tipped the total to 293 and failed CI. Add a reason per the
required '# noqa: CODE # <reason>' shape.
* fix(deepkeep): apply structured_messages redactions returned by the guardrail API
_build_return_inputs dropped any structured_messages the DeepKeep API returned and
always forwarded the original input, so redactions on that field never took effect.
Check the response first, same as texts/images/tools/tool_calls
* chore(ui): drop redundant preserve prop from the guardrail form
preserve defaults to true in rc-field-form (isMergedPreserve falls back to true when
unset), so the explicit prop changed nothing and only widened this PR's blast radius
to every guardrail provider in the shared form
* fix(deepkeep): stop extra_headers list from crashing the guardrail call and name the real firewall id config key
litellm_params.extra_headers is a list of header names to forward, so passing it
straight into dict.update raised ValueError and, under fail_closed, took the request
down with it. Only merge mapping values and warn otherwise
The docstring example and the missing-secret error both said firewall_id, but
initialize_guardrail only reads deepkeep_firewall_id, so anyone following them
had their value silently ignored
* refactor(proxy): drop normalize_callback change; split to its own PR (#33905)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yaniv Israel <yaniv@deepkeep.ai>
Co-authored-by: DK-yaniv <164404355+DK-yaniv@users.noreply.github.com>
Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The MCP OAuth DCR relay's client-facing /register response handed back LiteLLM's own
<base>/callback as the client's redirect_uris on every arm except the true bridge relay.
A spec-compliant OAuth 2.1 DCR client (e.g. Open WebUI over Streamable HTTP) adopts that
value for its subsequent /authorize calls, so /callback redirects to itself; the second
hit carries the client's opaque state, fails to decrypt, and surfaces as the LIT-4197
"oauth_state ... Incorrect padding" error, making pass-through MCP OAuth unusable against
real DCR-capable upstreams.
The short-circuit arm keeps registering the gateway callback upstream (unchanged) and now
echoes the client's own redirect_uris back to the client across all register arms. Since
the client then authorizes with its own separate-origin redirect, the /authorize rejection
hint now points operators to MCP_TRUSTED_REDIRECT_ORIGINS, the mechanism a legitimate
cross-origin OAuth client needs (auto-trusting a DCR-registered redirect would reintroduce
the VERIA-57 open-redirect vector, since dynamic registration is unauthenticated).
temp_budget_increase was only applied on the DB-fetch path of _user_api_key_auth_builder, so a key served from the auth cache reverted to its original max_budget and was wrongly blocked with BudgetExceededError once spend crossed the original budget while staying under the effective budget.
Move _update_key_budget_with_temp_budget_increase out of the DB-only branch so it runs for every resolved token regardless of source. The cache stores the original budget and each cache hit returns a fresh model_copy(), so this never double-applies.
Fixes#25760
Co-authored-by: shivam <shivam@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The JWT first-login upsert in get_user_object creates the user row by
merging default_internal_user_params straight into table.create, so a
configured budget_duration landed with budget_reset_at NULL. The reset
sweep now heals such rows (PR #33623), but until the next sweep the row
shows a null reset time and its first window starts at the sweep instead
of one full duration after creation. Compute budget_reset_at at creation
like every other write path (/user/new, UI SSO, /key/generate, /team/new)
already does
Internal users seeded from default_internal_user_params (SSO/JWT first-login
upsert, or /user/new without an explicit budget_reset_at) get budget_duration
set but budget_reset_at = NULL. The ResetBudgetJob user/team queries filter on
{"budget_reset_at": {"lt": now}}, which never matches NULL, so these rows are
never reset: their spend accumulates for the lifetime of the row and silently
exceeds max_budget with no periodic reset.
The budget-table query already handles this by OR-ing in a
{budget_reset_at IS NULL AND budget_duration IS NOT NULL} branch. Apply the
same pattern to the user and team reset queries in PrismaClient.get_data.
Adds a regression test asserting both the user and team reset queries select
NULL-budget_reset_at rows that have a budget_duration.
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Staging now contains #33153, whose final rounds made _extract_upstream_auth_failure a thin delegate
to upstream_auth_challenge and introduced the response-level iterator this branch predates. The
resolution completes the consolidation both branches were converging on: iter_exception_tree
(faults/traversal.py) is the one tree walk, _iter_upstream_responses is rebuilt on top of it instead
of carrying a second copy of the traversal, the manager keeps the delegate, and the semantic filter
port from this branch stands. Test conflicts were append-append and both sides are kept
