JWT auth built UserAPIKeyAuth without user_email even though the resolved
user row and the JWT email claim were both available, so the user_email
label on Prometheus metrics and user_api_key_user_email in
StandardLogging/SpendLogs metadata were always None for JWT traffic.
Plumb user_email through JWTAuthBuilderResult: auth_builder returns the
user row email when set, falling back to the user_email_jwt_field claim
(covers the scope-based proxy-admin path where no user row is loaded).
The JWT branch now stamps it on the proxy-admin return, the standard
valid_token, and the auto-registered virtual key object.
Resolves LIT-4238
The add_deployment and get_credentials background jobs that keep a multi-pod
deployment in sync with config-in-DB objects (models, credentials, guardrails,
general settings, etc.) polled the database on a hardcoded 30s interval, with
no way to trade convergence latency against DB load.
Expose it as the general_setting proxy_config_reload_interval_seconds (env
PROXY_CONFIG_RELOAD_INTERVAL_SECONDS parsed via get_env_int, default 30),
threaded like the existing proxy_batch_polling_interval knob, and surface it on
the admin general-settings page so it is reachable from the dashboard and
persists to the DB for all pods. Non-positive values are rejected at the UI
(gt=0) and fall back to 30s with a warning on the env/config/DB paths.
The async @client wrapper stamped the global litellm.num_retries onto the raised
exception via setattr(e, "num_retries", ...), even on router calls where the
request-level num_retries had already been popped and resolved. async_function_with_retries
then adopted that stamped global value, overwriting the request-level num_retries it had
correctly resolved. So a per-request num_retries (request body or x-litellm-num-retries
header) was silently ignored whenever litellm_settings.num_retries was set.
Only stamp num_retries on the exception when the call itself carried one (an explicit
request value or a deployment's litellm_params.num_retries), never the global fallback.
The router already resolves the global via self.num_retries, so leaving the exception
unset preserves the request-level value and lets the per-deployment path set it when present.
Resolves LIT-4516
save_config wrote the entire merged config to the DB config table on every call. Because get_config() resolves os.environ/ placeholders to plaintext and merges the environment_variables section, any endpoint that does get_config() then mutates one section then calls save_config() (/add/allowed_ip, delete_callback, model and cost-tracking settings, and others) accidentally persisted an environment_variables row holding all YAML/OS-sourced env vars. Once that row existed the DB overlay shadowed YAML and container env on every subsequent startup, so config/env changes were silently ignored
save_config now pops environment_variables from the DB write unless the caller passes include_env_vars=True. The dedicated /config/update path already writes env vars per-section via _upsert_section, so no current caller needs to opt in
Resolves LIT-2009
* fix(router): stop per-deployment num_retries from double-counting as provider max_retries
A model group with one deployment and num_retries set in the deployment's
litellm_params sent (1 + num_retries) ** 2 requests upstream instead of
1 + num_retries. The deployment's num_retries reached litellm.completion, which
copied it onto max_retries and set it on the provider client, so the provider SDK
retried num_retries times inside each of the Router's 1 + num_retries attempts.
The Router is the sole retry owner for routed calls, so completion() now forces the
provider-SDK max_retries to 0 whenever the call originates from the Router/proxy
(detected via model_group in the request metadata) and only keeps the num_retries
to max_retries alias for direct, non-routed litellm calls (the instructor use case).
This also stops a request- or deployment-level max_retries from nesting on top of
the Router's retries.
Resolves LIT-4385
* fix(router): make router-origin check robust and close test clients
Address review: detect the router marker in both metadata and litellm_metadata
independently (a non-empty metadata without model_group no longer hides a
model_group in litellm_metadata), and close the injected async clients in the
test fixture.
* test(e2e): add other suite covering master-key auth and health lifecycle
Covers the other.* holding-pen cells that were uncovered: master-key
valid_allows/invalid_denied on the admin /user/list gate, and the
lifecycle probes liveness.ping, readiness.public_probe,
readiness.reports_db_status, and readiness_details.authenticated_diagnostics.
New tests/e2e/other/ suite on the shared ProxyClient; the health probes
send no auth header to prove the public routes need no credential, and the
details route is asserted to reject an anonymous caller while exposing
version/db diagnostics to the master key.
* test(e2e): cover block_code_execution and openai_moderation guardrails
Extends the guardrails suite with two built-in guardrails registered per
request (default_on=False, opted in via the chat body's guardrails selector)
so neither intercepts unrelated traffic on the shared proxy.
block_code_execution.pre_call.blocks: a python code block plus a run-this
request is intercepted with the canned content-blocked message and the model
never runs, while the same code block asked about with don't-run-it reaches
the model. Verified live.
openai_moderations.pre_call.blocks: a flagged prompt is rejected 400 naming
the moderation policy while a benign prompt passes. The guardrail calls
OpenAI's moderation API; verifying it needs an OpenAI key with moderation
quota (this account currently 429s the moderation endpoint).
Adds a shared create_backend_model helper and a generic register() plus
per-request guardrails/max_tokens on the client so more built-ins can reuse
the same path.
* test(e2e): cover presidio PII masking (pre_call + post_call)
Registers a presidio guardrail per request (default_on=False) with the
analyzer/anonymizer bases supplied in the registration params, so the test
controls its own dependency and needs no proxy restart.
presidio.pre_call.masks: a repeat-verbatim request comes back with the
<EMAIL_ADDRESS> placeholder and never the raw email, proving the prompt was
anonymized before the model saw it.
presidio.post_call.masks: with apply_to_output the model's own emitted email
is masked on the way out, so the caller never receives the raw value.
Both verified live against real presidio analyzer + anonymizer containers.
logging_only is intentionally not covered: /spend/logs exposes no prompt
messages to read back the masked log, and a logging_only run also masked the
response, contradicting its contract; noted in the module docstring for a
follow-up.
* test(e2e): cover presidio logging_only masking via OTEL read-back
Adds the third presidio cell, guardrail.presidio.logging_only.masks. The
logging_only contract (mask what is logged, do not block) is verified by
reading the request's gen-AI span back from the real OTEL destination: the
span's gen_ai.input.messages attribute carries the <EMAIL_ADDRESS> placeholder,
never the raw email, and the call itself is not blocked.
Reads the trace via the shared OtelReader, promoted from logging/ to the suite
root so both suites use it. The masked prompt is polled to a deadline because
logging_only masks the payload asynchronously and the span can briefly export
before the mask lands. Drops the throwaway chat_send in favor of the existing
transport.send for the call-id capture.
* fix(e2e): tolerate cross-pod guardrail sync delay in team-opt-out test
Stage runs multiple gateway pods behind the shared key. POST /guardrails
registers a new default-on guardrail in-process immediately only on the
pod that served the create call; every other pod picks it up on its next
periodic DB sync (proxy_server.py, every 30s), so the very next chat call
can race a pod that has not synced yet. Poll to a 40s deadline instead of
asserting on the first response, matching the existing pattern in
test_budget_reset_advances_e2e.py.
* test(e2e): cover a guardrail on the MCP tool-call path (content_filter pre_mcp_call)
Adds guardrail.litellm_content_filter.pre_mcp_call.blocks: against the real
Datadog MCP server, a content_filter guardrail configured mode=pre_mcp_call
blocks a banned keyword in an MCP tool call's arguments with HTTP 400 attributed
to the pre_mcp_call hook, and lets a clean argument reach the upstream server.
The guardrail attaches with default_on because per-key/request guardrail
selection is dropped from the synthetic MCP request the hook sees; the banned
keyword is unique per run so default_on only intercepts this test's own call.
mode must be pre_mcp_call - a pre_call config silently no-ops on tools/call
because the event type is rewritten for call_mcp_tool.
Drives the tool directly via /mcp-rest/tools/call for a deterministic check of
the same pre_mcp_call enforcement the OpenAI-SDK chat path hits when a model
invokes an MCP tool.
* fix(e2e): mid-conversation messages test uses client.proxy not client.gateway
EndpointsClient exposes .proxy after the Gateway->ProxyClient rename; the
mid-conversation system test still referenced .gateway, which fails the e2e
basedpyright gate. Aligns it with the rest of the harness.
* test(e2e): address review on the guardrail coverage
MCP tool-call guardrail: poll the banned call until the guardrail is enforced
instead of asserting on the first call, so the control-plane -> data-plane
guardrail sync cannot race the check into a false pass-through; add a repeat
banned call after enforcement to guard against a partial-propagation state.
OpenAI moderation: distinguish a moderation-endpoint 429 (rate limit / no
moderation quota) from a guardrail failure, so an account-capability gap reads
as such rather than as "did not block". Runs green with a moderation-capable key.
* test(e2e): close partial-propagation false-pass in MCP guardrail block test
The single post-block repeat call could be load-balanced back to the same
already-synced data-plane pod, so the test could pass while another pod still
lacked the guardrail and let the banned MCP call reach Datadog. Anchor a wait to
the guardrail create time (every pod is guaranteed to have DB-synced only after a
full ~30s sync interval), then require the banned call to stay blocked across
several attempts; a pass-through after that window is a real leak, not a race.
* test(e2e): drop xfail-style rate-limit branch from openai_moderation test
OpenAI's /v1/moderations is free and returns 200 with the env key (verified
directly), so the RateLimitedError branch mislabeled the failure: a 429 there is
insufficient_quota (no account billing), not throttling. The branch also only
printed a softer message before failing anyway, an xfail-in-disguise the e2e rules
forbid. A 429 now falls through and fails loudly with the full result.
* refactor(auth): derive temp budget bump without mutation, tz-aware auth datetimes
_update_key_budget_with_temp_budget_increase mutated max_budget in place, so correctness depended on every resolution path handing it a fresh copy of the cached token; one future re-cache of a live token would compound the bump per request. Return a model_copy instead so no caller can leak an increased budget into shared state.
Also fixes the three remaining DTZ005 naive datetime.now() calls in user_api_key_auth.py (auth span start, builder start_time, service-log end_time; all consumers convert to epoch or subtract same-pair datetimes) and ratchets the DTZ005 strict budget 244 -> 241.
* test: pin non-mutation of the temp budget helper input
Adversarial mutation-testing showed reverting the helper to in-place mutation still passed every test: the cache's copy-on-read layer masks the mutation in the integration test and the direct unit test only inspected the return value. Assert the input object is left untouched and the result is a distinct object so the purity guarantee itself is load-bearing.
The mid-conversation-system messages test still referenced the removed
client.gateway attribute, so the tests/e2e basedpyright gate reported 9
errors and went red on every e2e PR. EndpointsClient exposes .proxy, so
point the /v1/messages post helper at client.proxy.transport.
Budgets reset at midnight in the configured timezone with no way to control
the time of day, so a drained daily budget surfaces as an overnight incident.
Add a litellm_settings.budget_reset_time option (e.g. "12:00") that shifts
day/week/month resets to a configurable wall-clock time in the existing
timezone, so the end of the budget window lands during business hours.
The reset time is parsed once into an immutable BudgetResetSettings and
injected into the reset job (constructor) and computation, rather than read
from a module-level global at call time. A malformed value fails fast at
startup. Sub-day durations ignore the offset. Unset preserves midnight resets.
Google added a queued value to Interaction.status in the live Interactions
OpenAPI spec, so the compliance canary test_status_enum_values started
failing on every open PR. The exact-match assertion is deliberate; it is
how we find out the spec moved, so this adds the new value rather than
loosening the check, and mirrors it into the generated Status enums so
InteractionStatus stays truthful.
* fix(model_armor): sanitize error details by default
Generated with AI
Co-Authored-By: Claude Code
* fix(model_armor): sanitize handler-raised HTTP errors and redact scanned content in guardrail logging
The async HTTP handler raises MaskedHTTPStatusError on any non-2xx via
raise_for_status, so the non-200 branch in make_model_armor_request never ran
against a live API and the raw upstream body reached callers and logs. Catch
the raised error and build the sanitized detail from the response status
Replace the empty-dict guardrail logging payload with field-level redaction of
the keys that echo scanned content (text, sanitizedText, findings) so guardrail
traces keep filter states and block reasons while scanned content stays out
Restore the upstream status code in the sanitized error detail, read guardrail
metadata from the same key the hooks write, and keep guardrail_status within
its typed literal values
* fix(model_armor): bound redactor recursion depth and allowlist it in the recursion detector
_redact_scanned_content walks provider JSON bounded by _REDACT_MAX_DEPTH=20 and
fails closed by returning the redaction sentinel at the cap
* fix(model_armor): honor fail_on_error for upstream API failures
API failures now raise a dedicated ModelArmorAPIError so hooks can tell them
apart from content-block HTTPExceptions; fail_on_error=False lets the request
proceed on a Model Armor outage again while fail-closed configs get the same
sanitized 400 as before
Also addresses review notes: sanitize_error_detail constructor annotation
matches the nullable config field, redaction is owned by the metadata write
sites so _process_response no longer re-applies it, and the request and
response debug log branches move into helpers
* test(model_armor): cover fail_on_error routing on during-call, post-call, streaming, and file-scan paths
* chore: remove accidentally committed pytest cache files
* fix(model_armor): keep sanitize_error_detail coerced across in-memory config reloads
update_in_memory_litellm_params assigns raw LitellmParams fields, so a hot
reloaded config carrying an explicit null would silently disable sanitization;
re-apply the only-explicit-False-opts-out coercion after the update
* fix(model_armor): redact matched malicious URIs and reuse the shared recursion depth constant
maliciousUriMatchedItems echoes the caller-supplied URL including path and
query, so it joins the scanned-content key set; the redactor depth cap now
comes from DEFAULT_MAX_RECURSE_DEPTH in litellm constants instead of a local
literal
* fix(model_armor): keep API failures out of the intervention trace status
Fail-closed upstream failures re-raise ModelArmorAPIError instead of
converting to HTTPException(400), so the shared guardrail logging keeps
recording them as guardrail_failed_to_respond while content blocks stay
guardrail_intervened. Callers see the same 500 shape as before this PR,
with the sanitized message
* chore(model_armor): drop explanatory comment per repository comment policy
---------
Co-authored-by: eugene-yao-zocdoc <eugene.yao@zocdoc.com>
* adding deepkeep as custom guardrail
* adding deepkeep as a custom guardrail
* adding deepkeep as a custom guardrail (hooks)
* adding litellm/proxy/_experimental/out/ to .gitignore
* adding deepkeep as custom guardrail in litellm
* removing sentinel_fortress
* comparing schema.prisma files
* fix(deepkeep): address greptile review comments
- extra_headers: fix type annotation (list -> Dict[str, str]) and actually
merge them into _build_request_headers() so user-configured headers
reach the DeepKeep API
- user_api_key_hash: only fall back to user_api_key_token when no
explicit hash is already set, avoiding silent overwrite
- apply_guardrail: preserve tool_calls and structured_messages in the
return value so downstream callers don't lose that content
Adds tests for all four fixes.
* fix(deepkeep): address greptile review comments
- extra_headers: fix type annotation (list -> Dict[str, str]) and actually
merge them into _build_request_headers() so user-configured headers
reach the DeepKeep API
- user_api_key_hash: only fall back to user_api_key_token when no
explicit hash is already set, avoiding silent overwrite
- apply_guardrail: preserve tool_calls and structured_messages in the
return value so downstream callers don't lose that content
Adds tests for all four fixes.
* fix: add missing __init__.py and allowlist entries for upstream merge
- tests/test_litellm/proxy/client/__init__.py: fixes pytest collection
collision with tests/test_litellm/models/test_models.py (same basename)
- tests/test_litellm/models/__init__.py: same fix
- backend/routes/allowlist.py: add /config_overrides/ and /v1/unified_access_group
prefixes for new routes added by upstream
* fix(ui/tests): resolve frontend-lint failures in new test files
- useLogDetails.test.ts: add Wrapper.displayName, replace 'null as any'
with null, type resolveCall promise resolver properly
- usePaginatedDailyActivity.test.ts: remove unused waitFor import,
add Wrapper.displayName, change Record<string,any> to Record<string,unknown>
- UsageViewSelect.adminFiltering.test.tsx: replace all props:any with
explicit SelectProps/BadgeProps/SelectOption types, replace (X as any).displayName
with direct X.displayName assignment
no-explicit-any count: 2034 (budget: 2040). Prettier check: clean.
* fix(ui): sync proxy/_experimental/out/ exactly to upstream
245 stale JS chunk files from earlier merges were left in the out/
directory but had been deleted in upstream. The Docker image in CI is
built by copying this directory verbatim, so the stale artifacts caused
the SERVER_ROOT_PATH redirect E2E to fail.
Synced by: git checkout upstream/litellm_internal_staging -- out/ (adds
new files) + git rm on every file present in HEAD but absent from
upstream.
* Update litellm/proxy/guardrails/guardrail_hooks/deepkeep/deepkeep.py
Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>
* fix(makefile): fall back to upstream/litellm_internal_staging for strict-budget gate
origin/litellm_internal_staging exists on BerriAI's CI but not on forks
that use a different remote name (e.g. Azure DevOps as origin). Fall
back to upstream/litellm_internal_staging when the origin ref is absent.
* linter reformat
* fix(deepkeep): apply guardrail tool/tool_call redactions from API response
When DeepKeep returns GUARDRAIL_INTERVENED with redacted tools or
tool_calls, the previous code ignored those redactions and forwarded
the original (potentially sensitive) values to the model — a guardrail
bypass for content embedded in tool schemas or function arguments.
Fix: prefer response_json["tools"] / response_json["tool_calls"] when
present, falling back to the originals only when the guardrail did not
return replacements — consistent with the existing pattern for texts and
images.
Refactor _build_return_inputs() into a private static helper to keep
apply_guardrail() under the PLR0915 statement limit (50).
Adds test_apply_guardrail_applies_tool_redactions_from_response to
assert that redacted tool payloads from the API response are used.
* Update litellm/proxy/guardrails/guardrail_hooks/deepkeep/deepkeep.py
Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>
* fix(lint): move base-ref fallback into ruff_strict_gate.py; revert Makefile
The previous Makefile fix had a shell bug: 'git rev-parse --verify'
writes the resolved SHA to stdout, so the $$(...) substitution captured
both the SHA and the echo output, handing '--base <sha>\norigin/...' as
two tokens to the Python script, causing exit code 1 in CI.
Fix: revert Makefile to its original single-line invocation and add
_resolve_base() to ruff_strict_gate.py. The function checks whether the
requested ref resolves; if not, it tries the 'upstream/' equivalent
before falling back to the original ref (letting git emit a clear error).
Behaviour in BerriAI CI: origin/litellm_internal_staging resolves → used
as before, no change.
Behaviour on forks with a different 'origin': falls back to
upstream/litellm_internal_staging transparently.
* fix(lint): fix UP006/UP045/F401 in changed files; add depth guard to check_any_discipline
- Replace Dict/List/Optional/Tuple typing imports with built-in equivalents
(UP006, UP045) across files touched in this PR diff, then clean up
the now-unused typing imports (F401).
- Add _MAX_CONTAINS_ANY_DEPTH guard to check_any_discipline.contains_any()
to prevent RecursionError on deeply-nested mypy types.
* fix(lint): resolve all three CI lint job failures
1. lint (ruff_strict_gate) — UP006/UP045/F401 violations introduced on
changed lines. Fixed Dict/List/Optional/Tuple → built-in equivalents
across every file in the PR diff; cleaned up now-unused typing imports.
2. any-discipline — RecursionError in check_any_discipline.contains_any()
on deeply-nested mypy types. Upstream fixed this by converting to an
iterative stack-based algorithm (merged). Also added deepkeep.py to
any-discipline-budget.json via 'make lint-any-budget-update' so the
new file's Any count is baselined instead of failing against the
zero-baseline default.
3. basedpyright reportMissingParameterType — **kwargs in DeepKeepGuardrail
__init__ lacked a type annotation. Added **kwargs: Any.
* Update litellm/deepkeep_tilt_config.yaml
Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>
* fix(lint): black reformat after merge
* fix(deepkeep): honour empty-list replacements in _build_return_inputs
When DeepKeep returns GUARDRAIL_INTERVENED with an intentional empty
replacement (e.g. texts:[], tool_calls:[]) the previous truthiness check
treated [] as absent and forwarded the original content downstream —
a guardrail bypass for any case where the firewall wants to fully clear
a field.
Fix: replace all response_json.get(field) truthiness checks with
'is not None' comparisons so that an empty list is respected as a
deliberate replacement. Applies to texts, images, tools, tool_calls,
and the original-input fallback guards.
Adds test_apply_guardrail_honours_empty_list_replacements.
* fix(test): replace live httpbin.org call with mocked transport in test_pass_through_with_httpbin_redirect
Root cause of OOM: the test made a real HTTP request to https://httpbin.org
inside a pytest-xdist worker. Under memory pressure the worker's httpx client
and redirect-following logic allocated enough virtual memory to trip the OOM
killer (confirmed by ulimit -v 16GB reproducing the crash with 'node down: Not
properly terminated' on this exact test).
Fix: replace the real network call with a custom httpx.AsyncBaseTransport that
returns a pre-built 302 -> 200 response sequence in-memory. The test now runs
hermetically with no network dependency and no excess memory allocation.
ulimit -v 16GB: 24,284 passed (0 crashes) after this fix.
* fix: merge upstream/litellm_internal_staging (197 commits), resolve conflicts
7 conflicts resolved:
- 6 Python files: upstream added new code with old-style typing (Optional,
Dict, List) on lines where we had ruff-fixed modern syntax (str | None,
dict, list). Took upstream's version then re-ran ruff UP006/UP045/F401
--fix to keep both the new content and ruff compliance.
- test_openapi_compliance.py: upstream replaced 'role' with 'steps' in
output_fields and updated the spec comment. Took upstream's version.
Also: added _resolve_base() fallback to type_check_gate.py and removed
the hard 'git fetch origin litellm_internal_staging' from the Makefile's
lint-basedpyright target (same pattern as ruff_strict_gate.py fix).
* fix: merge upstream (41 commits), resolve .gitignore conflict, fix BLE001
- .gitignore: upstream removed package.json/out/ ignore entries; took theirs
- deepkeep.py: added '# noqa: BLE001' on catch-all Exception handler
(BLE001 rule newly enforced in ruff-strict-budget)
- type_check_gate.py: added _resolve_base() fallback for basedpyright gate
- Makefile: removed hard 'git fetch origin' from lint-basedpyright target
* fix: merge upstream (57 commits), resolve conflicts
- Makefile: upstream added lint-fetch-base target; made it tolerant of
missing origin/litellm_internal_staging (git fetch || true)
- test_websearch_chat_completion.py: took upstream's new assertions and
skipif marker
- anthropic_cache_control_hook.py: upstream added new code using List/Dict/Tuple
which were undefined after our earlier UP006 cleanup; replaced with
built-in list/dict/tuple
* fix(coverage): revert ruff UP006/UP045 changes on upstream files
The previous ruff fixes (Dict→dict, Optional→X|None) on 7 upstream files
added ~500 changed lines of pure type-annotation no-ops to our PR diff.
codecov/patch penalised these uncovered lines, dropping patch coverage
to 51.35% (target 61.83%).
Fix: revert these files to exactly match upstream/litellm_internal_staging.
The ruff_strict_gate still passes because the violations exist equally in
both the base and HEAD (total == base_count → no breach).
* fix: merge upstream (130 commits), resolve Makefile + base_email conflicts
- Makefile: upstream changed lint deps to $(LINT_DEP_INSTALL)/$(LINT_DEP_BASE);
kept our --base removal (handled by _resolve_base in Python scripts)
- base_email.py: took upstream's dedup cache addition
- deepkeep.py: ruff format after merge
* chore: remove lint/format-only changes and non-feature files
Revert all lint-infra and black/ruff-reformat-only changes back to
upstream/litellm_internal_staging so the PR diff shows only the DeepKeep
guardrail feature:
- Makefile, scripts/ruff_strict_gate.py, scripts/type_check_gate.py
(lint-gate infra)
- credential_migration.py + enterprise/* + assorted test files
(black-reformat / xdist test-isolation drift)
- backend/routes/allowlist.py (merge glue)
Remove non-feature local artifacts: build-and-push.sh,
deepkeep_tilt_config.yaml, stray __init__.py collision shims, and
unrelated UI test files.
* fix(lint): add reason to BLE001 noqa to satisfy type-discipline gate (LIT003)
The type-discipline budget ratcheted LIT003's ceiling to 292 as upstream
fixed reasonless suppressions, so our '# noqa: BLE001' (code but no
reason) tipped the total to 293 and failed CI. Add a reason per the
required '# noqa: CODE # <reason>' shape.
* fix(deepkeep): apply structured_messages redactions returned by the guardrail API
_build_return_inputs dropped any structured_messages the DeepKeep API returned and
always forwarded the original input, so redactions on that field never took effect.
Check the response first, same as texts/images/tools/tool_calls
* chore(ui): drop redundant preserve prop from the guardrail form
preserve defaults to true in rc-field-form (isMergedPreserve falls back to true when
unset), so the explicit prop changed nothing and only widened this PR's blast radius
to every guardrail provider in the shared form
* fix(deepkeep): stop extra_headers list from crashing the guardrail call and name the real firewall id config key
litellm_params.extra_headers is a list of header names to forward, so passing it
straight into dict.update raised ValueError and, under fail_closed, took the request
down with it. Only merge mapping values and warn otherwise
The docstring example and the missing-secret error both said firewall_id, but
initialize_guardrail only reads deepkeep_firewall_id, so anyone following them
had their value silently ignored
* refactor(proxy): drop normalize_callback change; split to its own PR (#33905)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yaniv Israel <yaniv@deepkeep.ai>
Co-authored-by: DK-yaniv <164404355+DK-yaniv@users.noreply.github.com>
Co-authored-by: veria-ai[bot] <224490171+veria-ai[bot]@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The MCP OAuth DCR relay's client-facing /register response handed back LiteLLM's own
<base>/callback as the client's redirect_uris on every arm except the true bridge relay.
A spec-compliant OAuth 2.1 DCR client (e.g. Open WebUI over Streamable HTTP) adopts that
value for its subsequent /authorize calls, so /callback redirects to itself; the second
hit carries the client's opaque state, fails to decrypt, and surfaces as the LIT-4197
"oauth_state ... Incorrect padding" error, making pass-through MCP OAuth unusable against
real DCR-capable upstreams.
The short-circuit arm keeps registering the gateway callback upstream (unchanged) and now
echoes the client's own redirect_uris back to the client across all register arms. Since
the client then authorizes with its own separate-origin redirect, the /authorize rejection
hint now points operators to MCP_TRUSTED_REDIRECT_ORIGINS, the mechanism a legitimate
cross-origin OAuth client needs (auto-trusting a DCR-registered redirect would reintroduce
the VERIA-57 open-redirect vector, since dynamic registration is unauthenticated).
temp_budget_increase was only applied on the DB-fetch path of _user_api_key_auth_builder, so a key served from the auth cache reverted to its original max_budget and was wrongly blocked with BudgetExceededError once spend crossed the original budget while staying under the effective budget.
Move _update_key_budget_with_temp_budget_increase out of the DB-only branch so it runs for every resolved token regardless of source. The cache stores the original budget and each cache hit returns a fresh model_copy(), so this never double-applies.
Fixes#25760
Co-authored-by: shivam <shivam@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
