decrypt_value_helper logged `Unable to decrypt value={value}` at DEBUG, which
printed the raw secret whenever decryption failed (for example after a salt or
master key change). This is the same environment_variables config path the
db-config redaction covers, so a DATABASE_URL connection string could still
leak here when the module regex scrubber is bypassed. Drop the value; the key
already identifies the failing pair.
Regression forces a decrypt failure with the redaction filter disabled and
asserts the raw value never reaches a log record while the key stays visible.
general_settings.user_api_key_cache_ttl was ignored for every management-object
write into user_api_key_cache. The configured value is propagated to the cache's
default_in_memory_ttl at startup, but DualCache only applies that default when no
explicit ttl kwarg is passed, and every management-object writer passed
ttl=DEFAULT_MANAGEMENT_OBJECT_IN_MEMORY_CACHE_TTL (60s), which always won. So keys,
teams, users, budgets, object permissions, vector stores, JWT user syncs and MCP
caches all expired after 60s regardless of the setting.
Adds get_management_object_ttl(cache) in user_api_key_cache.py, which returns the
configured default_in_memory_ttl and falls back to the 60s constant only when no
default is set, and routes every management-object writer through it. The helper
takes a DualCache so it works at the many call sites that are typed UserApiKeyCache
but exercised with a bare DualCache.
Also covers the spend-update writeback in update_cache (async_set_cache_pipeline),
which hardcoded ttl=60 on the same key/user/team objects and reset an active key's
cache entry back to 60s on every priced request, so the configured TTL was never
observed for keys receiving traffic.
Resolves LIT-3338
Skip token counting in Router._pre_call_checks when no deployment in the
group declares max_input_tokens, and skip the full-body surrogate-repair
regex in _read_request_body above a configurable size, raising the existing
400 immediately.
Resolves LIT-3541
* feat(proxy): add configurable response headers middleware
Adds a small ASGI middleware that sets standard response headers
(X-Frame-Options, Content-Security-Policy frame-ancestors, X-Content-Type-Options)
on proxy and UI responses. Strict-Transport-Security is optional and gated
behind LITELLM_ENABLE_HSTS for HTTPS deployments. Values use setdefault so a
route that sets its own header is preserved.
* feat(proxy/ui): make login page credentials hint configurable
build_ui_login_form accepts a hide_default_credentials_hint parameter and
google_login reads LITELLM_HIDE_DEFAULT_CREDENTIALS_HINT (or general_settings)
so the legacy login page behaves consistently with the new UI. Also collapses
a duplicated branch and removes an unused variable and module-level constant.
* fix(proxy/ui): apply credentials hint flag on /fallback/login
The /fallback/login handler still rendered the default-credentials hint
regardless of LITELLM_HIDE_DEFAULT_CREDENTIALS_HINT. Collapse its duplicate
branch and forward the flag, matching google_login, so all login surfaces
behave consistently. Adds regression tests for /fallback/login and makes the
ui_sso test helper restore os.environ so env vars do not leak across tests.
* fix(datadog): pass callback_specific_params so DatadogCostManagementLogger receives cost_tag_keys (#29590)
* fix(datadog): pass callback_specific_params so DatadogCostManagementLogger receives cost_tag_keys
* test(proxy): regression test that load_config forwards callback_specific_params
* fix(proxy): guard lakera_prompt_injection callback_specific_params against non-dict
Addresses review feedback: forwarding callback_settings as callback_specific_params
(so DatadogCostManagementLogger receives cost_tag_keys) exposed the
lakera_prompt_injection branch, which did lakeraAI_Moderation(**callback_specific_params
["lakera_prompt_injection"]) with no type guard. A config like
`callback_settings: {lakera_prompt_injection: "any-string"}` then hit `**"any-string"`
-> TypeError: argument after ** must be a mapping, not str.
Guard the lakera branch with isinstance(dict), matching the existing presidio and
datadog_cost_management branches (non-dict values fall back to {}). Add a regression
test asserting initialize_callbacks_on_proxy ignores a non-dict value instead of crashing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: inject fake lakera_ai module to avoid importing the real one
CI fix for the lakera regression test: it stubbed litellm.proxy.proxy_server with
a SimpleNamespace and then monkeypatch.setattr'd the real lakera_ai module, which
forces importing it — and lakera_ai does `from litellm.proxy.proxy_server import
LiteLLM_TeamTable`, absent on the stub -> ImportError under proxy-infra tests.
Inject a fake lakera_ai module into sys.modules instead, so the callbacks branch's
`from ...lakera_ai import lakeraAI_Moderation` resolves to the stub without loading
the real module. The guard under test (isinstance(dict) in the lakera branch) is
unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(callbacks): guard compression/websearch interceptors against non-dict callback_settings (#30153)
#29590 forwards the full callback_settings dict into initialize_callbacks_on_proxy, which activates the compression_interception and websearch_interception consumers. Their initialize_from_proxy_config read the callback_settings subkey without an isinstance(dict) guard, so a non-dict value such as `compression_interception: true` reached from_config_yaml(...).get(...) and aborted proxy startup with AttributeError. #29590 added that guard for lakera_prompt_injection but not for these two
Mirror the isinstance(dict) guard already used by the lakera, presidio, and datadog branches so a non-dict value is ignored and the callback initializes with defaults. A parametrized test feeds every callback_settings consumer a non-dict value through initialize_callbacks_on_proxy to catch a future consumer that forgets the guard
* fix(callbacks): normalize non-dict callback_specific_params to empty dict
A blank callback_settings: key in YAML loads as None, and
config.get('callback_settings', {}) returns None because dict.get only
falls back to the default when the key is absent. Forwarding that value
verbatim to initialize_callbacks_on_proxy made the first
'<name>' in callback_specific_params membership test raise
TypeError: argument of type 'NoneType' is not iterable, aborting proxy
startup. Same failure for any non-dict root such as callback_settings: true.
Normalize the value at the function boundary so both callsites (and any
future ones) initialize callbacks with their defaults instead of crashing.
---------
Co-authored-by: Hedi Daoud <150018939+hdaoud23@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(ui): add budget duration to edit team member form
Editing a team member created a member budget with no duration, so the
budget never reset. This threads a budget reset period through the edit
flow end to end and reuses the shared duration dropdown so the options
stay in sync with the rest of the UI.
Resolves LIT-2651
* fix(proxy): validate member budget_duration and persist clears
Reject budget_duration values that can't be parsed, are non-positive, or overflow date math before any write, so a bad value can't be persisted and later crash the budget reset job.
Clearing the budget duration in the edit-member form now sends null and clears the column end to end, so the dropdown's clear control reflects a real change instead of being a no-op
* chore(ui): regenerate schema.d.ts for member budget_duration
Adds budget_duration to TeamMemberUpdateRequest/Response in the generated dashboard types so the Check UI API Types Sync gate passes
* fix(reset_budget): write only {spend, budget_reset_at} and stop pre-zeroing counter
ResetBudgetJob's batched update_data path shipped the full key/user/team
model on each reset. Prisma rejects object_permission_id and budget_limits
on the update input type, so any row carrying those fields detonated the
entire batch -- spend never reset, budget_reset_at never advanced. After
v1.84.0 started populating object_permission_id on UI-created keys, this
fires routinely.
_reset_budget_common also zeroed the cross-pod spend counter before the
DB write, so failed resets left enforcement reading 0 from the counter
while the DB still held the over-budget spend, admitting requests past
the cap until the counter naturally re-saturated from new reservations.
Switch the write to per-row narrow updates ({spend, budget_reset_at})
via db.batch_, and move the counter invalidation out of
_reset_budget_common so it only fires after the DB write commits. On
DB-write failure the counter is left untouched, enforcement continues
to block, and the next scheduler tick can retry without leaving a
bypass window.
Fixes#27730.
* fix(reset_budget): address Greptile review on #29358
- Strengthen the bypass-half regression test: replace the for-loop over
call_args_list (vacuously true when empty) with assert_not_called(),
so the test would actually flag a re-introduction of counter-zeroing
via any code path.
- Add the same explanatory docstring on _write_user_reset_updates and
_write_team_reset_updates that _write_key_reset_updates already has,
so all three helpers point future maintainers at #27730.
* test(reset_budget): update test_proxy_budget_reset for new batch-write path
Same shape as the previous test_reset_budget_job.py update: keys/users/teams
now write through prisma.db.batch_().<table>.update, not update_data, so the
tests need a batcher mock and updated assertions. Adds:
- _wire_batcher_for_test helper that returns a list which accumulates per-row
batch updates captured from prisma_client.db.batch_().
- _attrify helper that wraps dict fixtures so getattr(item, "token") works
alongside the dict item-access the fake_reset_* mocks rely on. The new
narrow-write helpers use getattr to pull out the row's id, and would
silently skip plain dicts otherwise.
- Updates 3 partial_failure tests to assert against the batch-call list
(rows by id, payload contains only {spend, budget_reset_at}) instead of
update_data.assert_awaited_once + data_list inspection.
- Updates test_reset_budget_continues_other_categories_on_failure: only
budget + enduser still flow through update_data; key/user/team go through
the batch path now.
- Wires the batcher mock into 3 service_logger_*_success tests so commit()
is actually awaitable and the success hook fires.
These tests were silently passing locally only because the editable install
in .venv pointed at the main repo, not the worktree — running pytest with
PYTHONPATH overridden to the worktree (matching CI) reproduces the failures.
* fix(proxy): strip LiteLLM policy tracking from OpenAI batch metadata
Batch create was failing with `Invalid type for 'metadata.applied_policies':
expected a string, but got an array instead` whenever a policy attachment
matched the request. The policy engine helpers wrote `applied_policies`,
`applied_guardrails`, and `policy_sources` into `data["metadata"]`
unconditionally, and `/v1/batches` forwarded that dict straight to OpenAI,
which only accepts string values.
- Route proxy-internal tracking into `litellm_metadata` for batch/file
routes via a shared `_get_or_create_proxy_metadata_bucket` helper.
- Sanitize `data["metadata"]` in `create_batch` to drop known internal
keys and non-string values before building the OpenAI request.
- Cover both behaviors with unit + endpoint tests.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(proxy): merge metadata buckets for batch policy response headers
Ensure get_logging_caching_headers reads both metadata and litellm_metadata so policy/guardrail headers are emitted on batch routes with user metadata, and log dropped non-string OpenAI metadata at debug level.
Co-authored-by: Cursor <cursoragent@cursor.com>
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
* chore(proxy): strict media-type match for form bodies (#27939)
* chore(proxy): strict media-type match for form bodies
``_read_request_body`` and ``get_request_body`` routed on
``"form" in content_type`` / ``"multipart/form-data" in content_type``,
which match any header containing the literal — ``application/form-json``,
``multiform/anything``, ``application/json; xform=1``. Starlette's
``request.form()`` returns an empty ``FormData`` for any non-canonical
type without consuming the body, so the auth-time pre-read saw ``{}``
and skipped the banned-param check while the handler's later
``request.body()`` saw the original JSON payload.
Parse the media type per RFC 7231 (substring before ``;``, trimmed,
lowercased) and accept only ``application/x-www-form-urlencoded`` and
``multipart/form-data``. Replace both substring sites with the shared
``_is_form_content_type`` helper.
Tests pin: case/whitespace/charset variants of the two real types
match; ``application/form-json`` and similar substring-match traps
fall through to the JSON parse path; real form POSTs continue to
route through ``request.form()``.
* chore(proxy): extract _is_json_content_type symmetric helper
Mirror ``_is_form_content_type`` for the JSON branch of
``get_request_body`` so both classifications share the same media-type
normalisation (strip params, trim, lowercase) and any future change
to the parsing rules has one place to update.
Adds tests for ``_is_json_content_type`` and for ``get_request_body``
covering the canonical JSON / form / unsupported / non-POST paths.
* chore(proxy): surface form-parse failures instead of caching empty body
Starlette's ``request.form()`` raises ``MultiPartException`` /
``ValueError`` / ``AssertionError`` on malformed multipart input
(missing boundary, malformed chunk encoding, etc.). The outer
``except Exception: return {}`` swallowed every form-parse failure
and cached an empty parsed body — auth-time pre-reads saw ``{}`` and
skipped every banned-param check while a later raw-body re-read in
the handler still saw the original payload. Same TOCTOU shape as the
substring-match bypass: the auth gate and the handler don't agree on
what the body is.
Wrap ``request.form()`` in a narrow ``try`` that converts any parse
failure to a 400 ``ProxyException``. The outer broad ``except`` is
retained for unrelated unexpected errors but no longer covers
form-parse-side bypass shapes.
Adds a regression test parametrised over the exception classes
Starlette can raise from ``request.form()``.
* chore(proxy): drop redundant _is_json_content_type test class
``_is_json_content_type`` is a 3-line wrapper around the shared
``_normalize_media_type`` helper. Positive coverage lives in
``TestGetRequestBody.test_json_with_charset_param_parses_as_json``;
negative coverage is covered transitively by
``TestIsFormContentType``'s non-form parametrize matrix (anything that
isn't a form type falls through to the JSON branch).
* chore(proxy): carry ASGI path into WebSocket auth synthetic Request (#27940)
``user_api_key_auth_websocket`` built a synthetic ``Request`` with a
two-key scope (``type`` + ``headers``) and set ``request._url =
websocket.url``. ``get_request_route`` reads ``scope.get("path", ...)``
and falls back to ``request.url.path`` only when ``path`` is absent.
For the WebSocket flow that fallback fires and resolves to the
Host-header-derived value (Starlette reconstructs ``websocket.url``
from the Host header), so a malformed Host collapses the resolved
route and lets the auth gate compare against the wrong value.
Carry the ASGI scope's ``path``, ``root_path``, and ``app_root_path``
into the synthetic scope so the lookup never reaches the fallback on
the legitimate path.
Regression test pins that the request handed to ``user_api_key_auth``
has ``scope["path"]`` equal to the ASGI scope's path.
---------
Co-authored-by: stuxf <70670632+stuxf@users.noreply.github.com>
* fix(proxy): resolve cache handling issues in _lookup_deprecated_key
- Updated the in-memory cache for deprecated key lookups to store a 3-tuple (active_token_id, cache_expires_at_ts, revoke_at_ts) instead of a 2-tuple, ensuring proper unpacking and backward compatibility.
- Removed duplicate cache reads and added logic to handle legacy cache entries gracefully.
- Enhanced unit tests to cover scenarios for cache hits, DB misses, and respect for revoke_at timestamps, ensuring robust handling of the grace-period key-rotation feature.
* refactor(proxy): streamline cache handling in _lookup_deprecated_key
- Simplified the cache retrieval logic by directly unpacking the 3-tuple cache entries, removing the need for backward compatibility checks for 2-tuple entries.
- Updated unit tests to ensure that pre-warmed 3-tuple cache entries are served correctly without unnecessary database lookups.
* chore(ci): add new unit test for deprecated key grace period
- Included `test_deprecated_key_grace_period.py` in the CI workflow to enhance coverage for deprecated key handling scenarios.
* fix(proxy): remove unnecessary check for revoke_at in _lookup_deprecated_key
- Eliminated the redundant check for None on revoke_at, streamlining the logic for handling deprecated keys in the cache. This change enhances the efficiency of the key lookup process.
* test(proxy): add end-to-end tests for deprecated key lookup behavior
- Introduced a new test class `TestDeprecatedKeyLookupDbE2E` to validate the behavior of deprecated key lookups against a real Prisma-backed database.
- The test ensures that old key hashes resolve correctly and that repeated lookups utilize the in-memory cache without errors.
- Cleaned up the `_lookup_deprecated_key` function by removing an unnecessary check for `revoke_at`, enhancing the efficiency of the key lookup process.
* fix: invalidate cached tag object on tag budget reset (#27481) (#27572)
Squash-merged by litellm-agent from oss-agent-shin's PR.
* chore(mcp): tighten stdio server registration paths (#27570)
Squash-merged by litellm-agent from stuxf's PR.
* fix(proxy): clear MCP OpenAPI mappings on server eviction; widen budget cache invalidation
Evict OpenAPI tools from global_mcp_tool_registry and strip tool_name_to_mcp_server_name_mapping entries when a server leaves the runtime registry (remove_server and approval-status eviction). Invalidate user_api_key_cache for keys, orgs, and team members on budget-tier spend resets alongside tags.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(mcp): align update_server eviction with remove_server name fallback
Document budget-reset test assertion flip (cross-pod cache staleness).
Greptile: eviction now pops by server_id then server_name like remove_server;
test docstring explains assert_not_awaited -> assert_any_await change.
Co-authored-by: Cursor <cursoragent@cursor.com>
* Fix org budget cache invalidation
---------
Co-authored-by: oss-agent-shin <ext-agent-shin@berri.ai>
Co-authored-by: stuxf <70670632+stuxf@users.noreply.github.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Two cleanups from the /simplify review pass:
* ``Response`` was imported inside the ``except OSError`` branch in
``/get_image`` and at the top of ``/get_favicon``. Per the project's
no-inline-imports rule (CLAUDE.md), hoisted to the existing
``from fastapi.responses import (...)`` block at the top of
``proxy_server.py``.
* The test class's ``_patches()`` helper returned a 2-element list of
patch context managers and tests indexed into them via
``self._patches(...)[0], self._patches()[1]`` — two distinct calls
with confusing aliasing semantics. Restructured to:
- module-level ``_patch_async_safe_get(...)`` that returns a single
patch context manager
- autouse fixture that patches ``get_async_httpx_client`` for every
test in the file (it's the same patch in every case)
- small ``_image_response(...)`` factory to deduplicate Mock setup
Tests now read as ``with _patch_async_safe_get(return_value=...):``
with no list-indexing or duplicate Mock construction.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three review items addressed:
* **Veria (Medium): SSRF via redirect.** ``fetch_validated_image_bytes``
was calling ``validate_url(url)`` once and then fetching with the
default httpx client, so a 3xx to an internal IP would have been
followed unvalidated. Switched to ``async_safe_get`` (the existing
SSRF primitive used elsewhere in the codebase) which walks each
redirect hop, re-validates, and rejects redirects to blocked
networks. Default ``litellm.user_url_validation`` is True so
protection is on out of the box.
* **Greptile (P2): SVG can embed JS.** Removed ``image/svg+xml`` from
the allowed-Content-Type set. The hardcoded response media type
(``image/jpeg`` / ``image/x-icon``) means a real SVG body wouldn't
render as SVG anyway in modern browsers — the allowlist entry was
giving up XSS surface for no actual SVG-rendering benefit. If real
SVG support is wanted later, that's a deliberate feature PR with CSP
/ nosniff bundled.
* **Greptile (P2): cache-write OSError drops validated bytes.** When
the upstream fetch succeeded but ``open(cache_path, "wb")`` raised
(read-only assets dir), the bytes were discarded and the default
logo was served — a silent regression for that deployment. Now
serve the validated bytes inline via ``Response(...)`` as a fallback
before falling back to default.
Tests:
- Replaced low-level mocks of ``validate_url`` with mocks of
``async_safe_get`` directly, exercising the helper's contract
rather than the SSRF primitive's internals.
- New ``test_rejects_svg_content_type`` confirms SVG is blocked.
- ``test_get_image_cache_logic`` fixture now sets
``mock_response.is_redirect = False`` so ``async_safe_get`` doesn't
treat the Mock's truthy attribute as a redirect.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The unauthenticated ``/get_image`` and ``/get_favicon`` endpoints accept
the admin-set env vars ``UI_LOGO_PATH`` and ``LITELLM_FAVICON_URL`` and
return whatever bytes they resolve to, with a hard-coded ``image/jpeg``
or ``image/x-icon`` content-type. Two attack shapes:
* ``UI_LOGO_PATH=/etc/passwd`` (or any other readable file path) — any
unauthenticated caller exfiltrates the file via ``GET /get_image``.
The previous gate was ``os.path.exists(logo_path)`` which fires on
every readable file. Same shape for the favicon endpoint.
* ``UI_LOGO_PATH=http://169.254.169.254/iam`` (or any internal HTTP
service the admin pointed at) — the proxy fetches it server-side
and streams the response body to the unauthenticated caller. No
URL validation, no Content-Type validation; ``application/json``
AWS metadata gets tunneled out under the ``image/jpeg`` wrapper.
New helper module ``litellm/proxy/common_utils/static_asset_utils.py``:
* ``resolve_local_asset_path(candidate, allowed_roots)`` — returns the
resolved absolute path only if it lives within one of the allowed
asset roots. Uses ``realpath`` so symlinks pointing outside the roots
are caught.
* ``fetch_validated_image_bytes(url)`` — runs the URL through
``validate_url`` (rejecting private / cloud-metadata / loopback
targets) and only returns the response body if the upstream
Content-Type is in a small allowlist of image MIME types.
Both ``/get_image`` and ``/get_favicon`` are wired through the helpers.
The SSRF gate is enforced unconditionally — these endpoints are
unauthenticated, so the admin-facing ``litellm.user_url_validation``
toggle does not apply (an admin who opted out of URL validation for
LLM provider paths shouldn't also expose ``/get_image`` to SSRF).
Tests:
- ``TestResolveLocalAssetPath``: 10 cases covering legitimate paths,
``/etc/passwd``, ``/proc/self/environ``, symlink-out, ``..``
traversal, directories, missing files, and root list edge cases.
- ``TestFetchValidatedImageBytes``: 7 cases covering SSRF block, non-
image content-type rejection, valid image passthrough, non-200
response, fetch exception, empty URL, and parametrized coverage of
every allowed image MIME type.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three issues surfaced in review of the previous commit:
1. **Veria — Medium**: ``litellm_params`` carries a nested
``litellm_embedding_config`` dict (auto-resolved from the model
registry on create / update) which itself holds ``api_key`` /
``aws_*`` / ``vertex_credentials``. The previous redactor only
inspected top-level keys, so the nested values passed through
unredacted. Recurse into nested dicts.
2. **Greptile — P2**: when ``litellm_params`` is a JSON-serialized
string (the in-memory registry occasionally stores it that way), the
previous redactor silently no-op'd via the ``isinstance(..., dict)``
guard and echoed the raw payload back. Now: parse, redact, re-serialize.
If the string is not valid JSON, replace it with the redaction
sentinel rather than echo it.
3. **mypy** flagged ``_redact_sensitive_litellm_params``'s
``Optional[Dict[str, Any]]`` signature as incompatible with the
``object``-typed call site. Widened to ``Any -> Any`` to reflect the
actual contract (the function now handles dict / str / None / other).
Also fixes a related test regression in
``test_remove_sensitive_info_from_deployment_with_excluded_keys``: the
``"credentials"`` plural addition to ``SensitiveDataMasker`` defaults
caused the first call (without ``excluded_keys``) to mutate the input
dict's ``litellm_credentials_name`` to a masked value. The second call
(with ``excluded_keys``) then saw the already-masked value rather than
the original. Construct fresh input for each call.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Updated instances of DualCache to UserApiKeyCache across multiple files to enhance cache handling for user API keys.
- Adjusted cache retrieval and storage methods to ensure proper serialization and deserialization of cached objects.
- Introduced a new UserApiKeyCache class to streamline caching logic and improve type safety.
- Updated relevant tests to reflect changes in caching behavior and ensure compatibility with the new cache implementation.
Mirrors the existing vertex_credentials handling for the newer
vertex_ai_credentials field, and extends the dynamic masker's sensitive
pattern set to recognize the plural form so other plural-named credential
fields are also covered.
The periodic budget-window reset job filtered keys/teams with
`where={"budget_limits": {"not": None}}`. The prisma-client-python
library does not support null-filtering on `Json?` columns (no
DbNull/JsonNull sentinel — upstream issue #714). The client drops the
`None` value during serialization and the engine rejects the query with
`MissingRequiredValueError: where.budget_limits.not: A value is
required but not set`, so neither the key nor team reset path runs.
Switch those two `find_many` calls to `query_raw` with
`WHERE budget_limits IS NOT NULL`, selecting only the PK and the
`budget_limits` column. Writes still go through the ORM. Add unit tests
covering the expired/unexpired paths for keys and teams, string-encoded
JSON payloads, empty payloads, error isolation between the two paths,
and a regression guard asserting the query still uses `IS NOT NULL`.
Previously, members added to a team without an explicit per-member budget were
all linked to the same `litellm_budgettable` row referenced by the team's
`metadata.team_member_budget_id`. Updating one member's budget via
`/team/member_update` mutated the shared row and silently changed every other
member's budget too.
Now both write paths produce a private, per-member budget:
- `add_new_member` clones the team's default budget into a fresh row when a
member is added without `max_budget_in_team`/`allowed_models`. If no team
default exists, the membership is created with no budget.
- `_upsert_budget_and_membership` detects when an existing membership still
points at the team's default budget id and clones-on-write, relinking the
membership to the new private budget before applying the update.
- `team_member_update` reads `team_member_budget_id` from team metadata and
passes it through so the helper can make this distinction.
Adds unit tests for clone-on-write, in-place update of a private budget, and
the no-default-no-budget add path.
Made-with: Cursor
Adds total_spend column to LiteLLM_TeamMembership that accumulates
continuously and is not zeroed by the budget cycle reset job. This
enables UI surfaces to distinguish current-cycle spend (the existing
spend column, which resets) from lifetime spend per team member.
Also exposes budget_reset_at on LiteLLM_BudgetTable so /team/info
callers can see when a member's budget window next resets. The field
was already stored in the DB but stripped by the response Pydantic
model.
Includes regression tests that:
- Guard the reset job against ever writing total_spend: 0
- Verify the spend writer increments both spend and total_spend in
one UPDATE statement.
* refactor: new agentic loop event hook
simplifies how to create logic for tool based multi llm calls
* fix: compress - make it work on anthropic input as well
* fix(compress.py): working prompt compression for claude code
ensures claude code messages can run through proxy easily
* docs: add agentic loop hook guide
* docs: add agentic_loop_hook to sidebar
* fix: fix multiple arguments error
* fix: fix tool call loop for compression on streaming /v1/messages
* fix: fix linting errors
* fix: fix ci/cd errors
* feat(litellm_pre_call_utils.py): use claude code session for litellm session id
allows claude code logs to be stitched together, making it easy to know they were all part of the same conversation
* fix: suppress incorrect mypy warning rE: module
* revert: drop PR's changes to litellm/proxy/_experimental/out/
Restores the 34 HTML files under _experimental/out/ to their pre-PR
paths (X/index.html -> X.html). All renames are R100 (content
unchanged); no other files are touched.
* fix: address greptile review comments on PR #25729
- Skip ``kwargs["tools"] = []`` injection when compression is a no-op —
Anthropic Messages rejects empty tool arrays on requests that did not
originally declare tools.
- Move agentic-loop safety guards (fingerprint cycle / max depth) out of
the per-callback try/except so they propagate instead of being swallowed
by the generic exception handler. Extracted _check_agentic_loop_safety.
- Gate generic ``x-<vendor>-session-id`` capture behind the
LITELLM_CAPTURE_VENDOR_SESSION_HEADERS env var (off by default) to
preserve backwards compatibility; explicit x-litellm-* headers are
unaffected.
- Fix monkeypatch target in pre-call-hook test to patch the actual
module-level binding
(litellm.integrations.compression_interception.handler.compress).
- Add regression tests for empty-tools skip and opt-in session capture.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* revert: drop LITELLM_CAPTURE_VENDOR_SESSION_HEADERS flag
Generic x-<vendor>-session-id header capture is a new feature and only
runs *after* the explicit x-litellm-trace-id / x-litellm-session-id
checks, so it does not change behavior for any existing caller that was
already using the LiteLLM headers — no backwards-incompatibility to gate.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor(compress): replace input_type with CallTypes call_type
Drop the bespoke ``CompressionInputType`` literal and use the existing
``litellm.types.utils.CallTypes`` enum instead. ``litellm.compress()``
now takes ``call_type: Union[CallTypes, str]`` (default
``CallTypes.completion``) — no new concept to learn, and the enum is
already the way the rest of the codebase talks about request shapes.
Supported values: ``completion`` / ``acompletion`` (OpenAI chat-completions
shape) and ``anthropic_messages`` (Anthropic structured content blocks).
Updated: compress(), the compression_interception handler, tests, docs,
and the two eval scripts.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
When litellm.max_end_user_budget_id is configured, implicitly-created end users
(via /chat/completions) have budget_id=NULL in the DB since the default budget
is only applied in-memory. The budget reset job filtered by budget_id, so these
users were never reset and eventually permanently blocked.
Fix: when the default budget is in the reset list, also query for and reset
end users with budget_id=NULL and spend > 0. This keeps the hot auth path
unchanged (no DB writes on every request).
Fixes#22019
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Post-merge audit found 6 adjacent variants of the VERIA-28 class. All
fixed here with regression tests:
1. Strip widened from 3 named keys to the full user_api_key_* prefix.
The proxy writes a dozen user_api_key_* fields (user_id, alias,
spend, team_id, request_route, end_user_id, …) into
data[_metadata_variable_name]; the 3-key strip left the rest
exploitable for identity/spend forgery in audit logs and guardrails.
2. proxy_server_request['body'] snapshot moved to AFTER the strip.
Was captured at line ~990 before the strip ran, so
standard_logging_object, lago, and spend_tracking readers saw the
attacker-forged payload even though the live data dict was clean.
3. get_tags_from_request_body (auth-time) now coerces JSON-string
metadata via safe_json_loads. Previously crashed with
AttributeError on string metadata (DoS; potential RBAC bypass if
a caller swallowed the exception).
4. get_end_user_id_from_request_body coerces JSON-string
metadata/litellm_metadata. Previously isinstance(dict) guard
caused end-user budget attribution to be silently skipped when
the caller sent metadata as a JSON string.
5. Four hand-rolled 'if data.get("metadata") is None: data["metadata"] = {}'
blocks in proxy_server.py (7160, 7341, 7590, 11375) now guard on
isinstance(dict). They crashed with TypeError when metadata was a
JSON string (DoS).
6. _get_admin_metadata defensively guards with isinstance(dict);
previously AttributeError'd on any leaked string metadata.
Also hoists the inline safe_json_loads import in _guardrail_modification_check
to module level per CLAUDE.md style.
