mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-12 23:01:41 +00:00
87 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
yuneng-jiang
|
6ff668c7aa
|
[Infra] Promote internal staging to main (#27245)
Some checks failed
CodeQL / Analyze (actions) (push) Has been cancelled
CodeQL / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL / Analyze (python) (push) Has been cancelled
CodSpeed Benchmarks / benchmarks (push) Has been cancelled
Helm unit test / unit-test (push) Has been cancelled
Read Version from pyproject.toml / read-version (push) Has been cancelled
Scorecard supply-chain security / Scorecard analysis (push) Has been cancelled
Unit Tests: Caching (Redis) / caching-redis (push) Has been cancelled
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Has been cancelled
Unit Tests: Security / security (push) Has been cancelled
GitHub Actions Security Analysis / zizmor (push) Has been cancelled
Unit Tests: Proxy DB Operations / auth-checks (push) Has been cancelled
Unit Tests: Proxy DB Operations / budgets (push) Has been cancelled
Unit Tests: Proxy DB Operations / custom-logging (push) Has been cancelled
Unit Tests: Proxy DB Operations / db-and-spend (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-runtime (push) Has been cancelled
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Has been cancelled
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Has been cancelled
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Has been cancelled
Unit Tests: Proxy DB Operations / key-generation (push) Has been cancelled
Unit Tests: Proxy DB Operations / logging-misc (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-server-core (push) Has been cancelled
Unit Tests: Proxy DB Operations / schema-migration (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-utils (push) Has been cancelled
* default requested_model to empty string on litellm-side rejects * Update litellm/router.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * fix: scope key access_group_ids override by team's assigned groups A team member could set any access_group_ids on their key (e.g. a group assigned only to a different team) and override the team's model restriction. Intersect the key's access_group_ids with team_object.access_group_ids in _key_access_group_grants_model so foreign groups are dropped before model expansion. Adds a regression test that asserts expansion is never called for foreign groups. * [Fix] Proxy: Skip Personal Budget Hook When Reservation Covers Counter The reservation path (PR #26845) atomically pre-fills `spend:user:{user_id}` and admits at the strict-`<` boundary. The legacy `_PROXY_MaxBudgetLimiter` pre-call hook re-reads the same counter with `>=`, so a reservation that fills the counter to exactly `max_budget` (e.g. a request without a `max_tokens` cap that falls back to reserving the smallest remaining headroom) is rejected by the hook even though the reservation already admitted it. Skip the hook when the request's active `budget_reservation` covers `spend:user:{user_id}`. The reservation is the source of truth for that counter cross-pod; the legacy `>=` path remains in place for requests without a reservation (e.g. paths that bypass the reservation entirely). Reproduces as `tests/otel_tests/test_prometheus.py::test_user_budget_metrics` on a fresh user with `max_budget=10` calling `fake-openai-endpoint` without `max_tokens`. Adds focused unit coverage in `tests/test_litellm/proxy/hooks/test_max_budget_limiter.py`. * harden bedrock file bucket validation * Fix syntax errors from botched merge in router.py * Fix Vertex batch output edge cases * [Fix] RBAC: Drop management_routes Write Fallback for Admin Viewer Greptile P1: the unsafe-method branch of `_check_proxy_admin_viewer_access` ended with a blanket `if route in management_routes: return`. That set is a mix of reads (info/list — handled via the safe-method GET branch above) and writes. The fallback let Admin Viewer POST to write endpoints not enumerated in `_ADMIN_VIEWER_BLOCKED_WRITE_ROUTES`, including: - /team/block, /team/unblock, /team/permissions_update - /jwt/key/mapping/{new,update,delete} - /key/bulk_update - /key/{key_id}/reset_spend Remove the fallback. The two remaining allow sets (admin_viewer_routes and global_spend_tracking_routes) are both read-only, so removal does not affect the legitimate POST-as-read cases (e.g. /spend/calculate, which is in spend_tracking_routes ⊂ admin_viewer_routes). Tests: - 8 new parametrized cases pinning each previously-leaking management write endpoint to 403 on POST for PROXY_ADMIN_VIEW_ONLY. * fix(tests): anchor VCR redis cassette key to repo root `os.path.relpath` with no `start` arg uses the current working directory, so running pytest from a subdirectory produced a different Redis key than running from the repo root. CI-recorded cassettes and locally-replayed runs would silently miss each other's cache. Anchor the path to the repo root (derived from `__file__`) so the key is stable regardless of CWD. https://claude.ai/code/session_018uCx7pcrkdUJZrCVMaTdPx * fix: gate key access_group override on group's own assignment Replaces the previous intersect-with-team.access_group_ids check, which made the override unreachable in practice (the team-gate fallback already covered every case the intersection allowed). The override now resolves each of the key's access_group_ids via get_access_object and accepts the group only if its assigned_team_ids includes the key's team_id, or its assigned_key_ids includes the key's token. This fulfills the original ask (a key can extend a team's allow-list via a group the admin granted to that team or that specific key) while still rejecting foreign groups referenced by team members of other teams. * [Fix] Proxy/Key Management: Honor team_member_permissions /key/list In /key/list Endpoint When a team grants /key/list via team_member_permissions, non-admin members should see all keys for that team — same as a team admin. Previously the classification in list_keys() only checked admin status, so permitted members fell into the service-account-only path and could not see other members' personal keys. Routes those members into the full-visibility set. * Fix access-group bypass via litellm-model fallback path When _get_all_deployments returns 0 candidates and the litellm-model fallback branch (_get_deployment_by_litellm_model) finds deployments that the access-group filter then empties, _access_group_filter_emptied_candidates remained False (it was captured before that branch ran). The router would then proceed to default fallbacks; the fallback model could have no access_groups and short-circuit the filter, silently serving a caller blocked by access-group restrictions. Update the flag inside the litellm-model branch when filtering empties a non-empty candidate set so the default-fallback guard still triggers. * fix(proxy): redact MCP server URL and headers for non-admin viewers (VERIA-8) Many MCP integrations (Zapier, etc.) embed an upstream API key directly in the server URL, e.g. ``https://actions.zapier.com/mcp/<api-key>/sse``. The list and single-server endpoints were returning the full URL to any authenticated user — `_redact_mcp_credentials` only stripped the explicit ``credentials`` field, and `_sanitize_mcp_server_for_virtual_key` only ran for restricted virtual keys. Non-admin internal users could read the dashboard, click the unmask toggle, and exfiltrate the raw token. Add `_sanitize_mcp_server_for_non_admin` that runs on top of the existing credential redaction and clears the credential-bearing fields: - ``url`` (the primary leak vector) - ``spec_path`` (OpenAPI spec URLs that may carry tokens) - ``static_headers`` / ``extra_headers`` (Authorization) - ``env`` (arbitrary secrets) - ``authorization_url`` / ``token_url`` / ``registration_url`` Identity fields (``server_id``, ``alias``, ``mcp_info``, etc.) are preserved so the UI can still list servers a non-admin's team has access to. Apply the new sanitizer in `fetch_all_mcp_servers` and the per-server fetch path right after the existing virtual-key branch. Update the existing `test_list_mcp_servers_non_admin_user_filtered` assertions that previously checked URL visibility. Frontend defense-in-depth: hide the URL unmask toggle on `mcp_server_view.tsx` unless the viewer is a proxy admin. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Fix runtime policy attachment initialization Mark runtime-created policies and attachments initialized so global policy attachments created from the policy builder apply immediately without requiring a restart. Co-authored-by: Cursor <cursoragent@cursor.com> * test(router): cover _try_early_resolve_deployments_for_model_not_in_names The router_code_coverage CI check requires every function in router.py to be referenced by at least one test under tests/{local_testing, router_unit_tests,test_litellm} in a file with "router" in its name. The recently-extracted helper had no direct test, so the check failed with "0.45% of functions in router.py are not tested". Add a focused test that exercises the four return paths: model already in self.model_names, no fallback applies, pattern-router match, and default_deployment substitution (also asserting the stored default isn't mutated). https://claude.ai/code/session_019AVp1XL7RT9RxRe4qRLkay * Fix policy registry teardown in tests Reset the policy ID index during policy engine test cleanup so stale policy versions cannot leak between tests. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(batches): count non-chat tokens, validate batch-file model access (VERIA-39) (#27015) * fix(batches): count non-chat tokens and validate every model in batch file Two security control bypasses on POST /v1/batches: 1. `_get_batch_job_input_file_usage` only summed tokens for `body.messages` (chat completions). Embedding (`input`) and text completion (`prompt`) batches reported zero, letting massive non-chat workloads slip past TPM rate limits. Extend the counter to handle string and list shapes for both fields. 2. The batch input file was forwarded to the upstream provider without inspecting the models named inside the JSONL — only the outer `model` query parameter was checked against the caller's allowlist. A caller restricted to gpt-3.5 could submit a batch targeting gpt-4o and the upstream would execute it under the proxy's shared API key. Add `_get_models_from_batch_input_file_content` (returns the distinct `body.model` values) and call it from `_enforce_batch_file_model_access` in the pre-call hook, which runs each model through `can_key_call_model` so the same allowlist semantics (wildcards, access groups, all-proxy-models, team aliases) the proxy enforces on `/chat/completions` apply here too. Any unauthorized model raises a 403 before the file is forwarded. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(batches): count pre-tokenized prompt/input shapes, classify 403 logs Two follow-ups from the Greptile review on the batch validation PR: 1. P1 TPM bypass via integer token arrays. The OpenAI batch schema accepts ``prompt`` and ``input`` as ``list[int]`` (a single pre-tokenized prompt) or ``list[list[int]]`` (multiple) in addition to the string and ``list[str]`` shapes. Pre-fix only the string shapes were counted, so a caller could submit a batch with hundreds of millions of pre-tokenized tokens and the rate limiter would record zero. Extract the per-field logic into ``_count_prompt_or_input_tokens`` and count each int as one token. 2. P2 access-denial logs were indistinguishable from I/O failures. ``count_input_file_usage`` caught every exception under a generic "Error counting input file usage" message, so an intentional 403 from ``_enforce_batch_file_model_access`` looked the same in the logs as a missing file or a Prisma timeout. Catch ``HTTPException`` separately and log 403s at WARNING level with a security-relevant message before re-raising. Tests cover the new shapes: single ``list[int]``, ``list[list[int]]`` (the worst-case bypass vector), and embeddings ``input`` with pre-tokenized arrays. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(proxy): re-validate user_id after /user/info re-parses query (#27009) * fix(proxy): re-validate user_id ownership after /user/info re-parses query The route-level access check in `RouteChecks.non_proxy_admin_allowed_routes_check` reads `request.query_params.get("user_id")`, which decodes literal `+` to spaces. The endpoint then re-parses the raw query string with `urllib.unquote` in `get_user_id_from_request` to preserve `+` characters (so plus-addressed emails work as user_ids). Those two paths produce different ids: a caller who registered a user_id containing a literal space could pass the route check and then read another user's row by sending the encoded `+` form. Add `_enforce_user_info_access` and call it after `_normalize_user_info_user_id` returns the final id. Proxy admin / view-only admin still bypass; everyone else must match the resolved user_id (or have no user_id, which falls back to the caller's own id later in the handler). Tests cover the admin bypass, owner-match path, and the cross-user lookup that this change blocks. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(proxy): apply user_info ownership check to PROXY_ADMIN_VIEW_ONLY `_enforce_user_info_access` was bypassing both PROXY_ADMIN and PROXY_ADMIN_VIEW_ONLY, but the upstream route check in `RouteChecks.non_proxy_admin_allowed_routes_check` only treats PROXY_ADMIN as a true admin for the `/user/info` route — view-only admins go through the `user_id == valid_token.user_id` enforcement along with regular users. Mirroring that asymmetry left the same encoded-`+` bypass open for view-only admins whose user_id contains a literal space. Drop the PROXY_ADMIN_VIEW_ONLY exemption so the post-decode re-check matches the upstream rule. Update tests: a view-only admin must now be blocked from cross-user lookups but still allowed to read their own row. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: yuneng-jiang <yuneng@berri.ai> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(spend-logs): opt-in suppression of stack traces in spend-tracking error logs Adds LITELLM_SUPPRESS_SPEND_LOG_TRACEBACKS env var. When set to true and the proxy log level is INFO or above, spend-tracking error paths emit a single ERROR line without the full traceback. Stack traces are preserved at DEBUG and the Sentry / proxy_logging_obj.failure_handler path is unchanged. The new spend_log_error helper is wired through the spend write hot path: - DBSpendUpdateWriter (update_database, _update_*_db, batch upsert, redis-commit fallbacks) - _ProxyDBLogger._PROXY_track_cost_callback - get_logging_payload exception path - update_spend / update_daily_tag_spend / spend logs queue monitor Resolves LIT-2704. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * fix(spend-logs): preserve no-traceback behavior for update_daily_tag_spend This call site previously logged a single-line error via verbose_proxy_logger.error() with no traceback. Switching it to spend_log_error(..., exc=e) caused a full stack trace to render by default (when LITELLM_SUPPRESS_SPEND_LOG_TRACEBACKS is unset), which contradicts the PR goal of leaving default behavior unchanged. Revert this specific site to the original error log call. * fix(spend-logs): preserve no-traceback behavior for update_daily_tag_spend Bugbot caught a regression: the previous error log here was a single-line verbose_proxy_logger.error(...) with no traceback. spend_log_error attaches the active exception's traceback by default (when the suppression env var is unset), so swapping it in changed default behavior. Revert this one site to its original .error() call to keep the PR strictly opt-in. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * feat(spend-logs): suppress traceback in SpendLogs error_information row Extend LITELLM_SUPPRESS_SPEND_LOG_TRACEBACKS to the failure callback so the per-row Metadata pane in the UI no longer shows the stack trace when the opt-in env var is set, matching the existing console-side suppression. https://claude.ai/code/session_014dztoRbRnRvq54HL9EyHx6 * [Fix] Proxy: Repair Merge Fallout In Router-Override Fallback Auth Conflict resolution for #26968 dropped the `Iterator` typing import (NameError at module load), left a dead `fallback_models = cast(...)` block, and the new tests called `_enforce_key_and_fallback_model_access` without the now-required `request` kwarg. * isolate dual OTEL handlers * harden cloud file compatibility path * harden cloud file compatibility path * [Fix] Proxy/Key Management: Align Key-Org Membership Checks On Generate And Regenerate Mirrors the membership rule on /key/update so that /key/generate and /key/{key}/regenerate apply the same `_validate_caller_can_assign_key_org` gate when the caller specifies an `organization_id`. Proxy admins bypass. The check no-ops when `organization_id` is not being set. * thread trusted params through vertex file content * trust only server legacy file flag * chore(proxy): keep public AI hub unauthenticated * fix(proxy): preserve low-detail readiness status * [Test] Anthropic: Replace Legacy Claude-4-Sonnet Alias With Haiku 4.5 Three live-API tests pinned to claude-4-sonnet-20250514, which is a non-canonical alias of claude-sonnet-4-20250514. Anthropic's main API no longer resolves the legacy form under freshly issued keys, so the tests fail with not_found_error. The token counter test pinned to claude-sonnet-4-20250514 itself (deprecation_date 2026-05-14, two weeks out) was on borrowed time too. Bump all four to claude-haiku-4-5-20251001 — capability superset for what these tests exercise (streaming, parallel tool calling, extended thinking, token counting), no upcoming deprecation, cheaper per-token. * chore(proxy): move URL-valued model/file_id guard from SDK to proxy The previous per-provider guards in HuggingFace, Oobabooga, and Gemini files lived in the SDK layer, breaking SDK callers who legitimately pass URL-valued model identifiers. Move the check to the proxy boundary in add_litellm_data_to_request so SDK users keep working while proxy users default-deny URL-valued model and file_id, with admin opt-in via litellm.provider_url_destination_allowed_hosts. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [Chore] Proxy/UI: Drop stray _experimental/out/chat/index.html This file is a regenerable UI build artifact that should not be tracked in source. Removing so the merge into litellm_internal_staging stays clean. * [Test] Anthropic Passthrough: Bump Streaming Cost-Injection Test To Haiku 4.5 test_anthropic_messages_streaming_cost_injection hits the proxy's /v1/messages route, which routes via the anthropic/* wildcard to api.anthropic.com. The 404 surfaced in the test was Anthropic's own not_found_error propagated back through the proxy (visible from the x-litellm-model-id hash on the response — the proxy did route). Same root cause as the prior commit: the legacy claude-4-sonnet-20250514 alias is no longer recognized by Anthropic's main API under the new key. Swap to claude-haiku-4-5-20251001 — same routing path, canonical model. * fix(proxy): handle ownership-recording failures after upstream create If record_container_owner raises after the upstream container is created, the user previously got a 500 with no usable container — they were billed for an unreachable resource. Move ownership recording into the create path's exception handling and split the two failure modes: - HTTPException from the recorder (auth conflicts) propagates verbatim so the client sees the real status code, not a generic LLM error. - Unexpected exceptions are logged and swallowed; the response is returned to the caller so they aren't billed for a container they can't address. The DB row stays untracked until an operator reconciles. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(guardrails): close post-call coverage gaps * fix(types): add /team/permissions_bulk_update to management_routes The blocklist check in _check_proxy_admin_viewer_access only fires for routes that match LiteLLMRoutes.management_routes — the bulk-update endpoint was missing from that list, so the test for view-only admins on /team/permissions_bulk_update fell through to "allow." Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * [Test] Anthropic Passthrough: Bump Thinking Tests Off Legacy Sonnet 4 Alias base_anthropic_messages_test.test_anthropic_messages_with_thinking and test_anthropic_streaming_with_thinking still pinned to claude-4-sonnet-20250514 — the same legacy alias Anthropic no longer recognizes under freshly issued keys. The other four tests in this base class already use claude-sonnet-4-5-20250929; these two were missed. Bump to claude-haiku-4-5-20251001 (supports_reasoning=true, no upcoming deprecation). Subclasses including TestAnthropicPassthroughBasic inherit these methods. * fix(guardrails): cover multi-choice output variants * fix(proxy): preserve public ai hub ui setting * fix(scim): cascade FK cleanup on user delete and surface block status in UI SCIM DELETE /Users/{id} previously called litellm_usertable.delete without clearing rows that FK back to the user, so Postgres rejected the delete with LiteLLM_InvitationLink_user_id_fkey and the SCIM caller saw a 500. Add a helper to drop invitation_link, organization_membership, and team_membership rows before the user delete (mirrors /user/delete in internal_user_endpoints). Also add a Status column to the Virtual Keys and Internal Users tables so admins can see at a glance which keys are blocked and which users SCIM has deactivated. SCIM-blocked keys carry a tooltip explaining the origin. Pin the dashboard's Node version to 20 via .nvmrc to match CI. * chore: update Next.js build artifacts (2026-05-02 03:21 UTC, node v20.20.2) * perf(proxy): cache container/skill ownership reads on the hot path Container ownership and skill rows are looked up on every retrieve / delete / list / file-content / chat-completion-with-skill call. The new stores wrapped raw Prisma queries with no cache, putting one DB round-trip on each request. Add an in-process TTL'd cache mirroring the _byok_cred_cache pattern in mcp_server/server.py: per-key (value, monotonic_timestamp), 60s TTL, 10000-entry cap with full-clear on overflow, invalidated by every write. Negative results (`None`) are cached too so untracked-resource checks also skip the DB. Tests cover: cache-after-first-hit, negative caching, write invalidation, no-caching-on-DB-error, TTL expiry, capacity eviction. 56 tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: update Next.js build artifacts (2026-05-02 03:39 UTC, node v20.20.2) * fix: remove traceback key instead of it being "" * fix: linting error * fix(scim): preserve scim_active on PUT when client omits the field A SCIM PUT may legally omit `active` (full-replace with the field absent). Pydantic fills the SCIMUser.active default of True, so the PUT handler was overwriting metadata.scim_active with True even when the client never sent it — silently reactivating a previously SCIM-blocked user and unblocking their keys. Use model_fields_set to detect whether the client actually sent `active`. If omitted, preserve the prior scim_active value and skip the cascade to virtual keys. Also drop comments added in this PR that just narrate what the code does; keep only the docstrings and the SQL-NULL pitfall note that explain non-obvious behaviour. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * fix(proxy): use set lookup for permitted agent filters * fix(mcp): redact command fields for non-admin server views * fix(proxy): forward decoded container ids after ownership checks * fix(caching): handle stale isolated Redis semantic index * fix(cloudflare): support response_text in streaming chunk parser Newer Cloudflare Workers AI models (e.g. Nemotron) emit 'response_text' instead of 'response' on streamed chunks. The non-streaming path was already updated to fall back to 'response_text' (#26385), but the streaming chunk parser still only read 'response', which caused streaming requests against those models to silently produce empty content. Mirror the non-streaming fallback in CloudflareChatResponseIterator.chunk_parser and add a streaming test for the response_text shape. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * Fix code qa * Address bugbot: drop dead encode/decode helpers; preserve empty custom_id - Remove unused _encode_gcp_label_value / _decode_gcp_label_value singular helpers; only the _chunks variants are actually called. - Use 'is not None' check for custom_id so empty-string custom_ids are still labeled and round-trip through batch outputs. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * Forward Vertex file content logging context * test vertex file content logging forwarding Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com> * Fix Vertex batch output logging mutation * fix: don't mutate caller's logging_obj in _try_transform_vertex_batch_output_to_openai The method was overwriting logging_obj.optional_params, logging_obj.model, and logging_obj.start_time on the caller's Logging instance. When invoked from llm_http_handler.py's generic framework path, the framework's own logging_obj (which already went through pre_call) had its properties clobbered, causing model and start_time to reflect the last batch line's values rather than the original call context. Fix: create a fresh local Logging instance for the per-line transformation instead of mutating the incoming logging_obj. The caller's object is now left entirely untouched regardless of whether a logging_obj was passed in or not. Regression tests added to verify model, start_time, and optional_params are not mutated on the caller's logging_obj. Co-authored-by: Sameer Kankute <Sameerlite@users.noreply.github.com> * feat: add opt-out flag for Vertex batch output transformation Adds litellm.disable_vertex_batch_output_transformation (default False). When True, afile_content returns raw Vertex predictions.jsonl untouched so users that parse candidates/modelVersion directly are not broken. * fix(anthropic,bedrock): omit thinking/output_config when reasoning_effort="none" Setting reasoning_effort="none" on Anthropic chat models (direct, Bedrock Invoke, Bedrock Converse, Vertex AI Anthropic, Azure AI Anthropic) crashed LiteLLM with: litellm.APIConnectionError: 'NoneType' object has no attribute 'get' Both the Anthropic chat transformation and Bedrock Converse called ``AnthropicConfig._map_reasoning_effort`` and assigned the ``None`` it returns for ``"none"`` directly to ``optional_params["thinking"]``. Downstream ``is_thinking_enabled`` then did ``optional_params["thinking"].get("type")`` and crashed. Pop ``thinking`` (and on Claude 4.6/4.7, ``output_config``) instead of assigning ``None``, restoring the documented contract that ``reasoning_effort="none"`` means "do not enable thinking". This also prevents downstream Anthropic 400s ("thinking: Input should be an object", "output_config.effort: Input should be ...") if the bug were ever masked. Verified end-to-end against the live Anthropic API and Bedrock Converse on claude-opus-4-{5,6,7} and claude-sonnet-4-6, plus Bedrock Invoke for Claude 4.5/4.6. Vertex AI Anthropic and Azure AI Anthropic inherit the fixed ``map_openai_params`` from ``AnthropicConfig`` and need no further changes. * fix(vertex-ai): set response=null on batch error entries per OpenAI spec The Vertex batch output transformer was emitting both a populated 'response' and 'error' for failed batch entries. The OpenAI Batch output spec defines them as mutually exclusive: on error 'response' MUST be null. This broke any consumer using 'result["response"] is None' to detect failures. * test(vertex-ai): cover transformation_error path emits response=null * fix(security): sandbox jinja2 in gitlab/arize/bitbucket prompt managers DotpromptManager was hardened to render through ImmutableSandboxedEnvironment. The three sibling managers (gitlab, arize, bitbucket) were missed and still instantiate plain jinja2.Environment(), leaving the same attribute-traversal SSTI primitive open: a template fetched from a GitLab/BitBucket repo or Arize Phoenix workspace can reach __class__.__init__.__globals__ and execute arbitrary Python on the proxy host. Match the dotprompt pattern by switching all three to ImmutableSandboxedEnvironment. The sandbox blocks the dunder-traversal chain while leaving normal {{ var }} substitution intact, so the template surface is unchanged for legitimate use. Adds tests/test_litellm/integrations/test_prompt_manager_ssti.py (18 cases) verifying each manager's jinja_env is a sandbox, that classic SSTI payloads raise SecurityError, and that ordinary variable rendering still works. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(proxy): drop client-supplied pricing fields from request bodies The proxy currently forwards request-body pricing parameters (the fields on `CustomPricingLiteLLMParams`, plus `metadata.model_info`) into the core call path. Those fields belong to deployment configuration, not to per-request input — sending them from a client mutates the request's recorded cost and, via `litellm.completion` → `register_model`, the process-wide `litellm.model_cost` map for every later caller in the worker. Strip them at the boundary. The strip set is built from `CustomPricingLiteLLMParams.model_fields` so pricing fields added later are covered automatically. Operators who do want clients to supply per-request pricing can opt back in per key or team via `metadata.allow_client_pricing_override = true`, mirroring the existing `allow_client_mock_response` and `allow_client_message_redaction_opt_out` flags. Tests cover the strip set's coverage, root and metadata strips, the opt-in skip on both key and team metadata, and a regression check that the global `litellm.model_cost` map is unmutated after a stripped request. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(proxy): log stripped pricing fields at debug for operator visibility Operators upgrading would otherwise see client-supplied pricing overrides silently stop applying with no diagnostic. Emit a debug-level line listing the dropped fields and pointing at the opt-in flag when any are stripped; stay silent on the no-op path so the log isn't filled with noise. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(proxy): move pricing strip below the litellm_metadata JSON-string parse The strip ran before the proxy parses ``litellm_metadata`` from a JSON string into a dict (a path used by multipart/form-data and ``extra_body`` callers), so ``isinstance(metadata, dict)`` was False and ``model_info`` survived the strip. Move the call to the same post-parse position the ``user_api_key_*`` strip already uses for the same reason. Adds a regression test exercising the JSON-string ``litellm_metadata`` path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(responses): replace legacy claude-4-sonnet alias in multiturn tool-call test Anthropic's main API no longer resolves the non-canonical 'claude-4-sonnet-20250514' alias for freshly issued keys, returning 404 not_found_error. PR #27031 already swept three other live tests pinned to this alias to claude-haiku-4-5-20251001 but missed test_multiturn_tool_calls in the responses API suite, which is now failing reliably on PR CI runs (e.g. PR #27074, job 1603363). Bump the two model references in test_multiturn_tool_calls to the same claude-haiku-4-5-20251001 snapshot used by PR #27031 -- it covers everything this test exercises (tool calling, multi-turn) and isn't on a deprecation schedule. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * chore(proxy): close callback-config and observability-credential side channels Two related gaps in the proxy's request bouncer: 1. ``is_request_body_safe`` (auth_utils.py) walked the request-body root and the ``litellm_embedding_config`` nested dict, but not ``metadata`` or ``litellm_metadata``. The same fields it bans at root — Langfuse / Langsmith / Arize / PostHog / Braintrust / Phoenix / W&B Weave / GCS / Humanloop / Lunary credentials and routing — were silently accepted when the caller put them inside metadata, retargeting observability callbacks to a caller-controlled host with caller-supplied creds. Walk both metadata containers (and parse the JSON-string form sent via multipart / ``extra_body``) through the same banned-params helper, so the existing ``allow_client_side_credentials`` opt-in covers both paths consistently. 2. The banned-params list was hand-maintained and lagged the canonical ``_supported_callback_params`` allow-list in ``initialize_dynamic_callback_params``. Derive the observability bans from that allow-list (minus a small ``_SAFE_CLIENT_CALLBACK_PARAMS`` set for informational fields like ``langfuse_prompt_version`` and ``langsmith_sampling_rate``) so future integrations are covered automatically; ``_EXTRA_BANNED_OBSERVABILITY_PARAMS`` carries the handful of fields integrations read but the allow-list hasn't caught up to. A guard test fails CI if a new entry is added to ``_supported_callback_params`` without an explicit safe-list decision. Separately in ``litellm_pre_call_utils.py``: add ``callbacks``, ``service_callback``, ``logger_fn``, and ``litellm_disabled_callbacks`` to ``_UNTRUSTED_ROOT_CONTROL_FIELDS``. The first three are appended to worker-wide ``litellm.{input,success,failure,_async_*,service}_callback`` lists / ``litellm.user_logger_fn`` from inside ``function_setup`` — one request poisons every subsequent caller in that worker. The last is the inverse primitive: the legitimate path reads it from key/team metadata, the request-body version silently disables admin-configured audit / observability for the call. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(auth): per-param allow must continue, not return early A pre-existing logic bug in ``_check_banned_params``: when the deployment-level ``configurable_clientside_auth_params`` permitted one banned field, the loop ``return``-ed on the first match instead of ``continue``-ing, so any other banned param later in the same body or metadata dict was never checked. This PR's metadata walk multiplies the surface where that bypass matters — a body pairing an allowed ``api_base`` with an observability credential like ``langfuse_host`` would silently pass. Proxy-wide ``allow_client_side_credentials`` keeps ``return`` (it's a global opt-in for every banned param). The per-param branch becomes ``continue`` so only the one explicitly-permitted field is skipped. Adds a regression test that exercises the api_base + langfuse_host pair. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(vector_store): resolve embedding config at request time, never persist creds The vector store create/update path previously called ``_resolve_embedding_config`` against the admin-configured router/DB model and persisted the resolved ``litellm_embedding_config`` dict (``api_key`` / ``api_base`` / ``api_version``) into the ``litellm_managedvectorstorestable.litellm_params`` column. Because the resolver expanded ``os.environ/...`` references via ``get_secret``, the DB row carried cleartext provider credentials, and the ``/vector_store/{new,info,update,list}`` responses returned them to any authenticated caller who could supply a known admin model name. Move the auto-resolve out of ``create_vector_store_in_db`` and out of the update path. Persist only the user-supplied ``litellm_embedding_model`` reference. Resolve at request-handling time inside ``_update_request_data_with_litellm_managed_vector_store_registry`` so the resolved config lives in the per-request ``data`` dict and is garbage-collected after the response. Legacy rows that were created by an earlier proxy version and already carry a resolved ``litellm_embedding_config`` skip the re-resolution and pass through unchanged so embedding calls keep working. The ``new_vector_store`` response now also runs the existing ``_redact_sensitive_litellm_params`` masker (already used by ``info``, ``update``, and ``list``), defending against caller-supplied cleartext on the create path and against legacy rows whose persisted credentials are still in the database. Existing tests that asserted the old write-time-resolve behaviour are updated to assert the new persistence shape (no embedding config stored, just the model reference). Two new tests cover the use-time path: one asserting fresh resolution happens when a row carries only the model reference, the other asserting legacy rows with persisted config skip re-resolution and continue to work. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(vector_store): tighten registry-mutation comment and dedupe test helpers * fix(vector_store): cache use-time embedding-config resolution Hold the resolved config in a process-memory TTL cache so the request-handling path doesn't run litellm_proxymodeltable.find_first on every vector-store call. * fix(anthropic,bedrock,vertex): forward output_config.effort + 400 on garbage reasoning_effort Follow-up bugs surfaced by the QA sweep on PR #27039 (https://github.com/BerriAI/litellm/pull/27039#issuecomment-4363363610). 1. Stop stripping output_config.effort on Bedrock + Vertex adaptive routes. - Vertex AI Claude 4.6/4.7 accepts output_config.effort on rawPredict (verified end-to-end against us-east5 / global). The strip helper now no-ops for effort. - Bedrock Converse routes output_config into additionalModelRequestFields for anthropic base models so the requested adaptive tier (low/medium/ high/xhigh/max) actually reaches the wire instead of all collapsing to identical thinking. - Bedrock Invoke chat transformation (AmazonAnthropicClaudeConfig) stops popping output_config from the post-AnthropicConfig request body. - Bedrock Invoke /v1/messages allowlist (BedrockInvokeAnthropicMessagesRequest) now lists output_config so the runtime allowlist filter forwards it. 2. Validate effort across Bedrock Converse so 'disabled' / 'invalid' / '' / unsupported tiers (xhigh/max on Sonnet 4.6 or budget-mode 4.5 models) surface as a clean 400 BadRequestError instead of 500. 3. ValueError -> BadRequestError throughout (AnthropicConfig.map_openai_params, _apply_output_config, AmazonConverseConfig._handle_reasoning_effort_parameter). Empty-string effort is now rejected (was silently passing the 'if effort and ...' short-circuit). 4. Floor reasoning_effort='minimal' at the Anthropic provider minimum (1024 budget_tokens) via new ANTHROPIC_MIN_THINKING_BUDGET_TOKENS so it's a usable tier on direct Anthropic / Azure AI Anthropic / Vertex AI Anthropic / Bedrock Invoke (all of which 400 below 1024). 5. model_prices: dedupe duplicate supports_max_reasoning_effort key on claude-opus-4-7 / claude-opus-4-7-20260416. Adds regression tests across all five affected paths; existing tests asserting the silent-strip behavior were updated to reflect the new pass-through and clean 400 surfaces. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * fix(constants): make ANTHROPIC_MIN_THINKING_BUDGET_TOKENS a plain constant The documentation CI test (tests/documentation_tests/test_env_keys.py) asserts every os.getenv() key in the source has a matching entry in the litellm-docs config_settings.md table. ANTHROPIC_MIN_THINKING_BUDGET_TOKENS tracks Anthropic's published wire-protocol minimum (1024) — it's not a user-tunable, so making it env-overridable was wrong anyway. Drop the os.getenv() wrapper; the value is now a plain literal. Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * fix(anthropic,bedrock): correct effort error message and dedupe effort_map - Remove 'none' from the Bedrock _validate_anthropic_adaptive_effort error message; it was listed as a valid value but rejected by the membership check, leaving users in a feedback loop if they tried 'none'. - Hoist the duplicated reasoning_effort -> output_config.effort mapping out of AnthropicConfig.map_openai_params and AmazonConverseConfig._handle_reasoning_effort_parameter into a single AnthropicConfig.REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT class constant so the two routes cannot drift. * fix(anthropic): translate reasoning_effort on /v1/messages route Closes the remaining QA-sweep gap on PR #27074: Bedrock Invoke /v1/messages was silently ignoring ``reasoning_effort`` because the shared param filter only kept native Anthropic keys, so every effort tier collapsed to the same behavior on the wire (27/231 cells failing across opus-4-5 / opus-4-6 / sonnet-4-6). Map ``reasoning_effort`` to native Anthropic ``thinking`` / ``output_config.effort`` at the ``AnthropicMessagesConfig`` layer so all four /v1/messages routes (direct Anthropic, Azure AI, Vertex AI, Bedrock Invoke) inherit the same translation: - Add ``reasoning_effort`` to ``AnthropicMessagesRequestOptionalParams`` so the param filter in ``AnthropicMessagesRequestUtils.get_requested_anthropic_messages_optional_param`` no longer drops it before the transformation runs. - Add ``_translate_reasoning_effort_to_anthropic`` and call it from ``transform_anthropic_messages_request``. Mirrors ``AnthropicConfig.map_openai_params`` on the chat completion path (re-uses ``_map_reasoning_effort`` and ``REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT``) so the two routes cannot drift. Pops ``reasoning_effort`` so it never reaches the wire. - Caller-supplied native ``thinking`` / ``output_config.effort`` always win — same precedence as ``_translate_legacy_thinking_for_adaptive_model``. - Garbage values (``""``, ``"disabled"``, ``"invalid"``) raise ``AnthropicError(status_code=400)`` instead of falling through and surfacing as 500s from the provider. - ``"none"`` clears thinking + output_config so callers can opt out per request. Also restores the non-adaptive-model test coverage on Bedrock Invoke /v1/messages that the previous commit lost when ``test_bedrock_messages_strips_output_config`` was renamed to the ``forwards`` variant on Opus 4.7. Adds a new test file ``test_reasoning_effort_translation.py`` covering the translation at the shared config level (adaptive + non-adaptive models, none, garbage, caller precedence) so all four /v1/messages routes are exercised by a single suite. Adds parametrized + behavioral tests on the Bedrock Invoke /v1/messages suite covering: minimal/low/medium/high/xhigh/max mapping for adaptive models, thinking-budget mapping for non-adaptive Opus 4.5, ``none`` clears both, garbage raises 400, explicit ``output_config`` wins. Refs: https://github.com/BerriAI/litellm/pull/27074 * fix(anthropic,bedrock): reject unmapped reasoning_effort at mapping site Both the chat completion path (AnthropicConfig.map_openai_params) and the Bedrock Converse path (_handle_reasoning_effort_parameter) used REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT.get(value, value) which falls back to the raw input on unmapped keys. Combined with _map_reasoning_effort returning type='adaptive' for any string on Claude 4.6/4.7, garbage values (e.g. 'disabled') could leak into optional_params['output_config']['effort'] unvalidated if map_openai_params ran without the downstream transform_request or _validate_anthropic_adaptive_effort check. Mirror the /v1/messages pattern: use .get(value) (no fallback) and raise BadRequestError immediately when the value is unmapped, co-locating validation with the mapping for defense in depth. * style: black formatting Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com> * fix(anthropic): stop class-attr leak; gate xhigh/max on every route The reasoning-effort mapping dict was a public class attribute on AnthropicConfig, so BaseConfig.get_config returned it as a request parameter and every Anthropic-backed call (Anthropic / Azure / Vertex / Bedrock Invoke) hit a 400 'REASONING_EFFORT_TO_OUTPUT_CONFIG_EFFORT: Extra inputs are not permitted' from the provider. Move the mapping to a module-level constant. _supports_effort_level only looked the model up under custom_llm_provider='anthropic', so bedrock-prefixed model ids (e.g. bedrock/invoke/us.anthropic.claude-opus-4-7) returned False for both 'max' and 'xhigh' even when the underlying model entry has the flag set. Strip known provider prefixes and retry the lookup against litellm.model_cost directly so per-model gating works on every route. Mirror the per-model xhigh/max gate from AnthropicConfig._apply_output_config in AnthropicMessagesConfig._translate_reasoning_effort_to_anthropic so the /v1/messages route also raises a clean 400 instead of forwarding the unsupported tier. * feat(anthropic,bedrock): strip output_config under drop_params for non-effort models When a proxy fronts Claude Code (which always sends `output_config.effort`) at a pre-4.5 Anthropic model — haiku-3, sonnet-3.5, opus-3, sonnet-4 — the forwarded knob causes a forced 400 the client can't fix. Gating a strip behind the existing `drop_params` flag lets operators opt into silent fixup once and stop worrying about per-model param hygiene. Default (`drop_params=False`) still forwards and surfaces the provider's error, preserving the strict, debuggable contract from #27074. Per https://platform.claude.com/docs/en/build-with-claude/effort the supporting set is Opus 4.5+, Sonnet 4.6+, and Mythos Preview; everything else is dropped (with a verbose_logger warning so the strip is visible). Recognition uses model-name patterns plus a fallback to any `supports_*_reasoning_effort` flag in the model map for forward compatibility with new entries. https://claude.ai/code/session_01WjHq31rvXT6xYNdVmSJvRp (cherry picked from commit |
||
|
Michael Riad Zaky
|
4e26835098 | Reorder counter invalidation to run after DB write | ||
|
Michael Riad Zaky
|
fed5f36a3d | Invalidate spend counters on budget reset | ||
|
yuneng-jiang
|
15b7386859
|
Merge pull request #26815 from stuxf/fix/get-image-lfi-ssrf
chore(proxy): contain UI_LOGO_PATH / LITELLM_FAVICON_URL on unauthenticated asset endpoints |
||
|
user
|
215f538d4f | fix(static-assets): browser-load remote branding assets | ||
|
user
|
c112bdf2c1
|
chore(static-assets): /simplify pass — top-level Response import + cleaner test fixture
Two cleanups from the /simplify review pass:
* ``Response`` was imported inside the ``except OSError`` branch in
``/get_image`` and at the top of ``/get_favicon``. Per the project's
no-inline-imports rule (CLAUDE.md), hoisted to the existing
``from fastapi.responses import (...)`` block at the top of
``proxy_server.py``.
* The test class's ``_patches()`` helper returned a 2-element list of
patch context managers and tests indexed into them via
``self._patches(...)[0], self._patches()[1]`` — two distinct calls
with confusing aliasing semantics. Restructured to:
- module-level ``_patch_async_safe_get(...)`` that returns a single
patch context manager
- autouse fixture that patches ``get_async_httpx_client`` for every
test in the file (it's the same patch in every case)
- small ``_image_response(...)`` factory to deduplicate Mock setup
Tests now read as ``with _patch_async_safe_get(return_value=...):``
with no list-indexing or duplicate Mock construction.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
user
|
75d1a0116e
|
fix(static-assets): use async_safe_get; drop SVG; serve bytes inline on cache miss
Three review items addressed: * **Veria (Medium): SSRF via redirect.** ``fetch_validated_image_bytes`` was calling ``validate_url(url)`` once and then fetching with the default httpx client, so a 3xx to an internal IP would have been followed unvalidated. Switched to ``async_safe_get`` (the existing SSRF primitive used elsewhere in the codebase) which walks each redirect hop, re-validates, and rejects redirects to blocked networks. Default ``litellm.user_url_validation`` is True so protection is on out of the box. * **Greptile (P2): SVG can embed JS.** Removed ``image/svg+xml`` from the allowed-Content-Type set. The hardcoded response media type (``image/jpeg`` / ``image/x-icon``) means a real SVG body wouldn't render as SVG anyway in modern browsers — the allowlist entry was giving up XSS surface for no actual SVG-rendering benefit. If real SVG support is wanted later, that's a deliberate feature PR with CSP / nosniff bundled. * **Greptile (P2): cache-write OSError drops validated bytes.** When the upstream fetch succeeded but ``open(cache_path, "wb")`` raised (read-only assets dir), the bytes were discarded and the default logo was served — a silent regression for that deployment. Now serve the validated bytes inline via ``Response(...)`` as a fallback before falling back to default. Tests: - Replaced low-level mocks of ``validate_url`` with mocks of ``async_safe_get`` directly, exercising the helper's contract rather than the SSRF primitive's internals. - New ``test_rejects_svg_content_type`` confirms SVG is blocked. - ``test_get_image_cache_logic`` fixture now sets ``mock_response.is_redirect = False`` so ``async_safe_get`` doesn't treat the Mock's truthy attribute as a redirect. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
user
|
0166992f6b
|
fix(proxy): contain UI_LOGO_PATH and LITELLM_FAVICON_URL to allowed asset roots
The unauthenticated ``/get_image`` and ``/get_favicon`` endpoints accept the admin-set env vars ``UI_LOGO_PATH`` and ``LITELLM_FAVICON_URL`` and return whatever bytes they resolve to, with a hard-coded ``image/jpeg`` or ``image/x-icon`` content-type. Two attack shapes: * ``UI_LOGO_PATH=/etc/passwd`` (or any other readable file path) — any unauthenticated caller exfiltrates the file via ``GET /get_image``. The previous gate was ``os.path.exists(logo_path)`` which fires on every readable file. Same shape for the favicon endpoint. * ``UI_LOGO_PATH=http://169.254.169.254/iam`` (or any internal HTTP service the admin pointed at) — the proxy fetches it server-side and streams the response body to the unauthenticated caller. No URL validation, no Content-Type validation; ``application/json`` AWS metadata gets tunneled out under the ``image/jpeg`` wrapper. New helper module ``litellm/proxy/common_utils/static_asset_utils.py``: * ``resolve_local_asset_path(candidate, allowed_roots)`` — returns the resolved absolute path only if it lives within one of the allowed asset roots. Uses ``realpath`` so symlinks pointing outside the roots are caught. * ``fetch_validated_image_bytes(url)`` — runs the URL through ``validate_url`` (rejecting private / cloud-metadata / loopback targets) and only returns the response body if the upstream Content-Type is in a small allowlist of image MIME types. Both ``/get_image`` and ``/get_favicon`` are wired through the helpers. The SSRF gate is enforced unconditionally — these endpoints are unauthenticated, so the admin-facing ``litellm.user_url_validation`` toggle does not apply (an admin who opted out of URL validation for LLM provider paths shouldn't also expose ``/get_image`` to SSRF). Tests: - ``TestResolveLocalAssetPath``: 10 cases covering legitimate paths, ``/etc/passwd``, ``/proc/self/environ``, symlink-out, ``..`` traversal, directories, missing files, and root list edge cases. - ``TestFetchValidatedImageBytes``: 7 cases covering SSRF block, non- image content-type rejection, valid image passthrough, non-200 response, fetch exception, empty URL, and parametrized coverage of every allowed image MIME type. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
user
|
294ac8383e
|
fix(vector-stores): recurse into nested litellm_params; handle JSON-string shape
Three issues surfaced in review of the previous commit: 1. **Veria — Medium**: ``litellm_params`` carries a nested ``litellm_embedding_config`` dict (auto-resolved from the model registry on create / update) which itself holds ``api_key`` / ``aws_*`` / ``vertex_credentials``. The previous redactor only inspected top-level keys, so the nested values passed through unredacted. Recurse into nested dicts. 2. **Greptile — P2**: when ``litellm_params`` is a JSON-serialized string (the in-memory registry occasionally stores it that way), the previous redactor silently no-op'd via the ``isinstance(..., dict)`` guard and echoed the raw payload back. Now: parse, redact, re-serialize. If the string is not valid JSON, replace it with the redaction sentinel rather than echo it. 3. **mypy** flagged ``_redact_sensitive_litellm_params``'s ``Optional[Dict[str, Any]]`` signature as incompatible with the ``object``-typed call site. Widened to ``Any -> Any`` to reflect the actual contract (the function now handles dict / str / None / other). Also fixes a related test regression in ``test_remove_sensitive_info_from_deployment_with_excluded_keys``: the ``"credentials"`` plural addition to ``SensitiveDataMasker`` defaults caused the first call (without ``excluded_keys``) to mutate the input dict's ``litellm_credentials_name`` to a masked value. The second call (with ``excluded_keys``) then saw the already-masked value rather than the original. Construct fresh input for each call. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
yuneng-jiang
|
761e124c17
|
Merge pull request #26460 from BerriAI/litellm_expired_dashboard_key_cleanup
feat(proxy): Add cleanup job for expired LiteLLM dashboard session keys |
||
|
Yuneng Jiang
|
7503f14f3f
|
fix: harden /model/info redaction to cover plural credential field names
Mirrors the existing vertex_credentials handling for the newer vertex_ai_credentials field, and extends the dynamic masker's sensitive pattern set to recognize the plural form so other plural-named credential fields are also covered. |
||
|
Milan
|
22439a119e
|
Handle cleanup delete races and accurate counts
Made-with: Cursor |
||
|
Milan
|
69c5840e5f
|
Test cleanup of multiple expired UI session keys
Made-with: Cursor |
||
|
Milan
|
21cf42f568
|
Add expired UI session key cleanup job
Made-with: Cursor |
||
|
Yuneng Jiang
|
c41567eaa0
|
fix(budget_reset): use raw SQL for IS NOT NULL filter on Json? columns
The periodic budget-window reset job filtered keys/teams with
`where={"budget_limits": {"not": None}}`. The prisma-client-python
library does not support null-filtering on `Json?` columns (no
DbNull/JsonNull sentinel — upstream issue #714). The client drops the
`None` value during serialization and the engine rejects the query with
`MissingRequiredValueError: where.budget_limits.not: A value is
required but not set`, so neither the key nor team reset path runs.
Switch those two `find_many` calls to `query_raw` with
`WHERE budget_limits IS NOT NULL`, selecting only the PK and the
`budget_limits` column. Writes still go through the ORM. Add unit tests
covering the expired/unexpired paths for keys and teams, string-encoded
JSON payloads, empty payloads, error isolation between the two paths,
and a regression guard asserting the query still uses `IS NOT NULL`.
|
||
|
ryan-crabbe-berri
|
c4c1861389
|
Merge pull request #26195 from BerriAI/litellm_team_member_total_spend
Track per-member total spend on team memberships |
||
|
shivam
|
5770af0068
|
Merge branch 'litellm_internal_staging' into litellm_individual-team-member-budgets | ||
|
shivam
|
27a105bcf9
|
fix: give each team member an independent budget instead of sharing the team default
Previously, members added to a team without an explicit per-member budget were all linked to the same `litellm_budgettable` row referenced by the team's `metadata.team_member_budget_id`. Updating one member's budget via `/team/member_update` mutated the shared row and silently changed every other member's budget too. Now both write paths produce a private, per-member budget: - `add_new_member` clones the team's default budget into a fresh row when a member is added without `max_budget_in_team`/`allowed_models`. If no team default exists, the membership is created with no budget. - `_upsert_budget_and_membership` detects when an existing membership still points at the team's default budget id and clones-on-write, relinking the membership to the new private budget before applying the update. - `team_member_update` reads `team_member_budget_id` from team metadata and passes it through so the helper can make this distinction. Adds unit tests for clone-on-write, in-place update of a private budget, and the no-default-no-budget add path. Made-with: Cursor |
||
|
Ryan Crabbe
|
e5f3e15969
|
Track per-member total spend on team memberships
Adds total_spend column to LiteLLM_TeamMembership that accumulates continuously and is not zeroed by the budget cycle reset job. This enables UI surfaces to distinguish current-cycle spend (the existing spend column, which resets) from lifetime spend per team member. Also exposes budget_reset_at on LiteLLM_BudgetTable so /team/info callers can see when a member's budget window next resets. The field was already stored in the DB but stripped by the response Pydantic model. Includes regression tests that: - Guard the reset job against ever writing total_spend: 0 - Verify the spend writer increments both spend and total_spend in one UPDATE statement. |
||
|
Krrish Dholakia
|
386f334fee
|
Prompt Compression - add it to the proxy (#25729)
* refactor: new agentic loop event hook simplifies how to create logic for tool based multi llm calls * fix: compress - make it work on anthropic input as well * fix(compress.py): working prompt compression for claude code ensures claude code messages can run through proxy easily * docs: add agentic loop hook guide * docs: add agentic_loop_hook to sidebar * fix: fix multiple arguments error * fix: fix tool call loop for compression on streaming /v1/messages * fix: fix linting errors * fix: fix ci/cd errors * feat(litellm_pre_call_utils.py): use claude code session for litellm session id allows claude code logs to be stitched together, making it easy to know they were all part of the same conversation * fix: suppress incorrect mypy warning rE: module * revert: drop PR's changes to litellm/proxy/_experimental/out/ Restores the 34 HTML files under _experimental/out/ to their pre-PR paths (X/index.html -> X.html). All renames are R100 (content unchanged); no other files are touched. * fix: address greptile review comments on PR #25729 - Skip ``kwargs["tools"] = []`` injection when compression is a no-op — Anthropic Messages rejects empty tool arrays on requests that did not originally declare tools. - Move agentic-loop safety guards (fingerprint cycle / max depth) out of the per-callback try/except so they propagate instead of being swallowed by the generic exception handler. Extracted _check_agentic_loop_safety. - Gate generic ``x-<vendor>-session-id`` capture behind the LITELLM_CAPTURE_VENDOR_SESSION_HEADERS env var (off by default) to preserve backwards compatibility; explicit x-litellm-* headers are unaffected. - Fix monkeypatch target in pre-call-hook test to patch the actual module-level binding (litellm.integrations.compression_interception.handler.compress). - Add regression tests for empty-tools skip and opt-in session capture. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * revert: drop LITELLM_CAPTURE_VENDOR_SESSION_HEADERS flag Generic x-<vendor>-session-id header capture is a new feature and only runs *after* the explicit x-litellm-trace-id / x-litellm-session-id checks, so it does not change behavior for any existing caller that was already using the LiteLLM headers — no backwards-incompatibility to gate. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor(compress): replace input_type with CallTypes call_type Drop the bespoke ``CompressionInputType`` literal and use the existing ``litellm.types.utils.CallTypes`` enum instead. ``litellm.compress()`` now takes ``call_type: Union[CallTypes, str]`` (default ``CallTypes.completion``) — no new concept to learn, and the enum is already the way the rest of the codebase talks about request shapes. Supported values: ``completion`` / ``acompletion`` (OpenAI chat-completions shape) and ``anthropic_messages`` (Anthropic structured content blocks). Updated: compress(), the compression_interception handler, tests, docs, and the two eval scripts. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Yuneng Jiang
|
f483f1e800
|
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_yj_apr17 | ||
|
shivam
|
fad884e9a7
|
Merge branch 'litellm_internal_staging' into litellm_persist_default_router_end_budget | ||
|
Krrish Dholakia
|
bb9955beca
|
[Fix] Budget reset job now resets implicitly-created end users with NULL budget_id
When litellm.max_end_user_budget_id is configured, implicitly-created end users (via /chat/completions) have budget_id=NULL in the DB since the default budget is only applied in-memory. The budget reset job filtered by budget_id, so these users were never reset and eventually permanently blocked. Fix: when the default budget is in the reset list, also query for and reset end users with budget_id=NULL and spend > 0. This keeps the hot auth path unchanged (no DB writes on every request). Fixes #22019 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Yuneng Jiang
|
11c3270cdc
|
Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_yj_apr17
# Conflicts: # litellm/__init__.py |
||
|
Ishaan Jaffer
|
e8461b5b97
|
style: run black formatter on files from main merge | ||
|
Ishaan Jaffer
|
f31d4faa87
|
Merge origin/main into litellm_ishaan_april6 | ||
|
user
|
b4e98d190a
|
fix(proxy): close 6 more metadata/tag variant bypasses
Post-merge audit found 6 adjacent variants of the VERIA-28 class. All
fixed here with regression tests:
1. Strip widened from 3 named keys to the full user_api_key_* prefix.
The proxy writes a dozen user_api_key_* fields (user_id, alias,
spend, team_id, request_route, end_user_id, …) into
data[_metadata_variable_name]; the 3-key strip left the rest
exploitable for identity/spend forgery in audit logs and guardrails.
2. proxy_server_request['body'] snapshot moved to AFTER the strip.
Was captured at line ~990 before the strip ran, so
standard_logging_object, lago, and spend_tracking readers saw the
attacker-forged payload even though the live data dict was clean.
3. get_tags_from_request_body (auth-time) now coerces JSON-string
metadata via safe_json_loads. Previously crashed with
AttributeError on string metadata (DoS; potential RBAC bypass if
a caller swallowed the exception).
4. get_end_user_id_from_request_body coerces JSON-string
metadata/litellm_metadata. Previously isinstance(dict) guard
caused end-user budget attribution to be silently skipped when
the caller sent metadata as a JSON string.
5. Four hand-rolled 'if data.get("metadata") is None: data["metadata"] = {}'
blocks in proxy_server.py (7160, 7341, 7590, 11375) now guard on
isinstance(dict). They crashed with TypeError when metadata was a
JSON string (DoS).
6. _get_admin_metadata defensively guards with isinstance(dict);
previously AttributeError'd on any leaked string metadata.
Also hoists the inline safe_json_loads import in _guardrail_modification_check
to module level per CLAUDE.md style.
|
||
|
user
|
c2b3b62996
|
test: add unit tests for path_utils safe_join and safe_filename | ||
|
Dmitriy Alergant
|
e1bf114591
|
fix(budget): align budget table reset times with standardized calendar schedule (#25440)
Budget table entries (team members, end-users) used duration_in_seconds() for a sliding-window reset, while keys/users/teams used calendar-aligned get_budget_reset_time(). This made "30d" and "1mo" mean different things depending on entity type. Now both paths use get_budget_reset_time() for consistent calendar-aligned resets (e.g. "30d" → 1st of next month). Fixes #25432 Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Ishaan Jaffer
|
1c238b630f
|
fix(tests): update upsert tests to reflect new update-in-place behavior for existing budget_id | ||
|
ishaan-berri
|
51876292a0
|
Litellm ishaan april4 2 (#25150)
* feat(router): integrate allowed_fails_policy into health check failures (#24988) * feat(router): integrate allowed_fails_policy into health check failures Health check failures now increment the same per-deployment failure counters used by allowed_fails_policy, so users can control how many health check failures of each error type are required before a deployment enters cooldown. - ahealth_check() preserves the original exception in its return dict - run_with_timeout() returns a litellm.Timeout on health check timeout - _perform_health_check() propagates exceptions to unhealthy endpoints - _write_health_state_to_router_cache() calls _set_cooldown_deployments for each unhealthy endpoint that has an exception - When allowed_fails_policy is set, the binary health check filter is bypassed so cooldown is the sole routing exclusion mechanism - Safety net: if all deployments are in cooldown with enable_health_check_routing=True, the cooldown filter is bypassed Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(router): add health_check_ignore_transient_errors flag When enabled, health check failures with 429 (rate limit) or 408 (timeout) status codes are skipped from the cooldown pipeline. These are transient load issues, not broken deployments. Auth errors (401), 404, and 5xx errors still increment counters and trigger cooldown as before. Config (general_settings): health_check_ignore_transient_errors: true Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(router): also exclude 429/408 from health state cache when ignore_transient_errors set The previous fix only skipped cooldown counter increments. The health state cache was still marking 429/408 endpoints as is_healthy=False, causing the binary health check filter to exclude them from routing. Now, when health_check_ignore_transient_errors=True, 429/408 endpoints are also excluded from the unhealthy list passed to build_deployment_health_states(), so the binary filter treats them as unaffected (not unhealthy). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs(router): add health check driven routing guide New standalone page covering the full health check routing feature: allowed_fails_policy integration, health_check_ignore_transient_errors, architecture SVG, step-by-step setup, and gotchas (TTL, AllowedFails semantics). Replaces the inline section in health.md with a link to the new page. Added to the Routing & Load Balancing sidebar. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(health-check-routing): fix three CI failures - Add "exception" to ILLEGAL_DISPLAY_PARAMS in health_check.py so the exception object is stripped before the health endpoint serializes results to JSON (fixes TypeError: 'URL' object is not iterable) - Add allowed_fails_policy = None to FakeRouter stubs in test_router_health_check_routing.py (fixes AttributeError) - Add health_check_ignore_transient_errors to config_settings.md router settings reference table (fixes documentation test) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix litellm/tests/proxy_unit_tests/test_proxy_server.py * fix(router): address greptile review comments - Narrow cooldown safety-net bypass: only fires when allowed_fails_policy is set (cooldown is health-check driven). Without a policy, cooldowns are from real request failures and must not be bypassed. - Restore cooldown deployments DEBUG log that was accidentally removed. - Fix test_health TypeError: move exception extraction to a separate exceptions_by_model_id dict returned alongside endpoints, so exception objects never appear in the endpoint dicts that get JSON-serialized by the /health response. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(health-check-routing): properly isolate exceptions from health response Return exceptions_by_model_id as a separate third value from _perform_health_check / perform_health_check so exception objects (which contain non-JSON-serializable httpx URL types) never appear in the endpoint dicts that get serialized by the /health response. Callers updated: _health_endpoints.py, shared_health_check_manager.py, proxy_server.py background loop. All use the exceptions dict only for cooldown integration, not for display. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(shared-health-check): fix remaining 2-value return sites and update type annotation * fix(health-check-routing): fix P0 cooldown integration never firing The cooldown loop was reading endpoint.get("exception") which is always None because exceptions are now returned via exceptions_by_model_id, not stored in endpoint dicts. Fixed to use _exceptions.get(model_id). Also fixes the transient-error filter to use _exceptions instead of endpoint.get("exception"), and fixes all remaining 2-value return sites in shared_health_check_manager.py. Tests updated to pass exceptions via exceptions_by_model_id parameter instead of endpoint dicts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(health-check-routing): fix P1 transient-error filter broken on cache hits When SharedHealthCheckManager returns cached results, exceptions_by_model_id is always {} so the transient-error filter defaulted to status 500 for all endpoints, incorrectly marking 429/408 endpoints as unhealthy. Fix: store integer exception_status on each unhealthy endpoint dict in _perform_health_check. _get_endpoint_exception_status() uses the live exception object when available (direct path) and falls back to the stored integer (cache-hit path). The integer is JSON-serializable and survives the shared cache round-trip. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(health-check-routing): gate cooldown loop behind allowed_fails_policy Without the policy, cooldown is not the routing exclusion mechanism. Firing _set_cooldown_deployments for all enable_health_check_routing users was a backwards-incompatible change — 401s would immediately cooldown deployments that the binary filter would have recovered on the next cycle. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * revert: undo allowed_fails_policy gate on cooldown loop Cooldown integration via health checks is intentional for all enable_health_check_routing users, not just those with allowed_fails_policy. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(docs+tests): fix health_check_ignore_transient_errors doc section and test coverage - Move health_check_ignore_transient_errors from router_settings to general_settings in config_settings.md (code reads it from general_settings) - Remove duplicate enable_health_check_routing / health_check_staleness_threshold entries that were incorrectly listed under router_settings - Replace TestHealthCheckEndpointExceptionPropagation tests with ones that exercise the real _perform_health_check code path via mocked ahealth_check, verifying exceptions appear in exceptions_by_model_id and NOT in endpoint dicts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(tests+docs): fix tuple unpacking and docs test failures - Update test mocks that return (healthy, unhealthy) to return (healthy, unhealthy, {}) to match the new 3-value signature - Update test unpackings of perform_shared_health_check to use healthy, unhealthy, _ = ... - Add health_check_ignore_transient_errors to router_settings section in config_settings.md (it is a Router constructor param, so the doc test requires it there; it also lives in general_settings for proxy use) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix CodeQL errors * fix(tests): fix 2-value unpackings of _perform_health_check in test_health_check.py * fix(tests): fix mock _perform_health_check returning 2-tuple instead of 3 * fix team routing --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add distributed lock for key rotation job (#23364) * fix: add distributed lock for key rotation job * fix: address Greptile review feedback on key rotation lock (#23834) * fix: address Greptile review feedback on key rotation lock * fix req changes greptile * feat(proxy): Optional on_error for guardrail pipeline (API / technical failures) (#24831) * guardrails fallback * docs * docs: add LITELLM_KEY_ROTATION_LOCK_TTL_SECONDS to environment variables reference * fix(mypy): accept Union[Dict, Any] in _get_deployment_order and use typed list to fix min() type error * fix(mypy): use Optional[str] for api_base in PydanticAI provider to match superclass signature --------- Co-authored-by: Sameer Kankute <sameer@berri.ai> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Harshit Jain <48647625+Harshit28j@users.noreply.github.com> Co-authored-by: Shivam Rawat <shivam@berri.ai> Co-authored-by: yuneng-jiang <yuneng@berri.ai> |
||
|
yuneng-jiang
|
b314e8d20a
|
Merge pull request #20688 from BerriAI/litellm_budget_tier_enforcement_for_keys
[Fix] Budget-linked keys never had spend reset |
||
|
Harshit28j
|
b39218059a | fix req change | ||
|
Harshit28j
|
0b7d8b9a0d | fix: edge case when key alias empty | ||
|
Sameer Kankute
|
af3b6f3334 | Fix: litellm/tests/test_litellm/proxy/common_utils/test_http_parsing_utils.py | ||
|
Ryan Crabbe
|
53c10b0b64 |
Merge origin/main and address Greptile review feedback
- Resolve merge conflict in pass_through_endpoints.py - Add .copy() to proxy_server_request headers to prevent cache corruption - Add test for request.state unavailable fallback path |
||
|
Sameer Kankute
|
57af8e6a93
|
Merge pull request #21924 from BerriAI/main
merge main in oss 22 02 |
||
|
Krish Dholakia
|
52585eb2d7
|
Revert "fix(vertex_ai): enable context-1m-2025-08-07 beta header (#21870)" (#21876)
This reverts commit
|
||
|
Edwin Isac
|
bce078a796
|
fix(vertex_ai): enable context-1m-2025-08-07 beta header (#21870)
* server root path regression doc * fixing syntax * fix: replace Zapier webhook with Google Form for survey submission (#21621) * Replace Zapier webhook with Google Form for survey submission * Add back error logging for survey submission debugging --------- Co-authored-by: Ishaan Jaff <ishaanjaffer0324@gmail.com> * Revert "Merge pull request #21140 from BerriAI/litellm_perf_user_api_key_auth" This reverts commit |
||
|
LeeJuOh
|
50f36d9ca6
|
fix(budget): fix timezone config lookup and replace hardcoded timezone map with ZoneInfo (#21754)
* fix(budget): fix timezone config lookup and replace hardcoded timezone map with ZoneInfo * fix(budget): update stale docstring on get_budget_reset_time |
||
|
Ishaan Jaffer
|
2270a3aaf3 |
Revert "Merge pull request #21140 from BerriAI/litellm_perf_user_api_key_auth"
This reverts commit |
||
|
Ryan Crabbe
|
c7ad8053b1 |
Merge origin/main into litellm_perf_user_api_key_auth
Resolve conflicts: - pass_through_endpoints.py: take main's version, re-apply MAPPED_PASS_THROUGH_PREFIXES startswith(tuple) optimization - test_user_api_key_auth.py: keep both auth optimization regression tests and JWT admin identity field tests |
||
|
Ryan Crabbe
|
f0a542c55d |
fix: add .copy() to create_request_copy and tests for header caching
Protect cached headers from mutation in create_request_copy sites and add unit tests for _safe_get_request_headers caching behavior. |
||
|
Ryan Crabbe
|
3275fb09cb |
fix: add State() to mock requests in http_parsing_utils tests
The _safe_get_request_headers caching uses request.state._cached_headers, which returns a truthy MagicMock on bare MagicMock() objects instead of None, breaking content-type detection for form-data tests. |
||
|
Shivam Rawat
|
c45a17bad4
|
Merge branch 'main' into litellm_budget_tier_enforcement_for_keys | ||
|
shivam
|
0117b35a6b | added more tests, fixed tests | ||
|
Harshit Jain
|
f33a7ca7d6
|
fix: as per request changes | ||
|
Harshit Jain
|
f87b12a2f7
|
fix grace period with better error handling on frontend and as per best practices | ||
|
Harshit Jain
|
673b7d1fea
|
Merge branch 'main' into litellm_fix-virtual-key-grace-period | ||
|
Varun Chawla
|
e587370f67 |
fix(proxy): add regression tests for #20441 - <script> tags in messages (#20573)
* fix: empty guardrails/policies arrays should not trigger enterprise license check (#20304) The UI sends empty arrays for enterprise-only fields (guardrails, policies, logging) even when the user has not configured these features. The backend `is not None` check treated `[]` as a truthy intent to use the feature, falsely requiring an enterprise license for basic team operations. Backend: Add `and updated_kv[field] != [] and updated_kv[field] != {}` guards in `_update_metadata_fields` so empty collections are skipped. UI: Conditionally omit guardrails, logging, and policies from the payload when empty instead of defaulting to `[]`. Fixes #20304 * fix: allow clearing fields with empty collections while skipping enterprise check Address PR review feedback: 1. Move the empty-collection guard into _update_metadata_field (singular) so that empty lists/dicts skip only the premium license check but still get written into metadata. This lets users intentionally clear a previously-set field (e.g. guardrails: []) without being blocked, while the UI's default empty arrays still don't trigger a false enterprise error. 2. Remove sys.path hack from test file; use standard imports that work with pytest discovery. 3. Add tests verifying that empty collections are moved into metadata (field clearing works) even though they bypass the premium check. Fixes #20304 * fix(proxy): add regression tests for #20441 - ensure <script> tags in LLM messages are not blocked The 403 Forbidden error when sending messages containing `<script>` is caused by external WAF/reverse proxy infrastructure (confirmed by the standard nginx HTML 403 response format), not by LiteLLM's own content filtering. However, these regression tests ensure that: 1. The content filter guardrail's built-in patterns do not match HTML tags 2. Messages containing <script> and other HTML tags pass through the content filter unchanged when no explicit HTML-blocking rules are configured 3. The HTTP request body parser correctly handles JSON payloads containing HTML content without modification These tests guard against accidentally introducing HTML/XSS filtering that would break legitimate LLM API usage (e.g., discussing HTML/JavaScript code). Closes #20441 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |