* fix(logging): stop dumping the request body at every layer under --detailed_debug
The proxy dumped the full request body at every layer it passed through
(proxy entry, pre-call, router, SDK call echo, param mapping, logging
object), so one Codex "hi" wrote over 500KB of debug log. Keep the inbound
request line and the outgoing provider request, summarize large values in
the SDK call echo, and stop per-request lines that report nothing
(callback disable checks with no disable header, cost map name probes,
response id decode misses, vector store path misses).
* fix(logging): summarize tools in the param mapping debug lines
Non-Default params and Final returned optional params still printed full
tool definitions, so a Codex-style request repeated them twice more. Use
the same size summary as the SDK call echo and cover tools in the test.
* perf(logging): build the param mapping debug summary only when debug is on
* fix(guardrails): make hide-secrets honor skip_tool_message_in_guardrail
With the flag on, per guardrail or in litellm_settings, hide-secrets leaves
role: tool messages and Anthropic tool_result blocks as sent, as the
unified guardrails do. Tool call arguments are still masked.
* test(guardrails): type the skip-tool walker test and assert the whole request
* fix(guardrails): redact secrets in tool traffic and Anthropic system
* fix(guardrails): bound tool payload scans and stop nested tool_result recursion
Scan each tool value beside its key instead of an indented dump, read
tool_result content one level deep, keep non-finite tool arguments as
raw text, and give read-only guardrails one fragment per tool payload
* test(guardrails): cover read-only guardrail consumers of tool and system text
* fix(guardrails): keep tool argument parsing, scanning and tests within bounds
Treat tool arguments that decode to a lone surrogate as raw text, decode
JSON-escaped detections back to the leaf value, quote unkeyed leaves
without repeating a parent key per list item, drop cycle tracking that
parsed JSON cannot reach, and use the CI license instead of a fake
license server
* fix(guardrails): scan list items beside a capped parent key
List items keep their parent key in the scanned JSON line, capped to its
last 64 characters, so a credential key over a list is still detected
while a long key over many items no longer multiplies the scan text
* fix(guardrails): cap the scanned key after JSON escaping
A key of control characters escaped to six times its length after the
cap, so every list item's scan line grew with it
* fix(guardrails): collapse whitespace in the scanned key context
Space-heavy keys made every inherited list item scan several times slower
than a short key
* fix(guardrails): build each key's scan context once per payload
Repeating the collapse, escape and cut for every list item made a long key
over many items cost key length times item count
* fix(guardrails): cover Responses tool call items and instructions
Responses carries the same tool call arguments and system prompt as chat
and messages under function_call.arguments, custom_tool_call.input and
instructions, so secrets there reached the provider unscanned
* perf(guardrails): skip rescanning text hide-secrets already found clean
Agents resend their whole conversation on every turn, so each turn
rescanned every earlier tool result. Each guardrail now keeps a bounded
set of digests of scan inputs that held no secret and skips those; text
with a secret is always rescanned and no secret material is kept
* fix(guardrails): size the clean-scan cache for many sessions and replace longest secrets first
4096 digests held about 20 long agent sessions per worker, so interleaved
sessions evicted each other and every resend rescanned its history;
65536 digests (about 10 MB) hold a few hundred. Tool payload secrets are
now replaced longest first after JSON unescaping, so a shorter overlapping
match cannot leave part of a secret behind. The key-context test checks
the work it bounds instead of wall-clock time, which CI could not meet
* fix(guardrails): keep unmasked tool argument text and redact keyword matches cut inside an escape
* fix(guardrails): write a tool visitor's whole payload when it changes more than string values
* fix(guardrails): read message text before tool and system text so a vendor error on the new text cannot skip it
* fix(guardrails): moderate tool and system text in its own call so a long payload cannot hide a flagged message
* fix(guardrails): skip the empty moderation call when only tool or system text is present
* fix(guardrails): scan long multi-line tool strings as their own lines so the scan stays linear
* fix(guardrails): keep the key beside every line of a long multi-line tool string
* fix(guardrails): confine tool-traffic coverage to hide-secrets
Restore iter_message_text to its main behavior, so banned keywords, Azure
Content Safety, IBM detectors and the OpenAI and Google moderation hooks
read exactly what they read on main. Only walk_user_text, which
hide-secrets alone calls, still reaches system, instructions and tool
payloads.
* test(guardrails): stop the tool-traffic integration tests from flaking on a busy host
Use the shared per-case timeout for the rig client, and open a new
connection per call in the worker-kill burst so a pooled keep-alive
socket to the killed worker is never reused.
* test(guardrails): drop the wall-clock check from the deep tool arguments unit test
The integration test test_deeply_nested_tool_arguments_do_not_stall_the_worker
already covers the timing; the unit test keeps the redaction check.
---------
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* refactor(typing): narrow Any sources across providers, proxy and guardrails (wave 1)
* refactor(typing): narrow Any sources across hubs, providers and proxy (wave 2)
* refactor(typing): validate RunwayML task JSON and let dotprompt loaders accept exported prompts
* fix: restore pass-through behavior on paths the typing pass tightened
MCP client IP extraction trusts X-Forwarded-For only when use_x_forwarded_for is True, dotprompt skips a malformed JSON prompt entry instead of raising, RunwayML task payloads pass through unvalidated, and a malformed provider body raises the same AttributeError as before at the CompactifAI, Gemini, and mock transport parse sites
* fix(runwayml): keep a malformed 2xx task body on the 500 path
* fix(edenai,cli): keep a non-object body in the error and fail on a malformed migration report
EdenAI's embedding and image generation parse dropped a non-object body from the error
they raised, so the message no longer named what the provider sent. A helper now raises
with the body in the message, and both paths have a regression test.
`lite encryption migrate` exited 0 with no message when the proxy's report was not an
object or its residual_legacy was not a number, where base crashed. It now fails with a
message naming the value.
The CompactifAI and Gemini image-edit non-object body tests accept TypeError or
AttributeError, which both base and this branch raise, so they guard against a
ValidationError instead of pinning one class. RunwayML's async parse gets the same
non-object body test as the sync one.
* fix: keep base's error classes, duck-typed checks and fail-closed guardrails the typing pass changed
* test: annotate the new regression tests' locals with Final
* test: annotate the remaining new test locals with Final
* refactor(types): prove runtime types at harness, search, rag and client boundaries
Replace Any with adapter-validated types in the litellm.agent() harness, the
search provider transformations, RAG ingestion and query, the vector store
pre-call hook and registry, the galileo and opik logging integrations and the
proxy client CLI. Each boundary gets unit tests for well-formed and malformed
payloads.
* chore(typing): prove types at more provider boundaries and restore search transformations
Second pass over a2a, embedding, rerank, image, audio and small provider
modules. Search transformations go back to their previous form because
validating their response bodies would change the proxy status for
malformed upstream bodies from 400 to 500.
* refactor(types): prove types at logging, files, rerank, image, audio and management boundaries
Replace Any with validated or annotation-only types in 37 more files: logging
integrations, token counters, provider files/rerank/image generation/audio
transcription transformations, pass-through logging handlers and management
endpoints. No proxy HTTP status or error type changes.
* refactor(types): prove types at repository, spend, files and router boundaries
Replace Any with repository table accessors, validated mappings and
annotation-only types in 33 more files: Prisma repositories, the enterprise
batch and responses cost checkers, budget reservation, files endpoints,
management endpoints, the policy registry, the adaptive and complexity routers
and the secret managers. No proxy HTTP status or error type changes.
* test(types): run the aiohttp transformation test in-process and cover repository row conversion
The aiohttp chat transformation test no longer starts a server. It feeds the
transformation a response whose json() returns the body under test.
The proxy unit shards now exercise stored model rows whose params are JSON
strings and the object permission create and update paths.
Response cache reads and writes open cache.get llm_response and cache.set llm_response phase spans with their Redis spans nested underneath, on the Python path and on the native Rust path, and deployment selection runs inside a route {model_group} phase so the cooldown, usage and model-id reads the router issues nest under it before chat {model}. The autorouter classifier call nests under that route phase as well and carries its typed internal origin on litellm.request.purpose, so it is told apart from the provider attempt. Service spans are named {service}.{verb} {target} from a low-cardinality key family the producer declares (llm_response, auth_objects, spend_counters, router_cooldowns, claude_code_session_router_binding, rate_limits, pod_lock, budget_reset, ...) instead of the raw method or a per-request pipeline length; a pipeline flush is targeted by the one family its ops share or by mixed with the sorted families on litellm.redis.families, a batch op keeps the family it was declared under whichever pipeline or standalone read settles it, and the ambient family labels Redis spans only, never the DB write-back a task spawned inside that context performs later. The raw method stays on litellm.service.call_type and on the Prometheus and Datadog labels. Caller attribution is carried across asyncio task boundaries on a ContextVar so forwarder-only chains no longer surface, the raw cache key is dropped from Redis span metadata, pipeline op counts land as an integer attribute, every call_type the Redis cache layer emits maps to a verb, and a scan over litellm/ and enterprise/ fails when a Redis producer, batch reservation included, declares no key family.
A V2 logger built for a key or team logging entry while the operator's V2 logger is already registered keeps only the exporters its own preset contributed, whether or not the operator holds credentials for that backend, so every chat span no longer reaches the operator's collector twice. A span the success callback has to open itself, with no pre-call carrier, starts at the provider handoff (api_call_start_time) instead of the logging object's creation.
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(lint): remove the LIT002 mutable-construction rule
Drop LIT002 from scripts/check_type_discipline.py along with its helpers,
its budget entry, its unit tests, and the AGENTS.md and gate docstring
mentions. `# mutable-ok` now only suppresses LIT001, so the markers that
only existed to silence LIT002 became LIT013 stale suppressions and are
removed. The files whose layout depended on those trailing comments are
reformatted with ruff format.
Every other LIT rule count is unchanged and the ASTs of all touched
litellm/ files match main apart from one docstring.
* chore(lint): keep the leftover mutable-ok markers for a follow-up
Restore the ~1.4k `# mutable-ok` markers stripped in the previous commit
so this PR only touches the checker, its tests, the budget, and docs.
Those markers no longer suppress anything, so `# mutable-ok` is exempt
from LIT013 until a follow-up strips them.
* Revert "chore(lint): keep the leftover mutable-ok markers for a follow-up"
This reverts commit c35bc0b84e.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(proxy): email alerts at configured percentages of a team member budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(alerting): label team member budget crossings as team member budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(auth): cover the team member alert dispatch from _check_team_member_budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(email): drop the emoji from the team member budget alert template
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): ignore team member alert thresholds outside 1 to 100 on both the backend and the dashboard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): bound team member alert threshold key length before int parsing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): drop the legacy covers marker from the team member alert test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(team): reject malformed team_member_max_budget_alert_emails on team writes
Thresholds outside 1-100, non-list recipients, and invalid emails now return 422 on
/team/new, /team/update and PATCH /team/{id} instead of being stored and silently
ignored. The value is stored canonically. Read-side LiteLLM_TeamTable is unchanged,
and the PATCH body stays a raw merge patch so a null threshold still deletes it.
* fix(auth): enforce and alert on team member budgets only in common_checks
The builder re-checked the team member budget inline before common_checks ran the same
check, so one request that crossed a team_member_max_budget_alert_emails threshold
dispatched two alerts. Drop the inline check; common_checks is the single authorization
point and already covers per-member rows, the team default member budget, zero-cost
skips and the cross-pod spend counter. Its 422 message now uses the TeamMember=user:team
form the builder and budget reservation already returned.
* Revert "fix(team): reject malformed team_member_max_budget_alert_emails on team writes"
This reverts commit 703e754b46.
* fix(alerting): keep BaseBudgetAlertType.get_event_message zero-arg
Requiring user_info broke existing callers and out-of-tree subclasses. The team member
label now comes from SlackAlerting.budget_alerts, so the interface and its Readme are
unchanged from main.
* fix(mcp): keep team member budget enforcement on the MCP OAuth auth dependency
The MCP OAuth dependency stops at _user_api_key_auth_builder and never reaches common_checks, so removing the builder's inline member budget check would have let over-budget members through there. Enforce it explicitly for that caller.
* fix(auth): keep main's team member budget enforcement, alert once per request
Restore the builder's team member budget check and 422 message exactly as on main and drop the MCP-only gate. The builder sends the member alert only on the request it rejects; common_checks sends it for requests that get past the builder, so no request alerts twice.
* test(integration): read team member alert deliveries without a shared accumulator
* test(integration): match team member alert deliveries by subject so other alerts cannot race the count
* refactor(proxy): build the team member alert threshold config without mutable collections
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): collapse the alert recipient isinstance checks into one call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): read the SMTP sink through lock-guarded snapshots and assert the exact deliveries
---------
Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
- apply_db_row only clears runtime values for keys the row actually changed, so an env-resolved DB-owned setting survives a reload
- DELETE /config/field/delete refuses a key the config file owns instead of silently rewriting the row
- GET /config/field/info reports the declared value of a config-owned key, not the env-resolved secret
- SettingsStore gains a short-circuiting __bool__ so truthiness checks stop at the first key
- _initialize_jwt_auth resolves os.environ refs into a local mapping instead of mutating the shared general_settings dict
- rejected_writes compares against the resolved value, matching what __setitem__ accepts
- a stored value identical to the config template is no longer reported as shadowed
- the enterprise email-settings and coordination-redis writers go through reject_config_owned_writes
* fix(hide-secrets): restore credential coverage lost to the 4.5 entropy limit
Shannon entropy is bounded by log2(length), so the 4.5 limit #39879 shipped
cannot score any value shorter than 23 characters, and it catches a random
32-character base64 credential only about two thirds of the time. A line like
REDIS_PASSWORD=aB3dE6gH9jK2mN5p therefore reaches the provider in the clear.
Add a keyword plugin that yields the credential-shaped value assigned to a
credential-named key, reusing detect_secrets' own maintained denylist so
camelCase, snake_case and SCREAMING_CASE all work with no local word list, and
re-run the assignment-quoting transform detect_secrets skips once its first
pass has matched.
The entropy limits are untouched, so #39879's false-positive fix still holds.
* fix(hide-secrets): read the assignments in a prompt that is mostly prose
configparser aborts the whole parse on the first line it cannot read, so a
message like "Here is my config, can you review it?" followed by
REDIS_PASSWORD=... lost every assignment to that one prose line. Hand the
parser only the lines it can read, dedent the assignments inside a pasted
config, and keep each key distinct by line number so a config naming api_key
once per model keeps every value instead of only the last.
* fix(hide-secrets): drop the plugin docstrings and pin the block-scalar shapes
* fix(hide-secrets): keep a comment or an indented header from closing an open value
* fix(hide-secrets): drop the explanatory comments from the new scan helpers
* fix(hide-secrets): accept punctuation in a credential value
The value filter only allowed the URL-safe Base64 alphabet, so a password
such as hunter2!brahms or p@ssw0rd!2026 passed through unredacted while
the upstream keyword plugin had already matched it. The filter now rejects
only whitespace and brackets, which keeps function calls, subscripts and
sentences out while letting symbol-heavy passwords through.
* fix(hide-secrets): redact every credential on a line and skip timestamps and plain urls
replaces the inherited first-match scan with finditer over every keyword
match, drops iso 8601 timestamps and userinfo-free urls from credential
values, and threads the parser's open-option state through
itertools.accumulate instead of rebinding it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): scan the first token of an assignment and ignore surrounding punctuation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(hide-secrets): drop the unreachable configparser error fallback
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): keep prose after a credential key out of the keyword detector
A bare value followed by ordinary words (secret_sauce: Worcestershire sauce)
is prose, so the synthetic assignment is only built when the value stands
alone or is followed by a shell operator, comment, or another assignment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): scan the first token of shell-style assignments regardless of what follows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): keep spaced assignments in scope when shell text follows the value
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(hide-secrets): drop docstrings that restate the test names
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): stop reading a comparison operator as a trailing assignment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): keep dashed flags as assignment trailers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(hide-secrets): stop redacting benign identifiers and make redaction deterministic
The OpenAI key detector matched `sk-` anywhere inside a word, so `<task-notification>`
became `<ta[REDACTED]>`, and the Base64 entropy limit of 3.0 flagged ordinary quoted
identifiers such as `"application/json"` and model ids. Redaction also iterated a
hash-seeded set, so the same request produced different bytes on different workers and
broke prompt caching.
- require a standalone `sk-`/`sk_` token with a digit (still catches sk-proj-/sk-ant-)
- raise Base64HighEntropyString limit from 3.0 to the detect-secrets default 4.5
- redact overlapping matches longest-first in a stable order
Resolves LIT-7049
* fix(hide-secrets): treat separators as key boundaries and defer sk_live_ to the stripe detector
The standalone-token boundary also rejected keys glued to a preceding `_`, `-`
or percent-encoded delimiter (`openai_sk-…`, `key-sk-…`, `Bearer%20sk-…`), which
the old pattern redacted, and `sk_live_…` was counted by both the OpenAI and the
Stripe detector.
* fix(hide-secrets): keep the openai key scan linear on repeated sk separators
The digit requirement was a lookahead, so every `sk` inside a long
`[a-zA-Z0-9_-]` run re-scanned the rest of that run looking for a digit.
100 KB of `-sk-` took over 5s in the worker's event loop and the proxy
closed the connection without a response. The check now runs once per
match in `analyze_string` instead.
* chore(hide-secrets): remove redundant performance test comment
* fix(hide-secrets): consume complete openai key tokens
* chore(hide-secrets): remove redundant fixture comment
* chore(hide-secrets): remove redundant test docstrings
* fix(hide-secrets): redact whole stripe live keys
* style(hide-secrets): wrap secret sorting key
* Fix hide-secrets guardrail: playground redaction, UI dropdown entry, spend-log telemetry
The hide-secrets guardrail never implemented apply_guardrail, so the UI test
playground echoed secrets verbatim; it was missing from the Add Guardrail
dropdown; and it recorded no guardrail_information, so Spend Logs could not
distinguish a redacted request from a clean one.
- implement apply_guardrail (unified interface) with use_native_lifecycle_hooks
so proxied traffic stays on async_pre_call_hook (per-key opt-out and
data["prompt"] handling live only there)
- record standard_logging_guardrail_information (allow/mask + masked_entity_count)
via _process_response/_process_error; opted-out keys and legacy nameless
callback instances record nothing
- advertise hide-secrets in /guardrails/ui/add_guardrail_settings (pre_call only)
and /guardrails/ui/provider_specific_params with a config model
Resolves LIT-3548
* Fix hide-secrets passthrough telemetry and JSON config input
* fix(guardrails): validate hide-secrets object config before submit
- apply_guardrail treats empty-string-only texts as no input, so no
false allow is recorded
- the UI object field keeps raw text while editing and blocks submission
until it parses to a JSON object, instead of posting a string to an
object-only API
- supported_modes_by_provider keeps its dict[str, list[str]] value type
* fix(guardrails): record no hide-secrets telemetry when nothing was inspected
walk_user_text and the prompt redaction now report how many non-empty
strings they visited; when neither inspected anything (image-only
content, empty strings), the run records no guardrail entry instead of
an 'allow' row that counts a check which never saw any text.
* feat(logging): add user and team level spend and budget to StandardLoggingPayload metadata
* fix(logging): include user and team budget fields in dummy standard logging payload
* refactor(ui): point invitation links at the dedicated /onboarding route
Invitation and reset-password links were built as /ui?invitation_id=..., which lands on the dashboard index and renders the onboarding form inline. They now point at the standalone /ui/onboarding route, so the index no longer has to special-case invitations. Old links keep working unchanged; the index still renders onboarding inline for ?invitation_id until the migration closeout removes that branch.
Updates the three generators (the enterprise email builder, bulk user create, and the invitation/reset-password modal) and extracts the modal's URL building into a pure, unit-tested buildOnboardingUrl
Refs LIT-3687
* refactor(ui): guard buildOnboardingUrl against a missing invitation id
Return "" instead of emitting an invitation_id=undefined link when the id is not yet available, matching the existing empty-baseUrl guard. Placed after the SSO branch so the SSO link, which does not use the id, is unaffected
Refs LIT-3687
* fix: prevent duplicate budget alert emails on concurrent threshold crossings
Budget alert emails were sent more than once for a single threshold crossing. The email dedup guard read the "already sent" marker, awaited the send, then wrote the marker, so concurrent requests crossing the same threshold within the send window all saw no marker and each sent. This affected the multi-threshold path (default_key_max_budget_alert_emails), the legacy single-threshold path (EMAIL_BUDGET_ALERT_MAX_SPEND_ALERT_PERCENTAGE), and the soft budget path, all in EmailBaseCallback.budget_alerts
All three branches now claim the send slot atomically before sending via async_increment_cache, which is atomic per event loop for the in-memory cache and across workers via Redis INCR; only the caller that observes a count of 1 sends. On send failure the marker is released with async_delete_cache so a transient failure does not suppress the alert for the full 24h TTL
* fix: harden budget alert claim release and skip-path event allocation
Addresses review feedback on the claim-before-send change. The claim release in each send-failure handler now logs the send error first and releases the claim best-effort through a shared helper, so a transient cache error during async_delete_cache cannot propagate out of the fire-and-forget budget_alerts task, drop the send-failure log, and leave the claim stuck for the full 24h TTL. In the multi-threshold branch the increment claim now runs before the WebhookEvent is built, so skipped concurrent crossings no longer construct and discard the event, matching the single-threshold and soft budget branches
Several guardrail hooks short-circuit when ``message.content`` is a list
or when the request uses the Responses-API ``input`` field instead of
``messages``. Centralise the content-walking logic in a shared helper and
update the affected hooks so list-format and Responses-API payloads no
longer skip inspection.
Also: Aim's ``async_post_call_success_hook`` now inspects every choice
(via ``asyncio.gather``) instead of only ``choices[0]`` — the prior
behaviour let ``n>1`` callers hide content in subsequent completions.
Hooks updated to use the new helper:
- aim, lakera_ai_v2, lasso (post a synthesised messages list to a remote
guardrail service)
- azure_content_safety, ibm_detector, banned_keywords, openai_moderation,
google_text_moderation (iterate text fragments locally)
- secret_detection (walk-and-rewrite to redact in place)
Drive-by fix: the legacy ``data["prompt"]`` list-handling path in
secret_detection rebound the loop variable instead of mutating the list,
leaving secrets unredacted on text-completion calls; corrected to index
back into the list.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
send_max_budget_alert_email previously guarded with `is not None`, which
accepts `[]` and then crashes on `recipient_emails[0]` inside
_get_email_params. The current caller (_handle_multi_threshold_max_budget_alert)
already filters empty lists upstream, but the public method signature makes
no such guarantee — a future caller passing [] would hit IndexError.
Switch to truthiness so both None and [] fall through to the single-recipient
path.
- Add early return guard in _handle_multi_threshold_max_budget_alert
for None max_budget_alert_emails and max_budget
- Add explicit type annotation on alert_email_config in auth_checks
- Guard empty recipients in _handle_multi_threshold_max_budget_alert:
log warning and skip instead of falling through to old path error loop
- Widen max_budget_alert_emails type to Dict[str, Union[str, List[str]]]
to match _parse_email_list runtime behavior (accepts comma-separated strings)
- Pre-filter asyncio.create_task with min threshold check to avoid
unnecessary task allocation on every request when spend is below
all configured thresholds
Users can set metadata.max_budget_alert_emails as a JSON map of threshold
percentages to email recipients on virtual keys. When configured, the email
handler loops over each threshold, checks per-threshold dedup cache, and
sends to the configured recipients (auto-including the key owner's email).
When no map is set, the existing single 80% threshold behavior is preserved
unchanged. Teams support is out of scope for this v0.
* fix(vertex_ai): support pluggable (executable) credential_source for WIF auth (#24700)
The WIF credential dispatch in load_auth() only handled identity_pool and
aws credential types. When credential_source.executable was present (used
for Azure Managed Identity via Workload Identity Federation), it fell
through to identity_pool.Credentials which rejected it with MalformedError.
Add dispatch to google.auth.pluggable.Credentials for executable-type
credential sources, following the same pattern as the existing identity_pool
and aws helpers.
Fixes authentication for Azure Container Apps → GCP Vertex AI via WIF
with executable credential sources.
* feat(logging): add component and logger fields to JSON logs for 3rd p… (#24447)
* feat(logging): add component and logger fields to JSON logs for 3rd party filtering
* Let user-supplied extra fields win over auto-generated component/logger, tighten test assertions
* Feat - Add organization into the metrics metadata for org_id & org_alias (#24440)
* Add org_id and org_alias label names to Prometheus metric definitions
* Add user_api_key_org_alias to StandardLoggingUserAPIKeyMetadata
* Populate user_api_key_org_alias in pre-call metadata
* Pass org_id and org_alias into per-request Prometheus metric labels
* Add test for org labels on per-request Prometheus metrics
* chore: resolve test mockdata
* Address review: populate org_alias from DB view, add feature flag, use .get() for org metadata
* Add org labels to failure path and verify flag behavior in test
* Fix test: build flag-off enum_values without org fields
* Gate org labels behind feature flag in get_labels() instead of static metric lists
* Scope org label injection to metrics that carry team context, remove orphaned budget label defs, add test teardown
* Use explicit metric allowlist for org label injection instead of team heuristic
* Fix duplicate org label guard, move _org_label_metrics to class constant
* Reset custom_prometheus_metadata_labels after duplicate label assertion
* fix: emit org labels by default, remove flag, fix missing org_alias in all metadata paths
* fix: emit org labels by default, no opt-in flag required
* fix: write org_alias to metadata unconditionally in proxy_server.py
* fix: 429s from batch creation being converted to 500 (#24703)
* add us gov models (#24660)
* add us gov models
* added max tokens
* Litellm dev 04 02 2026 p1 (#25052)
* fix: replace hardcoded url
* fix: Anthropic web search cost not tracked for Chat Completions
The ModelResponse branch in response_object_includes_web_search_call()
only checked url_citation annotations and prompt_tokens_details, missing
Anthropic's server_tool_use.web_search_requests field. This caused
_handle_web_search_cost() to never fire for Anthropic Claude models.
Also routes vertex_ai/claude-* models to the Anthropic cost calculator
instead of the Gemini one, since Claude on Vertex uses the same
server_tool_use billing structure as the direct Anthropic API.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix(anthropic): pass logging_obj to client.post for litellm_overhead_time_ms (#24071)
When LITELLM_DETAILED_TIMING=true, litellm_overhead_time_ms was null for
Anthropic because the handler did not pass logging_obj to client.post(),
so track_llm_api_timing could not set llm_api_duration_ms. Pass
logging_obj=logging_obj at all four post() call sites (make_call,
make_sync_call, acompletion, completion). Add test to ensure make_call
passes logging_obj to client.post.
Made-with: Cursor
* sap - add additional parameters for grounding
- additional parameter for grounding added for the sap provider
* sap - fix models
* (sap) add filtering, masking, translation SAP GEN AI Hub modules
* (sap) add tests and docs for new SAP modules
* (sap) add support of multiple modules config
* (sap) code refactoring
* (sap) rename file
* test(): add safeguard tests
* (sap) update tests
* (sap) update docs, solve merge conflict in transformation.py
* (sap) linter fix
* (sap) Align embedding request transformation with current API
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) mock commit
* (sap) run black formater
* (sap) add literals to models, add negative tests, fix test for tool transformation
* (sap) fix formating
* (sap) fix models
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) commit for rerun bot review
* (sap) minor improve
* (sap) fix after bot review
* (sap) lint fix
* docs(sap): update documentation
* fix(sap): change creds priority
* fix(sap): change creds priority
* fix(sap): fix sap creds unit test
* fix(sap): linter fix
* fix(sap): linter fix
* linter fix
* (sap) update logic of fetching creds, add additional tests
* (sap) clean up code
* (sap) fix after review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) add a possibility to put the service key by both variants
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) update test
* (sap) update service key resolve function
* (sap) run black formater
* (sap) fix validate credentials, add negative tests for credential fetching
* (sap) fix validate credentials, add negative tests for credential fetching
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) fix after bot review
* (sap) lint fix
* (sap) lint fix
* feat: support service_tier in gemini
* chore: add a service_tier field mapping from openai to gemini
* fix: use x-gemini-service-tier header in response
* docs: add service_tier to gemini docs
* chore: add defaut/standard mapping, and some tests
* chore: tidying up some case insensitivity
* chore: remove unnecessary guard
* fix: remove redundant test file
* fix: handle 'auto' case-insensitively
* fix: return service_tier on final steamed chunk
* chore: black
* feat: enable supports_service_tier to gemini models
* Fix get_standard_logging_metadata tests
* Fix test_get_model_info_bedrock_models
* Fix test_get_model_info_bedrock_models
* Fix remaining tests
* Fix mypy issues
* Fix tests
* Fix merge conflicts
* Fix code qa
* Fix code qa
* Fix code qa
* Fix greptile review
---------
Co-authored-by: michelligabriele <gabriele.michelli@icloud.com>
Co-authored-by: Josh <36064836+J-Byron@users.noreply.github.com>
Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: milan-berri <milan@berri.ai>
Co-authored-by: Alperen Kömürcü <alperen.koemuercue@sap.com>
Co-authored-by: Vasilisa Parshikova <vasilisa.parshikova@sap.com>
Co-authored-by: Lin Xu <lin.xu03@sap.com>
Co-authored-by: Mark McDonald <macd@google.com>
Co-authored-by: Sameer Kankute <sameer@berri.ai>
The hanging-response constructor was fixed but the sibling failed-response
constructor at line 104 was still missing this field.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add missing `user_api_key_project_alias` key to SpendLogsMetadata and
PagerDutyInternalEvent constructors, and cast `reasoning_items` to list
for safe iteration in responses transformation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(generic_api_callback.py): make generic api OSS + support multiple generic API's
Enables https://github.com/BerriAI/litellm/pull/17094#discussion_r2562832967
* feat(callback_utils.py): support custom generic api callbacks
* feat(generic_api_callback.py): support specifying which event types to run the generic api for
* fix(litellm_logging.py): log system prompt for anthropic messages
* feat(generic_api_callback.py): support generic api compatible api's - e.g. rubrik agent cloud
* docs(sidebars.js): document new OSS generic api
* docs(generic_api.md): document new OSS Generic API
* docs(custom_webhook_api.md): document custom webhook api integration tutorial
* docs(custom_webhook_api.md): cleanup
* docs(custom_webhook_api.md): document what get's logged to custom webhook api
* Refactor: Pass callback config to GenericAPILogger
Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>
* Fix: Handle empty messages list in logging payload
Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>
* Checkpoint before follow-up message
Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>
* feat: Cache GenericAPILogger instances to improve performance
Co-authored-by: krrishdholakia <krrishdholakia@gmail.com>
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
