Admin password sets on /user/update and per-user /user/bulk_update stay
supported and policy-enforced. The request model hides the password from
repr so management alerts never format the plaintext, and the all_users
bulk path rejects passwords instead of writing one plaintext value to
every row.
/user/bulk_update awaited a separate HIBP lookup for each user in the
batch, so a degraded-slow HIBP (5s timeout per lookup) could stretch a
500-user batch to ~2500s and time out the request after some updates
had already persisted.
validate_passwords_bulk dedupes the batch's passwords, strength-checks
first, then fires every needed HIBP lookup concurrently, bounding the
worst case at one 5s timeout window. bulk_update_processed_users now
screens the whole batch before the serial update loop, so a rejected
password fails only its own entry and validation failures precede any
persistence.
Lock affected teams in sorted order inside a single 60s transaction so a
failure on any team rolls back every rewrite and every user row delete.
PrismaClient.tx() gains an optional timeout for the larger batch.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
/user/bulk_delete now deletes the users' keys, invitation links, org and team
memberships and user rows in one transaction and reports a rolled-back batch
per row instead of leaving partial deletes behind. Both bulk endpoints evict
the deleted keys (and deleted user objects) from the auth cache, so a deleted
key stops authenticating immediately rather than at TTL expiry.
/team/bulk_member_delete rejects member rows that carry both user_id and
user_email, reports repeated rows as duplicates, and only cleans up keys and
memberships of members it actually matched.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Batch user deletion that also removes each user from every team they belong to, and batch removal of many members from one team. Each touched team is rewritten once under the team advisory lock from a roster re-read under that lock
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Rows opting into auto_create_key lost blocked, permissions, aliases, config,
agent_id, budget_fallbacks and budget_limits before reaching the key helper.
When create_many commits but the response is lost, re-read which ids landed
and retry only the rest so committed rows report success and get their teams.
Regenerate schema.d.ts and allowlist the endpoint in the Terraform audit.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Creates up to 500 internal users in one request with set-based validation,
a single create_many for user rows, and one locked write per referenced team.
Rows fail independently, keys are opt-in per row via auto_create_key, and
send_invite_email is rejected for the batch.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Adds a nullable tpd_limit column and field to keys, teams, budgets and end users. The batch submission limiter swaps the per-minute RPM/TPM descriptor of any scope that has a tpd_limit for a token-only 24h descriptor, so batch traffic is budgeted per day while online traffic keeps the existing per-minute limits. The Admin UI exposes the field on key, team and budget create/edit forms
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The org cache entries written by prefetch_auth_objects carry the 5s
DEFAULT_IN_MEMORY_TTL. The first @log_db_metrics getter lazily imports
litellm.proxy.proxy_server, which on a cold CI runner can take longer
than 5s, so the org entry expired before get_org_object read it and the
getter fell through to the MagicMock database. Inject a frozen clock
into InMemoryCache so the test asserts the join, not import latency.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The rebuild's tool-call selection and its text-only fast path only looked at
choice 0 of each chunk, so a chunk that packs several choices (Gemini with
candidateCount above 1) lost a tool call carried by a later candidate, and a
chunk whose later choice had no tool calls at all made the rebuild raise.
Both now consider every choice in the chunk.
Move the BaseLLMException import into _map_error_event_exception so the
module no longer imports it at load time, clearing the module-level cyclic
import CodeQL flagged. The class is used only on the cold error path.
Replace the mutable list-append test collector with aiter/anext so the
regression tests read the stream immutably.
RouterRateLimitError now carries the model group's deployment ids so it
can tell when every deployment is cooled down, and exposes that as
type=all_deployments_in_cooldown with an explicit message. A partial
cooldown keeps type=rate_limit_error. Either way the proxy no longer
reports type=internal_server_error next to code 429
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Mid-stream error events on the streaming Responses API were all raised as
APIError, so a content_policy_violation event never matched the router's
content-policy fallback dispatch and the client got the raw error instead
of the fallback model's answer. Map each error event's code and status
through the existing exception_type mapping, matching the non-streaming
path, and unwrap the typed ContentPolicyViolationError and
ContextWindowExceededError so the router routes them to the configured
content_policy_fallbacks and context_window_fallbacks.
The generic 400 branch of the OpenAI exception mapper dropped the wire body and no
branch carried the response headers, so an application calling a LiteLLM proxy through
a litellm_proxy/ model could not tell a guardrail block from any other failure without
walking __cause__. BadRequestError now takes headers, filled for a litellm_proxy
upstream, and the generic branch passes the body. The proxy edge treats the literal
"None" type and param an older proxy sends as absent and stops forwarding an upstream
proxy's date and server headers.
Removes the runtime import of proxy_server from login_throttle so the
throttle module no longer participates in the import cycle CodeQL
flagged (py/cyclic-import). Callers pass general_settings and
redis_usage_cache explicitly; behavior is unchanged.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The rebuilt streamed response keyed tool-call fragments by tool index alone, so
on n>1 chat streams the two choices' argument fragments were concatenated into
one string and post_call guardrails scanned garbled JSON. Fragments are now
keyed by (choice index, tool index).
When a guardrail's rewrite cannot be written back to the stream (multi-choice
streams, a rewrite that adds or drops a tool call, legacy-hook shapes the
translation cannot rescan), the pipeline now logs a warning naming the
guardrail and the exact reason before releasing the original stream.
Also commits the regenerated dashboard API types that make check produced.
CodeQL flagged two module-level cyclic imports introduced by defining
CachedTokensDetails in litellm.types.llms.openai and importing it from
litellm.types.utils and litellm.cost_calculator. The class now lives in
litellm.types.llms.base, which imports nothing from litellm, and every
user imports it from there.
Also pins that combining realtime usages where only one response.done
carries cached_tokens_details keeps the earlier modality split in both
orders, and commits the regenerated dashboard API types.
A guardrail that rewrites text per chat message now returns the rewritten
rows as structured_messages instead of only texts, so the Responses and
chat handlers write the rewrite back through the structured path. The
generic guardrail API response accepts an optional structured_messages
list, Prompt Security modify builds one from modified_messages, and rows a
server echoes back exactly as shown are restored to the original row
objects because the request model drops undeclared keys. Texts-only
per-message answers keep the named rejection on both endpoints.
