The auto-bridge that moves gpt-5.4+ requests carrying function tools and no
reasoning_effort onto /v1/responses only fired when the resolved api_base was
the literal https://api.openai.com/v1, so a deployment pointed at an OpenAI
PrivateLink hostname (<region>.privatelink.api.openai.com) or a port-qualified
or trailing-slash default stayed on Chat Completions and got OpenAI's 400 back.
Gate on the resolved URL's hostname instead: api.openai.com or any subdomain of
it bridges, every other custom base still stays on chat
test_router_timeout, test_timeout_streaming and test_openai_embedding_timeouts
asked api.openai.com for a response in 10 to 100 microseconds and asserted the
resulting exception was a timeout. No connect can finish in that window, so
socket.create_connection always walked the whole address list, and because it
re-raises only the LAST address's error, the assertion was decided by the order
getaddrinfo happened to return.
api.openai.com is dual-stack and the CI container has no usable IPv6, so a
trailing AAAA record made the last attempt fail with an OSError. httpcore maps
socket.timeout to ConnectTimeout but OSError to ConnectError, so the expected
APITimeoutError arrived as APIConnectionError and the job went red. The three
tests were really measuring DNS ordering, not litellm.
Point them at the fake OpenAI endpoint the suite already runs, ask for the
slow-endpoint model it already delays on, and give them a timeout comfortably
under that delay. The embeddings route did not honour slow-endpoint yet, so it
now delays the same way chat and text completions already do.
Each test also gained a failure on the success path. Without it a request that
returned instead of timing out fell out of the try block and the test passed on
a result it was written to reject.
* feat(auto-router): support classifier reasoning effort
* fix(auto-router): harden classifier reasoning effort
* fix(ui): satisfy classifier config lint limits
* refactor(auto-router): simplify classifier effort support
* fix(auto-router): clear frontend-lint and type-discipline gates, trim LOC
---------
Co-authored-by: Tin Chi Lo <tin@berri.ai>
* fix(ui): clearing the organization picker no longer sends organization_id="" on key create
* fix(proxy): paginate Request Logs by conversation and aggregate session type counts and models server-side
* fix(proxy): keep access groups in sync when a model is renamed or deleted
* fix(proxy): cap the Request Logs conversation total like the row total
* fix(proxy): judge access group backing by the database for db models
A worker whose router has not polled the database yet still lists a sibling under its old
name, so a delete or rename handled there kept the stale name in every access group. Only
config-sourced deployments count as router backing now; db models are counted in the table.
* fix(ui): keep the conversation badge when an MCP call represents a conversation
A conversation that straddles the bounded page window can be represented by one of its MCP
rows, which showed a plain MCP badge and hid the session counts. The badge now reads the
server aggregates whenever the conversation has more than one call.
* fix(proxy): list every model of a conversation in Request Logs and keep the conversation badge for MCP representatives
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): type session spend aggregates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): satisfy request logs lint budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): cap per-session model aggregation in request logs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: ratchet type-discipline budget after staging merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): send an explicit null when the key edit form clears the organization
Clearing the Organization picker in the key edit form wrote undefined into
the form value, and JSON.stringify drops undefined-valued keys, so
/key/update never saw the field and the key kept its old organization.
Writing null instead survives serialization, and the backend's
model_dump(exclude_unset=True) preserves it, so the column is set to NULL.
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(cli): enable Claude Code gateway model discovery by default in lite claude
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(cli): build agent env declaratively and document discovery key for lite up
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(cli): keep build_agent_env within LIT002 type-discipline budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
prisma's `delete` returns None when the `where` clause matched no row
instead of raising, and the handler never looked at the return value. It
went straight on to filter an in-memory list that never held the name and
answered 200 "Credential deleted successfully", so an operator scripting
credential cleanup could not tell a real deletion from a typo.
Look at what the repository returned and answer 404 with the name, the
same rejection PATCH /credentials/{credential_name} already gives. A
credential that only exists in the config yaml is never written to the
table, so it now answers 404 too, which is honest: reporting success for
it is the same lie, since it comes back on the next proxy boot.
Adds regression tests for the delete 404, the still-working delete, the
config-yaml-only credential, and for the raise-not-return fix on both
DELETE /credentials/{credential_name} and GET /credentials, which
serialized a rejection as the 200 response body.
The config stub shared one list object between save_config and get_config, so the
DB overlay handed the endpoint back the very list it had just appended to and both
tests passed with the product fix reverted. Store the settings as JSON the way the
litellm_config row does, and check the duplicate guard against a list that only
ever existed in the DB.
`POST /v1/agents/{id}/make_public` appended the agent id to
`litellm.public_agent_groups` and only then called `get_config()`, which
re-applies the DB's `litellm_settings` over the module globals and threw the
append away. The config it saved was therefore a no-op: the endpoint answered
200 with an empty `public_agent_groups`, the agent never reached
`GET /public/agent_hub`, and re-publishing never hit the "already public" 400.
Read the config first, derive the new list from the refreshed globals, save it,
then update the global
Also fixes the e2e model hub spec, which is flaky for a second reason: the
"Make Models Public" modal preselects the groups that are already public, so a
blind click on "Select All" cleared them and left "Next" disabled for the full
15s action timeout. Check the box instead of toggling it, and wait for "Next"
to be enabled before clicking
Covers the reintroduction of a second module-level cache for the guardrail
translation mappings: remapping the loader between two pre-call hooks must
change which handler runs, and the module must expose no assignable map of
its own.
Streaming /v1/messages against a model served through the chat-completions
bridge (every non-Anthropic provider other than OpenAI) minted its msg_ id
inside the stream wrapper, so the spend row landed under the provider's own
completion id and the caller could not find the call by the only id it saw.
The wrapper now mints the id once in its constructor and hands it to the
logging object, the same way the Responses-API bridge does.
The documented DISABLE_AIOHTTP_TRANSPORT env var already selects the httpx transport, so the extra module-global write was redundant. Types the monkeypatch fixture while here.
The unified guardrail cached the endpoint translation mappings in its own
module global on top of the loader's cache in litellm/llms. Tests wrote to
that second copy directly, so a teardown that restored a stale snapshot left
a test double installed for every later test on the same xdist worker, and
proxy-endpoints went red on whichever guardrail streaming test happened to
land after it.
Read through load_guardrail_translation_mappings() at each call site and give
the tests one seam to patch, so pytest owns every restore.
Failing closed at the sensitive-data masker's depth of 10 turned an ordinary
nested tool JSON schema into REDACTED leaves, because a list level costs two
depth. The walk now bounds on DEFAULT_MAX_RECURSE_DEPTH, which no real payload
reaches, and the masker's own limit is left alone.
A streaming /v1/messages call against a non-Anthropic model is served an SSE
message_start frame carrying a msg_ id the adapter mints locally, since the
Responses API upstream only issues a resp_ id. That value never left the
adapter, so the spend row was keyed on the bridged response id and
GET /spend/logs?request_id=msg_... came back empty.
The adapter now hands the id it minted to the logging object, and the
/v1/messages logging path keys the row on it.
The "Final returned optional params" line printed whatever the caller nested inside
extra_body, so a credential tucked in there reached stdout in plaintext one line after
the request line that already redacts it.
The call site now runs redact_credentials_in_payload behind a guard reading both of
print_verbose's consumers, litellm.set_verbose and the LiteLLM logger's DEBUG level, so
the line prints in exactly the cases it did before and the walk costs nothing when
nothing would read it.
Background streaming emits event types with no typed model, which arrive
as GenericEvent holding a plain dict. Only typed events had their nested
response id rewritten, so those frames advertised the raw internal id
while their siblings advertised the encrypted one. The raw shape skips
the ownership check, so any other key could retrieve or cancel that
response.
Rewrite the advertised id wherever an event carries one, whichever shape
it arrives in, so a future event type cannot reopen this.
redact_credentials_in_payload only recursed into mappings, so a
credential-named key one level inside a list or tuple, the shape
extra_body and metadata routinely carry, still reached stdout under
set_verbose. Rebuild sequences element by element too, keeping the
container's own type so the printed repr is unchanged apart from the
secret.
`litellm.set_verbose = True` printed the caller's kwargs verbatim to stdout, so
`api_key` and its siblings landed in terminals and container log drains in
plaintext while the same statement's logger emission was already redacted.
Mask the kwargs at the source with a shared helper in
`litellm_core_utils/sensitive_data_masker.py`, reusing the existing
`SensitiveDataMasker` key classification and the `REDACTED` marker
`secret_redaction.py` already owns, so both debug surfaces agree.
A blocked guardrail (and any other HTTP error the proxy converts) came back
with "type": "None" and "param": "None", because the converters passed the
string "None" as the getattr default instead of None. OpenAI types error.type
as a required string and error.param as nullable, so type now falls back to
the type its status code stands for and param serializes as JSON null.
Covers the non-streaming body, the SSE error frame, the client-disconnect
frame, and the unclassified-exception path, so every unified LLM endpoint and
the anthropic endpoints return the same shape.
AmazonMoonshotConfig.transform_request called
_get_boto_credentials_from_optional_params purely for its side effect of
popping the aws_* keys off optional_params, then threw the result away. On
a box whose default AWS profile uses login_session without botocore[crt],
that call raises, so a bearer-token bedrock/invoke/moonshot.* deployment
still 500s with MissingDependencyException even after the rest of this
branch skips the chain.
It now filters the aws_* keys into a local dict the way the Qwen, OpenAI
and Claude 3 invoke transformations already do, so no credentials are
resolved and the caller's optional_params keeps the keys sign_request
reads afterwards.
/v1/messages against a non-Anthropic model answers with the Responses id,
but the spend row was built from a fresh ModelResponse, so it landed on a
chatcmpl- uuid nobody can look up. Carry that id through the same way the
Anthropic branch now does, and make the passthrough spend assertions fail
on an empty lookup instead of skipping past it.
POST /v1/messages returns an Anthropic-shaped body whose `id` is the only
request id the caller ever sees, but the spend row was written with a
`chatcmpl-<uuid>` (non-streaming) or the bare `litellm_call_id` (streaming and
the /anthropic/v1/messages passthrough), so
GET /spend/logs?request_id=msg_... returned [].
The logging conversion now carries the provider's response id through:
_handle_anthropic_messages_response_logging seeds the ModelResponse it builds
with the Anthropic id, and the passthrough logging handler prefers the id it
read off the response body or the message_start chunk over litellm_call_id.
get_spend_logs_id already prefers response_obj["id"], so the spend row and
standard_logging_object["id"] now both carry the id the client holds.
Every field of a multipart form arrives as a string, so `n` reached the
provider as "2" and Bedrock Nova Canvas rejected the request with
"expected type: Number, found: String". Restore the type the request
schema declares at the boundary where the form is parsed, driven by the
schema's own type hints so the helper covers any int- or float-typed
field on any multipart endpoint.
Resolve type-discipline-budget.json by taking the lower limit per rule so no
ceiling ratchets back up.
Staging's 66a3d24b3f left a duplicate embedding_executor parameter in the
Bedrock KB fake handler, which makes ruff fail on the whole tests tree. Drop
the duplicate here so this branch compiles; #39502 makes the same change on
staging.
Resolves two conflicts:
- tests/test_litellm/vector_stores/test_main.py: staging moved search() to a
RouterVectorStoreEmbeddingExecutor while this branch parametrized the same
test over query; keep both the executor assertions and the parametrize.
- tests/logging_callback_tests/test_bedrock_knowledgebase_hook.py: staging
carries a duplicate embedding_executor kwarg that makes the file a
SyntaxError; drop the trailing duplicate.
* Fix hide-secrets guardrail: playground redaction, UI dropdown entry, spend-log telemetry
The hide-secrets guardrail never implemented apply_guardrail, so the UI test
playground echoed secrets verbatim; it was missing from the Add Guardrail
dropdown; and it recorded no guardrail_information, so Spend Logs could not
distinguish a redacted request from a clean one.
- implement apply_guardrail (unified interface) with use_native_lifecycle_hooks
so proxied traffic stays on async_pre_call_hook (per-key opt-out and
data["prompt"] handling live only there)
- record standard_logging_guardrail_information (allow/mask + masked_entity_count)
via _process_response/_process_error; opted-out keys and legacy nameless
callback instances record nothing
- advertise hide-secrets in /guardrails/ui/add_guardrail_settings (pre_call only)
and /guardrails/ui/provider_specific_params with a config model
Resolves LIT-3548
* Fix hide-secrets passthrough telemetry and JSON config input
* fix(guardrails): validate hide-secrets object config before submit
- apply_guardrail treats empty-string-only texts as no input, so no
false allow is recorded
- the UI object field keeps raw text while editing and blocks submission
until it parses to a JSON object, instead of posting a string to an
object-only API
- supported_modes_by_provider keeps its dict[str, list[str]] value type
* fix(guardrails): record no hide-secrets telemetry when nothing was inspected
walk_user_text and the prompt redaction now report how many non-empty
strings they visited; when neither inspected anything (image-only
content, empty strings), the run records no guardrail entry instead of
an 'allow' row that counts a check which never saw any text.
The v2 migration resolver gave `prisma migrate deploy` four attempts, and
every recovery path ended in a bare `continue`, so each one burned an attempt.
A database first brought up with `--use_prisma_db_push` has a full schema and
no migrations ledger, so the baseline spent attempt one and the first three
migrations whose objects already existed spent the rest. The proxy then exited
before binding its port, and that database could never be moved onto the
resolver.
The retry budget now counts only attempts that got nowhere. Creating the
baseline, and each migration newly marked applied, leaves the budget alone, so
a push-created database works through its pre-existing objects one pass at a
time. Timeouts, deadlock rollbacks, advisory-lock waits, and a repeat of a
recovery that already ran still spend an attempt, so a run that stops making
progress gives up exactly as before.
Two branches independently added embedding_executor to the same fake
search handler in this file, #39472 in the middle of the signature and
#39474 at the end. Neither conflicted with the other, so both edits
merged and the function ended up declaring the parameter twice.
Python rejects that at compile time, so the whole module fails to
import and every test in the file is uncollectable, taking the
logging_testing job down on staging.
Keep the earlier of the two, which sits where the real handler declares
the parameter.
* fix(sso): resolve multi-valued role claims to the highest privilege role
A role claim carrying several roles used to resolve to whichever one the IdP
listed first, so a user holding both proxy_admin_viewer and internal_user lost
org-level spend visibility depending on claim ordering alone.
get_litellm_user_role now picks the highest privilege role out of a list-valued
claim, and the Entra app_roles path shares that same resolution instead of
keeping its own copy of the hierarchy. SAML assertions carrying several role
values go through the same path rather than taking the first value.
* test(sso): lock ranked-over-unranked role resolution for mixed claims
org_admin, team and customer sit outside the privilege ladder. Pin the
resolution for a claim that mixes one of them with a ranked role so the
asymmetry is covered rather than implicit.
* fix(sso): label the claim-sequence cast for the type-discipline gate
* fix(sso): resolve claim entries without recursing
The repo's recursive-function gate rejects self-recursion here, and a role
claim is flat anyway. Pull the single-value lookup into its own helper so the
list branch maps over it instead of calling back into itself.
