A blocked guardrail (and any other HTTP error the proxy converts) came back
with "type": "None" and "param": "None", because the converters passed the
string "None" as the getattr default instead of None. OpenAI types error.type
as a required string and error.param as nullable, so type now falls back to
the type its status code stands for and param serializes as JSON null.
Covers the non-streaming body, the SSE error frame, the client-disconnect
frame, and the unclassified-exception path, so every unified LLM endpoint and
the anthropic endpoints return the same shape.
litellm_settings keys are set on the litellm module with no allowlist, so a typo
in vector_store_search_failure_mode reached assert_never and turned every
vector-store request into a 500. Validate the configured value and fall back to
the permissive default with a warning naming the supported modes.
AmazonMoonshotConfig.transform_request called
_get_boto_credentials_from_optional_params purely for its side effect of
popping the aws_* keys off optional_params, then threw the result away. On
a box whose default AWS profile uses login_session without botocore[crt],
that call raises, so a bearer-token bedrock/invoke/moonshot.* deployment
still 500s with MissingDependencyException even after the rest of this
branch skips the chain.
It now filters the aws_* keys into a local dict the way the Qwen, OpenAI
and Claude 3 invoke transformations already do, so no credentials are
resolved and the caller's optional_params keeps the keys sign_request
reads afterwards.
/v1/messages against a non-Anthropic model answers with the Responses id,
but the spend row was built from a fresh ModelResponse, so it landed on a
chatcmpl- uuid nobody can look up. Carry that id through the same way the
Anthropic branch now does, and make the passthrough spend assertions fail
on an empty lookup instead of skipping past it.
A vector store search that fails is swallowed by the pre-call hook, so the
request goes to the model with an un-augmented prompt and the caller gets a
200 answering from the model's own knowledge with no way to tell the
knowledge base was skipped.
Failed searches now ride the same channel their successes already use: a
vector_store_search_failures entry on provider_specific_fields naming the
store id, provider, and error. That is additive and always on. For callers
who would rather fail than answer ungrounded, litellm_settings
vector_store_search_failure_mode: error raises VectorStoreSearchError (400)
instead; the default stays annotate, today's permissive behavior.
The hook's outer catch-all also now names the requested vector store ids in
its log line, and only wraps the augmentation itself, so the fail-closed
raise is not swallowed by it.
* fix(ui): keep Virtual Keys list state in the URL so it survives leaving the page
The search term, sort, pagination and drawer filters lived in component
state, so navigating away from Virtual Keys and back reset the table to an
unfiltered first page. Move them into query state alongside the existing
?key= deep link, which also makes a filtered view shareable.
* fix(ui): namespace the Virtual Keys filter params and bound page inputs
The unprefixed team_id filter hijacked the /api-keys create-key deep link,
which already takes team_id as a prefill, so ?create=true&team_id=X silently
filtered the list underneath the modal. Prefix the four drawer filters.
Now that page and page_size come from the address bar, clamp them to what
/key/list accepts instead of forwarding 0, negatives or an int64-overflowing
page straight through, and trim filter values arriving from a URL the same
way the drawer already trims them.
* fix(ui): fall back to a sortable column when the URL names an unknown one
A hand-edited or stale sort_by reached /key/list, which 400s it, leaving the
Virtual Keys page on its loading skeleton with no error. Validate it against
the fields the table's own headers can produce, and clear sort_by rather than
blanking it when a sort is reset so the URL stays clean.
Also replaces a default-state URL assertion that ran before any query-state
write could land, so it could not fail for the regression it named.
* fix(ui): use TanStack's functionalUpdate instead of a hand-rolled updater resolver
The local helper narrowed typeof updater === "function" against an
unconstrained T, which TypeScript cannot do because T itself may be a function
type, so next build failed to type check. table-core already exports the same
helper.
Review turned up two real problems in the TTS path.
Router.aspeech forwarded voice=None whenever the caller omitted it, which overwrote a
voice set in the deployment's litellm_params, so a configured fallback voice was
ignored on voice-less requests. It now leaves the key alone when no voice is passed.
get_complete_url also fell back to MISTRAL_API_BASE, but speech() always receives a
non-null api_base from get_llm_provider, whose mistral branch only reads
MISTRAL_AZURE_API_BASE and otherwise hardcodes the public host. That branch could
never run, and its unit test asserted a behavior the real path does not have. The
working override is api_base on the deployment, now pinned by an end-to-end test
Address the two Greptile P2 notes: construct the chunk through the shared
creator instead of mutating its choices afterwards, and drop the decorative
divider comment from the new tests.
POST /v1/messages returns an Anthropic-shaped body whose `id` is the only
request id the caller ever sees, but the spend row was written with a
`chatcmpl-<uuid>` (non-streaming) or the bare `litellm_call_id` (streaming and
the /anthropic/v1/messages passthrough), so
GET /spend/logs?request_id=msg_... returned [].
The logging conversion now carries the provider's response id through:
_handle_anthropic_messages_response_logging seeds the ModelResponse it builds
with the Anthropic id, and the passthrough logging handler prefers the id it
read off the response body or the message_start chunk over litellm_call_id.
get_spend_logs_id already prefers response_obj["id"], so the spend row and
standard_logging_object["id"] now both carry the id the client holds.
Every field of a multipart form arrives as a string, so `n` reached the
provider as "2" and Bedrock Nova Canvas rejected the request with
"expected type: Number, found: String". Restore the type the request
schema declares at the boundary where the form is parsed, driven by the
schema's own type hints so the helper covers any int- or float-typed
field on any multipart endpoint.
Resolve type-discipline-budget.json by taking the lower limit per rule so no
ceiling ratchets back up.
Staging's 66a3d24b3f left a duplicate embedding_executor parameter in the
Bedrock KB fake handler, which makes ruff fail on the whole tests tree. Drop
the duplicate here so this branch compiles; #39502 makes the same change on
staging.
_setup_database_v2 decided the next attempt budget inline in eight
branches, each rebinding budget before continuing. The branches now live
in _budget_after_deploy_failure, which returns the budget the next pass
runs under, and the two identical idempotent-recovery blocks share
_mark_migration_applied. The loop backs off whenever a pass spent an
attempt, which is the same set of paths that slept before.
Resolves two conflicts:
- tests/test_litellm/vector_stores/test_main.py: staging moved search() to a
RouterVectorStoreEmbeddingExecutor while this branch parametrized the same
test over query; keep both the executor assertions and the parametrize.
- tests/logging_callback_tests/test_bedrock_knowledgebase_hook.py: staging
carries a duplicate embedding_executor kwarg that makes the file a
SyntaxError; drop the trailing duplicate.
Resolves the tests/test_litellm/test_main.py collision, where both sides appended a
new test at the end of the file, by keeping both.
Also carries the one-line fix from #39502: staging arrived with a duplicate
embedding_executor kwarg in the Bedrock KB fake handler, which ruff rejects as a
syntax error, so every commit here would otherwise fail lint. The change is byte
identical to #39502, so that PR merges cleanly once it lands.
OCIStreamWrapper.chunk_creator built every chunk straight from the apiFormat
handlers, so it never reached model_response_creator and OCI streams came back
with a fresh chatcmpl id, a drifting created value and no model on every chunk.
Both exits now go through the shared creator.
The GENERIC apiFormat also closes its stream with a literal `data: [DONE]` line,
which chunk_creator json-parsed and turned into a 500 on every OCI streaming
completion. It is skipped now.
* Fix hide-secrets guardrail: playground redaction, UI dropdown entry, spend-log telemetry
The hide-secrets guardrail never implemented apply_guardrail, so the UI test
playground echoed secrets verbatim; it was missing from the Add Guardrail
dropdown; and it recorded no guardrail_information, so Spend Logs could not
distinguish a redacted request from a clean one.
- implement apply_guardrail (unified interface) with use_native_lifecycle_hooks
so proxied traffic stays on async_pre_call_hook (per-key opt-out and
data["prompt"] handling live only there)
- record standard_logging_guardrail_information (allow/mask + masked_entity_count)
via _process_response/_process_error; opted-out keys and legacy nameless
callback instances record nothing
- advertise hide-secrets in /guardrails/ui/add_guardrail_settings (pre_call only)
and /guardrails/ui/provider_specific_params with a config model
Resolves LIT-3548
* Fix hide-secrets passthrough telemetry and JSON config input
* fix(guardrails): validate hide-secrets object config before submit
- apply_guardrail treats empty-string-only texts as no input, so no
false allow is recorded
- the UI object field keeps raw text while editing and blocks submission
until it parses to a JSON object, instead of posting a string to an
object-only API
- supported_modes_by_provider keeps its dict[str, list[str]] value type
* fix(guardrails): record no hide-secrets telemetry when nothing was inspected
walk_user_text and the prompt redaction now report how many non-empty
strings they visited; when neither inspected anything (image-only
content, empty strings), the run records no guardrail entry instead of
an 'allow' row that counts a check which never saw any text.
The v2 migration resolver gave `prisma migrate deploy` four attempts, and
every recovery path ended in a bare `continue`, so each one burned an attempt.
A database first brought up with `--use_prisma_db_push` has a full schema and
no migrations ledger, so the baseline spent attempt one and the first three
migrations whose objects already existed spent the rest. The proxy then exited
before binding its port, and that database could never be moved onto the
resolver.
The retry budget now counts only attempts that got nowhere. Creating the
baseline, and each migration newly marked applied, leaves the budget alone, so
a push-created database works through its pre-existing objects one pass at a
time. Timeouts, deadlock rollbacks, advisory-lock waits, and a repeat of a
recovery that already ran still spend an attempt, so a run that stops making
progress gives up exactly as before.
The staging merge resolved three budget conflicts by keeping this branch's
older, higher numbers, which turned budget-ratchet red. Nothing on the branch
adds violations for those rules, so the base's limits hold.
Two branches independently added embedding_executor to the same fake
search handler in this file, #39472 in the middle of the signature and
#39474 at the end. Neither conflicted with the other, so both edits
merged and the function ended up declaring the parameter twice.
Python rejects that at compile time, so the whole module fails to
import and every test in the file is uncollectable, taking the
logging_testing job down on staging.
Keep the earlier of the two, which sits where the real handler declares
the parameter.
* fix(sso): resolve multi-valued role claims to the highest privilege role
A role claim carrying several roles used to resolve to whichever one the IdP
listed first, so a user holding both proxy_admin_viewer and internal_user lost
org-level spend visibility depending on claim ordering alone.
get_litellm_user_role now picks the highest privilege role out of a list-valued
claim, and the Entra app_roles path shares that same resolution instead of
keeping its own copy of the hierarchy. SAML assertions carrying several role
values go through the same path rather than taking the first value.
* test(sso): lock ranked-over-unranked role resolution for mixed claims
org_admin, team and customer sit outside the privilege ladder. Pin the
resolution for a claim that mixes one of them with a ranked role so the
asymmetry is covered rather than implicit.
* fix(sso): label the claim-sequence cast for the type-discipline gate
* fix(sso): resolve claim entries without recursing
The repo's recursive-function gate rejects self-recursion here, and a role
claim is flat anyway. Pull the single-value lookup into its own helper so the
list branch maps over it instead of calling back into itself.
* fix(ui): stop the create team form resetting organization and models
The organization preselect ran in an effect keyed on the organizations
query, so any refetch of that list while the Create Team modal was open
overwrote the user's organization pick, which in turn cleared their
models pick. The models field was also cleared whenever the available
models fetch resolved.
Preselect the organization when the modal opens instead, and clear the
models only when the user picks a different organization. An org admin
whose admin orgs narrow to one while the form is open can still pick,
rather than facing a locked empty field.
* fix(ui): block team create when the picked organization is no longer available
An organization picked in the Create Team form now survives a refetch of
the organization list, so it can outlive the admin's access to it. Refuse
the create with a message on the field rather than letting the request
fail authorization at the proxy.
* fix(ui): keep the team create organization field usable when the pick goes stale
Locking the field on a single admin organization also locked it while it
held a rejected organization, so an admin who lost access could not pick
the one organization left. Lock it only while it holds that organization.
* test(ui): hoist the created team fixture out of the mock call
The inline object pushed the repo past its no-large-inline-object-arg
lint budget, which has no headroom.
Clippy never links, so python-config's pyo3/auto-initialize needs no
libpython and the gateway clippy step can cover every feature at once.
The test step stays on --features server because cargo test does link
and this job installs no Python.
The check list existed in three places that had already drifted apart;
CLAUDE.md is now the only copy and the other two point at it.
The Dockerfile asked cargo for --features python-config, which cannot
select the litellm-ai-gateway bin target: that target carries
required-features = ["server"], so cargo silently built nothing and the
later COPY of /build/litellm-rust/target/release/litellm-ai-gateway had
no file to copy. Turn the server feature on and name the bin explicitly
so a future required-features drift fails at the cargo step instead of
silently producing an empty release dir.
