* fix(proxy): discover upstream LiteLLM Gateway wildcard models
* fix(proxy): use the typed shared client for gateway discovery
* fix(ui): show the selected LiteLLM Gateway logo
* feat(proxy): hash UI passwords with PBKDF2 in FIPS mode
A FIPS OpenSSL refuses scrypt, so every password write fails there and
nobody can sign in with a password. When OpenSSL enforces FIPS, passwords
now use PBKDF2-HMAC-SHA256 at 600k iterations with a salt from OpenSSL's
DRBG, stored as "pbkdf2:" plus base64 of salt and key. Any other row is
refused with one error log and left as it is, and the boot migration does
nothing, because FIPS mode is for new deployments.
PBKDF2 at 600k takes about five times as long as scrypt, so in FIPS mode the
five request paths run it on one thread instead of the event loop. A single
thread keeps a login burst from using up a CPU quota, which is what stalls
the loop with asyncio.to_thread. A login that queued for that thread asks the
throttle again before it answers, so a burst of guesses can't get past a block
that the failures ahead of it just set.
The breached password check is off in FIPS mode, because it runs the
password through SHA-1.
A standard OpenSSL reads False at every new branch, so standard mode keeps
scrypt, inline, on the same rows.
Resolves LIT-8425
* fix(proxy): take FIPS logins one at a time and recheck the throttle first
A login that waited for the KDF thread asked the throttle again only after
deriving, so a burst for one user kept that thread busy long after the user
was blocked. Logins now wait for their turn on a lock and ask the throttle
when it comes, so a blocked login costs no PBKDF2 and only one login at a time
holds a place in the KDF queue. Password changes and invite claims no longer
wait behind a whole burst of logins.
* fix(proxy): keep PBKDF2 rows on a standard boot and warn when FIPS skips the breach check
A standard boot treated a pbkdf2: row as plaintext and hashed its text, so the
real password stopped working and the old row text signed in. The migration now
leaves a row of that exact shape alone. Any other pbkdf2: text still migrates.
FIPS mode now logs a warning when it skips a breach check that is turned on.
The one-thread test no longer depends on timing, and the new test variables are
annotated Final.
* fix(proxy): keep the breached password check on in FIPS mode
* fix(proxy): choose the password KDF from fips_mode instead of the host OpenSSL
* test(proxy): fake the scrypt probe in the FIPS fixture now that the gate checks scrypt
* build(deps): lock filelock 4.2.4 on Python 3.11+ and 3.20.3 on 3.10 to clear osv-scan
Co-authored-by: Gilberto Plaza <keko950@hotmail.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* build(deps): bump @azure/identity to 4.14.1 and @azure/msal-node to 7.1.2 to clear osv-scan
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* build(deps): bump oauthlib to 4.0.1 to clear osv-scan
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Gilberto Plaza <keko950@hotmail.com>
Co-authored-by: mateo <277851410+mateo-berri@users.noreply.github.com>
Chirp and other Cloud TTS models now call us-texttospeech or eu-texttospeech.googleapis.com when the deployment's vertex_location is a US or EU location, keep the global host otherwise, and an explicit api_base still wins. Gemini TTS accepts more US and EU regions.
* fix(bedrock): forward the per-turn-control beta to Bedrock Invoke
Claude Code sends `anthropic-beta: per-turn-control-2026-07-01` together
with a mid-conversation `role: system` message carrying a message-level
`output_config` (for example an effort change). The beta header config
mapped the beta to null for `bedrock`, so LiteLLM dropped it while
keeping the message in place on models flagged
`supports_mid_conversation_system`, and Bedrock Invoke rejected the turn
with "messages.N.output_config: Extra inputs are not permitted".
Direct Bedrock InvokeModel probes with a synthetic request show Claude
Opus 5.5 accepts that message shape only with this beta, and that Claude
Haiku 4.5, Sonnet 4.6, Opus 4.8, Opus 5, and Sonnet 5 accept the beta on
a plain request, so map it to itself for `bedrock`. `bedrock_converse`
is unchanged.
Generated with AI
Co-Authored-By: Claude Code
* test(anthropic): expect the per-turn-control beta to reach Bedrock Invoke
Generated with AI
Co-Authored-By: Claude Code
* test(bedrock): drop the duplicate per-turn-control filter test and cite the live check
* fix(rust): allow the per-turn-control beta on bedrock in the Rust beta table
The Rust table must match litellm/anthropic_beta_headers_config.json cell for cell, and the JSON now maps the beta for bedrock. Also type the new Bedrock Invoke test.
* fix(bedrock): forward per-turn-control on Bedrock Invoke only where the cost map flags the model
Bedrock InvokeModel accepts per-turn-control-2026-07-01 on Claude 4.5 and newer and answers 400 invalid beta flag
on Claude 3 Haiku. Released proxies load the beta map from main, so forwarding it in the bedrock row would turn
Claude 3 Haiku requests on every running release from 200 into 400. Keep that row null and forward the client's
beta from the Invoke transformation when a new claude-per-turn-control cost map rule sets supports_per_turn_control.
* fix(bedrock): keep the per-turn-control beta through the messages handler filter on Bedrock Invoke
The /v1/messages handler filtered client betas against the beta map before the
Bedrock Invoke config could add per-turn-control back for models the cost map
flags, so the beta never reached Bedrock. Provider configs now name betas their
model accepts beyond the beta map, and the handler filter keeps those
* fix(bedrock): keep a deployment's per-turn-control flag on that deployment
supports_per_turn_control is now deployment-scoped, so a deployment's opt-in or opt-out no longer changes the beta forward for other deployments of the same Bedrock model. The forward reads the deployment's own model_info first, then the model map.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(redis): route the router, rate limiters and auth through the native store families
* style(redis): satisfy the python lint gates
Format the rate limit store, sort its __all__ and give the routing read batch and two tests concrete list and dict types.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(router): give the registry's default usage store a memory-only handle
RouterUsageStore now takes a handle, so the default passed None and any usage read before RustCore swapped in its store raised AttributeError.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(redis): build stores from native handles and wait for router tasks
Tests still used the pre-refactor store constructors (Store(None), handle= factories, NativeIncrement, tuple scopes). The router outcome tests counted 20 loop ticks, too few now that the allowed-fails increment resolves on the native runtime, so settle() waits for the scheduled tasks instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(ui): let admins set tpm_limit and rpm_limit on tags
The tag create modal and edit form said tag TPM/RPM limits were unsupported and never sent them, while the proxy has enforced tag-level rpm_limit and tpm_limit since #41807. Replace the notice with TPM Limit and RPM Limit inputs wired to /tag/new and /tag/update, validated as nonnegative integers, with a cleared limit sent as null so the proxy drops it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): prefill tag budget and limit fields from the Edit Tag button
The edit form only seeded max_budget and budget_duration when opened from the row's Edit action, so the Edit Tag button inside the tag view showed empty budget and limit inputs and a cleared field could not be sent as null. Seed from the loaded tag on both paths
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): accept null budget fields when editing a tag
A tag whose budget row has max_budget or a limit set to null failed the edit form's zod schema with 'expected string, received null' once the fields were prefilled. Treat null like an empty input and type Tag's budget fields as nullable
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): accept a null description when editing a tag
A tag created without a description comes back from /tag/info with description null, which the edit form's zod schema rejected with 'expected string, received null' on save. Allow null and type Tag.description as nullable
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): ignore invalid limits left in a collapsed tag budget section
An invalid value typed in Budget & Rate Limits and then hidden by collapsing the section blocked Create Tag and Save Changes with the error out of sight. When the section is collapsed and the only errors are on its fields, drop them and submit the visible values, which already discards the hidden budget fields
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): name the expected payload so the inline-object lint budget holds
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(ui): move remaining outlier text onto i18n
Moves 146 English messages that the earlier i18n batches missed into en.json.
They live in helpers, zod schemas, label fallbacks and shortcut descriptions
that the batch scope script did not count, so no batch picked them up. The
English on screen is unchanged.
* refactor(ui): keep exported compliance error text in English
The compliance results' "Error: ..." text is also written to the CSV export,
so it stays a literal. The policy table's unnamed fallback now reads i18n.t
inside its helper instead of taking t as a new parameter.
* test(e2e): add a Codex row that calls a local MCP tool on a first and a resumed turn
* test(e2e): assert the codex mcp row got each value back from the mcp tool call itself
* test(e2e): check the codex mcp row exits cleanly on both turns
* test(e2e): drop the unused content type from the codex mcp result model
* feat(team): let default_team_params attach access_group_ids to every new team
* fix(types): keep default access_group_ids as Sequence for the type discipline gate
* fix(settings): chunk the default access group existence lookup
* test(team): prove default access groups reach the saved team row and type the settings fixtures
* fix(team): check default access groups on the writer so a fresh group is not rejected behind a read replica
* feat(spend-logs): opt-in latest_turn mode that stores only the current turn of stateless requests
* fix(spend-logs): keep bridged Responses rows and classifier audit in latest_turn mode, reject unknown values at boot
* fix(spend-logs): drop the recursive call in the latest_turn trim and type the stateless test cases
* test(spend-logs): format the stateless latest_turn cases
* test(spend-logs): wrap the classifier audit fixture
* fix(spend-logs): type the latest_turn trim inputs
* fix(spend-logs): leave the raw request body unvalidated before the latest_turn trim
* fix(spend-logs): trim classifier rows and ignore continuation keys off /v1/responses in latest_turn mode
* fix(spend-logs): take the classifier's originating request from the logging payload and redact a copy of vector store results
* fix(spend-logs): redact the body selected for storage, after the latest_turn trim
* feat(proxy): give Codex accurate model entries for every model in /v1/models
* fix(router): read a deployment's upstream model without a not-required key access
* fix(proxy): keep Codex tiers for unpriced deployments and match stock entries by declared provider
A deployment with no token price no longer votes a Codex service tier off,
an undescribed model keeps Codex's default reasoning summary parameter,
a Bedrock deployment naming its provider in custom_llm_provider gets Codex's
stock entry, and Bedrock provider namespaces follow the AWS model id format.
* fix(proxy): import cost tracking and provider helpers lazily in the Codex catalog
lite codex imports the catalog before the SDK is initialized, so importing
cost tracking, the Bedrock helpers or provider resolution at module level hit
a circular import and broke every lite command.
* fix(proxy): drop Codex's fast speed tier wherever its priority tier is dropped
Codex shows /fast for a leftover fast speed tier even when service_tiers
no longer offers priority, so an alias, an empty configured tier list or
a configured list without priority showed a /fast toggle that sends nothing.
One helper now sets service tiers, speed tiers and the default together.
* fix(proxy): give an older Codex a translated entry for models past its version
Codex's own entries name the oldest release that can read them in minimal_client_version. A release older than that can fail to decode the entry, such as a reasoning effort it has no name for, which fails the whole catalog, so it now gets the translated entry, whose levels are trimmed to that release
Reworks the redis-client, connect, resolve and testing crates, adds python-compat json loads and
TTL seconds, replaces the redis-store memory tier with a DualCache plus a shared failure log, and
gives the Python side the Redis timeout log throttle and a request OTEL parent span for service events
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(proxy): let admins allow specific S3 buckets and endpoints in request bodies, block bucket settings otherwise
Client requests can no longer pick s3_bucket_name, s3_output_bucket_name or allow_legacy_cloud_file_ids unless the
value is on general_settings.allowed_client_side_s3_buckets (buckets) or the admin opted in the usual way.
s3_endpoint_url, already blocked, can now be allowed per value through general_settings.allowed_client_side_s3_endpoint_urls
* refactor(proxy): drive the client-side S3 allowlist from one typed field-to-setting map
* fix(proxy): treat a null client-side S3 allowlist in config as empty instead of failing the request
* fix(gemini): omit raw response body from audio transcription error messages
* fix(gemini): raise incomplete transcription interactions as non-retryable 400 and drop response bodies from Vertex transcription errors
* fix(audio_transcription): keep response bodies out of transcription errors
Keep the upstream status on a Gemini interaction that did not complete, so
the router still retries it as it does on main. Gemini stops the same long
clip on some tries and not on others, so a retry can recover it.
Stop quoting the reply in the Deepgram, ElevenLabs, Watsonx, Mistral,
OVHcloud and Scaleway parse errors, and hide pydantic input values on the
Gemini and Vertex response models.
* fix(audio-transcription): keep reply bodies out of watsonx, whisper, xai and validation errors
Watsonx and whisper no longer print the parsed reply after Got: when it has no text field. TranscriptionResponse and the xAI response model hide input in validation errors, so a malformed field cannot quote transcript fragments. Deepgram and ElevenLabs keep the original error text, which is now safe.
* fix(audio-transcription): keep transcripts out of the shared converter's parse errors
* test(audio-transcription): compare wrapped lookup errors instead of pinning upstream wording
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(router): drop supports_web_search false deployments for Anthropic web_search_* and Gemini googleSearch tools
* test(integration): build the web search filter proxy config without mutating the shared config
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): add a beta language setting to the account menu
The account menu gets a Language field, marked Beta, that lists each language by its own name with the English name beside it. The choice is saved per browser in localStorage and applied with a page reload, so memoized and imperative translations are read fresh.
English stays the default. An unset value, English, an unknown or unsupported code, a locale that fails to load, and any string a locale has not translated all show English. The field stays hidden until a second locale is registered in LOCALES, so this change is invisible until the first translation lands.
* fix(ui): keep the help translate link text inside the link
<link> is a void HTML tag, so the Trans parser dropped its children and rendered an empty anchor. The note now uses <githubLink>, and the language list is at least as wide as its trigger.
* fix(proxy): probe scrypt instead of MD5 in the fips_mode OpenSSL check
* test(proxy): fake a working scrypt so the fips_mode check tests pass on FIPS hosts
* feat(bedrock): enable native structured output for Opus 5 models
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): keep govcloud and bare model rows off native output
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): retry rejected native schemas with the JSON tool on default-native models
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): keep the rejection retry on the unforceable-tool native path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): retry rejected schemas on the Messages path for default-native models
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(bedrock): keep JSON-tool streaming coverage on unchanged route
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(bedrock): format Messages native output retry
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(bedrock): allow profile-only native output metadata
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): only disable default-native output on schema-rejection retries
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(bedrock): type fixture parameters in native output retry tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): retry with the JSON tool when Bedrock rejects a schema's grammar size
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): drop strict tools after a native-output retry hits the grammar limit
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(bedrock): format native output fallback
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The banner inherits the Alert primitive's icon layout, which spans two grid rows and nudges the icon down 2px so it lines up with a title plus description. The banner only has a one-line title, so the empty second row and the nudge left the icon 3px below the text. Keep the icon on the title's row with no offset
The /chat request errors and deep research notice, and Aria's step limit
and no answer replies, are written into assistant messages that are sent
to the model as history on the next turn. Pin them to English with
i18n.getFixedT("en") so a future UI language never changes the model
payload. English output is unchanged.
* feat(packaging): ship litellm-proxy wheels with the proxy extra
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(packaging): keep litellm-proxy out of the proxy extra until PyPI has it
The current release pipeline publishes litellm with whatever pyproject.toml lists, so pinning an unpublished package would break pip install litellm[proxy]. Images and CI still install the locally built gateway wheel
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(types): define the responses compact call types so during_call guardrails and deployment hooks see them
* test: regenerate dashboard api types and skip compact in the azure sdk client sweep
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The primary stream's mid-stream stall reaches failure_exits as a mapped litellm.Timeout (408), which cools the deployment on the Rust core. Keep that behavior and update the integration test that expected the Python core's exemption. Fallback hops stay exempt through is_stream_stall.
* fix(router): write the prompt caching pin when turn_off_message_logging is on
The success event PromptCachingDeploymentCheck reads had its messages redacted in place by
turn_off_message_logging before any callback ran, so the pin was keyed off redacted-by-litellm,
never written, and every follow-up bounced across the group. Callbacks that route on the request
and never log it now opt out of that redaction and get the request messages back, while every
other callback and the spend log keep the redacted payload.
* fix(logging): type the request-messages payload helper as dict[str, object]
The bare dict return grew the LIT001 count past the merge base and left the returned fields unchecked
* test(integration): type the pin rig's Redis client as Redis[bytes]
The untyped generic grew the basedpyright reportUnknownMemberType count past the merge base
* test(integration): quote the Redis[bytes] annotation so the rig collects at runtime
redis.Redis is not generic at runtime, so the unquoted subscript raised TypeError on import
* fix(logging): read the request messages once as object in the exempt payload helper
Two partially unknown reads of self.messages grew the basedpyright reportUnknownMemberType count past the merge base
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The Rust core built its Scheduler without a Redis cache, so the priority queue lived only in each worker's memory and was never shared across workers or instances. Build it on the routing cache's Redis cache, as the Python core did
* fix(responses): preserve native stream spend on guardrail rejection
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(responses): type deferred logging dispatch once
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(responses): expect guarded rounds on one failure row
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(tests): format guarded billing regression
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(tests): preserve existing MCP unit formatting
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A bare model name that no team pattern matches as spelled is now tried again as provider/model against the caller's team pattern routers, as PatternMatchRouter.get_pattern did in the Python core, with the routed deployment's model filled from the prefixed name.
The fallback loggers got the walk's own metadata in place of the request's, so prometheus fallback counters lost the caller's key and team labels. Lay the walk's stamps over a copy of the request's bucket, as run_async_fallback did
* feat(interactions): continue requires_continuation long decodes with continuation_token hops
A foreground Interactions request stops at Google's per-hop output cap with
status requires_continuation and a continuation_token. litellm now re-posts the
request with previous_interaction_id and continuation_token until the status
moves on, merging the hops into one interaction (last id, every hop's steps,
summed usage with initial_input_tokens from hop 1), for non-streaming and
streaming calls. continuation_token is accepted as a request param, and
requires_continuation is a known status for billing.
* refactor(gemini interactions): validate streaming chunks with model_validate
* fix(interactions): keep the held completion across stream hops, stop on a repeated continuation_token, follow the spec redirect
* fix(interactions): add grounding_tool_count per tool type, stop on any seen continuation_token, keep streaming past a [DONE] frame when a hop is pending
* ci(interactions): point the spec canary at aistudio.google.com, where Google moved the Interactions OpenAPI spec
* test(interactions): scripted Google ends each long decode hop with a [DONE] frame
* refactor(interactions): pass requires_continuation and continuation_token through instead of looping hops in the proxy
The caller drives the continuation, the same as against Google directly. /interactions accepts continuation_token in place of input so a resume request needs no input.
* test(proxy): assert the accepted /interactions bodies pass the required-param check
* fix(proxy): keep the router default fallback for input on /interactions, continuation_token stands in for it
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
