Replace the hand-rolled combobox in the Cmd/Ctrl+K palette with shadcn's
base-vega Command and CommandDialog (cmdk) and bind the shortcut with
react-hotkeys-hook. cmdk now owns arrow keys, wraparound, Home/End and IME
composition, Base UI keeps the dialog mounted for its closing animation and
returns focus to the trigger, and mod+k is ignored while focus is inside
another dialog
Results keep their fixed Keys, Actions, Pages order with shouldFilter
disabled. The highlight resets to the first row whenever debounced key
results load, so Enter after typing still opens the top key
cmdk 1.1.1 leaves aria-activedescendant empty or stale after it picks a row
on its own (dip/cmdk#413), so a MutationObserver keeps it on the row that is
actually aria-selected until upstream fixes it
* fix(proxy): let the Admin UI cancel /team/spend/by_user at 60s with an opt-in statement timeout
GET /team/spend/by_user takes an optional timeout_seconds (1 to 600). When set, the raw
LiteLLM_SpendLogs aggregation runs under SET LOCAL statement_timeout and Postgres cancels it,
and the route returns 503 instead of letting the query run for hours. Callers that omit the
param are unchanged. The Admin UI sends timeout_seconds=60 and the Spend Per User Within Team
card shows the server's error instead of an empty table
* fix(proxy): cancel /team/spend/by_user in Postgres after 30s
Prisma's HTTP client already gives up on the statement at 30s, so the
query now runs under SET LOCAL statement_timeout = 30000 for every caller
and the opt-in timeout_seconds param is gone. The Admin UI card shows the
request error instead of the empty-state text.
* fix(proxy): keep /team/spend/by_user on the read replica inside its timeout transaction
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat: show user names in access group details
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat: show user names on project, org, model and prompt details
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): look up user names by exact id
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): fall back to user list for org admins
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): fall back when /v2/user/info returns a different user id
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): let org admins read their org members in /v2/user/info
Org admins could list a user through /user/list but got a 404 from
/v2/user/info for that same user, so the dashboard fell back to a
substring /user/list lookup that could miss the user. /v2/user/info
now allows an org admin to read users in organizations they administer,
and the dashboard's single-user lookup calls only that endpoint.
Expected 4xx responses from /v2/user/info no longer log a traceback.
* fix(proxy): drop unbounded IN from org admin check in /v2/user/info
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(cost-map): record the provider's recommended replacement as successor_model on deprecating rows and report it from /model/deprecations
* fix(cost-map): pick the alias row's successor by its own shutdown date when the family has several
* chore(ui): regenerate schema.d.ts for the /model/deprecations successor_model wording
* fix(proxy): prefer a configured successor_model over the cost map's when deployments share a name, and keep the cost map's under a hand-set deprecation_date
* test(cost-map): allow successor_model in the cost map row schema
* feat(cost-map): infer successor_model as the next version in the same family for 136 dated rows with no published replacement
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(ui): move teams, users and orgs text onto i18n
Moves the page-authored English on /teams, /users and /organizations,
plus the shared member table, user search modal, model select and budget
schedule helpers they render, into react-i18next t() calls backed by
en.json. The rendered English is unchanged
Helpers that files outside this batch or untouched tests still call keep
their signatures and take an optional t that defaults to the shared
instance, the same as usageUnits.ts. Text that existing tests pin, values
compared against the API, code samples, CSV column names and date formats
stay as they were
* refactor(ui): escape member ids in team member dialogs and merge split org and team strings
The reset spend and reset budget dialogs interpolate the member id into a Trans string, so an id containing strong, i, p or br tags rendered as markup. Escaping the values and unescaping text nodes keeps it literal, as it was before the move to i18n. Also merges the org budget lines and the team metadata hint into whole sentences and drops two constants nothing imports anymore
* refactor(ui): pass the member id into the team member reset dialogs as an element
Escaping the id still let Trans fill in placeholders like {{budget}} written inside it a second time. Rendering the id through its own component keeps it out of the translated string, so any id shows exactly as it did before the move to i18n
* feat(realtime): support latest OpenAI audio models
* fix(transcription): type SDK stream duration metadata explicitly
* fix(realtime): preserve transport settings and translation duration
* refactor(realtime): reuse SDK client initialization
* fix(audio): bill finalized streams and authorize transcription models
* refactor(audio): keep transcription pricing selection together
* fix(realtime): reconcile model metadata and test contracts
* fix(realtime): preserve provider duration and model capabilities
* fix(realtime): keep transcription validation compatible with cost map guard
* fix(realtime): address transcription and billing review findings
* fix(realtime): preserve input usage and provider routing
* fix(realtime): bill retained output and route Azure aliases
* fix(realtime): reject malformed provider durations
* fix(realtime): preserve batch cache rates and Azure v1 routes
* fix(transcription): reject nonfinite provider durations
* refactor(realtime): align annotations with current lint rules
* fix(realtime): preserve translation deployment pricing and input usage
* test(transcription): verify synchronous stream finalization
* fix(realtime): retain missing translation input usage at close
* fix(transcription): reject streams that bypass output guardrails
* fix(realtime): reconcile audio protocols billing and current CI
* fix(realtime): trust confirmed formats and authorize default models
* fix(realtime): preserve absent models and reject unsupported guardrails
* fix(realtime): reject translation policy pipelines
* chore(ci): rerun transient infrastructure timing failure
* fix(realtime): reconcile audio protocols with current upstream
* fix(audio): validate provider responses at SDK boundaries
* fix(realtime): preserve minimum OpenAI SDK compatibility
* fix(realtime): preserve deferred Live session authorization
* fix(realtime): preserve upstream authentication and transcription behavior
* fix(audio): scope provider duration precedence to configured models
* fix(typing): avoid Final declarations inside transcription cost loop
* refactor(audio): reuse transcription mapping and duration resolution
* fix(realtime): enforce model budgets and preserve translation routing
* fix(audio): log streamed transcripts whose done event reports detected languages
* fix(audio): pass stream, languages and keywords through unchecked for non-OpenAI transcription providers
Providers without a streaming transcription handler got UnsupportedParamsError
for stream=True, languages and keywords, where the merge base forwarded them as
provider-specific params. Only openai and azure validate them now.
* fix(audio): bill streamed gpt-transcribe from the provider's reported duration
The stream path computed cost after logging had replaced the transcription's
duration usage, so it fell back to the measured file length while the
unstreamed call billed the provider's seconds.
* fix(transcription): log streamed transcription usage from the provider response
The stream logging path built the spend payload from the response after
its duration usage was transformed away, so whisper-1 with stream=true and
streamed gpt-transcribe logged an empty usage object. Bill and log from the
provider's TranscriptionResponse in both the sync and async paths.
* fix(realtime): keep main's routing, protocol and transport behavior around the new translation routes
- Send OpenAI realtime and translation client_secrets and calls through litellm's httpx client again, so ssl_verify and custom CA settings apply and upstream errors keep main's shape
- Use the input transcription model as the session model only for transcription sessions
- Drop the new /openai/v1 translation aliases and record each lazy feature's own routes, so the /openai pass-through and the MCP routes resolve as on main
- Let a configured Azure realtime_protocol win for every model, with GA as the default only when none is set
- Return x-litellm headers on streamed transcriptions and mark the request done
- Refuse translation sessions before connecting upstream when realtime guardrails apply or the provider is not OpenAI or Azure
- Accept json and text for gpt-transcribe, and drop other formats under drop_params instead of failing
- Bill translation sessions on output audio seconds only, at the single published rate
- Document the 201 SDP answer on the translation calls routes and the transcription 200 response
* fix(realtime): restore main behavior for guardrails, Azure transcription, and realtime clients
Streamed transcription runs unstreamed when a deployment post_call guardrail applies, so the
guardrail still sees the transcript. Azure transcription maps v1, latest and preview to the
default dated version on the deployments route for every model. The SDK client secret keeps
session.model ahead of model, realtime WebSockets connect to api_base again, realtime metadata
keeps only guardrails, and managed agents need an explicit transcription model. Reverts the two
unrelated streamed Responses error codes and the duplicate cached_tokens_details field, and the
gpt-transcribe format check only reads the request provider's cost map entry.
* fix(realtime): keep main's transcription, client secret and route behavior where the PR changed it
Transcription reaches every provider as on main again, sends stream only when asked, forwards language and
languages together, keeps form key order, bills provider seconds only when above zero, honors base_model for the
new flags, keeps main's Azure api_version on the proxy path, and streams carry chat's headers, an error event on a
mid-stream failure, and a spend row when the stream ends early. Translation cost reads both per-second rates from
one source and spend rows carry the billed seconds.
Realtime transcription WebSockets keep model in the upstream URL outside OpenAI's hosts, client secrets forward
explicit nulls, accept conversation and delay, and answer 400 on a malformed body, the /openai translation paths
are back to main, translation refuses only for guardrails realtime sessions run, nested aliases resolve and later
session.update models are authorized, translation client secrets run per-model auth, beta clients bridged to GA
get beta-shaped session events, and transcription sessions send the deployment model in both shapes.
* fix(realtime): address review findings on transcription billing, streaming and realtime auth
Per-second pricing replaces token billing only when the priced entry sets no token rates, translation rates fall back per key to the cost map, transcription cache keys include languages and keywords, a stream request a post_call guardrail forces unstreamed comes back as redacted SSE, Azure beta clients are bridged only for GA-only deployments, nested transcription models on the realtime WebSocket are checked against the key, and translation spend carries key and team metadata
* fix(azure): pick the GA realtime protocol for deployments whose model_info.base_model is GA-only
* fix(realtime): preserve prior merge resolutions after rebase
* fix(realtime): enforce translation policies and normalize Azure transcription URLs
* fix(cookbook): stop translation once output idles after the upload ends
* fix(realtime): bill translation on input audio and drop pre-read stream cost headers
Translation spend now uses the input audio seconds the session sent, priced
through input_cost_per_second, instead of the translated output duration.
A streamed transcription no longer returns cost headers before the stream is
read, since the cost is only known after the last event. Also removes an
unused realtime guardrail helper.
* fix(realtime): drop top-level session nulls, bill dropped transcription streams, and bridge beta response fields
Client secret, translation and transcription session bodies drop top-level nulls again, as main did, while nested nulls such as turn_detection still pass through. Transcription client secrets send the deployment model as session.model. A streamed transcription the client drops still runs its success callbacks, and the stream ends with data: [DONE] like OpenAI's. The beta to GA bridge maps max_response_output_tokens, speed and noise reduction, remaps response.create, and drops temperature under drop_params. The Entra token for realtime HTTP routes is fetched off the event loop
* test(e2e): cover realtime translation, live transcription, client secrets, and streamed transcription
* fix(realtime): close review gaps in translation billing, transcription streams, and Azure GA
Bill translation sessions from the deployment's own per-second rates, add per-second prices for
Azure gpt-4o transcribe models, keep stream-scoped guardrails seeing the caller's stream flag, map
mid-stream transcription errors to LiteLLM exceptions with a [DONE] frame, map GA responses back to
the beta shape for bridged beta clients, honor the minting key's guardrail opt-out on translation
calls, restore the Azure transcription session URL, and flag GA-only Azure realtime models
* fix(realtime): bill a translation rate the deployment omits from the session model
* test(e2e): check translated speech is not English instead of exact French wording
* fix(realtime): keep main's behavior for non-bridged backends, stream error frames, gateway URLs and Azure transcribe prices
Remap beta events to GA only for bridged beta clients and translation sessions, bridge beta clients on GA-pinned Azure deployments, log error-only upstream sessions as failures, flag dated Azure GA realtime models, drop the unpublished Azure per-second transcribe rates, log a mid-stream transcription failure once, keep main's SSE error frame outside transcription, drop unsupported transcription params under drop_params, keep the caller's Azure auth headers, keep a gateway /v1 path, let key opt-outs release string-callback guardrails on translation, and keep the cookbook waiting while a long upload is still sending
* fix(realtime): keep the proxy's realtime session gate off translation sessions
A key that opts out of a legacy callback guardrail now gets a working translation session, since the transcription-guardrail gate no longer sends a realtime session.update the translation upstream rejects
* fix(realtime): keep websocket model limits, bill measured transcription length, route transcription sessions on the allowed model
A WebSocket opened without a query model now authorizes as an empty model, transcription cost prefers the measured upload length over the provider's reported seconds, transcription sessions overwrite nested models with the authorized top-level model again, and a non-boolean transcription stream value answers 400 or is dropped under drop_params.
* fix(ci): copy the rust bridge extension into the openapi docs gate's base worktree
* fix(model-catalog): accept the realtime translation endpoints in the cost map
* fix(cookbook): close the translation session and read until session.closed
* fix(realtime): treat an empty transcription stream as unset and pass the cost-map guard
An empty `stream` form value runs the transcription unstreamed again, as
OpenAI does, instead of answering 400. The four translate cost-map entries
drop the translation routes from supported_endpoints until the main
branch's catalog parser accepts them, and the nested transcription model
lookup uses the router registry entry point.
* test(proxy): drop the mid-stream error frame pin that main's error shape change replaced
* fix(transcription): bill uploaded seconds when a stream fails mid-transcript
* fix(proxy): bill uploaded seconds when a streamed transcription breaks mid-stream
* fix(audio): keep transcription's positional arguments and redact streamed error details
- Move the new languages, keywords and stream options to the end of transcription's parameters, so a prompt or format passed by position keeps its meaning
- Redact internal hostnames, paths and tracebacks from a streamed transcription's mid-stream error event, as chat streams already do
- Move the CA-bundle check for the realtime HTTP routes, which serves real TLS on loopback, from the unit suite to the SDK integration suite
* fix(audio): stream the guarded transcript to SDK callers that asked for a stream
* fix(logging): keep a streamed transcript out of callbacks that turned off message logging
A streamed transcription's assembled response reached success callbacks with its text intact under turn_off_message_logging or a callback's message_logging=False, because the shared streaming redactor had no transcription branch. It now redacts the text, as the unstreamed call already does
* fix(proxy): stream a transcription the deployment streamed when the client sent no stream field
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(decisions): add Parallel AI web-grounded decisions
* fix(decisions): name the Parallel model decision
* fix(parallel_ai): restrict environment credentials to trusted decision endpoints
* fix(decisions): keep Parallel evidence fields off other providers and handle missing confidence
Only fields a provider names reach the OpenAI response, so existing providers keep
their base shape. A malformed api_base answers 400 on both credential paths. The
complexity router classifier and the Playground handle a provider that omits
confidence. Regenerates the lazy OpenAPI snapshot and the dashboard API types
* chore(cost-map): declare the Decisions routes on parallel_ai/decision
The provider endpoint files are generated from the cost map, so the row must
name both routes for the generated System One flag to hold
* fix(decisions): report no classifier confidence when a provider omits it
A choice answer with no confidence is a valid classifier answer, so the
complexity router picks its tier and leaves classifier_confidence and the
confidence signal out instead of reporting a different metric in its place.
Also show the Parallel AI logo in the provider picker and add a Parallel
translation fixture captured from the live API for both Decisions routes.
* fix(decisions): keep a token-priced row on token billing when it also sets a query price
A plain input_cost_per_query bills flat only on a row with no token price, and input_cost_per_query_decisions always bills flat, so a custom Decisions row that already bills tokens costs the same as before this PR
* fix(decisions): bill a deployment's query price and accept Parallel bases that end in a route
A proxy request logs through an object built before the router picks the deployment, so a query price on litellm_params never reached Decisions billing. Parallel Search accepts a base ending in /v1/search and shares PARALLEL_AI_API_BASE with Decisions, so Decisions now accepts that base and one ending in /v1/decisions.
* fix(decisions): keep per-query billing and optional confidence on Parallel only
A query price on any other Decisions provider moved that provider off token
billing, and a choice or score answer with no confidence became a 200 with a
null for every provider. Both now apply to Parallel alone, so other providers
bill and fail as they do on main.
A Parallel deployment with explicit zero token rates and a query price read as
free, so every budget check was skipped while it billed per call. It now counts
as priced.
The dashboard key for Parallel is now PARALLEL_AI so a served bundle that
predates the provider still sends parallel_ai as the provider slug, and the
README table lists the provider.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(responses): reproduce comparison response ID ownership and wire failures
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(responses): guard stale retries and malformed comparison envelopes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(responses): authorize and unwrap comparison response ids
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(responses): require comparison IDs to follow response security policy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(responses): deny raw comparison response ids by default
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(responses): reject raw comparison response ids by default
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(responses): enforce managed ownership with unmanaged ID opt-in
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: Return Anthropic error envelopes on Messages auth failures
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
* fix: Answer Anthropic envelopes on Messages auth failures without relabelling unmapped statuses
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
* fix(proxy): answer the Anthropic error envelope on the /anthropic pass-through
The pass-through relays upstream errors in Anthropic's shape, but the proxy's
own refusals there (bad key, model access, budget) still answered the OpenAI
body, so an Anthropic SDK pointed at /anthropic could not read them
* fix(proxy): answer the Anthropic envelope for 404, 405 and unhandled 500 errors on Messages routes
A disabled Claude Code gateway 404, a wrong-method 405 and an unhandled 500 on the
Messages and /anthropic/ routes kept FastAPI's detail body or the OpenAI shape. The
central HTTP and unhandled-exception handlers now pick the Anthropic envelope by route.
The dashboard playground toast titles Anthropic error types when the body has no code.
* fix(proxy): keep LiteLLM error types and Anthropic envelopes on Messages routes
Error bodies on the Messages routes now carry error.litellm_error_type when the
LiteLLM type differs from the Anthropic type, so clients and the Admin UI keep
causes like budget_exceeded. Unlisted 4xx statuses map to invalid_request_error
per Anthropic's errors page, model-list requests that send anthropic-version get
the Anthropic envelope, and the admission, decompression and request-size
middleware errors use it on these routes too.
* test(proxy): expect litellm_error_type in Messages integration tests and type middleware tests
* fix(proxy): keep handler error bodies unchanged and timeouts as api_error
Errors the Messages and count_tokens handlers already returned in the Anthropic
shape keep their original bodies, so litellm_error_type now appears only on
errors the proxy raises before the handler. A 408 stays api_error as before,
and the playground toast titles the new Anthropic types and ignores long
numbers in front of an embedded error
* fix(proxy): keep the base error types inside the Messages handler and scope toast titles to Anthropic bodies
The wider status table (402 billing_error, 409 conflict_error, 504 timeout_error,
other unlisted 4xx invalid_request_error) now applies only to errors the proxy
raises before the handler. Errors inside the Messages handler, streamed error
frames and the SDK streaming adapters keep the merge base's table.
The dashboard toast reads a status before embedded JSON and titles by Anthropic
type only when the body is an Anthropic error envelope, so other pages keep
their titles.
* test(anthropic): mark the new test variables Final
---------
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* refactor(ui): move admin and settings text onto i18n
Route the page-authored English on the admin panel, caching, access
groups, tag management, logging and alerts, and router settings pages
through react-i18next, with 1,183 messages added to en.json. The
rendered English is unchanged
* fix(ui): guard i18n label lookups against prototype-named values
A server value such as "toString" used as a key into a label map
resolved to an Object.prototype member and was passed to t(), which
throws. Check own properties before translating so those values fall
back to the raw string as before
Move the page-authored English on the agents, vector stores, prompts,
search tools, memory and skills pages, plus the shared skill detail,
make-skill-public and model selector components, into en.json behind t().
The English on screen is unchanged. Keys are added under agents,
vectorStores, prompts, searchTools, memory, skills, claudeCodePlugins and
modelSelector, inserted at the top of en.json
Strings that could only move with a structural change stay English,
along with API values, code samples, paths and names
Moves the text the /guardrails page writes itself onto react-i18next under a
new top-level `guardrails` block in en.json, placed right before
`guardrailsMonitor`. Catalog data (garden cards, custom code templates, option
lists) stores ParseKeys and translates at render time. Provider and vendor
names, API values, code samples and test payloads stay raw
guardrail_info.tsx and custom_code/CustomCodeModal.tsx cross the 800-line
max-lines cap only because Prettier wraps the new t() calls, so each gets a
max-lines suppression entry instead of a split. The react/no-unescaped-entities
entry for add_guardrail_form.tsx is pruned because its apostrophes moved into
en.json
Route the page-authored English on /mcp-servers, /mcp/oauth/callback and the
shared MCP permission, server selector and BYOK components through t() and
Trans, with 1,094 messages added to en.json under mcpServers,
mcpServerManagement, mcpTools and mcpOAuthCallback. The rendered English is
unchanged.
The unchecked track used --input, which is nearly white in the light theme, so an off switch read as a white knob on a white page. The light unchecked track now uses --muted-foreground, and dark mode keeps its own override
* feat(model_info): add supports_continuation capability flag for Gemini long decode continuation
* fix(router): keep a deployment's supports_continuation override on that deployment
A supports_continuation value set in one deployment's model_info was written onto the shared provider/model cost map key, so every other deployment of that model reported it too. List it with the other per-deployment capability flags, and cite Google's long decode page next to the cost map assertion
* fix(cost-map): flag vertex_ai/gemini-4-argon-cyber-permissive with supports_continuation
Google's Vertex Gemini 4 Argon guide gives gemini-4-argon-cyber-permissive the same long decode limit as gemini-4-argon, and LiteLLM already continues its long requests, so the cost map now says so in both copies.
* fix(router): count only a boolean true supports_continuation when building a model group
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(bedrock, gemini, deepseek, databricks, gigachat, anthropic): answer 400 for a hosted MCP tool a route cannot run, drop it under drop_params
A `type: "mcp"` tool sent to a route that rebuilds tools for its provider
(Bedrock Converse and Nova, Bedrock Invoke Claude, Vertex AI Claude, Vertex AI
and Google AI Studio Gemini, DeepSeek, Databricks Claude, GigaChat) was dropped
silently and the request still answered 200. These routes now answer
UnsupportedParamsError (400) naming the model, route and server, and pointing
to the LiteLLM MCP gateway. Under drop_params the tool is dropped with a
warning, other tools are kept, and no empty tools field is sent.
Anthropic, Azure AI Claude and Claude Platform on AWS keep mapping the tool to
the MCP connector. Native Responses providers and raw OpenAI-compatible
forwarders are unchanged.
* fix(mcp): check hosted MCP tools once per request and keep base tool_choice handling
Each route now runs one request-level step before its param mapping. When the drop leaves no tools,
the request sends no tools, tool_choice or parallel_tool_calls. Converse and GigaChat go back to their
base tool_choice and empty-tools handling.
* refactor(deepseek): hold the sent tools in a tuple
* fix(mcp): keep tool_choice for legacy functions when every hosted MCP tool drops
* fix(mcp): answer 400 for MCP servers a route cannot run on Messages, Ollama and the Responses playground
/v1/messages requests with mcp_servers on a chat-adapter or Responses-bridge
route now answer 400 naming each server, or drop it with a warning under
drop_params, instead of being forwarded or silently ignored. Ollama generate
no longer pastes a hosted MCP tool (headers included) into the prompt, and
ollama_chat filters hosted MCP tools the same way. The Admin UI Responses
playground now sends litellm_proxy/mcp references like the other endpoints.
* fix(ollama): leave JSON mode off when drop_params drops every hosted MCP tool
Ollama generate set format json for any request with tools, even after the
hosted MCP check dropped all of them, so a plain answer came back as JSON or
as a call to a tool the model never had. The check now runs before the JSON
mode switch, which stays on only while a tool is left for the prompt.
* fix(messages): honor additional_drop_params for mcp_servers and read the beta map only for hosted MCP tools
* fix(mcp_security): accept gateway server names and toolsets the gateway resolves
The guardrail compared litellm_proxy/mcp/<name> refs against registry keys, which are server ids, so a saved server referenced by name, alias, or toolset name was blocked although the gateway routes it. It now resolves each ref the way the gateway does and still blocks unknown names.
* fix(tool_fields): keep MCP drop warnings on one log line
Server labels, model names, and routes come from the caller, so a line break in one could forge extra log lines. The drop warnings now strip CR and LF from each value.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(proxy): make /update/sso_settings a true PATCH and send only changed SSO fields from the UI
* fix(proxy): mask SSO client secrets in the /update/sso_settings response
* fix(proxy): keep SSO values that only live in the environment_variables row on a partial SSO PATCH
The SSO PATCH strips every SSO env var from the environment_variables config
row. Now that the UI only sends changed fields, an omitted value that lived
only in that row was dropped from both tables and lost on restart. Carry those
values into the SSO row before stripping them
* test(ui): pin CLEARED_SSO_SETTINGS to exactly the backend SSOConfig fields
With PATCH semantics an omitted key keeps its stored value, so a field
missing from the Clear list would silently survive Clear SSO settings
* feat(ovalix): file and tool scanning, app discovery from the key alias
Mirror of BerriAI/litellm#34132 at 5ffc268585, rebuilt on main.
* fix(ovalix): keep inspecting requests whose key has no user
The tracker allows a checkpoint call with an empty actor without inspecting
it, so a key with no user (or a /v1/messages or /v1/responses request that
carries its own metadata object) got no Ovalix enforcement. Fall back to
"unknown" as the guardrail did before, so every request is inspected.
* refactor(ovalix): check texts without mutation and test discovery through tracker requests
- _check_texts builds its result from per-text verdicts checked newest first by _check_text
- discovery and routing cache unit tests answer the tracker routes instead of replacing private methods
- integration test adds streaming cells and a tracker outage during a concurrent burst
* fix(ovalix): resolve tool result names in one pass
Each tool result rescanned every earlier message for its call id, so a request with thousands of
tool results spent seconds in the event loop. Declarations are now indexed once per request and
each result takes the nearest earlier one by bisection
* test(ovalix): annotate new test bindings and build the burst plan with product
* fix(ovalix): drop app discovery from the key alias
A config with no application_id no longer boots in discovery mode, where
the [Name] prefix of the key alias picked the Ovalix application and so its
policy. Anyone who could rename a key could move it to a weaker app, or to
an unknown name the tracker created with its default policy. The guardrail
now needs application_id at startup again, as on main, and every checkpoint
call goes by the configured application and checkpoint ids.
Removes enable_routing_cache and fail_if_no_application, the
app_name_regex and resolve_application tracker calls, and the per-alias
routing cache. Restores main's startup check that a requested pre_call or
post_call hook has its checkpoint id.
---------
Co-authored-by: Shalom Jamil <shalom@ovalix.io>
* fix(ui): point the Spend Per entity docs link at the cost tracking page
* fix(docs): replace the remaining docs/proxy/enterprise links with current pages
* revert(terraform): leave the team tags docs link for a code-owned follow-up
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Deletes 17 exported calls that nothing references anymore. The Old Usage page removal left 10 behind, the move of agent traces to the Lens service left 5, and modelInfoV1Call and getUISettings lost their last callers in earlier refactors
Seven names that are only used inside the file stop being exported. buildModelGroupTestRequest stays exported because networking.test.ts calls it directly
Also drops the two test mocks that still stubbed deleted calls and prunes the lower eslint suppression counts for the file
* test(e2e): add gemini tool call id length repro for LIT-8912
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(gemini): add thought_signature_in_tool_call_id to keep tool call ids short for clients that cap them
* fix(gemini): pair a tool result with its call when the client re-adds the thought signature to the call id
Strands, which the AgentCore harness runs, puts provider_specific_fields.thought_signature back on the
assistant tool call id as <id>__thought__<signature> while its tool result keeps the bare id, so the Gemini
conversion found no matching call and answered 500. Ids now match with the signature suffix stripped.
Also adds thought_signature_in_tool_call_id to the litellm_params inventory test and gives the pass-through
tests' mocked logging objects the litellm_params a real Logging object always has.
* fix(gemini): keep the thought signature on streamed Messages and Responses tool calls and accept a string setting
* fix(gemini): read the thought signature back from Responses input and coerce the per-request setting
A Responses function_call input item now keeps its provider_specific_fields, so a
client that sends the item back gets its real thought signature replayed to Gemini.
A per-request thought_signature_in_tool_call_id of "false" or 0 now keeps ids short,
and a value that is not a boolean answers 400, or is dropped under drop_params,
instead of a 500.
The e2e test now uses the sdk fixture and @meta subjects
* fix(caching): key the response cache on thought_signature_in_tool_call_id
* fix(responses): send thought signature fields only with short ids and drop malformed routing kwargs whole
* fix(main): validate routing kwargs after the locals snapshot so fallbacks and model_list calls keep working
The e2e test now uses gemini-3.8-flash with enough max_tokens to finish thinking before the tool call.
* fix(gemini): keep non-chat routes tolerant of a malformed thought_signature_in_tool_call_id, match tool results by exact id first, and keep the deployment's value when drop_params drops a malformed override
* fix(router): validate strict litellm params through a helper so star imports cannot re-bind a public Final
* perf(router): skip strict validation when a request overrides no deployment param
* fix(responses): keep a dict tool call's thought signature on the streamed function_call done item
* refactor(main): return invalid supplemental params as a value and test the streaming tool call with a real response
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(anthropic): map pause_turn and return refusal explanations on chat and responses
pause_turn now maps to finish_reason length (Python and Rust) and keeps the raw
stop_reason in provider_specific_fields. Responses reports a paused turn as
incomplete with max_output_tokens and shows the unrun server search as an
in_progress web_search_call instead of a client function_call.
A refusal's stop_details.explanation now fills message.refusal and the final
stream delta.refusal on chat, and becomes a refusal content part on Responses,
streamed or not.
* fix(anthropic): let a chat client resend a paused turn so Claude continues it
* fix(anthropic): replay a paused chat turn without its thinking blocks, and drop its unrun search before a new message
Chat cannot say where Claude wrote each thinking block in a paused web search turn, and
Anthropic answers 400 when one moves but accepts the turn without them. A new message after
a paused turn also 400s on a server_tool_use with no result, so that unrun search is dropped.
* fix(responses): send the streamed refusal part's type and text in content_part.added
* fix(anthropic): keep pause_turn and Claude refusal explanations out of the Messages adapter's text
A paused Claude turn now ends on stop_reason pause_turn instead of max_tokens, streamed or not. A Claude refusal keeps its explanation in stop_details with Claude's category and no longer repeats it as a text block, so a guardrail mask on a streamed refusal is delivered. A paused turn sent back with list content drops its thinking blocks like string content does
* fix(anthropic): replay a paused turn with one unrun search and no thinking on chat and Responses
* fix(responses): report a paused Claude turn as completed so Codex and the Agents SDK keep going
* fix(responses): mark a paused turn's message item completed
* fix(responses): mark every item of a paused turn completed
* fix(responses): keep a refusal's explanation when a refused turn is sent back
A refusal part in a Responses input message now goes to the model as text, so the assistant turn is no longer dropped from the history. The integration test now expects a paused Claude turn on Responses to report completed, and the paused-turn test helper is typed
* fix(responses): scan refusal parts in Responses guardrails
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat: add Hostinger Router as an OpenAI-compatible provider
Adds hostinger_router (https://router.hostinger.com/v1) to the
JSON-configured openai_like providers with chat completions, responses,
messages, and embeddings endpoints, cost-map entries for claude-sonnet-5,
gpt-5.2 and the two text-embedding-3 models (credits converted at
1 credit = $0.01, sourced from the public
https://router.hostinger.com/v1/models catalog), the endpoint-support
matrix, Admin UI add-model form fields and logo, and respx-mocked unit
tests covering all declared endpoints.
Also routes /v1/embeddings for JSON-configured providers that declare it
in providers.json; previously embedding calls for these providers fell
through to LiteLLMUnknownProvider.
* fix(hostinger_router): match prices to the live catalog, drop retired max_tokens, regenerate the endpoint matrix
* fix(hostinger_router): keep api_base detection off, send embeddings headers as HTTP headers, drop generic key fallback
Unprefixed models called with Hostinger's api_base keep their own provider, JSON provider
embeddings no longer borrow OPENAI_LIKE_API_KEY or litellm.api_key, and the openai_like
embedding handler sends extra_headers as request headers instead of in the JSON body
* fix(openai_like): route Hostinger messages and responses per model, keep the configured key header
Native /v1/messages and /v1/responses calls for a JSON provider now also follow the
model's cost-map supported_endpoints, so Hostinger models without the route go
through chat completions instead of a 403. Embedding key headers replace
extra_headers case-insensitively, and pinstripes stops declaring /v1/embeddings.
* fix(openai_like): let embeddings extra_headers override the key's headers as chat does
Embeddings now match chat through the OpenAI SDK, so an extra_headers entry replaces the key's header of the same name in any letter case. The Hostinger capability test checks that the supports_* lookups agree with each cost-map row instead of pinning vendor flags
* fix(openai_like): send numeric embeddings extra_headers as text and keep Hostinger's 1h cache TTL
A config.yaml header like X-Org-Id: 12345 loads as an int, which the embeddings header check rejected with a 500. Int and float values now go out as their decimal text. Hostinger Router accepts and bills a 1h cache_control TTL on /v1/messages, so its provider entry opts into keeping it.
* fix(openai_like): send Hostinger's 1h cache TTL as the 5-minute default so cache writes bill at its charge
* fix(openai_like): route Hostinger natives per model from providers.json, refuse non-text embeddings headers unless drop_params
* fix(openai_like): refuse Host, Content-Length and Transfer-Encoding in embeddings extra_headers unless drop_params
* fix(openai_like): refuse malformed embeddings extra_headers and ignore transport headers
Header names must be HTTP tokens and values printable ASCII on one line, so a
colon in a name or a line break in a value answers 400 unless drop_params drops
it. Host, Content-Length and Transfer-Encoding are ignored, since a 400 for them
sends the call to the deployment's fallback with the same headers.
* fix(openai_like): trim embeddings extra_headers values and drop values HTTP cannot carry
HTTP ignores spaces and tabs around a header value, so values are trimmed
before they are sent. A value that is not printable ASCII on one line after
trimming is dropped with a warning instead of failing the call, since the
proxy adds headers itself from key aliases and forwarded client headers that
the caller cannot fix. A malformed header name or a non-text value still
answers 400 unless drop_params drops it.
* fix(openai_like): ignore hop-by-hop extra_headers and trim line breaks around values
TE, Upgrade, Connection, Keep-Alive, Proxy-Connection and Trailer in
extra_headers are now ignored like Host, Content-Length and
Transfer-Encoding, since litellm builds the request and TE: gzip fails
HTTP/2 while Upgrade makes some upstreams hang. Line breaks at the ends of a
value are trimmed along with spaces and tabs, so a secret read from a file
with a trailing newline is sent instead of dropped. The drop warning now
names the model.
* fix(embeddings): keep one request's extra_headers out of the deployment's headers and ignore Expect
embedding() merged a request's extra_headers into the headers dict it got
from the deployment's litellm_params. The router only shallow-copies those
params, so the merge wrote the caller's headers into the deployment itself,
and later embeddings calls through that deployment sent them too. Build a
new dict instead, and drop the None checks that can no longer be true.
Also treat Expect as a transport header that hostinger_router embeddings
never forwards, and name the model with %r in the dropped-value warning.
* fix(embeddings): keep name and value pairs working in headers and answer 400 for other non-object values
The previous commit merged headers and extra_headers with dict unpacking,
which only takes a mapping, so a list of name and value pairs that
dict.update used to accept now raised a TypeError and the proxy answered
500. Merge through dict() so pairs keep working, and answer 400 for any
other value, or drop it under drop_params, as the litellm convention does
for an unsupported param.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* build(deps): bump pyasn1 from 0.6.4 to 0.6.5
Bumps [pyasn1](https://github.com/pyasn1/pyasn1) from 0.6.4 to 0.6.5.
- [Release notes](https://github.com/pyasn1/pyasn1/releases)
- [Changelog](https://github.com/pyasn1/pyasn1/blob/main/CHANGES.rst)
- [Commits](https://github.com/pyasn1/pyasn1/compare/v0.6.4...v0.6.5)
---
updated-dependencies:
- dependency-name: pyasn1
dependency-version: 0.6.5
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
* build(deps): bump pyasn1, pypdf and minimatch to clear osv-scan
pyasn1 0.6.4 -> 0.6.5 and pypdf 6.21.2 -> 6.21.4 in uv.lock, with a dated
exclude-newer-package cutoff that admits pypdf 6.21.4 before it clears the
3-day window. minimatch has no fixed 10.x release, so the dashboard's override
and a new vscode-extension override move it to 11.0.2.
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(ui): resolve auto router templates through ids a * catch-all lists over a partial wildcard
* fix(ui): resolve wildcard-listed ids by their routed model in the Simple view and named aliases
A catch-all's fixed model or base_model no longer keys every listed id as that one model, a named alias like claude/* over anthropic/claude-* keys its ids by the model the router sends them to, and the Simple view suggests a pair from those ids too
* fix(ui): credit a wildcard-listed id only through the pattern and provider the router uses
When several named wildcards match a listed id, only the one the router's
specificity sort picks is credited. A captured route counts only when its
provider is one /model_group/info reports for the group, so an alias that
points elsewhere keeps its own name. Paused catch-alls credit nothing, and
a name with its own text before the star captures like the proxy lists it.
* fix(ui): resolve model group aliases before wildcards in auto router templates
An alias that shadows an id the catch-all lists now counts as the model the router sends it to, and paused deployments credit no template model
* fix(ui): wait for aliases, read them as admin only, and skip paused wildcard routes in auto router templates
* fix(ui): let slashless wildcard model names take part in route precedence
The router puts every model_name holding a star into its pattern list, with
or without a slash, so a listed id like opus-5-5 under a "*" catch-all goes
to a more specific opus-* deployment first. The dashboard skipped slashless
names when it picked the route, so a paused opus-* still credited opus-5-5
through the catch-all and a template could fill a tier that answers 403.
The old "derives no pattern from a slashless wildcard model_name" test
described /models listing, not routing, and is replaced by the routed case.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(proxy): pin admin-owned team keys to their team's MCP servers and toolsets
A virtual key on a real team owned by a proxy_admin or proxy_admin_viewer, with no MCP list of its
own, currently reaches every MCP server in the registry and any toolset, ignoring its team. These
tests expect it to resolve only what its team grants, like the same key owned by an internal user,
while the admin dashboard session keeps the whole registry.
* fix(proxy)!: bind admin-owned team keys to their team's MCP servers
A virtual key on a real team now resolves MCP servers and toolsets through its team even when its
owner is a proxy_admin or proxy_admin_viewer, the way team model access already binds it. The
unscoped admin view stays for the master key, admin keys with no team, the admin dashboard
session, and admitted user subjects. The toolset grant checks on /toolset/{name}/mcp and on the
Responses litellm_proxy MCP tool use the same rule.
BREAKING CHANGE: a team key owned by a proxy admin or admin viewer no longer reaches MCP servers or
toolsets its team does not grant. Grant them to the team or use a key with no team.
* fix(proxy): answer 403 for a toolset the key cannot use on LLM routes
A litellm_proxy MCP tool on /v1/responses, /v1/chat/completions or /v1/messages that named a toolset
the key's team was never granted was skipped, which left the server filter empty and handed the model
every tool the key could reach. The denied name now stays in the filter, so the request gets the same
403 an unknown server name gets.
Admin credentials on a real team stay bound to it whatever carries them, a lite login session included,
the way the team already bounds their model access. The tests pin that and drop an unused patch.
* fix(proxy): keep a denied toolset out of the scoped key beside a granted one
* test(responses): cover toolset name fallbacks and align MCP admin docstrings
* test(mcp): cover no-team admins' toolset access on LLM routes and drop a dead admin-check clause
* fix(responses): keep a whole-gateway MCP entry serving the key's tools beside a denied toolset
* test(responses): check the tools a whole-gateway request lists beside a denied toolset
* test(responses): keep the toolset lookup real in the whole-gateway listing test
* test(responses): run the toolset gateway tests on a real MCP server manager
* fix(responses): keep denied toolset names out of the MCP server filter
A denied toolset named like a server the key holds was matched to that server
by the loose server-name lookup. Denied names now never reach the filter, and a
request naming only denied toolsets answers 403 directly, each name once
* fix(responses): let a held MCP access group outrank a same-named ungranted toolset and give denied toolsets the generic 403
* test(responses): pin access group and granted toolset precedence for LLM-route MCP names
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Moves the page-authored English on /policies, /tool-policies and
/guardrails-monitor into react-i18next t() calls backed by en.json, and
adds a small data test for en.json. The rendered English is unchanged.
Runs one tool-calling conversation per selected endpoint (/v1/chat/completions,
/v1/responses, /v1/messages) against a saved deployment or unsaved litellm_params
and reports conversation, prompt_caching and cost_tracking verdicts, including
whether LiteLLM_SpendLogs rows landed
* test(ui): cover separate Meta Model API provider onboarding
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): add Meta Model API provider onboarding
Co-authored-by: lostmartian <sahilgangurde08@gmail.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): align Meta coverage with provider keys and listed registry
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show Meta model placeholder
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: lostmartian <sahilgangurde08@gmail.com>
* feat(keys): allow assigning a project to an unassigned key on /key/update
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(keys): suppress LIT002 on project assignment error payloads
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(keys): format project assignment error detail
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(keys): assert accepted project assignment flows into update payload
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(keys): drop stale LIT002 suppressions removed by main
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(keys): mock writer_db team lookup used by check_db_only path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(keys): write project assignment only while key row is still unassigned
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(keys): satisfy type gate on guarded project assignment write
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(keys): default soft-budget update path to unguarded write
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(keys): run guarded project assignment in one writer transaction
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(keys): include object permission link in assignment guard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(keys): return permission-linked payload without mutating the update dict
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(keys): fix type errors and stale noqa after main merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(keys): use Sequence in assignment guard model annotation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(keys): type update fields as a mapping in the key update path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(keys): place cast reason on the cast call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(keys): honor explicit null max_budget when assigning a project
* test(keys): assert explicit null max_budget flows into update payload
* fix(keys): match NULL models column in guarded project assignment
* fix(keys): authorize caller before project lookup on /key/update
* fix(keys): drop explicit null object_permission before project assignment write
---------
Co-authored-by: jesus <jesus@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): show ai&'s name and logo in the dashboard
The Add Model form already lists ai& from its provider metadata, but the dashboard had no entry for it, so the provider showed no logo and model tables printed the raw aiand slug. Adds ai& to the dashboard provider list with its official mark, plated in dark mode so the red and black mark keep their colours, and the aiand/openai/gpt-oss-120b model placeholder
* fix(aiand): retire deepseek-v4-flash, glm-5.2 and kimi-k2.7-code
ai& retired these three models on 2026-10-27 and https://api.aiand.com/v1/models now lists ten. The rows keep their prices with a deprecation_date, so earlier spend still prices, and aiand/* stops listing them, the same way baseten handles retired rows. The other ten rows already match the live prices, context windows and output limits
* feat(aiand): add deepseek-v4.1-flash-1017
ai& launches deepseek-ai/deepseek-v4.1-flash-1017 on 2026-11-02. It is served on https://staging-api.aiand.com/v1/models with the same price, context window, output limit and capabilities as deepseek-v4.1-flash, so its row copies that one
* fix(aiand): retire deepseek-v4.1-flash
ai& retires deepseek-ai/deepseek-v4.1-flash on 2026-10-30, now that deepseek-v4.1-flash-1017 is live in production on https://api.aiand.com/v1/models with the same price, limits and capabilities. Its row keeps its prices with a deprecation_date, like the other three retired ai& rows
* fix(aiand): retire deepseek-v4.1-flash on 2026-11-06
https://api.aiand.com/v1/models now gives deepseek-v4.1-flash a deprecation_date of 2026-11-05T15:00Z, which is 2026-11-06 00:00 in Japan, so its row uses 2026-11-06 instead of 2026-10-30 and aiand/* keeps listing it until then. The wildcard test now leaves a model out only once its retirement date has passed
* test(aiand): derive retired models from the rows instead of pinning dates
The wildcard test now reads each row's deprecation_date, so it holds whichever dates ai& sets, and the test that only restated the JSON dates is gone
* fix(aiand): retire models on their UTC date while the proxy runs, and add muse-spark-open
The wildcard listing now checks each row's deprecation_date when it expands, against the UTC date the field
documents, so a model drops on its day without a restart. deepseek-v4.1-flash now carries 2026-11-05, the
UTC day ai& retires it, and meta/muse-spark-open gets its price row from ai&'s live /v1/models
* fix(aiand): hide retired models only from bundled listings, including the wildcard health check
* fix(aiand): share the UTC retirement check with Together AI and the health check sort
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(streaming): normalize provider logprobs to prevent MockValSer serialization crash
* fix(streaming): convert upstream chunk logprobs to litellm ChoiceLogprobs so the finish chunk serializes
A streaming chat completion with logprobs on the OpenAI SDK path attached the
SDK's own ChoiceLogprobs object to the outgoing chunk when the chunk had no
content to rebuild from, which is the provider's finish chunk (empty delta,
logprobs for the end token, finish_reason). The SDK parses stream chunks
without building that nested class, so its pydantic serializer is still a
MockValSer and model_dump_json fails. The proxy then sent an error frame in
place of the finish chunk, and when the same chunk also carried usage the SDK
wrapper itself raised MidStreamFallbackError.
The attach site now validates upstream logprobs through a TypeAdapter that
dumps any non-litellm pydantic model before validating into litellm's
ChoiceLogprobs, and passes litellm instances through unchanged.
* fix(streaming): keep upstream logprobs that don't fit ChoiceLogprobs as plain JSON so the stream still finishes
* fix(streaming): build ChoiceLogprobs' serializer up front and keep nonconforming logprobs as JSON on chunk rebuilds
ChoiceLogprobs inherits the openai SDK's deferred build, so a fresh worker
whose first logprobs arrive on the finish chunk still had no serializer.
Foreign logprobs models now dump in full, so a missing top_logprobs becomes
[] and SDK callers keep attribute access, and chunk rebuilds for content and
usage keep logprobs that don't fit as plain JSON instead of raising.
* fix(streaming): keep only the logprobs keys upstream sent and type refusal logprobs
* fix(types): pass nonconforming non-streaming logprobs through instead of failing the response
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(router): apply content_policy_fallbacks when router settings update at runtime
* test(router): annotate the fallback regression test's locals as Final
* fix(proxy): validate router_settings.content_policy_fallbacks on /config/update like its siblings
* test(proxy): annotate the content_policy_fallbacks config update test locals as Final
* fix(proxy): keep litellm_settings content policy fallbacks, accept dict targets, skip invalid stored rules
A runtime content_policy_fallbacks rule now layers over the litellm_settings rules instead of replacing
them, so models the runtime rules do not name keep their file fallbacks. Fallback targets may be
deployment dicts as well as model names. A stored rule that fails validation is logged and skipped at
sync, so the other router settings still apply. Per-user token discovery now reads every fallback key
from key and team router_settings.
* fix(proxy): validate content policy fallback rules one by one and keep a config-owned list alone
An invalid stored rule is now skipped on its own instead of dropping every runtime rule, a router_settings list from the config file stays the router's only rules across DB syncs, and GET /fallback names the model of a dict target.
* fix(proxy): answer an error for a string-valued fallback rule on GET /fallback instead of its characters
* fix(proxy): keep config-file content-policy rules as Router init accepted them and log a skipped rule once per change
* fix(proxy): log a skipped content-policy rule whenever the saved rules change
* fix(proxy): keep general fallbacks when no content-policy rule is saved, name a model-less target's model, and take only model names in key rules
* fix(proxy): reject an empty dict target in a key's content-policy rule
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
