Moves the English left in the Aria assistant, the MCP tool permission
groups, the shared chat composer, the Beta badge, the user banner and
the metadata key fields onto react-i18next. English output is unchanged.
Text that Aria sends to the model (system prompt, tool descriptions,
tool results and usage chart payloads) stays English byte for byte.
Where the same string is also shown on screen, a display map keyed by
the operation name or by the chart's English value translates it at
render time, and the model copy is left alone.
Compliance framework and category names, the Compare endpoint labels, the
Decisions presets and schema messages, chat metrics, MCP events, reasoning
toggle, conversation date groups, model filters and the MCP boolean argument
labels now resolve through en.json. Ids, stored values and text sent to a
model are unchanged
Constants and option lists on the keys, teams, agents, budgets, cost
tracking, router settings and vector store pages now hold message keys,
and the code that renders them translates. Values, ids, sort keys and
payloads are unchanged, and lookups keyed by server values check
Object.hasOwn first
Five tests now resolve the expected English through i18n.t instead of
reading the constant directly
Moves the leftover English in the sidebar, page descriptions, account
menu, usage card and a handful of shared widgets into en.json. Option
lists and label maps now hold message keys and are translated where they
render, while every option value, id and stored setting stays the same
string. Lookups keyed by a server value check Object.hasOwn first
* refactor(ui): move remaining models text onto i18n
Moves the leftover English on the models pages, mostly labels and hints
held in constants, onto react-i18next. Constants now hold message keys and
the code that renders them translates. Where a string is both a value and
a label (tier names, health check sentinels), the English value stays for
comparisons, sorting and payloads, and a separate label key drives display.
Lookups keyed by server values check Object.hasOwn first.
* refactor(ui): translate the tier name in display name inputs
Logs and usage option lists, filter chip labels and label maps now hold
message keys that their readers translate at render time. Tile titles, the
"Claude (unspecified)" agent label and the endpoint chart categories stay
English ids and get display maps keyed by them. Lookups keyed by URL or
server values check Object.hasOwn first.
* fix(accounting): keep one undecodable delivery from blocking the ledger
A request whose metadata carried a 401-digit tokens_saved stored that number in
its delivery payload. The query engine could not decode it when the worker
claimed the batch, so every later delivery on the proxy stayed pending
The worker now claims the payload as text and decodes each row in Python. A row
that still cannot be decoded is parked alone while the rest of the batch
applies. The payload builder also clamps integers to the JSON-safe range, so
caller-supplied counts can no longer reach the engine at that size
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): take spend locks in one order so the ledger cannot deadlock
A ledger delivery updates many spend tables in one transaction, so it can
deadlock with any writer that locks the same rows in another order. It took the
team advisory lock after the user and team rows, while team roster writers take
it first
The delivery now takes every team advisory lock before any row, then writes
tables in one canonical order with ids sorted. It also runs under the rollup
lock_timeout, applies 500 deliveries per batch by default, and spreads retries
with jitter
The writers that could still collide with it now follow the same order. The
budget reset cascade, model allowlist rename, global spend reset, bulk user
update, org delete and the delete-by-list paths lock their rows in key order
first, and access group endpoints update teams and keys in one sorted pass
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(accounting): prove a ledger delivery waits instead of deadlocking
Real Postgres tests hold the first lock of a team roster change, an access group
update and a budget reset cascade, start a delivery, and let the other writer
finish. The delivery must wait and then charge once. Before the lock order fix
each case ends in a Postgres deadlock
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(accounting)!: make the SpendLogs ledger the default accounting path
use_spend_logs_ledger now defaults to true, so every request is charged through
LiteLLM_SpendLogDelivery. Setting it to false rolls back to the in-memory and
Redis spend queues, which stay in place. Tests that cover those legacy writers
pin the setting off, and the rest check the queued charge the ledger stores
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(proxy): let the mocked client open a transaction for key and team deletes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(proxy): take team locks first when a model rename rewrites allowlists
A rename locked team rows before user rows while a roster change holds the team
advisory lock and a user row, so the two could deadlock. The rename now takes the
advisory lock for every team that lists the model, sorted, before any row lock.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): drain a backlog in one worker run, one batch per transaction
One run applied a single fresh batch, so at 500 per batch charges fell behind any
load above that rate. The worker now keeps claiming fresh batches while they come
back full, for up to five seconds, and each batch still commits on its own.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): queue only the charge when a request's spend log is suppressed
With logging off for a request, the ledger still queued the full row, so a failed
flush or a shutdown could leave its prompt and response in Redis. The queue now
holds only the fields the charge needs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* style(proxy): format the rename team listing query
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): remove expired delivery receipts as fast as the worker applies them
The worker now applies several batches per run but still removed one cleanup batch
of applied receipts, so the table could outgrow its retention window. Cleanup now
repeats while each pass removes a full batch, for up to five seconds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(proxy): retry a model rename when a team starts listing the model mid-lock
The team advisory locks come from a snapshot, so a team that gained the old name
before its row lock was locked without its advisory lock. That attempt now commits
without writing and the next one locks the fresh set of teams.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat: opt-in model deprecation emails for affected owners
* fix: annotate deprecation sender initialization
* fix: clarify deprecation emails and use consistent settings toggles
* fix: exclude former team admins from deprecation emails
* feat: clarify provider deprecations and model migration steps
* feat(ui): show model deprecation banner on the models page
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(ui): format model deprecation banner
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): refresh model deprecations when models change
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): address review feedback on model deprecation banner
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): cover deprecation role gate and agent builder refresh
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): separate deprecation banner toggle from alerts link
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): dedupe and allow dismissing the model deprecation banner
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): refresh table once and key banner entries on successor
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: quiet successor logs, count banner models, use local dates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): pin banner test date and bucket upcoming entries locally
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: cover mixed-successor log level through the deprecations endpoint
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): move deprecation banner cases into integration tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): keep deprecation banner when a refetch fails
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The key edit form read react-hook-form's dirtyFields only inside the submit
handler. formState is a proxy that only tracks what the component reads while
rendering, so on the first save of an edit session that read returned the
initial empty object. A team admin editing a member's key then sent only the
key id to /key/update, got a 200 and a "Key updated successfully" toast, and
nothing changed: a max_budget edit was silently dropped, and a rename went
through as an empty update instead of the "team admins can only change budget
fields" error.
Read dirtyFields during render so the form subscribes to it and the submit
handler sees the fields the user actually changed.
* feat(accounting): add opt-in SpendLogs ledger delivery
* fix(accounting): preserve missing organization daily spend parity
* fix(accounting): preserve ledger receipts when replaying migration
* fix(accounting): preserve callback windows and tune ledger batches
* fix(accounting): preserve window charges and bound ledger statements
* test(accounting): verify budget callback registration order
* test(accounting): verify captured charges after repeated startup
* feat(accounting): store durable payloads independently of spend logs
* fix(accounting): preserve first delivery and trace ledger writes
* fix(accounting): keep SDK imports independent of proxy tracing
* fix(accounting): honor log retention during batch takeover
* test(accounting): cover retained batch alerts and invalid settings
* test(accounting): verify downgrade command against pending work
* fix(accounting): drain retries without starving fresh deliveries
* test(accounting): cover delivery projection, metrics and window key reads
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(accounting): read delivery metrics without the shared registry
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): charge a leftover $0 batch row when its claim fails
A failed batch claim buffers the retrieve's row and reports it charged. At flush the
row conflicted with the pre-upgrade $0 SpendLogs row, so its delivery was rolled back
and the charge never reached the spend tables. The flush now takes that row over in the
same transaction and keeps the delivery.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): trace the batch takeover at flush as a SpendLogs update
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): keep local spend logs when SPEND_LOGS_URL has no export client
The export client is never created, so main always writes SpendLogs to the proxy's own
database even with SPEND_LOGS_URL set. The ledger skipped local rows whenever the URL was
set, which dropped those request logs. It now treats a flush as an export only when the
client exists, and batch claims write local rows as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): keep every request's charge when its log row is shared, evicted or rejected
- Key non-batch deliveries on a per-request delivery_id so a reused client call id is still
charged once per request, while buffer replays stay deduplicated
- Persist the redacted api_key instead of an opaque bearer token
- Keep the accounting payload when the spend-log queue evicts a row or Postgres rejects it
- Strip NUL from payloads and skip non-text tags so projection never parks a paid request
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): isolate a raw-SQL spend log row Postgres rejects instead of requeueing the batch
A rejected row on the ledger insert path raises RawQueryError, which the exact DataError check missed, so the whole batch was requeued every flush. Data and integrity SQLSTATE classes now bisect like before, while missing tables and deadlocks keep retrying
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): keep charging a request whose tag is too long for the daily tag index
A tag past Postgres's btree entry limit failed the DailyTagSpend upsert, and on the ledger path that rolled back the request's key, team and user charges with it. Such tags are now left out of the daily tag rollup only
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): store only guardrail billing fields in the accounting payload
The ledger payload copied whole guardrail records, including responses and match details, into LiteLLM_SpendLogDelivery even when spend logs were disabled. It now keeps the id, name, usage and cost fields the projection reads. NUL stripping reuses safe_json_structure instead of a recursive walk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(proxy): move spend payload stamping out of update_database
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(proxy): restore the modules the import-layering guard evicts
The guard popped litellm.proxy.utils from sys.modules for good, so later tests on the same xdist worker patched a fresh module while calling functions bound to the old one, and spend queue tests failed depending on scheduling
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(proxy): keep update_database's payload stamping on its own local
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(accounting): type guardrail entries and use a public key redaction helper
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): keep charging requests whose end-user id is too long for an index
An oversized end-user id failed the SpendLogs insert with SQLSTATE 54000, which was requeued forever, and failed the end-user upsert in the delivery, which parked the key and team charge with it. Class 54 rows are now isolated like other rejected rows, and such ids are left out of the end-user rollups only
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(proxy): type the spend log queue snapshot
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): leave out any projected row Postgres cannot index or store
Caller-supplied ids and counters, such as a long tag or end-user id or an out-of-range compression count, failed the delivery transaction and rolled back the request's key, team and user charges. One check in project() now drops only the row that cannot be stored, replacing the per-field cutoffs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): keep the guardrail fields compression savings are counted from
Slimming the guardrail records dropped the provider and the tokens_saved figure that Headroom compression savings are read from, and dropped a bare-dict record entirely. The payload now keeps both while still leaving out guardrail content
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): cap projected counters so daily totals cannot overflow
Counters such as a caller-supplied compression count fit one by one but overflowed the bigint daily total once summed across requests, rolling back the key, team and user charges in the same transaction. Each counter is now capped at 2^40, far below any total that could overflow
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): let a buffered batch charge take over its zero-cost receipt
A batch charge queued after a failed lookup hit the $0 receipt an earlier poll left behind, and ON CONFLICT dropped it. The buffered insert now runs the same receipt takeover as a live retrieve
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): read a rescued batch row's spend from its accounting payload
Rows rescued from an overflowing queue keep only the accounting payload, so the buffered takeover never saw their positive spend and left the $0 receipt uncharged
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): read budget windows through the delivery transaction
Window seed and existing-row reads used a separate connection while the delivery transaction held one, so with a one-connection pool they waited on their own transaction until it timed out. Inside an enclosing transaction they now share its connection
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): retry a transient failure while keeping a dropped row's charge
The rescue swallowed every error, so a database blip right after a row was rejected lost the unpaid charge. Only a rejected row is still logged and skipped, anything else propagates so the flush retries
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(accounting): give delivery projection payloads their charged entities
project() now reads entities only from accounting_entities, so the hand-built
delivery payloads carry them like real stored payloads do.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(accounting): own ledger rows by their payload and key batches on the batch id
Drop the accounting_path stamp and its SpendLogs column. A buffered row that
carries an accounting payload is ledger owned, one without it is legacy, and
spend counters get a plain windows_from_ledger flag. Window seeding tells
applied ledger rows apart by their delivery row.
A ledger batch is plain transaction A keyed by the batch id, so a repeated
completed retrieve conflicts and charges nothing. A $0 retrieve records no
delivery, a batch a legacy retrieve already charged is not charged again, and
the legacy batch path is unchanged from main apart from refusing a batch the
ledger already owns. Batch takeover and its delta bookkeeping are removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(accounting): drop a projection test duplicated while rebasing
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(accounting): drop previous_payload now that deliveries are never replaced
Batch takeover was the only writer of previous_payload, so the worker applies
each delivery's payload as stored and the column leaves the migration.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(accounting): drop the ledger downgrade checklist script
The use_spend_logs_ledger flag is the rollback path, so a separate downgrade
gate is no longer needed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): name the window seed reads so their Prisma span resolves
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(accounting): remove the unused accounting event types and protocols
types.py, protocols.py and measurements.py were left from the old event
scaffolding. Nothing outside the package imports them, so they go with their
tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): park only deliveries whose own data fails
A delivery that keeps failing on a database outage, lock timeout or deadlock
stays pending and retries with capped backoff, so a recovered database applies
it. Only a data error or an invalid payload parks the isolated row.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): isolate a spend log row the insert statement rejects
The raw SpendLogs insert validates columns and JSON before Postgres sees the
row, so a row that fails there is now bisected out and dropped alone while its
charge is kept, instead of failing and replaying the whole flush.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): carry a job's model window on every completed poll in ledger mode
The ledger keeps the poll whose delivery claims the job, which need not be the
poll that won the limiter's in-memory charge. Every completed poll now carries
the job's window, so the delivery that lands always has it, while the live
model budget is still charged once.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(accounting): leave model-budget windows on the existing window writer
Model-budget windows are written by the model budget limiter rather than
derived from a request's spend, so they stay on main's window queue like live
budget counters do. This drops window capture from the accounting payload,
the deferred limiter registration and the job-window capture, and the ledger
keeps owning key and team budget windows.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): keep a delivery pending when the database is unreachable
Prisma reports a can't reach database server outage as a DataError, so the
worker now checks for an unavailable database before treating a data error
as the delivery's own fault.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): isolate only rows the raw insert cannot encode
Catching every ValueError also swallowed unexpected insert errors that main
re-raises. The raw insert now raises RejectedSpendLogRow for an unknown column
or non-JSON value, and only that is isolated as a rejected row.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): store zero savings when caller data makes them uncomputable
A huge caller-supplied tokens_saved overflowed the savings math inside
build_accounting_payload, which dropped the request's charge in ledger mode.
Savings that cannot be computed are now recorded as zero and the charge lands.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Every string the Models + Endpoints, Model Insights and Transform Request
pages write themselves now goes through t() or Trans, including the shared
auto-router settings those pages render on other routes. 1,651 English
messages are added under 28 new top-level keys in en.json.
This is a pure text move. The English on screen does not change, option
lists and constants keep their shape, and text that could not move without
restructuring stays English. ModelInfoEditForm.tsx, ComplexityRouterConfig.tsx
and build_complexity_router_config.ts grew past the 800 line limit and get
max-lines entries in eslint-suppressions.json instead of being split.
Move batch B1 of the admin dashboard onto react-i18next: the dashboard
shell (navbar, command palette, deployment health, header, layouts), the
entry pages (login, onboarding, change password, home, API reference,
UI theme, workflows), the public AI hub and the AI hub table, the shared
components and helpers in src/components/shared, common_components,
networking toasts, the toast titles and the shadcn primitive labels.
Pure text move into t() and Trans with 1,056 new en.json keys. The
English on screen is unchanged.
* fix(guardrails): check Anthropic-format request tools in tool_permission pre_call
The pre_call hook only collected tools shaped as {"type": "function", "function": {...}},
so flat tools from /v1/messages ({"name", "input_schema"}) and /v1/responses
({"type": "function", "name"}) were never checked and deny rules silently passed.
Reuse the shared anthropic_tool_names helper so every request tool name is checked,
and handle Anthropic tool_choice {"type": "tool"} in both block and rewrite modes.
Fixes#40583
* fix(guardrails): detect Responses API named tool_choice in tool_permission
Responses API named choices put the name at the top level ({"type": "function", "name": ...}),
so rewrite mode removed the denied tool but left a tool_choice pointing at it.
* test(guardrails): run tool_permission pre_call tests through real default_on gating
Build the guardrail as an always-on pre_call guardrail instead of patching should_run_guardrail.
* test(guardrails): type the new tool_permission pre_call test parameters
* fix(guardrails): keep the Responses API custom tool type distinct
Anthropic's user-defined tools carry type "custom", so pre_call mapped
that to "function" for tool_type rules. The Responses API has its own
custom tool type, and mapping it let a custom tool slip past a
tool_type deny rule when the default action is allow.
Normalize only for the formats where custom means a plain function, and
read tool names directly in the rewrite path, which never needed a type.
* test(guardrails): cover flat tools through Messages and Responses
Add blocked and permitted flat-tool cases through the running proxy, with
endpoint coverage and metadata for the current e2e harness
Require a guardrail policy verdict when checking denied tools so an
unrelated provider error cannot satisfy the regression test
Co-authored-by: Christopher Krah <christopher.krah@plexito.de>
* fix(guardrails): check namespace and nameless tools and relax tool_choice in tool permission rewrite
* fix(guardrails): check forced, grouped, loaded, and hosted tools by their real type
* fix(guardrails): drop references to removed tools and count loaded tools in tool permission rewrite
* fix(guardrails): check tools sent outside top-level tools in tool permission
* fix(guardrails): judge namespace members by their bare name as a fallback and keep rewrites valid for Gemini, Bedrock and hosted tools
A namespace member is checked by namespace__name first, and by its bare name when no rule names the joined one. Rewrite mode now prunes Gemini allowedFunctionNames and turns off forced calling when no function is left, drops a removed hosted tool from allowed_tools, reads Bedrock toolSpec and systemTool blocks by name, skips cachePoint markers, and leaves a tools field that is not a list for the provider to reject instead of failing with a 500
* fix(guardrails): let only name rules decide a namespace member's bare-name fallback
* fix(guardrails): judge malformed tool types, empty Gemini name lists and pass-through Anthropic tools like the endpoints do
A tool or tool_choice whose type is not a string now reads as untyped instead of crashing the guardrail with a 500, and a null type key is never read as a Gemini tool part. An empty allowedFunctionNames list no longer turns off the functions a rewrite kept. An Anthropic custom tool reads as a function by its input_schema on every route, so a pass-through route judges it the same as /v1/messages. A Responses WebSocket frame now forwards the extra_body tools a rewrite stripped
* fix(guardrails): judge a named hosted tool by its type and keep a nested chat custom tool custom
* fix(guardrails): keep judging Anthropic client tools and type-named server tools by their name
* fix(guardrails): judge a named tool by its name alone only when the name is its type's own
* fix(guardrails): judge the function a tool names beside a declaration list or a top-level name
* fix(guardrails): judge Gemini hosted keys whatever the tool type and a namespace's function
* fix(guardrails): judge legacy functions entries like tools and send Gemini a namespace's members
* fix(guardrails): judge a namespace member by every name a provider reads and send Gemini its schema converted
* fix(guardrails): judge a legacy functions entry as a function and relax a tool_choice in extra_body
* fix(gemini): keep a namespace whole when inlining its members would repeat a function name
* fix(guardrails): judge a namespace member by every bare name that shares its qualified name
* fix(vertex_ai): declare a repeated namespace member name once instead of sending namespace names to Gemini
* fix(guardrails): judge a tool_choice sent in extra_body like a top-level one
A forced name only extra_body.tool_choice carried was never checked, so block mode let a denied name through to the provider
* fix(guardrails): judge a Gemini namespace member returned by its bare name and force a named namespace's members
* test: keep formatter churn out of the tool_permission and Gemini test files
* fix(guardrails): judge extra_body.input tools, argument-aware alias rules, and exact forced Gemini functions
Tools loaded in extra_body.input are now checked and stripped like top-level input. A returned
namespace member is judged by the alias whose first name rule its arguments satisfy. A forced
Gemini name that matches a top-level function keeps that function. Alias grouping and Gemini
declaration dedupe now run in one pass instead of quadratic scans.
* fix(guardrails): judge and rewrite every tool field sent inside extra_body
extra_body is merged into the upstream body, so functions, function_call,
mcp_servers and toolConfig nested there reached the model unjudged. Treat
extra_body as a second request layer that is collected and rewritten the
same way as the top-level request.
* fix(guardrails): judge Gemini hosted keys by key and count only the tools a provider receives after extra_body
* fix(guardrails): relax a tool_choice that names a dropped namespace member by its bare name
Gemini and Anthropic chat send namespace members under their bare names, so rewrite mode now treats a dropped member's bare name, and an emptied namespace's own name, as removed when no kept tool still carries it
* fix(gemini,guardrails): keep a top-level function over a same-named namespace member and leave malformed tool values to the provider
Gemini and Vertex chat now declare a top-level function over a namespace member that shares its name, so a forced call gets the top-level schema. post_call judges a returned name a top-level tool carries as that tool, and pre_call no longer raises on an unhashable or non-string tool name, input item type, or function_call name.
* fix(guardrails): judge a returned namespace member named like a hosted MCP server as that member
* fix(gemini,guardrails): send a namespace as main did when another tool has a member's name
Gemini chat inlines a namespace's members only when no other tool, namespace
member, or legacy functions entry declares the same name, and otherwise sends the
namespace by its own name as main did. A tool_choice naming such a namespace
forces the namespace itself. The tool permission post_call reads top-level names
the way Gemini declares them and judges a call the Responses bridge tags with a
namespace as that member.
* fix(gemini): send colliding namespace members as namespace__name functions and judge them in post_call
When a namespace member shares a name with another tool, Gemini chat now gets that namespace's members as namespace__name functions instead of one function named after the namespace, so every function it can call is a name the rules judged. Declared names are counted once, and a forced member maps to each holder's qualified name.
post_call judges a returned namespace__name as that member, by its qualified or bare name the way pre_call does, so a bare-name allowlist or denylist applies to it.
* fix(gemini): keep generated member names unique and let a top-level function own its exact name
A namespace that listed a member twice, or a generated namespace__name that
repeated a top-level function, sent Gemini two declarations with one name, and
Gemini answered 400 where main answered 200. A name a namespace lists twice is
now one member, and a top-level function keeps its exact name.
post_call now judges a returned name a top-level function declares as that
function, never as the namespace member the name could also mean.
* fix(gemini): qualify namespace members by the namespace's top-level name
The guardrail judges a member as `name__member` from the namespace's top-level
name. Gemini built the name from `function.name` when the tool also carried one,
so a request could pass pre-call and reach Gemini under a name no rule judged.
* fix(vertex_ai): force only the namespace members Gemini is sent
A forced namespace or member on Gemini chat could still allow a generated name that a top-level function or an earlier namespace owns, so tool_choice forced an unrelated function. The forced names now come from the same collision checks the declarations use
* fix(vertex_ai): reject a forced namespace Gemini gets no member of, and resolve forced names in one pass
* fix(guardrails): judge a namespace member by a rule on its namespace's name
---------
Co-authored-by: Christopher Krah <christopher.krah@plexito.de>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Moves the page-authored English on /playground, the /chat app and /moyai
into t() and Trans with keys in en.json, as a pure text move. The rendered
English is unchanged
Route the page-authored English on the Logs, Usage and Telemetry pages,
the log detail drawer and the shared usage widgets through t() and Trans,
with 1,250 messages added to en.json under 10 new top-level keys. The
rendered English is unchanged.
Moves the page-authored English on the virtual keys, projects, cost tracking,
cost optimization and budgets pages, plus the create key form and the shared
key and team pickers, into t() and Trans with 1,318 messages under 32 new
top-level keys in en.json. The English on screen is unchanged
key_edit_view.tsx grows past 800 lines, so it gets a max-lines entry in
eslint-suppressions.json instead of a split
* feat(decisions): add EmpirioLabs AI as a decisions provider
empiriolabs/aplomb-1 now works with litellm.decisions and litellm.systemone, the proxy /v1/decisions and /v1/systemone routes, the decision model guardrail and the Admin UI. Requests go to POST /v1/decisions on the EmpirioLabs AI API with the System One body, keyed by EMPIRIOLABS_API_KEY with an optional EMPIRIOLABS_API_BASE
Chat models keep routing through the existing OpenAI-compatible JSON provider, and a test pins that
* ci: run the EmpirioLabs AI decisions tests in the llms unit shard
* fix: align EmpirioLabs registration with CI gates
---------
Co-authored-by: Adam Dalloul <47503782+Adam-Dalloul@users.noreply.github.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(accounting): replace event scaffolding with SpendLogs projection
* test: preserve spend timestamps without removed capture callback
* fix: guard accounting event removal across schema update paths
* refactor(accounting): retain schema and prepare durable payloads
* fix(accounting): preserve dates in serialized payloads
* fix(accounting): retain schema push compatibility
* fix(accounting): retain span registration for the legacy schema
* test(accounting): remove obsolete capture acknowledgement test
* fix(accounting): date every daily projection row the way the model rollup does
A space-separated startTime kept the whole timestamp as the daily key, while the model usage rollup took the calendar date. Every daily row now uses the same date helper
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* chore(ui): regenerate dashboard API types after rebasing on main
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(accounting): project only from the stored accounting payload
project() now reads entities only from the payload's accounting_entities, so
the raw SpendLogs fallback branches are gone. build_accounting_payload() builds
the metadata once instead of dumping it to JSON and parsing it back. Guardrail
usage dates use a tolerant UTC parse, so a malformed startTime drops only the
guardrail rows instead of failing the whole projection.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(openai): bill gov.api.openai.com FedRAMP endpoint with 1.1x uplift
* chore(openai): trim FedRAMP uplift diff
* refactor(openai): bill the FedRAMP endpoint through its own fedramp_uplift_multiplier key
* fix(ui): regenerate fedramp_uplift_multiplier doc comment in schema.d.ts
* style: ruff format utils.py and reorder fedramp_uplift_multiplier in schema.d.ts
* fix(openai): infer data_residency from OPENAI_BASE_URL and litellm.api_base when api_base is unset
* test(cost): cover get_fedramp_uplift_multiplier fallback paths
* fix(cost): apply fedramp_uplift_multiplier in batch_cost_calculator
* fix(openai): derive data_residency from a supplied client's base_url
* test: type the supplied-client data_residency test params
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): rebuild dashboard navigation and Aria workspace
* refactor(ui): make Aria a single workspace and clarify dashboard controls
* fix(ui): preserve link semantics for the Aria shortcut
* style(ui): use Sparkle for Aria
* style(ui): simplify workspace menu rows
* fix(ui): keep worker controls compact on narrow headers
* style(ui): remove workspace selector dark fill
* feat(ui): rebuild the public AI Hub as a searchable list with a details pane
The public AI Hub at /ui/model_hub_table (and the older /ui/model_hub) now
renders PublicAiHub, built from shadcn parts only: a header with the
gateway's base URL and the admin's links, segmented tabs for models, agents,
MCP servers and skills, and a list beside a details pane on desktop that
becomes a bottom sheet on phones. The open item and tab live in the URL.
It keeps every feature the old page had: server-side search, provider, mode
and feature filters, sorting, paging, health, rate limits, prices, custom
metadata, supported params, usage snippets, the require-auth redirect and
the not-enabled state. cURL shows only for chat models, since the old one
called chat completions for every mode.
The navbar hides the community buttons below the sm breakpoint on every page,
and the public navbar uses a smaller logo on phones, so Docs and Blog no
longer collide with them.
* fix(ui): harden the public AI Hub after adversarial review
- Show "This AI Hub isn't available" only when every hub call fails; a
failed models call now shows an inline error with Try again and leaves
agents, MCP servers and skills usable (a gateway with no models answers
the models list with 400)
- Row links keep the page's other query params, so open-in-new-tab and
copy link no longer 404
- A shared ?open= link resolves models beyond the first page, keeps the
opened item when a search filters it out, and says when it is no
longer listed instead of showing a different item
- Searching, sorting and paging no longer blank the details pane or drop
the Operational badge
- Key MCP servers by their listed name, which is never null
- Keep sub-cent prices and show Free for zero cost, fold evaluation into
Decisions, sort agents, MCP servers and skills by name, restore
word-wise search, and show a skill's source, author and date added
- Drop the invented description for models without a mode, the empty
agent modes row, the stray auth dash on MCP rows and NaN dates
- Do not retry the UI config call, and close the 1023-1024px gap between
the sheet and the details pane
- Update the public hub Playwright spec for the new tabs and list, and
prune the lint suppressions the old route pages needed
* fix(ui): surface hub load errors and keep the open item steady
Agents, MCP servers and skills now load through apiClient so an HTTP error
shows the inline error instead of an empty "published yet" state, and an
all-failing gateway reaches the unavailable page. Try again shows loading
instead of an empty hub, the open model stays in the details pane when a
search or page change hides its row, a zero-result search keeps the pane,
phones get the not-listed sheet for a stale link, an empty open param means
nothing opened, free models read Free in the list, and an invalid skill date
is hidden
* fix(anthropic): answer a 503 as 529 overloaded_error on /v1/messages
Anthropic's error contract has no 503, so a client built against it (Claude Code) treats a 503 from /v1/messages as a plain 5xx and retries it ten times instead of taking its overload path. A Bedrock 503 now leaves /v1/messages and /v1/messages/count_tokens as 529 overloaded_error, for both a JSON answer and a stream that fails before its first byte.
An Anthropic 529 overloaded_error now maps to litellm.ServiceUnavailableError (503) instead of InternalServerError (500), so OpenAI-shaped routes answer it as 503, the status OpenAI documents for an overloaded engine, and router retry and allowed-fails policies key it as ServiceUnavailableError.
* test(proxy): expect overloaded_error on a 503 stream error after a keepalive ping
* fix(proxy): record the 529 the Anthropic routes answer and type the chat stream error frame
The messages route stamps the answered status on the exception before post_call_failure_hook runs, so the spend log row, the OTel server span, the failed-request metric, and alerting record the 529 the client saw instead of the deployment's 503, while the deployment failure metric and cooldown keep the deployment's own status. A chat completions stream error frame names the type its status stands for instead of the null .type ServiceUnavailableError carries. An Anthropic-shaped 503 envelope passes through as 529 overloaded_error through a validated TypedDict, and count_tokens answers every ProxyException as the Anthropic envelope. Five integration tests that pinned 503 on the messages route now expect 529
* test(proxy): expect the top-level Anthropic envelope from count_tokens errors
count_tokens now answers a ProxyException as a JSONResponse carrying the
Anthropic error envelope instead of raising HTTPException, which FastAPI
wrapped in {"detail": ...}. The three legacy error-format tests now read
the response body and check that no detail wrap is left.
* test(proxy): cover the 529 overloaded_error answer across routes, cooldown, and retry budgets
* fix(anthropic): keep Anthropic overloads as InternalServerError and answer 529 from the provider status
An upstream Anthropic 529 stays litellm.InternalServerError (status 500) so
InternalServerErrorRetries, InternalServerErrorAllowedFails, SDK handlers,
and the OpenAI-shaped routes behave as on main. The error now carries
provider_status_code, and the /v1/messages routes answer 529
overloaded_error from it.
* refactor(proxy): unwrap a mid-stream fallback error once instead of recursing for the provider status
* fix(anthropic): keep count_tokens' detail body, end keepalive streams with an Anthropic error event, and log any 529 as an overload
* test(integration): cover the audit round 3 cells and expect 529 on the count_tokens 503 cases
* fix(proxy): re-raise an unchanged stream HTTPException as itself and walk nested mid-stream fallbacks for the provider status
* fix(proxy): walk nested mid-stream fallbacks without recursion
* fix(anthropic): preserve 503s and answer upstream 529s on messages routes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(anthropic): answer an upstream Anthropic 529 as 529 on count_tokens
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(anthropic): use integration coverage for keepalive errors
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): send only shown SSO fields when editing, and keep env-row SSO values behind a blank stored value
* fix(ui): clear every competing SSO provider when the admin switches provider
Only the detected provider's fields were nulled on a switch, so a second stored
provider (e.g. Microsoft behind Google) kept outranking the new one at sign-in
Replace the hand-rolled combobox in the Cmd/Ctrl+K palette with shadcn's
base-vega Command and CommandDialog (cmdk) and bind the shortcut with
react-hotkeys-hook. cmdk now owns arrow keys, wraparound, Home/End and IME
composition, Base UI keeps the dialog mounted for its closing animation and
returns focus to the trigger, and mod+k is ignored while focus is inside
another dialog
Results keep their fixed Keys, Actions, Pages order with shouldFilter
disabled. The highlight resets to the first row whenever debounced key
results load, so Enter after typing still opens the top key
cmdk 1.1.1 leaves aria-activedescendant empty or stale after it picks a row
on its own (dip/cmdk#413), so a MutationObserver keeps it on the row that is
actually aria-selected until upstream fixes it
* fix(proxy): let the Admin UI cancel /team/spend/by_user at 60s with an opt-in statement timeout
GET /team/spend/by_user takes an optional timeout_seconds (1 to 600). When set, the raw
LiteLLM_SpendLogs aggregation runs under SET LOCAL statement_timeout and Postgres cancels it,
and the route returns 503 instead of letting the query run for hours. Callers that omit the
param are unchanged. The Admin UI sends timeout_seconds=60 and the Spend Per User Within Team
card shows the server's error instead of an empty table
* fix(proxy): cancel /team/spend/by_user in Postgres after 30s
Prisma's HTTP client already gives up on the statement at 30s, so the
query now runs under SET LOCAL statement_timeout = 30000 for every caller
and the opt-in timeout_seconds param is gone. The Admin UI card shows the
request error instead of the empty-state text.
* fix(proxy): keep /team/spend/by_user on the read replica inside its timeout transaction
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat: show user names in access group details
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat: show user names on project, org, model and prompt details
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): look up user names by exact id
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): fall back to user list for org admins
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): fall back when /v2/user/info returns a different user id
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): let org admins read their org members in /v2/user/info
Org admins could list a user through /user/list but got a 404 from
/v2/user/info for that same user, so the dashboard fell back to a
substring /user/list lookup that could miss the user. /v2/user/info
now allows an org admin to read users in organizations they administer,
and the dashboard's single-user lookup calls only that endpoint.
Expected 4xx responses from /v2/user/info no longer log a traceback.
* fix(proxy): drop unbounded IN from org admin check in /v2/user/info
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(cost-map): record the provider's recommended replacement as successor_model on deprecating rows and report it from /model/deprecations
* fix(cost-map): pick the alias row's successor by its own shutdown date when the family has several
* chore(ui): regenerate schema.d.ts for the /model/deprecations successor_model wording
* fix(proxy): prefer a configured successor_model over the cost map's when deployments share a name, and keep the cost map's under a hand-set deprecation_date
* test(cost-map): allow successor_model in the cost map row schema
* feat(cost-map): infer successor_model as the next version in the same family for 136 dated rows with no published replacement
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(ui): move teams, users and orgs text onto i18n
Moves the page-authored English on /teams, /users and /organizations,
plus the shared member table, user search modal, model select and budget
schedule helpers they render, into react-i18next t() calls backed by
en.json. The rendered English is unchanged
Helpers that files outside this batch or untouched tests still call keep
their signatures and take an optional t that defaults to the shared
instance, the same as usageUnits.ts. Text that existing tests pin, values
compared against the API, code samples, CSV column names and date formats
stay as they were
* refactor(ui): escape member ids in team member dialogs and merge split org and team strings
The reset spend and reset budget dialogs interpolate the member id into a Trans string, so an id containing strong, i, p or br tags rendered as markup. Escaping the values and unescaping text nodes keeps it literal, as it was before the move to i18n. Also merges the org budget lines and the team metadata hint into whole sentences and drops two constants nothing imports anymore
* refactor(ui): pass the member id into the team member reset dialogs as an element
Escaping the id still let Trans fill in placeholders like {{budget}} written inside it a second time. Rendering the id through its own component keeps it out of the translated string, so any id shows exactly as it did before the move to i18n
* feat(realtime): support latest OpenAI audio models
* fix(transcription): type SDK stream duration metadata explicitly
* fix(realtime): preserve transport settings and translation duration
* refactor(realtime): reuse SDK client initialization
* fix(audio): bill finalized streams and authorize transcription models
* refactor(audio): keep transcription pricing selection together
* fix(realtime): reconcile model metadata and test contracts
* fix(realtime): preserve provider duration and model capabilities
* fix(realtime): keep transcription validation compatible with cost map guard
* fix(realtime): address transcription and billing review findings
* fix(realtime): preserve input usage and provider routing
* fix(realtime): bill retained output and route Azure aliases
* fix(realtime): reject malformed provider durations
* fix(realtime): preserve batch cache rates and Azure v1 routes
* fix(transcription): reject nonfinite provider durations
* refactor(realtime): align annotations with current lint rules
* fix(realtime): preserve translation deployment pricing and input usage
* test(transcription): verify synchronous stream finalization
* fix(realtime): retain missing translation input usage at close
* fix(transcription): reject streams that bypass output guardrails
* fix(realtime): reconcile audio protocols billing and current CI
* fix(realtime): trust confirmed formats and authorize default models
* fix(realtime): preserve absent models and reject unsupported guardrails
* fix(realtime): reject translation policy pipelines
* chore(ci): rerun transient infrastructure timing failure
* fix(realtime): reconcile audio protocols with current upstream
* fix(audio): validate provider responses at SDK boundaries
* fix(realtime): preserve minimum OpenAI SDK compatibility
* fix(realtime): preserve deferred Live session authorization
* fix(realtime): preserve upstream authentication and transcription behavior
* fix(audio): scope provider duration precedence to configured models
* fix(typing): avoid Final declarations inside transcription cost loop
* refactor(audio): reuse transcription mapping and duration resolution
* fix(realtime): enforce model budgets and preserve translation routing
* fix(audio): log streamed transcripts whose done event reports detected languages
* fix(audio): pass stream, languages and keywords through unchecked for non-OpenAI transcription providers
Providers without a streaming transcription handler got UnsupportedParamsError
for stream=True, languages and keywords, where the merge base forwarded them as
provider-specific params. Only openai and azure validate them now.
* fix(audio): bill streamed gpt-transcribe from the provider's reported duration
The stream path computed cost after logging had replaced the transcription's
duration usage, so it fell back to the measured file length while the
unstreamed call billed the provider's seconds.
* fix(transcription): log streamed transcription usage from the provider response
The stream logging path built the spend payload from the response after
its duration usage was transformed away, so whisper-1 with stream=true and
streamed gpt-transcribe logged an empty usage object. Bill and log from the
provider's TranscriptionResponse in both the sync and async paths.
* fix(realtime): keep main's routing, protocol and transport behavior around the new translation routes
- Send OpenAI realtime and translation client_secrets and calls through litellm's httpx client again, so ssl_verify and custom CA settings apply and upstream errors keep main's shape
- Use the input transcription model as the session model only for transcription sessions
- Drop the new /openai/v1 translation aliases and record each lazy feature's own routes, so the /openai pass-through and the MCP routes resolve as on main
- Let a configured Azure realtime_protocol win for every model, with GA as the default only when none is set
- Return x-litellm headers on streamed transcriptions and mark the request done
- Refuse translation sessions before connecting upstream when realtime guardrails apply or the provider is not OpenAI or Azure
- Accept json and text for gpt-transcribe, and drop other formats under drop_params instead of failing
- Bill translation sessions on output audio seconds only, at the single published rate
- Document the 201 SDP answer on the translation calls routes and the transcription 200 response
* fix(realtime): restore main behavior for guardrails, Azure transcription, and realtime clients
Streamed transcription runs unstreamed when a deployment post_call guardrail applies, so the
guardrail still sees the transcript. Azure transcription maps v1, latest and preview to the
default dated version on the deployments route for every model. The SDK client secret keeps
session.model ahead of model, realtime WebSockets connect to api_base again, realtime metadata
keeps only guardrails, and managed agents need an explicit transcription model. Reverts the two
unrelated streamed Responses error codes and the duplicate cached_tokens_details field, and the
gpt-transcribe format check only reads the request provider's cost map entry.
* fix(realtime): keep main's transcription, client secret and route behavior where the PR changed it
Transcription reaches every provider as on main again, sends stream only when asked, forwards language and
languages together, keeps form key order, bills provider seconds only when above zero, honors base_model for the
new flags, keeps main's Azure api_version on the proxy path, and streams carry chat's headers, an error event on a
mid-stream failure, and a spend row when the stream ends early. Translation cost reads both per-second rates from
one source and spend rows carry the billed seconds.
Realtime transcription WebSockets keep model in the upstream URL outside OpenAI's hosts, client secrets forward
explicit nulls, accept conversation and delay, and answer 400 on a malformed body, the /openai translation paths
are back to main, translation refuses only for guardrails realtime sessions run, nested aliases resolve and later
session.update models are authorized, translation client secrets run per-model auth, beta clients bridged to GA
get beta-shaped session events, and transcription sessions send the deployment model in both shapes.
* fix(realtime): address review findings on transcription billing, streaming and realtime auth
Per-second pricing replaces token billing only when the priced entry sets no token rates, translation rates fall back per key to the cost map, transcription cache keys include languages and keywords, a stream request a post_call guardrail forces unstreamed comes back as redacted SSE, Azure beta clients are bridged only for GA-only deployments, nested transcription models on the realtime WebSocket are checked against the key, and translation spend carries key and team metadata
* fix(azure): pick the GA realtime protocol for deployments whose model_info.base_model is GA-only
* fix(realtime): preserve prior merge resolutions after rebase
* fix(realtime): enforce translation policies and normalize Azure transcription URLs
* fix(cookbook): stop translation once output idles after the upload ends
* fix(realtime): bill translation on input audio and drop pre-read stream cost headers
Translation spend now uses the input audio seconds the session sent, priced
through input_cost_per_second, instead of the translated output duration.
A streamed transcription no longer returns cost headers before the stream is
read, since the cost is only known after the last event. Also removes an
unused realtime guardrail helper.
* fix(realtime): drop top-level session nulls, bill dropped transcription streams, and bridge beta response fields
Client secret, translation and transcription session bodies drop top-level nulls again, as main did, while nested nulls such as turn_detection still pass through. Transcription client secrets send the deployment model as session.model. A streamed transcription the client drops still runs its success callbacks, and the stream ends with data: [DONE] like OpenAI's. The beta to GA bridge maps max_response_output_tokens, speed and noise reduction, remaps response.create, and drops temperature under drop_params. The Entra token for realtime HTTP routes is fetched off the event loop
* test(e2e): cover realtime translation, live transcription, client secrets, and streamed transcription
* fix(realtime): close review gaps in translation billing, transcription streams, and Azure GA
Bill translation sessions from the deployment's own per-second rates, add per-second prices for
Azure gpt-4o transcribe models, keep stream-scoped guardrails seeing the caller's stream flag, map
mid-stream transcription errors to LiteLLM exceptions with a [DONE] frame, map GA responses back to
the beta shape for bridged beta clients, honor the minting key's guardrail opt-out on translation
calls, restore the Azure transcription session URL, and flag GA-only Azure realtime models
* fix(realtime): bill a translation rate the deployment omits from the session model
* test(e2e): check translated speech is not English instead of exact French wording
* fix(realtime): keep main's behavior for non-bridged backends, stream error frames, gateway URLs and Azure transcribe prices
Remap beta events to GA only for bridged beta clients and translation sessions, bridge beta clients on GA-pinned Azure deployments, log error-only upstream sessions as failures, flag dated Azure GA realtime models, drop the unpublished Azure per-second transcribe rates, log a mid-stream transcription failure once, keep main's SSE error frame outside transcription, drop unsupported transcription params under drop_params, keep the caller's Azure auth headers, keep a gateway /v1 path, let key opt-outs release string-callback guardrails on translation, and keep the cookbook waiting while a long upload is still sending
* fix(realtime): keep the proxy's realtime session gate off translation sessions
A key that opts out of a legacy callback guardrail now gets a working translation session, since the transcription-guardrail gate no longer sends a realtime session.update the translation upstream rejects
* fix(realtime): keep websocket model limits, bill measured transcription length, route transcription sessions on the allowed model
A WebSocket opened without a query model now authorizes as an empty model, transcription cost prefers the measured upload length over the provider's reported seconds, transcription sessions overwrite nested models with the authorized top-level model again, and a non-boolean transcription stream value answers 400 or is dropped under drop_params.
* fix(ci): copy the rust bridge extension into the openapi docs gate's base worktree
* fix(model-catalog): accept the realtime translation endpoints in the cost map
* fix(cookbook): close the translation session and read until session.closed
* fix(realtime): treat an empty transcription stream as unset and pass the cost-map guard
An empty `stream` form value runs the transcription unstreamed again, as
OpenAI does, instead of answering 400. The four translate cost-map entries
drop the translation routes from supported_endpoints until the main
branch's catalog parser accepts them, and the nested transcription model
lookup uses the router registry entry point.
* test(proxy): drop the mid-stream error frame pin that main's error shape change replaced
* fix(transcription): bill uploaded seconds when a stream fails mid-transcript
* fix(proxy): bill uploaded seconds when a streamed transcription breaks mid-stream
* fix(audio): keep transcription's positional arguments and redact streamed error details
- Move the new languages, keywords and stream options to the end of transcription's parameters, so a prompt or format passed by position keeps its meaning
- Redact internal hostnames, paths and tracebacks from a streamed transcription's mid-stream error event, as chat streams already do
- Move the CA-bundle check for the realtime HTTP routes, which serves real TLS on loopback, from the unit suite to the SDK integration suite
* fix(audio): stream the guarded transcript to SDK callers that asked for a stream
* fix(logging): keep a streamed transcript out of callbacks that turned off message logging
A streamed transcription's assembled response reached success callbacks with its text intact under turn_off_message_logging or a callback's message_logging=False, because the shared streaming redactor had no transcription branch. It now redacts the text, as the unstreamed call already does
* fix(proxy): stream a transcription the deployment streamed when the client sent no stream field
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(decisions): add Parallel AI web-grounded decisions
* fix(decisions): name the Parallel model decision
* fix(parallel_ai): restrict environment credentials to trusted decision endpoints
* fix(decisions): keep Parallel evidence fields off other providers and handle missing confidence
Only fields a provider names reach the OpenAI response, so existing providers keep
their base shape. A malformed api_base answers 400 on both credential paths. The
complexity router classifier and the Playground handle a provider that omits
confidence. Regenerates the lazy OpenAPI snapshot and the dashboard API types
* chore(cost-map): declare the Decisions routes on parallel_ai/decision
The provider endpoint files are generated from the cost map, so the row must
name both routes for the generated System One flag to hold
* fix(decisions): report no classifier confidence when a provider omits it
A choice answer with no confidence is a valid classifier answer, so the
complexity router picks its tier and leaves classifier_confidence and the
confidence signal out instead of reporting a different metric in its place.
Also show the Parallel AI logo in the provider picker and add a Parallel
translation fixture captured from the live API for both Decisions routes.
* fix(decisions): keep a token-priced row on token billing when it also sets a query price
A plain input_cost_per_query bills flat only on a row with no token price, and input_cost_per_query_decisions always bills flat, so a custom Decisions row that already bills tokens costs the same as before this PR
* fix(decisions): bill a deployment's query price and accept Parallel bases that end in a route
A proxy request logs through an object built before the router picks the deployment, so a query price on litellm_params never reached Decisions billing. Parallel Search accepts a base ending in /v1/search and shares PARALLEL_AI_API_BASE with Decisions, so Decisions now accepts that base and one ending in /v1/decisions.
* fix(decisions): keep per-query billing and optional confidence on Parallel only
A query price on any other Decisions provider moved that provider off token
billing, and a choice or score answer with no confidence became a 200 with a
null for every provider. Both now apply to Parallel alone, so other providers
bill and fail as they do on main.
A Parallel deployment with explicit zero token rates and a query price read as
free, so every budget check was skipped while it billed per call. It now counts
as priced.
The dashboard key for Parallel is now PARALLEL_AI so a served bundle that
predates the provider still sends parallel_ai as the provider slug, and the
README table lists the provider.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(responses): reproduce comparison response ID ownership and wire failures
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(responses): guard stale retries and malformed comparison envelopes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(responses): authorize and unwrap comparison response ids
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(responses): require comparison IDs to follow response security policy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(responses): deny raw comparison response ids by default
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(responses): reject raw comparison response ids by default
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(responses): enforce managed ownership with unmanaged ID opt-in
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: Return Anthropic error envelopes on Messages auth failures
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
* fix: Answer Anthropic envelopes on Messages auth failures without relabelling unmapped statuses
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
* fix(proxy): answer the Anthropic error envelope on the /anthropic pass-through
The pass-through relays upstream errors in Anthropic's shape, but the proxy's
own refusals there (bad key, model access, budget) still answered the OpenAI
body, so an Anthropic SDK pointed at /anthropic could not read them
* fix(proxy): answer the Anthropic envelope for 404, 405 and unhandled 500 errors on Messages routes
A disabled Claude Code gateway 404, a wrong-method 405 and an unhandled 500 on the
Messages and /anthropic/ routes kept FastAPI's detail body or the OpenAI shape. The
central HTTP and unhandled-exception handlers now pick the Anthropic envelope by route.
The dashboard playground toast titles Anthropic error types when the body has no code.
* fix(proxy): keep LiteLLM error types and Anthropic envelopes on Messages routes
Error bodies on the Messages routes now carry error.litellm_error_type when the
LiteLLM type differs from the Anthropic type, so clients and the Admin UI keep
causes like budget_exceeded. Unlisted 4xx statuses map to invalid_request_error
per Anthropic's errors page, model-list requests that send anthropic-version get
the Anthropic envelope, and the admission, decompression and request-size
middleware errors use it on these routes too.
* test(proxy): expect litellm_error_type in Messages integration tests and type middleware tests
* fix(proxy): keep handler error bodies unchanged and timeouts as api_error
Errors the Messages and count_tokens handlers already returned in the Anthropic
shape keep their original bodies, so litellm_error_type now appears only on
errors the proxy raises before the handler. A 408 stays api_error as before,
and the playground toast titles the new Anthropic types and ignores long
numbers in front of an embedded error
* fix(proxy): keep the base error types inside the Messages handler and scope toast titles to Anthropic bodies
The wider status table (402 billing_error, 409 conflict_error, 504 timeout_error,
other unlisted 4xx invalid_request_error) now applies only to errors the proxy
raises before the handler. Errors inside the Messages handler, streamed error
frames and the SDK streaming adapters keep the merge base's table.
The dashboard toast reads a status before embedded JSON and titles by Anthropic
type only when the body is an Anthropic error envelope, so other pages keep
their titles.
* test(anthropic): mark the new test variables Final
---------
Signed-off-by: Alberto Perdomo <aperdomo@redhat.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* refactor(ui): move admin and settings text onto i18n
Route the page-authored English on the admin panel, caching, access
groups, tag management, logging and alerts, and router settings pages
through react-i18next, with 1,183 messages added to en.json. The
rendered English is unchanged
* fix(ui): guard i18n label lookups against prototype-named values
A server value such as "toString" used as a key into a label map
resolved to an Object.prototype member and was passed to t(), which
throws. Check own properties before translating so those values fall
back to the raw string as before
Move the page-authored English on the agents, vector stores, prompts,
search tools, memory and skills pages, plus the shared skill detail,
make-skill-public and model selector components, into en.json behind t().
The English on screen is unchanged. Keys are added under agents,
vectorStores, prompts, searchTools, memory, skills, claudeCodePlugins and
modelSelector, inserted at the top of en.json
Strings that could only move with a structural change stay English,
along with API values, code samples, paths and names
Moves the text the /guardrails page writes itself onto react-i18next under a
new top-level `guardrails` block in en.json, placed right before
`guardrailsMonitor`. Catalog data (garden cards, custom code templates, option
lists) stores ParseKeys and translates at render time. Provider and vendor
names, API values, code samples and test payloads stay raw
guardrail_info.tsx and custom_code/CustomCodeModal.tsx cross the 800-line
max-lines cap only because Prettier wraps the new t() calls, so each gets a
max-lines suppression entry instead of a split. The react/no-unescaped-entities
entry for add_guardrail_form.tsx is pruned because its apostrophes moved into
en.json
Route the page-authored English on /mcp-servers, /mcp/oauth/callback and the
shared MCP permission, server selector and BYOK components through t() and
Trans, with 1,094 messages added to en.json under mcpServers,
mcpServerManagement, mcpTools and mcpOAuthCallback. The rendered English is
unchanged.
