* refactor(ui): move remaining outlier text onto i18n
Moves 146 English messages that the earlier i18n batches missed into en.json.
They live in helpers, zod schemas, label fallbacks and shortcut descriptions
that the batch scope script did not count, so no batch picked them up. The
English on screen is unchanged.
* refactor(ui): keep exported compliance error text in English
The compliance results' "Error: ..." text is also written to the CSV export,
so it stays a literal. The policy table's unnamed fallback now reads i18n.t
inside its helper instead of taking t as a new parameter.
* feat(team): let default_team_params attach access_group_ids to every new team
* fix(types): keep default access_group_ids as Sequence for the type discipline gate
* fix(settings): chunk the default access group existence lookup
* test(team): prove default access groups reach the saved team row and type the settings fixtures
* fix(team): check default access groups on the writer so a fresh group is not rejected behind a read replica
* feat(spend-logs): opt-in latest_turn mode that stores only the current turn of stateless requests
* fix(spend-logs): keep bridged Responses rows and classifier audit in latest_turn mode, reject unknown values at boot
* fix(spend-logs): drop the recursive call in the latest_turn trim and type the stateless test cases
* test(spend-logs): format the stateless latest_turn cases
* test(spend-logs): wrap the classifier audit fixture
* fix(spend-logs): type the latest_turn trim inputs
* fix(spend-logs): leave the raw request body unvalidated before the latest_turn trim
* fix(spend-logs): trim classifier rows and ignore continuation keys off /v1/responses in latest_turn mode
* fix(spend-logs): take the classifier's originating request from the logging payload and redact a copy of vector store results
* fix(spend-logs): redact the body selected for storage, after the latest_turn trim
* feat(proxy): let admins allow specific S3 buckets and endpoints in request bodies, block bucket settings otherwise
Client requests can no longer pick s3_bucket_name, s3_output_bucket_name or allow_legacy_cloud_file_ids unless the
value is on general_settings.allowed_client_side_s3_buckets (buckets) or the admin opted in the usual way.
s3_endpoint_url, already blocked, can now be allowed per value through general_settings.allowed_client_side_s3_endpoint_urls
* refactor(proxy): drive the client-side S3 allowlist from one typed field-to-setting map
* fix(proxy): treat a null client-side S3 allowlist in config as empty instead of failing the request
* feat(ui): add a beta language setting to the account menu
The account menu gets a Language field, marked Beta, that lists each language by its own name with the English name beside it. The choice is saved per browser in localStorage and applied with a page reload, so memoized and imperative translations are read fresh.
English stays the default. An unset value, English, an unknown or unsupported code, a locale that fails to load, and any string a locale has not translated all show English. The field stays hidden until a second locale is registered in LOCALES, so this change is invisible until the first translation lands.
* fix(ui): keep the help translate link text inside the link
<link> is a void HTML tag, so the Trans parser dropped its children and rendered an empty anchor. The note now uses <githubLink>, and the language list is at least as wide as its trigger.
The banner inherits the Alert primitive's icon layout, which spans two grid rows and nudges the icon down 2px so it lines up with a title plus description. The banner only has a one-line title, so the empty second row and the nudge left the icon 3px below the text. Keep the icon on the title's row with no offset
The /chat request errors and deep research notice, and Aria's step limit
and no answer replies, are written into assistant messages that are sent
to the model as history on the next turn. Pin them to English with
i18n.getFixedT("en") so a future UI language never changes the model
payload. English output is unchanged.
* fix(types): define the responses compact call types so during_call guardrails and deployment hooks see them
* test: regenerate dashboard api types and skip compact in the azure sdk client sweep
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Moves the English left in the Aria assistant, the MCP tool permission
groups, the shared chat composer, the Beta badge, the user banner and
the metadata key fields onto react-i18next. English output is unchanged.
Text that Aria sends to the model (system prompt, tool descriptions,
tool results and usage chart payloads) stays English byte for byte.
Where the same string is also shown on screen, a display map keyed by
the operation name or by the chart's English value translates it at
render time, and the model copy is left alone.
Compliance framework and category names, the Compare endpoint labels, the
Decisions presets and schema messages, chat metrics, MCP events, reasoning
toggle, conversation date groups, model filters and the MCP boolean argument
labels now resolve through en.json. Ids, stored values and text sent to a
model are unchanged
Constants and option lists on the keys, teams, agents, budgets, cost
tracking, router settings and vector store pages now hold message keys,
and the code that renders them translates. Values, ids, sort keys and
payloads are unchanged, and lookups keyed by server values check
Object.hasOwn first
Five tests now resolve the expected English through i18n.t instead of
reading the constant directly
Moves the leftover English in the sidebar, page descriptions, account
menu, usage card and a handful of shared widgets into en.json. Option
lists and label maps now hold message keys and are translated where they
render, while every option value, id and stored setting stays the same
string. Lookups keyed by a server value check Object.hasOwn first
* refactor(ui): move remaining models text onto i18n
Moves the leftover English on the models pages, mostly labels and hints
held in constants, onto react-i18next. Constants now hold message keys and
the code that renders them translates. Where a string is both a value and
a label (tier names, health check sentinels), the English value stays for
comparisons, sorting and payloads, and a separate label key drives display.
Lookups keyed by server values check Object.hasOwn first.
* refactor(ui): translate the tier name in display name inputs
Logs and usage option lists, filter chip labels and label maps now hold
message keys that their readers translate at render time. Tile titles, the
"Claude (unspecified)" agent label and the endpoint chart categories stay
English ids and get display maps keyed by them. Lookups keyed by URL or
server values check Object.hasOwn first.
* fix(accounting): keep one undecodable delivery from blocking the ledger
A request whose metadata carried a 401-digit tokens_saved stored that number in
its delivery payload. The query engine could not decode it when the worker
claimed the batch, so every later delivery on the proxy stayed pending
The worker now claims the payload as text and decodes each row in Python. A row
that still cannot be decoded is parked alone while the rest of the batch
applies. The payload builder also clamps integers to the JSON-safe range, so
caller-supplied counts can no longer reach the engine at that size
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): take spend locks in one order so the ledger cannot deadlock
A ledger delivery updates many spend tables in one transaction, so it can
deadlock with any writer that locks the same rows in another order. It took the
team advisory lock after the user and team rows, while team roster writers take
it first
The delivery now takes every team advisory lock before any row, then writes
tables in one canonical order with ids sorted. It also runs under the rollup
lock_timeout, applies 500 deliveries per batch by default, and spreads retries
with jitter
The writers that could still collide with it now follow the same order. The
budget reset cascade, model allowlist rename, global spend reset, bulk user
update, org delete and the delete-by-list paths lock their rows in key order
first, and access group endpoints update teams and keys in one sorted pass
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(accounting): prove a ledger delivery waits instead of deadlocking
Real Postgres tests hold the first lock of a team roster change, an access group
update and a budget reset cascade, start a delivery, and let the other writer
finish. The delivery must wait and then charge once. Before the lock order fix
each case ends in a Postgres deadlock
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(accounting)!: make the SpendLogs ledger the default accounting path
use_spend_logs_ledger now defaults to true, so every request is charged through
LiteLLM_SpendLogDelivery. Setting it to false rolls back to the in-memory and
Redis spend queues, which stay in place. Tests that cover those legacy writers
pin the setting off, and the rest check the queued charge the ledger stores
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(proxy): let the mocked client open a transaction for key and team deletes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(proxy): take team locks first when a model rename rewrites allowlists
A rename locked team rows before user rows while a roster change holds the team
advisory lock and a user row, so the two could deadlock. The rename now takes the
advisory lock for every team that lists the model, sorted, before any row lock.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): drain a backlog in one worker run, one batch per transaction
One run applied a single fresh batch, so at 500 per batch charges fell behind any
load above that rate. The worker now keeps claiming fresh batches while they come
back full, for up to five seconds, and each batch still commits on its own.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): queue only the charge when a request's spend log is suppressed
With logging off for a request, the ledger still queued the full row, so a failed
flush or a shutdown could leave its prompt and response in Redis. The queue now
holds only the fields the charge needs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* style(proxy): format the rename team listing query
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): remove expired delivery receipts as fast as the worker applies them
The worker now applies several batches per run but still removed one cleanup batch
of applied receipts, so the table could outgrow its retention window. Cleanup now
repeats while each pass removes a full batch, for up to five seconds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(proxy): retry a model rename when a team starts listing the model mid-lock
The team advisory locks come from a snapshot, so a team that gained the old name
before its row lock was locked without its advisory lock. That attempt now commits
without writing and the next one locks the fresh set of teams.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat: opt-in model deprecation emails for affected owners
* fix: annotate deprecation sender initialization
* fix: clarify deprecation emails and use consistent settings toggles
* fix: exclude former team admins from deprecation emails
* feat: clarify provider deprecations and model migration steps
* feat(ui): show model deprecation banner on the models page
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(ui): format model deprecation banner
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): refresh model deprecations when models change
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): address review feedback on model deprecation banner
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): cover deprecation role gate and agent builder refresh
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): separate deprecation banner toggle from alerts link
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): dedupe and allow dismissing the model deprecation banner
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): refresh table once and key banner entries on successor
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: quiet successor logs, count banner models, use local dates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): pin banner test date and bucket upcoming entries locally
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: cover mixed-successor log level through the deprecations endpoint
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): move deprecation banner cases into integration tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): keep deprecation banner when a refetch fails
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The key edit form read react-hook-form's dirtyFields only inside the submit
handler. formState is a proxy that only tracks what the component reads while
rendering, so on the first save of an edit session that read returned the
initial empty object. A team admin editing a member's key then sent only the
key id to /key/update, got a 200 and a "Key updated successfully" toast, and
nothing changed: a max_budget edit was silently dropped, and a rename went
through as an empty update instead of the "team admins can only change budget
fields" error.
Read dirtyFields during render so the form subscribes to it and the submit
handler sees the fields the user actually changed.
* feat(accounting): add opt-in SpendLogs ledger delivery
* fix(accounting): preserve missing organization daily spend parity
* fix(accounting): preserve ledger receipts when replaying migration
* fix(accounting): preserve callback windows and tune ledger batches
* fix(accounting): preserve window charges and bound ledger statements
* test(accounting): verify budget callback registration order
* test(accounting): verify captured charges after repeated startup
* feat(accounting): store durable payloads independently of spend logs
* fix(accounting): preserve first delivery and trace ledger writes
* fix(accounting): keep SDK imports independent of proxy tracing
* fix(accounting): honor log retention during batch takeover
* test(accounting): cover retained batch alerts and invalid settings
* test(accounting): verify downgrade command against pending work
* fix(accounting): drain retries without starving fresh deliveries
* test(accounting): cover delivery projection, metrics and window key reads
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(accounting): read delivery metrics without the shared registry
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): charge a leftover $0 batch row when its claim fails
A failed batch claim buffers the retrieve's row and reports it charged. At flush the
row conflicted with the pre-upgrade $0 SpendLogs row, so its delivery was rolled back
and the charge never reached the spend tables. The flush now takes that row over in the
same transaction and keeps the delivery.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): trace the batch takeover at flush as a SpendLogs update
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): keep local spend logs when SPEND_LOGS_URL has no export client
The export client is never created, so main always writes SpendLogs to the proxy's own
database even with SPEND_LOGS_URL set. The ledger skipped local rows whenever the URL was
set, which dropped those request logs. It now treats a flush as an export only when the
client exists, and batch claims write local rows as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): keep every request's charge when its log row is shared, evicted or rejected
- Key non-batch deliveries on a per-request delivery_id so a reused client call id is still
charged once per request, while buffer replays stay deduplicated
- Persist the redacted api_key instead of an opaque bearer token
- Keep the accounting payload when the spend-log queue evicts a row or Postgres rejects it
- Strip NUL from payloads and skip non-text tags so projection never parks a paid request
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): isolate a raw-SQL spend log row Postgres rejects instead of requeueing the batch
A rejected row on the ledger insert path raises RawQueryError, which the exact DataError check missed, so the whole batch was requeued every flush. Data and integrity SQLSTATE classes now bisect like before, while missing tables and deadlocks keep retrying
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): keep charging a request whose tag is too long for the daily tag index
A tag past Postgres's btree entry limit failed the DailyTagSpend upsert, and on the ledger path that rolled back the request's key, team and user charges with it. Such tags are now left out of the daily tag rollup only
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): store only guardrail billing fields in the accounting payload
The ledger payload copied whole guardrail records, including responses and match details, into LiteLLM_SpendLogDelivery even when spend logs were disabled. It now keeps the id, name, usage and cost fields the projection reads. NUL stripping reuses safe_json_structure instead of a recursive walk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(proxy): move spend payload stamping out of update_database
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(proxy): restore the modules the import-layering guard evicts
The guard popped litellm.proxy.utils from sys.modules for good, so later tests on the same xdist worker patched a fresh module while calling functions bound to the old one, and spend queue tests failed depending on scheduling
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(proxy): keep update_database's payload stamping on its own local
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(accounting): type guardrail entries and use a public key redaction helper
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): keep charging requests whose end-user id is too long for an index
An oversized end-user id failed the SpendLogs insert with SQLSTATE 54000, which was requeued forever, and failed the end-user upsert in the delivery, which parked the key and team charge with it. Class 54 rows are now isolated like other rejected rows, and such ids are left out of the end-user rollups only
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(proxy): type the spend log queue snapshot
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): leave out any projected row Postgres cannot index or store
Caller-supplied ids and counters, such as a long tag or end-user id or an out-of-range compression count, failed the delivery transaction and rolled back the request's key, team and user charges. One check in project() now drops only the row that cannot be stored, replacing the per-field cutoffs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): keep the guardrail fields compression savings are counted from
Slimming the guardrail records dropped the provider and the tokens_saved figure that Headroom compression savings are read from, and dropped a bare-dict record entirely. The payload now keeps both while still leaving out guardrail content
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): cap projected counters so daily totals cannot overflow
Counters such as a caller-supplied compression count fit one by one but overflowed the bigint daily total once summed across requests, rolling back the key, team and user charges in the same transaction. Each counter is now capped at 2^40, far below any total that could overflow
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): let a buffered batch charge take over its zero-cost receipt
A batch charge queued after a failed lookup hit the $0 receipt an earlier poll left behind, and ON CONFLICT dropped it. The buffered insert now runs the same receipt takeover as a live retrieve
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): read a rescued batch row's spend from its accounting payload
Rows rescued from an overflowing queue keep only the accounting payload, so the buffered takeover never saw their positive spend and left the $0 receipt uncharged
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): read budget windows through the delivery transaction
Window seed and existing-row reads used a separate connection while the delivery transaction held one, so with a one-connection pool they waited on their own transaction until it timed out. Inside an enclosing transaction they now share its connection
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): retry a transient failure while keeping a dropped row's charge
The rescue swallowed every error, so a database blip right after a row was rejected lost the unpaid charge. Only a rejected row is still logged and skipped, anything else propagates so the flush retries
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(accounting): give delivery projection payloads their charged entities
project() now reads entities only from accounting_entities, so the hand-built
delivery payloads carry them like real stored payloads do.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(accounting): own ledger rows by their payload and key batches on the batch id
Drop the accounting_path stamp and its SpendLogs column. A buffered row that
carries an accounting payload is ledger owned, one without it is legacy, and
spend counters get a plain windows_from_ledger flag. Window seeding tells
applied ledger rows apart by their delivery row.
A ledger batch is plain transaction A keyed by the batch id, so a repeated
completed retrieve conflicts and charges nothing. A $0 retrieve records no
delivery, a batch a legacy retrieve already charged is not charged again, and
the legacy batch path is unchanged from main apart from refusing a batch the
ledger already owns. Batch takeover and its delta bookkeeping are removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(accounting): drop a projection test duplicated while rebasing
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(accounting): drop previous_payload now that deliveries are never replaced
Batch takeover was the only writer of previous_payload, so the worker applies
each delivery's payload as stored and the column leaves the migration.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(accounting): drop the ledger downgrade checklist script
The use_spend_logs_ledger flag is the rollback path, so a separate downgrade
gate is no longer needed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): name the window seed reads so their Prisma span resolves
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(accounting): remove the unused accounting event types and protocols
types.py, protocols.py and measurements.py were left from the old event
scaffolding. Nothing outside the package imports them, so they go with their
tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): park only deliveries whose own data fails
A delivery that keeps failing on a database outage, lock timeout or deadlock
stays pending and retries with capped backoff, so a recovered database applies
it. Only a data error or an invalid payload parks the isolated row.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): isolate a spend log row the insert statement rejects
The raw SpendLogs insert validates columns and JSON before Postgres sees the
row, so a row that fails there is now bisected out and dropped alone while its
charge is kept, instead of failing and replaying the whole flush.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): carry a job's model window on every completed poll in ledger mode
The ledger keeps the poll whose delivery claims the job, which need not be the
poll that won the limiter's in-memory charge. Every completed poll now carries
the job's window, so the delivery that lands always has it, while the live
model budget is still charged once.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(accounting): leave model-budget windows on the existing window writer
Model-budget windows are written by the model budget limiter rather than
derived from a request's spend, so they stay on main's window queue like live
budget counters do. This drops window capture from the accounting payload,
the deferred limiter registration and the job-window capture, and the ledger
keeps owning key and team budget windows.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): keep a delivery pending when the database is unreachable
Prisma reports a can't reach database server outage as a DataError, so the
worker now checks for an unavailable database before treating a data error
as the delivery's own fault.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): isolate only rows the raw insert cannot encode
Catching every ValueError also swallowed unexpected insert errors that main
re-raises. The raw insert now raises RejectedSpendLogRow for an unknown column
or non-JSON value, and only that is isolated as a rejected row.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): store zero savings when caller data makes them uncomputable
A huge caller-supplied tokens_saved overflowed the savings math inside
build_accounting_payload, which dropped the request's charge in ledger mode.
Savings that cannot be computed are now recorded as zero and the charge lands.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Every string the Models + Endpoints, Model Insights and Transform Request
pages write themselves now goes through t() or Trans, including the shared
auto-router settings those pages render on other routes. 1,651 English
messages are added under 28 new top-level keys in en.json.
This is a pure text move. The English on screen does not change, option
lists and constants keep their shape, and text that could not move without
restructuring stays English. ModelInfoEditForm.tsx, ComplexityRouterConfig.tsx
and build_complexity_router_config.ts grew past the 800 line limit and get
max-lines entries in eslint-suppressions.json instead of being split.
Move batch B1 of the admin dashboard onto react-i18next: the dashboard
shell (navbar, command palette, deployment health, header, layouts), the
entry pages (login, onboarding, change password, home, API reference,
UI theme, workflows), the public AI hub and the AI hub table, the shared
components and helpers in src/components/shared, common_components,
networking toasts, the toast titles and the shadcn primitive labels.
Pure text move into t() and Trans with 1,056 new en.json keys. The
English on screen is unchanged.
* fix(guardrails): check Anthropic-format request tools in tool_permission pre_call
The pre_call hook only collected tools shaped as {"type": "function", "function": {...}},
so flat tools from /v1/messages ({"name", "input_schema"}) and /v1/responses
({"type": "function", "name"}) were never checked and deny rules silently passed.
Reuse the shared anthropic_tool_names helper so every request tool name is checked,
and handle Anthropic tool_choice {"type": "tool"} in both block and rewrite modes.
Fixes#40583
* fix(guardrails): detect Responses API named tool_choice in tool_permission
Responses API named choices put the name at the top level ({"type": "function", "name": ...}),
so rewrite mode removed the denied tool but left a tool_choice pointing at it.
* test(guardrails): run tool_permission pre_call tests through real default_on gating
Build the guardrail as an always-on pre_call guardrail instead of patching should_run_guardrail.
* test(guardrails): type the new tool_permission pre_call test parameters
* fix(guardrails): keep the Responses API custom tool type distinct
Anthropic's user-defined tools carry type "custom", so pre_call mapped
that to "function" for tool_type rules. The Responses API has its own
custom tool type, and mapping it let a custom tool slip past a
tool_type deny rule when the default action is allow.
Normalize only for the formats where custom means a plain function, and
read tool names directly in the rewrite path, which never needed a type.
* test(guardrails): cover flat tools through Messages and Responses
Add blocked and permitted flat-tool cases through the running proxy, with
endpoint coverage and metadata for the current e2e harness
Require a guardrail policy verdict when checking denied tools so an
unrelated provider error cannot satisfy the regression test
Co-authored-by: Christopher Krah <christopher.krah@plexito.de>
* fix(guardrails): check namespace and nameless tools and relax tool_choice in tool permission rewrite
* fix(guardrails): check forced, grouped, loaded, and hosted tools by their real type
* fix(guardrails): drop references to removed tools and count loaded tools in tool permission rewrite
* fix(guardrails): check tools sent outside top-level tools in tool permission
* fix(guardrails): judge namespace members by their bare name as a fallback and keep rewrites valid for Gemini, Bedrock and hosted tools
A namespace member is checked by namespace__name first, and by its bare name when no rule names the joined one. Rewrite mode now prunes Gemini allowedFunctionNames and turns off forced calling when no function is left, drops a removed hosted tool from allowed_tools, reads Bedrock toolSpec and systemTool blocks by name, skips cachePoint markers, and leaves a tools field that is not a list for the provider to reject instead of failing with a 500
* fix(guardrails): let only name rules decide a namespace member's bare-name fallback
* fix(guardrails): judge malformed tool types, empty Gemini name lists and pass-through Anthropic tools like the endpoints do
A tool or tool_choice whose type is not a string now reads as untyped instead of crashing the guardrail with a 500, and a null type key is never read as a Gemini tool part. An empty allowedFunctionNames list no longer turns off the functions a rewrite kept. An Anthropic custom tool reads as a function by its input_schema on every route, so a pass-through route judges it the same as /v1/messages. A Responses WebSocket frame now forwards the extra_body tools a rewrite stripped
* fix(guardrails): judge a named hosted tool by its type and keep a nested chat custom tool custom
* fix(guardrails): keep judging Anthropic client tools and type-named server tools by their name
* fix(guardrails): judge a named tool by its name alone only when the name is its type's own
* fix(guardrails): judge the function a tool names beside a declaration list or a top-level name
* fix(guardrails): judge Gemini hosted keys whatever the tool type and a namespace's function
* fix(guardrails): judge legacy functions entries like tools and send Gemini a namespace's members
* fix(guardrails): judge a namespace member by every name a provider reads and send Gemini its schema converted
* fix(guardrails): judge a legacy functions entry as a function and relax a tool_choice in extra_body
* fix(gemini): keep a namespace whole when inlining its members would repeat a function name
* fix(guardrails): judge a namespace member by every bare name that shares its qualified name
* fix(vertex_ai): declare a repeated namespace member name once instead of sending namespace names to Gemini
* fix(guardrails): judge a tool_choice sent in extra_body like a top-level one
A forced name only extra_body.tool_choice carried was never checked, so block mode let a denied name through to the provider
* fix(guardrails): judge a Gemini namespace member returned by its bare name and force a named namespace's members
* test: keep formatter churn out of the tool_permission and Gemini test files
* fix(guardrails): judge extra_body.input tools, argument-aware alias rules, and exact forced Gemini functions
Tools loaded in extra_body.input are now checked and stripped like top-level input. A returned
namespace member is judged by the alias whose first name rule its arguments satisfy. A forced
Gemini name that matches a top-level function keeps that function. Alias grouping and Gemini
declaration dedupe now run in one pass instead of quadratic scans.
* fix(guardrails): judge and rewrite every tool field sent inside extra_body
extra_body is merged into the upstream body, so functions, function_call,
mcp_servers and toolConfig nested there reached the model unjudged. Treat
extra_body as a second request layer that is collected and rewritten the
same way as the top-level request.
* fix(guardrails): judge Gemini hosted keys by key and count only the tools a provider receives after extra_body
* fix(guardrails): relax a tool_choice that names a dropped namespace member by its bare name
Gemini and Anthropic chat send namespace members under their bare names, so rewrite mode now treats a dropped member's bare name, and an emptied namespace's own name, as removed when no kept tool still carries it
* fix(gemini,guardrails): keep a top-level function over a same-named namespace member and leave malformed tool values to the provider
Gemini and Vertex chat now declare a top-level function over a namespace member that shares its name, so a forced call gets the top-level schema. post_call judges a returned name a top-level tool carries as that tool, and pre_call no longer raises on an unhashable or non-string tool name, input item type, or function_call name.
* fix(guardrails): judge a returned namespace member named like a hosted MCP server as that member
* fix(gemini,guardrails): send a namespace as main did when another tool has a member's name
Gemini chat inlines a namespace's members only when no other tool, namespace
member, or legacy functions entry declares the same name, and otherwise sends the
namespace by its own name as main did. A tool_choice naming such a namespace
forces the namespace itself. The tool permission post_call reads top-level names
the way Gemini declares them and judges a call the Responses bridge tags with a
namespace as that member.
* fix(gemini): send colliding namespace members as namespace__name functions and judge them in post_call
When a namespace member shares a name with another tool, Gemini chat now gets that namespace's members as namespace__name functions instead of one function named after the namespace, so every function it can call is a name the rules judged. Declared names are counted once, and a forced member maps to each holder's qualified name.
post_call judges a returned namespace__name as that member, by its qualified or bare name the way pre_call does, so a bare-name allowlist or denylist applies to it.
* fix(gemini): keep generated member names unique and let a top-level function own its exact name
A namespace that listed a member twice, or a generated namespace__name that
repeated a top-level function, sent Gemini two declarations with one name, and
Gemini answered 400 where main answered 200. A name a namespace lists twice is
now one member, and a top-level function keeps its exact name.
post_call now judges a returned name a top-level function declares as that
function, never as the namespace member the name could also mean.
* fix(gemini): qualify namespace members by the namespace's top-level name
The guardrail judges a member as `name__member` from the namespace's top-level
name. Gemini built the name from `function.name` when the tool also carried one,
so a request could pass pre-call and reach Gemini under a name no rule judged.
* fix(vertex_ai): force only the namespace members Gemini is sent
A forced namespace or member on Gemini chat could still allow a generated name that a top-level function or an earlier namespace owns, so tool_choice forced an unrelated function. The forced names now come from the same collision checks the declarations use
* fix(vertex_ai): reject a forced namespace Gemini gets no member of, and resolve forced names in one pass
* fix(guardrails): judge a namespace member by a rule on its namespace's name
---------
Co-authored-by: Christopher Krah <christopher.krah@plexito.de>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Moves the page-authored English on /playground, the /chat app and /moyai
into t() and Trans with keys in en.json, as a pure text move. The rendered
English is unchanged
Route the page-authored English on the Logs, Usage and Telemetry pages,
the log detail drawer and the shared usage widgets through t() and Trans,
with 1,250 messages added to en.json under 10 new top-level keys. The
rendered English is unchanged.
Moves the page-authored English on the virtual keys, projects, cost tracking,
cost optimization and budgets pages, plus the create key form and the shared
key and team pickers, into t() and Trans with 1,318 messages under 32 new
top-level keys in en.json. The English on screen is unchanged
key_edit_view.tsx grows past 800 lines, so it gets a max-lines entry in
eslint-suppressions.json instead of a split
* feat(decisions): add EmpirioLabs AI as a decisions provider
empiriolabs/aplomb-1 now works with litellm.decisions and litellm.systemone, the proxy /v1/decisions and /v1/systemone routes, the decision model guardrail and the Admin UI. Requests go to POST /v1/decisions on the EmpirioLabs AI API with the System One body, keyed by EMPIRIOLABS_API_KEY with an optional EMPIRIOLABS_API_BASE
Chat models keep routing through the existing OpenAI-compatible JSON provider, and a test pins that
* ci: run the EmpirioLabs AI decisions tests in the llms unit shard
* fix: align EmpirioLabs registration with CI gates
---------
Co-authored-by: Adam Dalloul <47503782+Adam-Dalloul@users.noreply.github.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(accounting): replace event scaffolding with SpendLogs projection
* test: preserve spend timestamps without removed capture callback
* fix: guard accounting event removal across schema update paths
* refactor(accounting): retain schema and prepare durable payloads
* fix(accounting): preserve dates in serialized payloads
* fix(accounting): retain schema push compatibility
* fix(accounting): retain span registration for the legacy schema
* test(accounting): remove obsolete capture acknowledgement test
* fix(accounting): date every daily projection row the way the model rollup does
A space-separated startTime kept the whole timestamp as the daily key, while the model usage rollup took the calendar date. Every daily row now uses the same date helper
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* chore(ui): regenerate dashboard API types after rebasing on main
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(accounting): project only from the stored accounting payload
project() now reads entities only from the payload's accounting_entities, so
the raw SpendLogs fallback branches are gone. build_accounting_payload() builds
the metadata once instead of dumping it to JSON and parsing it back. Guardrail
usage dates use a tolerant UTC parse, so a malformed startTime drops only the
guardrail rows instead of failing the whole projection.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(openai): bill gov.api.openai.com FedRAMP endpoint with 1.1x uplift
* chore(openai): trim FedRAMP uplift diff
* refactor(openai): bill the FedRAMP endpoint through its own fedramp_uplift_multiplier key
* fix(ui): regenerate fedramp_uplift_multiplier doc comment in schema.d.ts
* style: ruff format utils.py and reorder fedramp_uplift_multiplier in schema.d.ts
* fix(openai): infer data_residency from OPENAI_BASE_URL and litellm.api_base when api_base is unset
* test(cost): cover get_fedramp_uplift_multiplier fallback paths
* fix(cost): apply fedramp_uplift_multiplier in batch_cost_calculator
* fix(openai): derive data_residency from a supplied client's base_url
* test: type the supplied-client data_residency test params
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): rebuild dashboard navigation and Aria workspace
* refactor(ui): make Aria a single workspace and clarify dashboard controls
* fix(ui): preserve link semantics for the Aria shortcut
* style(ui): use Sparkle for Aria
* style(ui): simplify workspace menu rows
* fix(ui): keep worker controls compact on narrow headers
* style(ui): remove workspace selector dark fill
* feat(ui): rebuild the public AI Hub as a searchable list with a details pane
The public AI Hub at /ui/model_hub_table (and the older /ui/model_hub) now
renders PublicAiHub, built from shadcn parts only: a header with the
gateway's base URL and the admin's links, segmented tabs for models, agents,
MCP servers and skills, and a list beside a details pane on desktop that
becomes a bottom sheet on phones. The open item and tab live in the URL.
It keeps every feature the old page had: server-side search, provider, mode
and feature filters, sorting, paging, health, rate limits, prices, custom
metadata, supported params, usage snippets, the require-auth redirect and
the not-enabled state. cURL shows only for chat models, since the old one
called chat completions for every mode.
The navbar hides the community buttons below the sm breakpoint on every page,
and the public navbar uses a smaller logo on phones, so Docs and Blog no
longer collide with them.
* fix(ui): harden the public AI Hub after adversarial review
- Show "This AI Hub isn't available" only when every hub call fails; a
failed models call now shows an inline error with Try again and leaves
agents, MCP servers and skills usable (a gateway with no models answers
the models list with 400)
- Row links keep the page's other query params, so open-in-new-tab and
copy link no longer 404
- A shared ?open= link resolves models beyond the first page, keeps the
opened item when a search filters it out, and says when it is no
longer listed instead of showing a different item
- Searching, sorting and paging no longer blank the details pane or drop
the Operational badge
- Key MCP servers by their listed name, which is never null
- Keep sub-cent prices and show Free for zero cost, fold evaluation into
Decisions, sort agents, MCP servers and skills by name, restore
word-wise search, and show a skill's source, author and date added
- Drop the invented description for models without a mode, the empty
agent modes row, the stray auth dash on MCP rows and NaN dates
- Do not retry the UI config call, and close the 1023-1024px gap between
the sheet and the details pane
- Update the public hub Playwright spec for the new tabs and list, and
prune the lint suppressions the old route pages needed
* fix(ui): surface hub load errors and keep the open item steady
Agents, MCP servers and skills now load through apiClient so an HTTP error
shows the inline error instead of an empty "published yet" state, and an
all-failing gateway reaches the unavailable page. Try again shows loading
instead of an empty hub, the open model stays in the details pane when a
search or page change hides its row, a zero-result search keeps the pane,
phones get the not-listed sheet for a stale link, an empty open param means
nothing opened, free models read Free in the list, and an invalid skill date
is hidden
* fix(anthropic): answer a 503 as 529 overloaded_error on /v1/messages
Anthropic's error contract has no 503, so a client built against it (Claude Code) treats a 503 from /v1/messages as a plain 5xx and retries it ten times instead of taking its overload path. A Bedrock 503 now leaves /v1/messages and /v1/messages/count_tokens as 529 overloaded_error, for both a JSON answer and a stream that fails before its first byte.
An Anthropic 529 overloaded_error now maps to litellm.ServiceUnavailableError (503) instead of InternalServerError (500), so OpenAI-shaped routes answer it as 503, the status OpenAI documents for an overloaded engine, and router retry and allowed-fails policies key it as ServiceUnavailableError.
* test(proxy): expect overloaded_error on a 503 stream error after a keepalive ping
* fix(proxy): record the 529 the Anthropic routes answer and type the chat stream error frame
The messages route stamps the answered status on the exception before post_call_failure_hook runs, so the spend log row, the OTel server span, the failed-request metric, and alerting record the 529 the client saw instead of the deployment's 503, while the deployment failure metric and cooldown keep the deployment's own status. A chat completions stream error frame names the type its status stands for instead of the null .type ServiceUnavailableError carries. An Anthropic-shaped 503 envelope passes through as 529 overloaded_error through a validated TypedDict, and count_tokens answers every ProxyException as the Anthropic envelope. Five integration tests that pinned 503 on the messages route now expect 529
* test(proxy): expect the top-level Anthropic envelope from count_tokens errors
count_tokens now answers a ProxyException as a JSONResponse carrying the
Anthropic error envelope instead of raising HTTPException, which FastAPI
wrapped in {"detail": ...}. The three legacy error-format tests now read
the response body and check that no detail wrap is left.
* test(proxy): cover the 529 overloaded_error answer across routes, cooldown, and retry budgets
* fix(anthropic): keep Anthropic overloads as InternalServerError and answer 529 from the provider status
An upstream Anthropic 529 stays litellm.InternalServerError (status 500) so
InternalServerErrorRetries, InternalServerErrorAllowedFails, SDK handlers,
and the OpenAI-shaped routes behave as on main. The error now carries
provider_status_code, and the /v1/messages routes answer 529
overloaded_error from it.
* refactor(proxy): unwrap a mid-stream fallback error once instead of recursing for the provider status
* fix(anthropic): keep count_tokens' detail body, end keepalive streams with an Anthropic error event, and log any 529 as an overload
* test(integration): cover the audit round 3 cells and expect 529 on the count_tokens 503 cases
* fix(proxy): re-raise an unchanged stream HTTPException as itself and walk nested mid-stream fallbacks for the provider status
* fix(proxy): walk nested mid-stream fallbacks without recursion
* fix(anthropic): preserve 503s and answer upstream 529s on messages routes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(anthropic): answer an upstream Anthropic 529 as 529 on count_tokens
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(anthropic): use integration coverage for keepalive errors
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): send only shown SSO fields when editing, and keep env-row SSO values behind a blank stored value
* fix(ui): clear every competing SSO provider when the admin switches provider
Only the detected provider's fields were nulled on a switch, so a second stored
provider (e.g. Microsoft behind Google) kept outranking the new one at sign-in
Replace the hand-rolled combobox in the Cmd/Ctrl+K palette with shadcn's
base-vega Command and CommandDialog (cmdk) and bind the shortcut with
react-hotkeys-hook. cmdk now owns arrow keys, wraparound, Home/End and IME
composition, Base UI keeps the dialog mounted for its closing animation and
returns focus to the trigger, and mod+k is ignored while focus is inside
another dialog
Results keep their fixed Keys, Actions, Pages order with shouldFilter
disabled. The highlight resets to the first row whenever debounced key
results load, so Enter after typing still opens the top key
cmdk 1.1.1 leaves aria-activedescendant empty or stale after it picks a row
on its own (dip/cmdk#413), so a MutationObserver keeps it on the row that is
actually aria-selected until upstream fixes it
