* feat(xai): use billed image cost and resolution prices
* test(xai): mark image generation tests as a package
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): format dashboard values through i18next formatters
Register named formats on i18next's formatter (usd, compact, duration,
relativetime, number and five date presets) with a cache keyed on the
language and options, add format.* keys to en.json, and move the usage
stat strip onto them.
* fix(ui): keep number formats in USD and the format cache bounded
Force USD on the number and compact formats, clean underscore language
codes like i18next does, fall back when options cannot be serialized,
keep formatted placeholders alone in format keys, and pin the remaining
stat strip values in the German test.
* refactor(ui): format durations with Intl.DurationFormat and load its polyfill on demand
Browsers without Intl.DurationFormat (Chrome and Edge before 129, Firefox before 136) load the pinned FormatJS polyfill through a dynamic import and re-render translations once it is installed. The milliseconds-to-units split and the 0s fallback stay, so the output matches the previous hand-built formatter.
* test(ui): cover duration edge cases on the polyfill path
* refactor(ui): drop the unused duration format and its polyfill
Nothing outside tests called format.duration, and its 1m 15s style doesn't match the decimal-second latencies the dashboard shows today. A duration format will come with the pages that need it, in the style they already use.
* fix(bedrock): run realtime guardrails on Nova Sonic sessions
* fix(bedrock): quote the spoken violation message and skip it after a blocked Nova Sonic tool output
* fix(bedrock): screen every message role, end Nova Sonic sessions on blocked transcripts, drop unparseable frames
* fix(realtime): apply guardrail masks, keep block reasons out of spoken notices, fail closed on guardrail crashes
* fix(realtime): screen joined text parts, keep the documented notice, honor key opt-outs
Text parts of one message are joined before screening again, so a phrase split
across parts is still blocked, and a mask collapses them into one part. The
spoken notice is realtime_violation_message or the block reason, quoted only on
Nova Sonic. Client-written output_text and text parts are screened. Key and team
guardrail opt-outs reach regular and translation realtime sessions. A non-string
upstream transcript skips screening as before.
* fix(realtime): honor guardrail role scope, clear stale response.create drops, screen split keywords, report Nova guardrail outages
* fix(realtime): keep every per-part mask when a joined text masks differently
* fix(realtime): screen one joined text per item and pass the session's tenant context to guardrails
Multi-part items reach each guardrail once as the space-joined text, as on main, and only an item a
guardrail rewrites gets its parts screened again to keep per-part masks. Guardrails now receive the
session request data with its metadata, so key and team policies apply, and a ModifyResponseException
block is reported as a block. Nova scopes every message item as the user turn it sends to Bedrock.
* fix(realtime): catch phrases split mid-word across text parts without re-screening single-part items
* fix(realtime): screen each joined text in its own guardrail call and keep a Nova opening message for the retry while it is screened
* fix(realtime): screen text parts in a loop so large messages keep their masks
* fix(realtime): fail closed on a non-string transcript under a guardrail and give each turn its own guardrail metadata
* fix(realtime): fail a non-string transcript only when a guardrail would screen the user's turn
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(ui): stop parsing formatted dates and spend back into values
Regenerating a key stored its new created_at as locale text and parsed it
back for display, which shows "Invalid Date" on browsers whose date format
JavaScript cannot parse. It now stores an ISO timestamp.
The model health table stored check times as locale text and sorted by
parsing that text, which misorders or drops rows outside US English. It now
keeps the raw timestamp and formats it in the table cell, so the shown text
is unchanged.
getSpendString decided whether spend rounds to zero by stripping commas from
its own formatted output. It now compares the number against half of the
smallest shown unit, which gives the same output for every value and no
longer depends on how the digits are rendered.
* fix(ui): keep getSpendString exact at high precision
0.5 / 10 ** decimals drifts by one ulp once 10 ** decimals is no longer exact
(past 22 decimals), so a value right at the threshold could flip from "< $..."
to a row of zeros. Parsing 5e-(decimals + 1) gives the correctly rounded
threshold for every precision.
* feat(router): add Switchyard Stage Router strategy
* feat(router): decision-model classifier and safer fallbacks for the Switchyard stage router
Moves to nemo-switchyard 0.4.0a1, which adds decision-model classifiers and makes
ModelCall.respond and fail async. The classifier is now a tagged union of an LLM
judge and a decision model, and decision calls go through the router's adecisions
so any decisions provider works. Algorithm misbehavior and library errors route to
the capable model instead of failing the request, and the session id plus client
session headers are always passed so capable holds apply without a classifier
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(router): keep the Switchyard router inside the type-check gate
The gate's environment did not install the optional switchyard extra, so every
libsy symbol counted as Unknown. nemo-switchyard joins proxy-dev like the other
optional runtime packages, and imports go through switchyard_rust.libsy, which
ships py.typed. Step handling moves into helpers so no Final is rebound in the
loop, and the dashboard API types are regenerated for the new routing fields
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(router): route Switchyard on every endpoint and reject unsafe router saves
The router now passes each request's call type to routing strategies, and
Switchyard converts the request to chat messages with the existing converter for
that call type before translating it. /v1/messages turns with tool or thinking
blocks used to fall back to the capable model, and now route on their signals.
The Anthropic converter can keep tool_result.is_error on an analysis copy, so
Switchyard sees tool failures there, while requests sent to models never carry it
Saves of a strategy router that lists itself now fail with a 400 instead of a
500 with a dead row, the capable and efficient models must differ, judge and
decision calls are not retried before failing open, the Switchyard routing
fields are classified for redaction, and the edit dialog returns the saved
router instead of the stale one
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(router): catch self-routing on patch and rename, and trim Switchyard tests
The self-routing check now reads the stored params through the decrypting
helper and runs when a patch carries no litellm_params, so a PATCH that points
a router at itself or renames it onto one of its targets returns 400 instead of
a 500 with a dead row. The Switchyard tests share one transcript builder and
router fixture, and reformatting noise in unrelated test files is reverted
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(ui): make Switchyard a strategy in the existing auto router form
Switchyard is now a card in the existing strategy picker. Picking it swaps the
tier and classifier fields for the Switchyard fields, and the shared name, team
and save button create it through one submit branch, the same flow as the other
strategies. The separate add form, the extracted picker and the hidden-form
swap are gone, and the edit dialog keeps its own Switchyard form
The judge's session_affinity and message_hash_fallback settings are removed,
since Switchyard rejects both for the stage router
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(ui): accept partial tool semantics and show server errors on model edit
The Switchyard tool semantics field rejected mappings that list only some
categories, although the server accepts them. Model PATCH errors were
reduced to a generic message, so it now goes through the shared API client
like model create and surfaces the server text
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(ui): show the routing strategy picker up front in the auto router form
Every strategy was hidden behind an Advanced configuration link, so the
quick setup read as the only option and never said it creates a heuristic
router. The picker now always shows with Heuristics as the labelled
default, the quick setup sits under it, and picking another strategy opens
that strategy prefilled with the chosen models
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(ui): show the server's reason when a model settings or API key update fails
Both screens replaced the rejection with a fixed message, so a 400 such as
a router routing to itself read only as a generic failure
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(router): route Switchyard on the Rust router core
main replaced the Python router core with the Rust one, which left
Switchyard routers registered nowhere. Register them with the other
pre-routing strategies, hand each strategy the endpoint the request came
through, and evict Switchyard routers on reload and delete like the rest
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(router): scope Switchyard sessions per API key and fix review gaps in the Switchyard form
Switchyard session state is keyed by a hash of the caller's key hash and session id, and client
session headers no longer reach Switchyard unscoped. The add form carries the quick setup models
into Switchyard, sends the team and access groups, clears a stale tool semantics error, and the
schema reads API-saved configs with nulls and structured decision instructions
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(router): make the Switchyard test folder a package
* fix(router): read the Switchyard session key hash only from the proxy's own metadata
* fix(proxy): keep nested nulls as null when masking deployment params
* fix(router): send Switchyard judge calls with the caller's own metadata bucket so tag routing applies
* fix(router): put Switchyard judge metadata in the bucket the judge endpoint reads
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(proxy): mixed Codex model picker and LiteLLM routing on native codex routes
The native Codex catalog now lists the key's other chat and responses models after
ChatGPT's, within Codex's catalog size limit, and drops the ETag so the merged list
refreshes on Codex's timer. A request for a model configured by exact name on another
provider goes through /v1/responses without the ChatGPT bearer or account header,
while ChatGPT names and wildcard-only matches keep the subscription passthrough.
Reasoning items another provider produced are dropped before forwarding to ChatGPT.
* fix(proxy): keep Codex's picker from turning reasoning off for LiteLLM models
Codex fills a missing default_reasoning_level with effort "none" when a model is picked in /model, while
`codex -m <model>` sends no effort at all. A thinking-only provider model then answers 400. On the native
/backend-api/codex/responses route, drop effort "none" for a LiteLLM model whose catalog entry names no
default, so the provider applies its own default exactly as `codex -m` does
* fix(proxy): harden Codex mixed-model routing on the native ChatGPT routes
Keep the ChatGPT credentials out of hook raw headers on the inner /v1/responses
request, route to LiteLLM and merge LiteLLM catalog rows only for keys that may
call /v1/responses, drop unpersisted reasoning ids on the passthrough when store
is not true, keep any slug the caller's ChatGPT account lists on the subscription
via a shared account catalog cache, and keep X-Models-Etag for keys whose catalog
added no LiteLLM rows. The guide covers the new behavior and the /v1 wildcard
precedence over a catch-all deployment.
* fix(proxy): keep Codex's native route on the subscription when the account list is unknown
A name OpenAI lists for the caller's ChatGPT account now goes to LiteLLM only when the
account's list is known and leaves it out. A failed catalog read is remembered for 60 s and
the Codex version comes only from a User-Agent whose product matches the originator header.
The merged catalog serves a stable ETag digest of OpenAI's ETag and the key's LiteLLM rows,
and /responses replaces X-Models-Etag with the same value, memoized per key for 300 s, so
Codex stops refetching or dropping its catalog cache.
LiteLLM-routed native replies wrap their reasoning with the model id, so an OpenAI or Azure
blob is dropped before chatgpt.com and unwrapped for LiteLLM, and the chat-completions bridge
decodes wrapped thinking blocks with their signature.
The access-group recipe uses chatgpt/ model names so /v1 routing does not change, and the
native route checks the chatgpt/ form of a name the key holds only through that group. A
none reasoning effort is dropped only under drop_params. The guide covers app tools, every
routing case, and the ETag, and the catalog logs how many models the byte limit left out.
* fix(fireworks_ai): move Codex additional_tools items into tools on the Responses API
* fix(proxy): keep Codex native turns working when the key's model listing fails
A listing callback that raises no longer turns chatgpt.com's catalog or
responses answer into a 500. The catalog keeps upstream's body and ETag, and
responses keep upstream's X-Models-Etag.
Under drop_params, a `none` reasoning effort is now dropped only when a
candidate deployment does not explicitly support it, so an alias to a model
like openai/gpt-5.4 keeps it.
The account slug list and its unavailable marker are keyed on the account id
plus the caller's ChatGPT Authorization header, so a failed fetch with one
bearer cannot turn off LiteLLM routing for another caller on the same account.
The auth step checks the Codex route before importing the Codex module.
* fix(proxy): keep Codex model switches working across LiteLLM providers
On the Codex native route, a LiteLLM model now gets only the encrypted reasoning every deployment it can route to is able to decrypt, so ChatGPT or another deployment's blobs no longer make xAI answer 400
Codex additional_tools items are hoisted into tools for every provider except the OpenAI API, and namespace tools become flat tools named mcp__docs__search, the shape xAI accepts
A JWT caller's models ETag is cached on a hash of its LiteLLM credential, so ChatGPT turns stop rebuilding the catalog rows
* fix(proxy): keep Codex namespace tool calls intact across LiteLLM providers
Fireworks and Bedrock Mantle accept Codex namespace tools natively, so the
native route now hoists them without flattening, like OpenAI and Azure and
the chat completions bridge. Other native Responses providers still get
flat names, now built with the bridge's namespace_tool_name_map, and each
function_call they return gets its tool name and namespace back in the
non-streaming body and in streamed output_item.added, output_item.done,
and completed/incomplete/failed events. Every other event passes through
unchanged
* fix(proxy): bridge Codex turns for providers whose Responses API rejects Codex tools
Codex sends its freeform exec tool as a custom tool on every turn, and xAI's
Responses API answers 422 for custom, namespace and additional_tools shapes.
On the Codex native route, a model whose deployments include a native
Responses provider other than OpenAI, Azure, Fireworks or Bedrock Mantle now
asks for the chat completions bridge with use_chat_completions_api, which
already turns custom and namespace tools into functions, hoists
additional_tools, and puts the namespace back on each returned call
OpenAI and Azure still get the request unchanged, and Fireworks and Bedrock
Mantle still get it on their own Responses API with additional_tools hoisted.
The route's namespace flattening and the response-side namespace restore are
removed, since no path sends flattened names anymore
* fix(proxy): drop reasoning ids a Codex turn's LiteLLM target cannot resolve
A reasoning item another deployment produced, replayed with store false and no blob the
target can decrypt, now reaches the target as its readable summary without id or status,
and is dropped when nothing readable is left. OpenAI answered 400 on the status field and
404 on the id for a glm-5p3 item replayed whole to an openai deployment. Same-deployment
replays and stored conversations keep the item whole
* refactor(codex): reach encrypted reasoning stripping through the router registry
The Codex route read Router.optional_callbacks and imported EncryptedContentAffinityCheck from
litellm.router_utils, which the router surface fence forbids outside the router. RouterRegistry
gains decryptable_reasoning(model_group, request_input, targets), which keeps the same rule: with
encrypted content affinity on for the group, reasoning one target can decrypt stays, otherwise
every target must decrypt it
* fix(proxy): keep Codex's priority tier off LiteLLM rows in the native model catalog
* fix(proxy): drop Codex's none effort for an alias whose native entry names no default
* fix(proxy): start a native Codex alias at its pinned reasoning effort and build the catalog off the event loop
* test(proxy): map the pinned reasoning efforts field in the router registry stub
* fix(proxy): honor admin-only /v1/responses and keep Codex's default tier off native aliases
Codex native routes now send a key to LiteLLM only when admin_only_routes allows it,
a native alias with configured service tiers starts with no tier picked, and the
account slug cache declares its Redis key family.
* fix(proxy): check the key's tool allowlist on native Codex LiteLLM turns and additional_tools items
* fix(proxy): check vector store grants on file_search tools inside additional_tools items
A Responses request can define file_search inside an additional_tools input item, which the native Codex route hoists for LiteLLM-served models and OpenAI runs as is on /v1/responses. The vector store check now reads those tools in both strict and legacy modes. The input item scan uses a Mapping guard instead of a pydantic error per item, since auth now runs it on every list input.
* fix(proxy): apply the proxy-wide allowed_routes to LiteLLM models on native Codex routes
* fix(proxy): cap native Codex account catalog reads per LiteLLM caller before rate limits apply
* fix(guardrails): check additional_tools items next to top-level function tools in tool_policy
ToolPolicyGuardrail read tools from the request body only when no top-level
function tool existed, so a blocked tool defined in an additional_tools item
reached Fireworks once the Responses transform moved the item into tools after
the pre-call hooks. Check every list on each request, including namespace
members inside items.
* fix(router): serve decryptable_reasoning from the Rust core's optional checks
The merge with main removed the Python router core, so the registry's host has no
optional_callbacks and no routing context, and native Codex requests with encrypted
reasoning raised RustCoreUnsupported. The filter now lives with the optional checks,
which own both. The ported pinned_reasoning_efforts field gets a test on the Rust core
Also give the static litellm.Router a declared constructor. Since the Python core was
deleted a bare __init__ assignment reads as RustCore's, so every Router(model_list=...)
is a type error and make lint fails on main. TYPE_CHECKING only, no runtime change
* fix(proxy): check the budget of a saved alias target on native Codex LiteLLM turns
Native /backend-api/codex/responses binds {endpoint}, so auth skipped the key's or team's model_group_alias and read budgets on the requested name, while the later /v1/responses handoff served the target unchecked. Auth now resolves the alias on a turn that routes to LiteLLM, as it does on /v1, and leaves ChatGPT-only names on the subscription. Also makes the tool_policy tests build a real ToolPolicyRegistry and type their fixtures.
* fix(proxy): keep a name the ChatGPT account lists on the subscription despite a saved alias
* fix(codex): keep LiteLLM keys off ChatGPT, enforce tool policy on namespaces and pin reasoning on aliases
Auth no longer asks ChatGPT for the account's model list when the bearer is
missing or is the LiteLLM key, so a key sent as the bearer never reaches it.
tool_policy now reads the members of a top-level namespace, which is where
the native route's hoist puts them.
A global alias now reads encrypted content affinity from its target group.
A non-list top-level tools value and a non-string function name now reach the
provider, which refuses them, instead of raising a 500.
The catalog ETag no longer depends on the order a process iterates priced
service tiers in, so workers and pods agree on it.
* fix(proxy): list a team's models in the order the team set them
get_team_models built the names in a set, so each process listed a team key's models in its own hash order and the native Codex catalog sent a different row order and ETag from each worker
* fix(proxy): apply JWT route policy before native Codex routes serve LiteLLM models
A standard JWT caller now gets a LiteLLM model on /backend-api/codex/models
and /backend-api/codex/responses only when the JWT auth flow would admit a
direct /v1/responses call for that model, so team_allowed_routes,
admin_allowed_routes and RBAC route limits hold on the handoff too.
The Fireworks Responses transform also leaves additional_tools input items
in place when the tools param is not a list, instead of raising while hoisting
* fix(fireworks): hoist additional_tools when tools is a tuple
litellm.responses passes a tuple tools argument through to the Fireworks
transform unchanged, so the non-list guard skipped hoisting and left the
additional_tools item in the input. Treat tuples like lists and cover both
shapes in the drop_params merge test
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(model-hub): show each deployment's full prices in the model hub
The model hub only showed one input and one output price per model, the highest across its
deployments. Each model group now lists every deployment's prices, read from the same entry
billing charges, including cached tokens, long-context tiers, flex, priority, batch and
tiered ranges. Both the logged-in and public hub detail views render them in one table per
deployment, labeled by provider only
* refactor(ui): move model hub pricing breakdown text onto i18n
Route every string the deployment pricing breakdown writes through t() under a new deploymentPricing group in en.json, plus the three hub headings that host it. The English on screen is unchanged
* fix(model-hub): show Vertex regional prices and explicit cached reads as billed
A Vertex deployment outside the global endpoint now shows its token prices with the regional
uplift billing applies at the deployment's location. The explicit cached read price shows per
1M tokens on its own row instead of a raw per-token value. The breakdown also says that
adjustments made per request, like off-peak discounts and regional or data residency
surcharges, aren't included
* fix(model-hub): show prices written as text the way billing reads them
A price written in config.yaml as 3e-7 arrives as text. Billing turns it into a number, but
the hub dropped it, so a deployment showed no price or the cost map's price instead of its own.
The hub now reads numeric text for regular prices and size-based price ranges, and skips text
that isn't a number
* style(ui): format public model hub test
* test(model-hub): name the deployment pricing fixture to stay within the inline object lint budget
* fix(anthropic): do not surface server_tool_use as client tool_calls
Anthropic server-executed tools (web search, tool search, code execution)
were returned in chat completions tool_calls with srvtoolu_ ids, so
OpenAI-protocol agent clients treated them as pending work and re-invoked
the model until their recursion limit.
Move them to provider_specific_fields.server_tool_calls on both the
non-streaming and streaming paths. History replay, the /v1/responses
bridge and the /v1/messages streaming guardrail read that field, so
multi-turn reconstruction, web_search_call output items and the
guardrail's fail-closed check on server tool blocks keep working.
* fix(responses): return Claude web_fetch calls as web_search_call open_page items
* fix(responses): keep Claude server web calls without a result as hosted items
A non-streamed Claude turn that calls web_search or web_fetch next to a
client tool can return the server call before its result. Build an
in_progress web_search_call for it, so the Responses bridge no longer
returns it as a client function_call
* fix(anthropic): keep pending server calls native and hide server calls from client tool paths
Replaying a server call that has no result yet now sends it back as
server_tool_use, so Anthropic accepts the next turn. The tool permission
guardrail checks server calls again and rewrite mode removes a denied one.
The /v1/messages adapter keeps Claude's server blocks, and the Responses
bridge never returns a server call as a client function_call.
* fix(anthropic): let guardrails, tool usage, tracing, and Responses replay see Claude's server calls
Claude's server tool calls moved out of tool_calls into provider_specific_fields.server_tool_calls,
so every reader of tool_calls stopped seeing them. The chat guardrail handler now passes them to
output guardrails and writes rewrites back to server_tool_calls (stream and non-stream), tool usage
and registry discovery count them, otel, Datadog and the Logs Tools panel list them, and a pending
Claude web search or fetch item replayed to /v1/responses goes back to Anthropic as its own
server_tool_use instead of running again
* fix(anthropic): let guardrails and loggers read Claude's server tool calls again
Guardrails, tool policies, tracing loggers and the Responses guardrail
translation read only client tool calls after server calls moved to
provider_specific_fields. They now read server calls first, then client
calls, through one shared helper, and the Responses guardrail treats
Claude's web_search_call items as web_search or web_fetch calls.
Non-streaming splits server calls by block type, as streaming does.
* fix(anthropic): keep Claude server calls visible to guardrails, redaction and tracing
A guardrail rewrite of a Claude server call now also updates its hosted web_search_calls copy,
streamed or not. Redaction with message logging off hides server call arguments and hosted web
call actions on chat and Responses. Pillar and Pangea scan server calls, transform-mode streaming
guardrails keep server call chunks, and OTEL v2 Responses spans list Claude's web calls.
* fix(anthropic): keep the legacy hook view of streamed /v1/messages web search after the main merge
Main's legacy post-call hooks see Claude's server call as tool_use with its search results first. The streamed /v1/messages response now keeps that view while native scans keep server_tool_use.
* fix(guardrails): let Cisco AI Defense scan Claude's web calls on Responses
Claude's web search and fetch now come back from /v1/responses as web_search_call items, so Cisco AI Defense reads and clears their query or URL the way it did for the old function_call arguments. The helpers that read and rewrite those items move to the shared guardrail utils so the Responses guardrail handler and Cisco use one copy.
* test(guardrails): inject Pillar's HTTP client and type the Cisco web call actions
* test(guardrails): type the fake Pillar client like its Protocol
* fix(anthropic): keep a pending server call and its thinking when the turn also waits on a client tool
Only a turn with no client tool_use is a paused turn. A tool_use turn that also started web_fetch or
web_search still needs that call and its thinking replayed, or Claude loses the fetch result
* fix(guardrails): scan replayed Claude server calls and hosted web calls
Request guardrails now see and rewrite server_tool_calls replayed in chat
messages. Cato and Purview scan Claude web_search and web_fetch calls on
Responses output, and a function tool call with an extra custom key is no
longer dropped from Cisco, Purview, Repello and PANW scans.
* fix(guardrails): scan replayed Claude server tool calls in Lasso, Lakera and Qualifire request checks
* fix(anthropic): apply streamed guardrail tool call rewrites to their own block when a server tool call follows
* test: annotate the new server tool call test locals with Final and type their parameters
* test: type the new guardrail test helpers with litellm's tool call types instead of Any
* test: narrow the remaining new test helper parameters from bare dict and object
* fix(anthropic): keep server calls in Responses streams, guardrail masks, OTEL and Logs
Streamed /v1/responses now emits Claude's tool search and paused web search calls, text guardrail masks rewrite replayed server calls, OTEL gen_ai messages and the Logs pretty view list server calls next to client calls, and integration tests read server calls from server_tool_calls
* fix(guardrails): keep hosted web call copies in sync on request masks and denied server calls, and accept null tool_calls in Lakera
* fix(anthropic): serialize masked hosted web call copies through their adapter so a replay before the OpenAI model is built does not 500
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(proxy): discover upstream LiteLLM Gateway wildcard models
* fix(proxy): use the typed shared client for gateway discovery
* fix(ui): show the selected LiteLLM Gateway logo
* feat(ui): let admins set tpm_limit and rpm_limit on tags
The tag create modal and edit form said tag TPM/RPM limits were unsupported and never sent them, while the proxy has enforced tag-level rpm_limit and tpm_limit since #41807. Replace the notice with TPM Limit and RPM Limit inputs wired to /tag/new and /tag/update, validated as nonnegative integers, with a cleared limit sent as null so the proxy drops it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): prefill tag budget and limit fields from the Edit Tag button
The edit form only seeded max_budget and budget_duration when opened from the row's Edit action, so the Edit Tag button inside the tag view showed empty budget and limit inputs and a cleared field could not be sent as null. Seed from the loaded tag on both paths
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): accept null budget fields when editing a tag
A tag whose budget row has max_budget or a limit set to null failed the edit form's zod schema with 'expected string, received null' once the fields were prefilled. Treat null like an empty input and type Tag's budget fields as nullable
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): accept a null description when editing a tag
A tag created without a description comes back from /tag/info with description null, which the edit form's zod schema rejected with 'expected string, received null' on save. Allow null and type Tag.description as nullable
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): ignore invalid limits left in a collapsed tag budget section
An invalid value typed in Budget & Rate Limits and then hidden by collapsing the section blocked Create Tag and Save Changes with the error out of sight. When the section is collapsed and the only errors are on its fields, drop them and submit the visible values, which already discards the hidden budget fields
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): name the expected payload so the inline-object lint budget holds
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(ui): move remaining outlier text onto i18n
Moves 146 English messages that the earlier i18n batches missed into en.json.
They live in helpers, zod schemas, label fallbacks and shortcut descriptions
that the batch scope script did not count, so no batch picked them up. The
English on screen is unchanged.
* refactor(ui): keep exported compliance error text in English
The compliance results' "Error: ..." text is also written to the CSV export,
so it stays a literal. The policy table's unnamed fallback now reads i18n.t
inside its helper instead of taking t as a new parameter.
* feat(team): let default_team_params attach access_group_ids to every new team
* fix(types): keep default access_group_ids as Sequence for the type discipline gate
* fix(settings): chunk the default access group existence lookup
* test(team): prove default access groups reach the saved team row and type the settings fixtures
* fix(team): check default access groups on the writer so a fresh group is not rejected behind a read replica
* feat(spend-logs): opt-in latest_turn mode that stores only the current turn of stateless requests
* fix(spend-logs): keep bridged Responses rows and classifier audit in latest_turn mode, reject unknown values at boot
* fix(spend-logs): drop the recursive call in the latest_turn trim and type the stateless test cases
* test(spend-logs): format the stateless latest_turn cases
* test(spend-logs): wrap the classifier audit fixture
* fix(spend-logs): type the latest_turn trim inputs
* fix(spend-logs): leave the raw request body unvalidated before the latest_turn trim
* fix(spend-logs): trim classifier rows and ignore continuation keys off /v1/responses in latest_turn mode
* fix(spend-logs): take the classifier's originating request from the logging payload and redact a copy of vector store results
* fix(spend-logs): redact the body selected for storage, after the latest_turn trim
* feat(proxy): let admins allow specific S3 buckets and endpoints in request bodies, block bucket settings otherwise
Client requests can no longer pick s3_bucket_name, s3_output_bucket_name or allow_legacy_cloud_file_ids unless the
value is on general_settings.allowed_client_side_s3_buckets (buckets) or the admin opted in the usual way.
s3_endpoint_url, already blocked, can now be allowed per value through general_settings.allowed_client_side_s3_endpoint_urls
* refactor(proxy): drive the client-side S3 allowlist from one typed field-to-setting map
* fix(proxy): treat a null client-side S3 allowlist in config as empty instead of failing the request
* feat(ui): add a beta language setting to the account menu
The account menu gets a Language field, marked Beta, that lists each language by its own name with the English name beside it. The choice is saved per browser in localStorage and applied with a page reload, so memoized and imperative translations are read fresh.
English stays the default. An unset value, English, an unknown or unsupported code, a locale that fails to load, and any string a locale has not translated all show English. The field stays hidden until a second locale is registered in LOCALES, so this change is invisible until the first translation lands.
* fix(ui): keep the help translate link text inside the link
<link> is a void HTML tag, so the Trans parser dropped its children and rendered an empty anchor. The note now uses <githubLink>, and the language list is at least as wide as its trigger.
The banner inherits the Alert primitive's icon layout, which spans two grid rows and nudges the icon down 2px so it lines up with a title plus description. The banner only has a one-line title, so the empty second row and the nudge left the icon 3px below the text. Keep the icon on the title's row with no offset
The /chat request errors and deep research notice, and Aria's step limit
and no answer replies, are written into assistant messages that are sent
to the model as history on the next turn. Pin them to English with
i18n.getFixedT("en") so a future UI language never changes the model
payload. English output is unchanged.
* fix(types): define the responses compact call types so during_call guardrails and deployment hooks see them
* test: regenerate dashboard api types and skip compact in the azure sdk client sweep
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Moves the English left in the Aria assistant, the MCP tool permission
groups, the shared chat composer, the Beta badge, the user banner and
the metadata key fields onto react-i18next. English output is unchanged.
Text that Aria sends to the model (system prompt, tool descriptions,
tool results and usage chart payloads) stays English byte for byte.
Where the same string is also shown on screen, a display map keyed by
the operation name or by the chart's English value translates it at
render time, and the model copy is left alone.
Compliance framework and category names, the Compare endpoint labels, the
Decisions presets and schema messages, chat metrics, MCP events, reasoning
toggle, conversation date groups, model filters and the MCP boolean argument
labels now resolve through en.json. Ids, stored values and text sent to a
model are unchanged
Constants and option lists on the keys, teams, agents, budgets, cost
tracking, router settings and vector store pages now hold message keys,
and the code that renders them translates. Values, ids, sort keys and
payloads are unchanged, and lookups keyed by server values check
Object.hasOwn first
Five tests now resolve the expected English through i18n.t instead of
reading the constant directly
Moves the leftover English in the sidebar, page descriptions, account
menu, usage card and a handful of shared widgets into en.json. Option
lists and label maps now hold message keys and are translated where they
render, while every option value, id and stored setting stays the same
string. Lookups keyed by a server value check Object.hasOwn first
* refactor(ui): move remaining models text onto i18n
Moves the leftover English on the models pages, mostly labels and hints
held in constants, onto react-i18next. Constants now hold message keys and
the code that renders them translates. Where a string is both a value and
a label (tier names, health check sentinels), the English value stays for
comparisons, sorting and payloads, and a separate label key drives display.
Lookups keyed by server values check Object.hasOwn first.
* refactor(ui): translate the tier name in display name inputs
Logs and usage option lists, filter chip labels and label maps now hold
message keys that their readers translate at render time. Tile titles, the
"Claude (unspecified)" agent label and the endpoint chart categories stay
English ids and get display maps keyed by them. Lookups keyed by URL or
server values check Object.hasOwn first.
* fix(accounting): keep one undecodable delivery from blocking the ledger
A request whose metadata carried a 401-digit tokens_saved stored that number in
its delivery payload. The query engine could not decode it when the worker
claimed the batch, so every later delivery on the proxy stayed pending
The worker now claims the payload as text and decodes each row in Python. A row
that still cannot be decoded is parked alone while the rest of the batch
applies. The payload builder also clamps integers to the JSON-safe range, so
caller-supplied counts can no longer reach the engine at that size
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): take spend locks in one order so the ledger cannot deadlock
A ledger delivery updates many spend tables in one transaction, so it can
deadlock with any writer that locks the same rows in another order. It took the
team advisory lock after the user and team rows, while team roster writers take
it first
The delivery now takes every team advisory lock before any row, then writes
tables in one canonical order with ids sorted. It also runs under the rollup
lock_timeout, applies 500 deliveries per batch by default, and spreads retries
with jitter
The writers that could still collide with it now follow the same order. The
budget reset cascade, model allowlist rename, global spend reset, bulk user
update, org delete and the delete-by-list paths lock their rows in key order
first, and access group endpoints update teams and keys in one sorted pass
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(accounting): prove a ledger delivery waits instead of deadlocking
Real Postgres tests hold the first lock of a team roster change, an access group
update and a budget reset cascade, start a delivery, and let the other writer
finish. The delivery must wait and then charge once. Before the lock order fix
each case ends in a Postgres deadlock
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(accounting)!: make the SpendLogs ledger the default accounting path
use_spend_logs_ledger now defaults to true, so every request is charged through
LiteLLM_SpendLogDelivery. Setting it to false rolls back to the in-memory and
Redis spend queues, which stay in place. Tests that cover those legacy writers
pin the setting off, and the rest check the queued charge the ledger stores
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(proxy): let the mocked client open a transaction for key and team deletes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(proxy): take team locks first when a model rename rewrites allowlists
A rename locked team rows before user rows while a roster change holds the team
advisory lock and a user row, so the two could deadlock. The rename now takes the
advisory lock for every team that lists the model, sorted, before any row lock.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): drain a backlog in one worker run, one batch per transaction
One run applied a single fresh batch, so at 500 per batch charges fell behind any
load above that rate. The worker now keeps claiming fresh batches while they come
back full, for up to five seconds, and each batch still commits on its own.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): queue only the charge when a request's spend log is suppressed
With logging off for a request, the ledger still queued the full row, so a failed
flush or a shutdown could leave its prompt and response in Redis. The queue now
holds only the fields the charge needs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* style(proxy): format the rename team listing query
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): remove expired delivery receipts as fast as the worker applies them
The worker now applies several batches per run but still removed one cleanup batch
of applied receipts, so the table could outgrow its retention window. Cleanup now
repeats while each pass removes a full batch, for up to five seconds.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(proxy): retry a model rename when a team starts listing the model mid-lock
The team advisory locks come from a snapshot, so a team that gained the old name
before its row lock was locked without its advisory lock. That attempt now commits
without writing and the next one locks the fresh set of teams.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat: opt-in model deprecation emails for affected owners
* fix: annotate deprecation sender initialization
* fix: clarify deprecation emails and use consistent settings toggles
* fix: exclude former team admins from deprecation emails
* feat: clarify provider deprecations and model migration steps
* feat(ui): show model deprecation banner on the models page
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(ui): format model deprecation banner
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): refresh model deprecations when models change
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): address review feedback on model deprecation banner
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): cover deprecation role gate and agent builder refresh
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): separate deprecation banner toggle from alerts link
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): dedupe and allow dismissing the model deprecation banner
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): refresh table once and key banner entries on successor
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: quiet successor logs, count banner models, use local dates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): pin banner test date and bucket upcoming entries locally
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: cover mixed-successor log level through the deprecations endpoint
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): move deprecation banner cases into integration tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): keep deprecation banner when a refetch fails
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Krrish Dholakia <krrishdholakia@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The key edit form read react-hook-form's dirtyFields only inside the submit
handler. formState is a proxy that only tracks what the component reads while
rendering, so on the first save of an edit session that read returned the
initial empty object. A team admin editing a member's key then sent only the
key id to /key/update, got a 200 and a "Key updated successfully" toast, and
nothing changed: a max_budget edit was silently dropped, and a rename went
through as an empty update instead of the "team admins can only change budget
fields" error.
Read dirtyFields during render so the form subscribes to it and the submit
handler sees the fields the user actually changed.
* feat(accounting): add opt-in SpendLogs ledger delivery
* fix(accounting): preserve missing organization daily spend parity
* fix(accounting): preserve ledger receipts when replaying migration
* fix(accounting): preserve callback windows and tune ledger batches
* fix(accounting): preserve window charges and bound ledger statements
* test(accounting): verify budget callback registration order
* test(accounting): verify captured charges after repeated startup
* feat(accounting): store durable payloads independently of spend logs
* fix(accounting): preserve first delivery and trace ledger writes
* fix(accounting): keep SDK imports independent of proxy tracing
* fix(accounting): honor log retention during batch takeover
* test(accounting): cover retained batch alerts and invalid settings
* test(accounting): verify downgrade command against pending work
* fix(accounting): drain retries without starving fresh deliveries
* test(accounting): cover delivery projection, metrics and window key reads
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(accounting): read delivery metrics without the shared registry
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): charge a leftover $0 batch row when its claim fails
A failed batch claim buffers the retrieve's row and reports it charged. At flush the
row conflicted with the pre-upgrade $0 SpendLogs row, so its delivery was rolled back
and the charge never reached the spend tables. The flush now takes that row over in the
same transaction and keeps the delivery.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): trace the batch takeover at flush as a SpendLogs update
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): keep local spend logs when SPEND_LOGS_URL has no export client
The export client is never created, so main always writes SpendLogs to the proxy's own
database even with SPEND_LOGS_URL set. The ledger skipped local rows whenever the URL was
set, which dropped those request logs. It now treats a flush as an export only when the
client exists, and batch claims write local rows as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): keep every request's charge when its log row is shared, evicted or rejected
- Key non-batch deliveries on a per-request delivery_id so a reused client call id is still
charged once per request, while buffer replays stay deduplicated
- Persist the redacted api_key instead of an opaque bearer token
- Keep the accounting payload when the spend-log queue evicts a row or Postgres rejects it
- Strip NUL from payloads and skip non-text tags so projection never parks a paid request
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): isolate a raw-SQL spend log row Postgres rejects instead of requeueing the batch
A rejected row on the ledger insert path raises RawQueryError, which the exact DataError check missed, so the whole batch was requeued every flush. Data and integrity SQLSTATE classes now bisect like before, while missing tables and deadlocks keep retrying
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): keep charging a request whose tag is too long for the daily tag index
A tag past Postgres's btree entry limit failed the DailyTagSpend upsert, and on the ledger path that rolled back the request's key, team and user charges with it. Such tags are now left out of the daily tag rollup only
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): store only guardrail billing fields in the accounting payload
The ledger payload copied whole guardrail records, including responses and match details, into LiteLLM_SpendLogDelivery even when spend logs were disabled. It now keeps the id, name, usage and cost fields the projection reads. NUL stripping reuses safe_json_structure instead of a recursive walk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(proxy): move spend payload stamping out of update_database
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(proxy): restore the modules the import-layering guard evicts
The guard popped litellm.proxy.utils from sys.modules for good, so later tests on the same xdist worker patched a fresh module while calling functions bound to the old one, and spend queue tests failed depending on scheduling
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(proxy): keep update_database's payload stamping on its own local
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(accounting): type guardrail entries and use a public key redaction helper
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): keep charging requests whose end-user id is too long for an index
An oversized end-user id failed the SpendLogs insert with SQLSTATE 54000, which was requeued forever, and failed the end-user upsert in the delivery, which parked the key and team charge with it. Class 54 rows are now isolated like other rejected rows, and such ids are left out of the end-user rollups only
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(proxy): type the spend log queue snapshot
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): leave out any projected row Postgres cannot index or store
Caller-supplied ids and counters, such as a long tag or end-user id or an out-of-range compression count, failed the delivery transaction and rolled back the request's key, team and user charges. One check in project() now drops only the row that cannot be stored, replacing the per-field cutoffs
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): keep the guardrail fields compression savings are counted from
Slimming the guardrail records dropped the provider and the tokens_saved figure that Headroom compression savings are read from, and dropped a bare-dict record entirely. The payload now keeps both while still leaving out guardrail content
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): cap projected counters so daily totals cannot overflow
Counters such as a caller-supplied compression count fit one by one but overflowed the bigint daily total once summed across requests, rolling back the key, team and user charges in the same transaction. Each counter is now capped at 2^40, far below any total that could overflow
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): let a buffered batch charge take over its zero-cost receipt
A batch charge queued after a failed lookup hit the $0 receipt an earlier poll left behind, and ON CONFLICT dropped it. The buffered insert now runs the same receipt takeover as a live retrieve
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): read a rescued batch row's spend from its accounting payload
Rows rescued from an overflowing queue keep only the accounting payload, so the buffered takeover never saw their positive spend and left the $0 receipt uncharged
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): read budget windows through the delivery transaction
Window seed and existing-row reads used a separate connection while the delivery transaction held one, so with a one-connection pool they waited on their own transaction until it timed out. Inside an enclosing transaction they now share its connection
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): retry a transient failure while keeping a dropped row's charge
The rescue swallowed every error, so a database blip right after a row was rejected lost the unpaid charge. Only a rejected row is still logged and skipped, anything else propagates so the flush retries
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(accounting): give delivery projection payloads their charged entities
project() now reads entities only from accounting_entities, so the hand-built
delivery payloads carry them like real stored payloads do.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(accounting): own ledger rows by their payload and key batches on the batch id
Drop the accounting_path stamp and its SpendLogs column. A buffered row that
carries an accounting payload is ledger owned, one without it is legacy, and
spend counters get a plain windows_from_ledger flag. Window seeding tells
applied ledger rows apart by their delivery row.
A ledger batch is plain transaction A keyed by the batch id, so a repeated
completed retrieve conflicts and charges nothing. A $0 retrieve records no
delivery, a batch a legacy retrieve already charged is not charged again, and
the legacy batch path is unchanged from main apart from refusing a batch the
ledger already owns. Batch takeover and its delta bookkeeping are removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(accounting): drop a projection test duplicated while rebasing
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(accounting): drop previous_payload now that deliveries are never replaced
Batch takeover was the only writer of previous_payload, so the worker applies
each delivery's payload as stored and the column leaves the migration.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(accounting): drop the ledger downgrade checklist script
The use_spend_logs_ledger flag is the rollback path, so a separate downgrade
gate is no longer needed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): name the window seed reads so their Prisma span resolves
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(accounting): remove the unused accounting event types and protocols
types.py, protocols.py and measurements.py were left from the old event
scaffolding. Nothing outside the package imports them, so they go with their
tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): park only deliveries whose own data fails
A delivery that keeps failing on a database outage, lock timeout or deadlock
stays pending and retries with capped backoff, so a recovered database applies
it. Only a data error or an invalid payload parks the isolated row.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): isolate a spend log row the insert statement rejects
The raw SpendLogs insert validates columns and JSON before Postgres sees the
row, so a row that fails there is now bisected out and dropped alone while its
charge is kept, instead of failing and replaying the whole flush.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): carry a job's model window on every completed poll in ledger mode
The ledger keeps the poll whose delivery claims the job, which need not be the
poll that won the limiter's in-memory charge. Every completed poll now carries
the job's window, so the delivery that lands always has it, while the live
model budget is still charged once.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(accounting): leave model-budget windows on the existing window writer
Model-budget windows are written by the model budget limiter rather than
derived from a request's spend, so they stay on main's window queue like live
budget counters do. This drops window capture from the accounting payload,
the deferred limiter registration and the job-window capture, and the ledger
keeps owning key and team budget windows.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): keep a delivery pending when the database is unreachable
Prisma reports a can't reach database server outage as a DataError, so the
worker now checks for an unavailable database before treating a data error
as the delivery's own fault.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): isolate only rows the raw insert cannot encode
Catching every ValueError also swallowed unexpected insert errors that main
re-raises. The raw insert now raises RejectedSpendLogRow for an unknown column
or non-JSON value, and only that is isolated as a rejected row.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(accounting): store zero savings when caller data makes them uncomputable
A huge caller-supplied tokens_saved overflowed the savings math inside
build_accounting_payload, which dropped the request's charge in ledger mode.
Savings that cannot be computed are now recorded as zero and the charge lands.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Every string the Models + Endpoints, Model Insights and Transform Request
pages write themselves now goes through t() or Trans, including the shared
auto-router settings those pages render on other routes. 1,651 English
messages are added under 28 new top-level keys in en.json.
This is a pure text move. The English on screen does not change, option
lists and constants keep their shape, and text that could not move without
restructuring stays English. ModelInfoEditForm.tsx, ComplexityRouterConfig.tsx
and build_complexity_router_config.ts grew past the 800 line limit and get
max-lines entries in eslint-suppressions.json instead of being split.
Move batch B1 of the admin dashboard onto react-i18next: the dashboard
shell (navbar, command palette, deployment health, header, layouts), the
entry pages (login, onboarding, change password, home, API reference,
UI theme, workflows), the public AI hub and the AI hub table, the shared
components and helpers in src/components/shared, common_components,
networking toasts, the toast titles and the shadcn primitive labels.
Pure text move into t() and Trans with 1,056 new en.json keys. The
English on screen is unchanged.
* fix(guardrails): check Anthropic-format request tools in tool_permission pre_call
The pre_call hook only collected tools shaped as {"type": "function", "function": {...}},
so flat tools from /v1/messages ({"name", "input_schema"}) and /v1/responses
({"type": "function", "name"}) were never checked and deny rules silently passed.
Reuse the shared anthropic_tool_names helper so every request tool name is checked,
and handle Anthropic tool_choice {"type": "tool"} in both block and rewrite modes.
Fixes#40583
* fix(guardrails): detect Responses API named tool_choice in tool_permission
Responses API named choices put the name at the top level ({"type": "function", "name": ...}),
so rewrite mode removed the denied tool but left a tool_choice pointing at it.
* test(guardrails): run tool_permission pre_call tests through real default_on gating
Build the guardrail as an always-on pre_call guardrail instead of patching should_run_guardrail.
* test(guardrails): type the new tool_permission pre_call test parameters
* fix(guardrails): keep the Responses API custom tool type distinct
Anthropic's user-defined tools carry type "custom", so pre_call mapped
that to "function" for tool_type rules. The Responses API has its own
custom tool type, and mapping it let a custom tool slip past a
tool_type deny rule when the default action is allow.
Normalize only for the formats where custom means a plain function, and
read tool names directly in the rewrite path, which never needed a type.
* test(guardrails): cover flat tools through Messages and Responses
Add blocked and permitted flat-tool cases through the running proxy, with
endpoint coverage and metadata for the current e2e harness
Require a guardrail policy verdict when checking denied tools so an
unrelated provider error cannot satisfy the regression test
Co-authored-by: Christopher Krah <christopher.krah@plexito.de>
* fix(guardrails): check namespace and nameless tools and relax tool_choice in tool permission rewrite
* fix(guardrails): check forced, grouped, loaded, and hosted tools by their real type
* fix(guardrails): drop references to removed tools and count loaded tools in tool permission rewrite
* fix(guardrails): check tools sent outside top-level tools in tool permission
* fix(guardrails): judge namespace members by their bare name as a fallback and keep rewrites valid for Gemini, Bedrock and hosted tools
A namespace member is checked by namespace__name first, and by its bare name when no rule names the joined one. Rewrite mode now prunes Gemini allowedFunctionNames and turns off forced calling when no function is left, drops a removed hosted tool from allowed_tools, reads Bedrock toolSpec and systemTool blocks by name, skips cachePoint markers, and leaves a tools field that is not a list for the provider to reject instead of failing with a 500
* fix(guardrails): let only name rules decide a namespace member's bare-name fallback
* fix(guardrails): judge malformed tool types, empty Gemini name lists and pass-through Anthropic tools like the endpoints do
A tool or tool_choice whose type is not a string now reads as untyped instead of crashing the guardrail with a 500, and a null type key is never read as a Gemini tool part. An empty allowedFunctionNames list no longer turns off the functions a rewrite kept. An Anthropic custom tool reads as a function by its input_schema on every route, so a pass-through route judges it the same as /v1/messages. A Responses WebSocket frame now forwards the extra_body tools a rewrite stripped
* fix(guardrails): judge a named hosted tool by its type and keep a nested chat custom tool custom
* fix(guardrails): keep judging Anthropic client tools and type-named server tools by their name
* fix(guardrails): judge a named tool by its name alone only when the name is its type's own
* fix(guardrails): judge the function a tool names beside a declaration list or a top-level name
* fix(guardrails): judge Gemini hosted keys whatever the tool type and a namespace's function
* fix(guardrails): judge legacy functions entries like tools and send Gemini a namespace's members
* fix(guardrails): judge a namespace member by every name a provider reads and send Gemini its schema converted
* fix(guardrails): judge a legacy functions entry as a function and relax a tool_choice in extra_body
* fix(gemini): keep a namespace whole when inlining its members would repeat a function name
* fix(guardrails): judge a namespace member by every bare name that shares its qualified name
* fix(vertex_ai): declare a repeated namespace member name once instead of sending namespace names to Gemini
* fix(guardrails): judge a tool_choice sent in extra_body like a top-level one
A forced name only extra_body.tool_choice carried was never checked, so block mode let a denied name through to the provider
* fix(guardrails): judge a Gemini namespace member returned by its bare name and force a named namespace's members
* test: keep formatter churn out of the tool_permission and Gemini test files
* fix(guardrails): judge extra_body.input tools, argument-aware alias rules, and exact forced Gemini functions
Tools loaded in extra_body.input are now checked and stripped like top-level input. A returned
namespace member is judged by the alias whose first name rule its arguments satisfy. A forced
Gemini name that matches a top-level function keeps that function. Alias grouping and Gemini
declaration dedupe now run in one pass instead of quadratic scans.
* fix(guardrails): judge and rewrite every tool field sent inside extra_body
extra_body is merged into the upstream body, so functions, function_call,
mcp_servers and toolConfig nested there reached the model unjudged. Treat
extra_body as a second request layer that is collected and rewritten the
same way as the top-level request.
* fix(guardrails): judge Gemini hosted keys by key and count only the tools a provider receives after extra_body
* fix(guardrails): relax a tool_choice that names a dropped namespace member by its bare name
Gemini and Anthropic chat send namespace members under their bare names, so rewrite mode now treats a dropped member's bare name, and an emptied namespace's own name, as removed when no kept tool still carries it
* fix(gemini,guardrails): keep a top-level function over a same-named namespace member and leave malformed tool values to the provider
Gemini and Vertex chat now declare a top-level function over a namespace member that shares its name, so a forced call gets the top-level schema. post_call judges a returned name a top-level tool carries as that tool, and pre_call no longer raises on an unhashable or non-string tool name, input item type, or function_call name.
* fix(guardrails): judge a returned namespace member named like a hosted MCP server as that member
* fix(gemini,guardrails): send a namespace as main did when another tool has a member's name
Gemini chat inlines a namespace's members only when no other tool, namespace
member, or legacy functions entry declares the same name, and otherwise sends the
namespace by its own name as main did. A tool_choice naming such a namespace
forces the namespace itself. The tool permission post_call reads top-level names
the way Gemini declares them and judges a call the Responses bridge tags with a
namespace as that member.
* fix(gemini): send colliding namespace members as namespace__name functions and judge them in post_call
When a namespace member shares a name with another tool, Gemini chat now gets that namespace's members as namespace__name functions instead of one function named after the namespace, so every function it can call is a name the rules judged. Declared names are counted once, and a forced member maps to each holder's qualified name.
post_call judges a returned namespace__name as that member, by its qualified or bare name the way pre_call does, so a bare-name allowlist or denylist applies to it.
* fix(gemini): keep generated member names unique and let a top-level function own its exact name
A namespace that listed a member twice, or a generated namespace__name that
repeated a top-level function, sent Gemini two declarations with one name, and
Gemini answered 400 where main answered 200. A name a namespace lists twice is
now one member, and a top-level function keeps its exact name.
post_call now judges a returned name a top-level function declares as that
function, never as the namespace member the name could also mean.
* fix(gemini): qualify namespace members by the namespace's top-level name
The guardrail judges a member as `name__member` from the namespace's top-level
name. Gemini built the name from `function.name` when the tool also carried one,
so a request could pass pre-call and reach Gemini under a name no rule judged.
* fix(vertex_ai): force only the namespace members Gemini is sent
A forced namespace or member on Gemini chat could still allow a generated name that a top-level function or an earlier namespace owns, so tool_choice forced an unrelated function. The forced names now come from the same collision checks the declarations use
* fix(vertex_ai): reject a forced namespace Gemini gets no member of, and resolve forced names in one pass
* fix(guardrails): judge a namespace member by a rule on its namespace's name
---------
Co-authored-by: Christopher Krah <christopher.krah@plexito.de>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
