* fix(ui): resolve auto router templates through ids a * catch-all lists over a partial wildcard
* fix(ui): resolve wildcard-listed ids by their routed model in the Simple view and named aliases
A catch-all's fixed model or base_model no longer keys every listed id as that one model, a named alias like claude/* over anthropic/claude-* keys its ids by the model the router sends them to, and the Simple view suggests a pair from those ids too
* fix(ui): credit a wildcard-listed id only through the pattern and provider the router uses
When several named wildcards match a listed id, only the one the router's
specificity sort picks is credited. A captured route counts only when its
provider is one /model_group/info reports for the group, so an alias that
points elsewhere keeps its own name. Paused catch-alls credit nothing, and
a name with its own text before the star captures like the proxy lists it.
* fix(ui): resolve model group aliases before wildcards in auto router templates
An alias that shadows an id the catch-all lists now counts as the model the router sends it to, and paused deployments credit no template model
* fix(ui): wait for aliases, read them as admin only, and skip paused wildcard routes in auto router templates
* fix(ui): let slashless wildcard model names take part in route precedence
The router puts every model_name holding a star into its pattern list, with
or without a slash, so a listed id like opus-5-5 under a "*" catch-all goes
to a more specific opus-* deployment first. The dashboard skipped slashless
names when it picked the route, so a paused opus-* still credited opus-5-5
through the catch-all and a template could fill a tier that answers 403.
The old "derives no pattern from a slashless wildcard model_name" test
described /models listing, not routing, and is replaced by the routed case.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(proxy): pin admin-owned team keys to their team's MCP servers and toolsets
A virtual key on a real team owned by a proxy_admin or proxy_admin_viewer, with no MCP list of its
own, currently reaches every MCP server in the registry and any toolset, ignoring its team. These
tests expect it to resolve only what its team grants, like the same key owned by an internal user,
while the admin dashboard session keeps the whole registry.
* fix(proxy)!: bind admin-owned team keys to their team's MCP servers
A virtual key on a real team now resolves MCP servers and toolsets through its team even when its
owner is a proxy_admin or proxy_admin_viewer, the way team model access already binds it. The
unscoped admin view stays for the master key, admin keys with no team, the admin dashboard
session, and admitted user subjects. The toolset grant checks on /toolset/{name}/mcp and on the
Responses litellm_proxy MCP tool use the same rule.
BREAKING CHANGE: a team key owned by a proxy admin or admin viewer no longer reaches MCP servers or
toolsets its team does not grant. Grant them to the team or use a key with no team.
* fix(proxy): answer 403 for a toolset the key cannot use on LLM routes
A litellm_proxy MCP tool on /v1/responses, /v1/chat/completions or /v1/messages that named a toolset
the key's team was never granted was skipped, which left the server filter empty and handed the model
every tool the key could reach. The denied name now stays in the filter, so the request gets the same
403 an unknown server name gets.
Admin credentials on a real team stay bound to it whatever carries them, a lite login session included,
the way the team already bounds their model access. The tests pin that and drop an unused patch.
* fix(proxy): keep a denied toolset out of the scoped key beside a granted one
* test(responses): cover toolset name fallbacks and align MCP admin docstrings
* test(mcp): cover no-team admins' toolset access on LLM routes and drop a dead admin-check clause
* fix(responses): keep a whole-gateway MCP entry serving the key's tools beside a denied toolset
* test(responses): check the tools a whole-gateway request lists beside a denied toolset
* test(responses): keep the toolset lookup real in the whole-gateway listing test
* test(responses): run the toolset gateway tests on a real MCP server manager
* fix(responses): keep denied toolset names out of the MCP server filter
A denied toolset named like a server the key holds was matched to that server
by the loose server-name lookup. Denied names now never reach the filter, and a
request naming only denied toolsets answers 403 directly, each name once
* fix(responses): let a held MCP access group outrank a same-named ungranted toolset and give denied toolsets the generic 403
* test(responses): pin access group and granted toolset precedence for LLM-route MCP names
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Moves the page-authored English on /policies, /tool-policies and
/guardrails-monitor into react-i18next t() calls backed by en.json, and
adds a small data test for en.json. The rendered English is unchanged.
* test(ui): cover separate Meta Model API provider onboarding
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): add Meta Model API provider onboarding
Co-authored-by: lostmartian <sahilgangurde08@gmail.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): align Meta coverage with provider keys and listed registry
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show Meta model placeholder
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: lostmartian <sahilgangurde08@gmail.com>
* feat(keys): allow assigning a project to an unassigned key on /key/update
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(keys): suppress LIT002 on project assignment error payloads
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(keys): format project assignment error detail
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(keys): assert accepted project assignment flows into update payload
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(keys): drop stale LIT002 suppressions removed by main
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(keys): mock writer_db team lookup used by check_db_only path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(keys): write project assignment only while key row is still unassigned
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(keys): satisfy type gate on guarded project assignment write
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(keys): default soft-budget update path to unguarded write
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(keys): run guarded project assignment in one writer transaction
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(keys): include object permission link in assignment guard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(keys): return permission-linked payload without mutating the update dict
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(keys): fix type errors and stale noqa after main merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(keys): use Sequence in assignment guard model annotation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(keys): type update fields as a mapping in the key update path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(keys): place cast reason on the cast call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(keys): honor explicit null max_budget when assigning a project
* test(keys): assert explicit null max_budget flows into update payload
* fix(keys): match NULL models column in guarded project assignment
* fix(keys): authorize caller before project lookup on /key/update
* fix(keys): drop explicit null object_permission before project assignment write
---------
Co-authored-by: jesus <jesus@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): show ai&'s name and logo in the dashboard
The Add Model form already lists ai& from its provider metadata, but the dashboard had no entry for it, so the provider showed no logo and model tables printed the raw aiand slug. Adds ai& to the dashboard provider list with its official mark, plated in dark mode so the red and black mark keep their colours, and the aiand/openai/gpt-oss-120b model placeholder
* fix(aiand): retire deepseek-v4-flash, glm-5.2 and kimi-k2.7-code
ai& retired these three models on 2026-10-27 and https://api.aiand.com/v1/models now lists ten. The rows keep their prices with a deprecation_date, so earlier spend still prices, and aiand/* stops listing them, the same way baseten handles retired rows. The other ten rows already match the live prices, context windows and output limits
* feat(aiand): add deepseek-v4.1-flash-1017
ai& launches deepseek-ai/deepseek-v4.1-flash-1017 on 2026-11-02. It is served on https://staging-api.aiand.com/v1/models with the same price, context window, output limit and capabilities as deepseek-v4.1-flash, so its row copies that one
* fix(aiand): retire deepseek-v4.1-flash
ai& retires deepseek-ai/deepseek-v4.1-flash on 2026-10-30, now that deepseek-v4.1-flash-1017 is live in production on https://api.aiand.com/v1/models with the same price, limits and capabilities. Its row keeps its prices with a deprecation_date, like the other three retired ai& rows
* fix(aiand): retire deepseek-v4.1-flash on 2026-11-06
https://api.aiand.com/v1/models now gives deepseek-v4.1-flash a deprecation_date of 2026-11-05T15:00Z, which is 2026-11-06 00:00 in Japan, so its row uses 2026-11-06 instead of 2026-10-30 and aiand/* keeps listing it until then. The wildcard test now leaves a model out only once its retirement date has passed
* test(aiand): derive retired models from the rows instead of pinning dates
The wildcard test now reads each row's deprecation_date, so it holds whichever dates ai& sets, and the test that only restated the JSON dates is gone
* fix(aiand): retire models on their UTC date while the proxy runs, and add muse-spark-open
The wildcard listing now checks each row's deprecation_date when it expands, against the UTC date the field
documents, so a model drops on its day without a restart. deepseek-v4.1-flash now carries 2026-11-05, the
UTC day ai& retires it, and meta/muse-spark-open gets its price row from ai&'s live /v1/models
* fix(aiand): hide retired models only from bundled listings, including the wildcard health check
* fix(aiand): share the UTC retirement check with Together AI and the health check sort
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(streaming): normalize provider logprobs to prevent MockValSer serialization crash
* fix(streaming): convert upstream chunk logprobs to litellm ChoiceLogprobs so the finish chunk serializes
A streaming chat completion with logprobs on the OpenAI SDK path attached the
SDK's own ChoiceLogprobs object to the outgoing chunk when the chunk had no
content to rebuild from, which is the provider's finish chunk (empty delta,
logprobs for the end token, finish_reason). The SDK parses stream chunks
without building that nested class, so its pydantic serializer is still a
MockValSer and model_dump_json fails. The proxy then sent an error frame in
place of the finish chunk, and when the same chunk also carried usage the SDK
wrapper itself raised MidStreamFallbackError.
The attach site now validates upstream logprobs through a TypeAdapter that
dumps any non-litellm pydantic model before validating into litellm's
ChoiceLogprobs, and passes litellm instances through unchanged.
* fix(streaming): keep upstream logprobs that don't fit ChoiceLogprobs as plain JSON so the stream still finishes
* fix(streaming): build ChoiceLogprobs' serializer up front and keep nonconforming logprobs as JSON on chunk rebuilds
ChoiceLogprobs inherits the openai SDK's deferred build, so a fresh worker
whose first logprobs arrive on the finish chunk still had no serializer.
Foreign logprobs models now dump in full, so a missing top_logprobs becomes
[] and SDK callers keep attribute access, and chunk rebuilds for content and
usage keep logprobs that don't fit as plain JSON instead of raising.
* fix(streaming): keep only the logprobs keys upstream sent and type refusal logprobs
* fix(types): pass nonconforming non-streaming logprobs through instead of failing the response
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(router): apply content_policy_fallbacks when router settings update at runtime
* test(router): annotate the fallback regression test's locals as Final
* fix(proxy): validate router_settings.content_policy_fallbacks on /config/update like its siblings
* test(proxy): annotate the content_policy_fallbacks config update test locals as Final
* fix(proxy): keep litellm_settings content policy fallbacks, accept dict targets, skip invalid stored rules
A runtime content_policy_fallbacks rule now layers over the litellm_settings rules instead of replacing
them, so models the runtime rules do not name keep their file fallbacks. Fallback targets may be
deployment dicts as well as model names. A stored rule that fails validation is logged and skipped at
sync, so the other router settings still apply. Per-user token discovery now reads every fallback key
from key and team router_settings.
* fix(proxy): validate content policy fallback rules one by one and keep a config-owned list alone
An invalid stored rule is now skipped on its own instead of dropping every runtime rule, a router_settings list from the config file stays the router's only rules across DB syncs, and GET /fallback names the model of a dict target.
* fix(proxy): answer an error for a string-valued fallback rule on GET /fallback instead of its characters
* fix(proxy): keep config-file content-policy rules as Router init accepted them and log a skipped rule once per change
* fix(proxy): log a skipped content-policy rule whenever the saved rules change
* fix(proxy): keep general fallbacks when no content-policy rule is saved, name a model-less target's model, and take only model names in key rules
* fix(proxy): reject an empty dict target in a key's content-policy rule
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(usage): show subscription-covered providers and account fees separately
* fix(usage): write only the named fee columns and sync the root schema
The fee update now writes just the columns the request names, so two admins changing
different fields at once both land, and the dashboard edit form sends only what changed.
The root schema.prisma carries the new table like the two copies, and a billing period
anchored in the calendar's last month ends on December 31, 9999 instead of failing.
* fix(usage): register the subscription table span and keep provider logic under llms
* fix(subscription_accounts): register the fee routes as management routes and reject an empty account id
* fix(usage): run the new tests in CI and close subscription card review gaps
Adds the subscription test file to its unit shard, sets the migration lock timeout, rejects infinite fees, keeps the card from rewriting login files, lists each deployment once without aliases, shows the provider badge on entity views, and asks before removing a fee
* fix(usage): resolve named-credential ChatGPT accounts and keep admin viewers read-only on the fee card
* fix(usage): count a subscription fee only for accounts behind a current deployment
Usage listed a fee row for an account no deployment uses while another ChatGPT
deployment existed and dropped it when none did. It now lists only accounts behind a
current deployment in both cases. Adds a real proxy and Postgres integration test for
the fee API and for ChatGPT Responses rows on a deployment's own login, and names the
card's multi-component test with the integration suffix.
* fix(usage): keep a stored subscription fee listed after its account's deployments are removed
The usage route lists every account behind a subscription deployment plus every account carrying a stored fee, and reads the fees on every proxy, so a fee stays visible, counted, and editable for past ranges after its last deployment goes. The integration test now polls every day the calls spanned, so a UTC midnight crossing no longer fails it.
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(proxy): apply published model rows on every worker before the debounced resync
Config sync messages now carry the ids of model rows a create, update, or upsert wrote. Every worker that receives one loads those rows by id right away, including while its debounced full resync is still waiting, so a group added one deployment at a time is complete on every worker as soon as the write returns. Deletes, bulk writes, and messages from older pods carry no ids and keep the full resync path
* fix(proxy): load published model rows from the primary under the reconcile lock
Read the by-id rows through the writer so a lagging read replica cannot put an old row back, and read them inside MODEL_RECONCILE_LOCK so a concurrent delete or reload stays the last word. Look the ids up in bounded chunks, reload DB credentials only when a row names one this worker lacks, and leave auto-router rows to the debounced full resync. Add an integration test that boots two proxies and needs the peer to fall back to a just-added backup within 4 s.
* test(proxy): cover a published model row that names no credential
* fix(proxy): keep the unknown model name read-through on the replica and outside the lock
Only the by-id load that follows a published write reads the primary inside
the reconcile lock. A request for a model name a worker does not know reads
through the replica routing before taking the lock again, as on main, so
unknown names put no load on the primary and do not wait behind a reload.
* fix(proxy): keep access group edits from dropping staying deployments and batch published config ids
* fix(proxy): read access group rows from the primary so quick edits are not lost to a lagging replica
* fix(proxy): install ordinary rows queued with an auto router and check access group edits before writing
A batch of published model ids that held one auto router skipped every row in it, so ordinary
rows written at the same moment waited for the full reload. The by-id load now leaves only the
auto router rows to that reload.
An access group update now reads every deployment from the primary, answers 404 for an unknown
group and 400 for an unknown model id or name before it writes anything, then removes the group
from the deployments that leave before it adds it to the ones that join, so a failed write leaves
the group narrower than asked, never wider.
* fix(proxy): write an access group update in one transaction after a fresh read
The update now checks model names first, reads every deployment from the
primary, and commits every join and leave in one batch right after that
read, so a failed write changes nothing and a field another admin set
during validation is kept. Other workers hear about the rows only after
the commit.
* fix(proxy): lock the model rows while an access group update reads and rewrites them
* test(proxy): run the model resync tests on the real installer and a real router
The by-id resync tests now install rows through ProxyConfig's real installer and credential loader on a real Router, faking only the database, and the config sync test row model is frozen.
* fix(proxy): lock only the rows an access group update touches and answer 404 before 400
An access group update locked every model row, so a lock on any unrelated
row stalled it for 30 s and then failed it with a 500, and unrelated model
edits on other proxies waited behind it. It now takes a per-group advisory
lock, locks only the rows that carry the group or that it selects, and
reads them again once locked. The group existence check runs before the
model name check again, so a missing group answers 404 as on main.
* fix(proxy): never wait on a locked row in an access group update
The update now takes its row locks with NOWAIT. When another edit holds one
of the group's rows, it rolls back, holding nothing, and tries again for about
10 s before answering 409. Holding rows while waiting deadlocked against team
deletes and let a concurrent model edit undo the update's change. The
per-group lock also moves to the two-key advisory space, apart from team ids.
* fix(proxy): retry access group row locks for 30 s with SKIP LOCKED and lock only the rows model_ids select
* fix(proxy): take the access group lock with pg_try_advisory_xact_lock and retry while another edit holds it
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(realtime): send Gemini Live setup on the client's first message so session.update modalities apply
Gemini and Vertex AI Live accept exactly one setup per session, and the proxy sent its
default audio setup at connect, so a client's session.update (GA output_modalities or
beta modalities) was dropped and every answer came back as audio. The setup now goes
out on the client's first message: a session.update becomes that setup, anything else
gets the model default. Bedrock Nova Sonic keeps its connect-time setup through a
provider hook. Text setups label deltas response.output_text.delta, and downgrading
TEXT to AUDIO on an audio-only model logs a warning
* fix(realtime): keep one Gemini Live response modality and skip the "inf" token limit in the setup
* fix(realtime): merge every session.update sent before the first content frame into the Gemini Live setup
Gemini and Vertex Live accept one setup per connection, so the bridge used to
turn only the first session.update into that setup and drop the rest without
telling the client. Now every session.update that arrives before the first
content frame is held, deep-merged (later values win per field), acknowledged
with a session.updated built from the setup it would produce, and the merged
session becomes the one setup sent with the first conversation item, audio
frame, or tool result. The session events also surface the systemInstruction
text as instructions, which the old string check never matched because the
setup carries it as a Content object.
* fix(realtime): cap the session.update merge depth so the recursive detector accepts it
_merge_session_fields recurses once per nested mapping level, so CI's
recursive_detector flagged it. Bound the descent at eight levels (the OpenAI
session object nests four deep at most), replace anything deeper wholesale
with the later session.update's value, and list the helper with the other
depth-capped functions the detector allows.
* fix(gemini-live): accept a string output modality and drop values Live cannot produce
A bare string output_modalities is read as a one-element list and modality values Gemini Live cannot produce are dropped with a warning instead of reaching the backend, so a malformed session.update no longer costs the session. Transcription guardrails now fold their VAD disable into the first held session.update only, so the merged setup carries it exactly once.
* fix(gemini realtime): answer an error for session updates Gemini Live cannot apply
An output modality the model cannot produce, or a session.update sent after the
first content, now gets an invalid_request_error event instead of a silent swap
or drop, unless drop_params is set. The synthetic session.created reports the
model's default modality, setupComplete no longer adds an extra session.updated,
and merging nested session updates is linear in their size
* fix(gemini realtime): send the setup before guardrail blocks and idle deadlines, mark push-to-talk turns
A pre_call guardrail blocking the first typed message now goes out after the
setup, so Gemini and Vertex no longer close the session with 1007.
Push-to-talk sessions (turn_detection null or create_response false) wrap each
audio turn in empty activityStart and activityEnd messages, which Gemini needs
when automatic activity detection is off.
A session with no content after 60 seconds sends its setup, since Vertex
refuses a setup 300 seconds after connect. Held session.update frames past
10 MB answer session_update_too_large, and deferred mode now errors on a
second session.update unless drop_params is set.
* fix(realtime): ack unchanged Gemini Live session.update resends and honor null turn_detection
An update that leaves the applied setup unchanged now gets session.updated
instead of session_update_not_applied, so clients that resend their session
with the first response keep working. Deferred setup mode merges held updates
like the default mode, turn_detection null turns Gemini's activity detection
off, drop_params drops a voice outside the documented Live voices, and the
lazy setup deadline moves to 240 seconds
* fix(realtime): keep Vertex activity detection off for null turn_detection and set up deferred sessions before text content
* fix(realtime): refuse unscreenable Gemini audio, send held setup after buffered audio, reject unknown Vertex voices
A realtime_input_transcription guardrail cannot screen Gemini Live audio before the answer starts, so audio input under one is refused with an audio_input_not_screenable error instead of reaching the model. In deferred setup mode a session.update that follows buffered audio is sent as setup at once. Only conversation content triggers the setup, so control events no longer lock in a default one. Vertex rejects an undocumented voice on the session.update, or drops it with drop_params, while Google AI Studio forwards any name.
* fix(realtime): hold first content so a following session.update joins the Gemini setup
Convert flat realtime tool schemas through Gemini's schema builder, and collapse a text and audio pair to what the model can produce on text-only models too
* fix(realtime): check AI Studio Live voices like Vertex
* fix(realtime): forward any voice to AI Studio Live models that accept it
* fix(realtime): map OpenAI TTS voices before the Live voice check, keep outage spend rows, share the playground session config
The Live voice gate now judges the Gemini voice an OpenAI TTS name maps to, so fable and onyx reach Vertex as Umbriel and Orus again. An upstream close before the held setup was sent is logged as a session, not a refusal. The playground sends one server_vad session config for typed and voice turns, and the e2e, integration, and dashboard tests follow lazy setup.
* fix(realtime): take the lazy setup clock as a dependency so its tests run on a stepped clock
The deadline and grace tests waited on real 50 ms timers, and the merge cost
check compared wall-clock runs. The session now reads an injected clock,
time.monotonic by default, the two tests step it, and the merge cost check
moves to the CodSpeed benchmarks. The Nova Sonic style handler test passes
the backend through wrap_backend, which main added after the test was written
* feat(lite): offer each model group's reasoning efforts in pi, OpenCode and Codex
lite pi, lite opencode and lite codex now read supported_reasoning_efforts from /model_group/info and write them into each client's model config. pi gets reasoning and a thinkingLevelMap, OpenCode gets reasoning_options plus one variant per effort, and Codex fallback entries get supported_reasoning_levels with a default level. The proxy-served Codex catalog on /v1/models?client_version= offers the same levels
Codex before 0.138.0 fails the whole catalog on an effort it cannot name, so those releases, and an unreadable version, get only the six efforts every release parses. A group with unknown efforts keeps today's output, and a group with none is written as non-reasoning
The strength order and nearest_declared_reasoning_effort move to an SDK-free module so the lite CLI, which loads without initializing the SDK, can import the Codex catalog
* fix(lite): describe team models by their public name and keep Codex's none default
/model_group/info now looks up a team's own deployments when the caller's
team serves the requested name, so a team model reports its reasoning levels.
Codex fallback entries default to none when the group lists it, and the proxy
builds the Codex catalog in a worker thread
* fix(lite): round Codex's default level down and describe a team's shared-name model from its own deployments
Move the reasoning strength helper to litellm_core_utils so lite and the Codex catalog import it without the SDK or the router, and let make lint-format-check-changed diff the working tree so a staged rename passes.
* fix(lite): keep pi's system role on reasoning model groups
pi switches its system prompt to the developer role for any model marked reasoning, so an operator's
cache_control_injection_points entry for role system stopped matching on Claude groups. lite pi now sets
compat.supportsDeveloperRole false on those entries, so pi sends system as it did before.
* fix(codex): treat a client_version too long to parse as an old Codex and keep undescribed stock levels
A client_version past Python's integer digit limit made the Codex catalog answer 500. It now gets the
levels every Codex parses. A stock Codex level with no description validates again, as on base, and
lite codex treats undecodable codex --version output as an unknown version
* fix(lite): keep REASONING_EFFORT_STRENGTH_ORDER importable from router_utils and skip the Codex sync on undecodable debug models output
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(pass-through): add eu/jev decision API pass-through with spend tracking
Adds a /eu_jev/{endpoint:path} pass-through route for eu/jev, an EU-hosted
decision API that speaks the same POST /v1/systemone contract as TypeSafe.
It reads EU_JEV_API_KEY and EU_JEV_API_BASE (defaults to
https://jev.bevel.software), books spend through the shared decision logging
handler, and prices eu_jev/eu-jeff-1.0.0 and eu_jev/eu-jeff-latest from the
cost map
* fix(pass-through): default eu_jev to its canonical eu-jev.com origin
* fix(pass-through): reject custom_body on the eu_jev route
A key limited to other models could send an eu/jev request through
custom_body, because auth checks the outer body while the shared
pass-through forwards custom_body upstream. The route now answers 400
before anything is forwarded, the same way validate_oss_request guards
the laya and bespoke routes
* fix(pass-through): give the eu_jev route request and 200 response schemas
The OpenAPI docs gate flagged POST /eu_jev/{endpoint} for OA001 and
OA003. The route now documents the eu/jev System One request body and
its 200 answer, usage and meta shape, taken from the eu/jev OpenAPI
document. The lazy OpenAPI snapshot and the dashboard API types are
regenerated with python -m litellm.proxy._lazy_openapi_snapshot and
npm run gen:api
* fix(pass-through): reject streaming on the eu_jev route
A body with stream=true switched the route to streaming, which skips the
decision spend tracking. Reject it with a 400, as validate_oss_request does
for laya and bespoke.
* fix(pass-through): type the eu_jev request body like the other decision routes
* fix(pass-through): check eu_jev decisions against the model eu/jev runs
Serve only POST /eu_jev/v1/systemone, eu/jev's one endpoint, resolve an omitted
model and the legacy names to eu-jeff-latest for key model checks and pricing,
check scopes under eu_jev/<model> like Laya and Bespoke, and answer 503 when
EU_JEV_API_KEY is unset
* fix(pass-through): apply eu_jev key limits, register eu_jev, price unlisted releases
Pre-call hooks and guardrails now see the body model as eu_jev/<release>, so
model_rpm_limit and model_tpm_limit set on eu_jev/eu-jeff-latest apply, and
upstream still gets the client's exact body. eu_jev is a registered decisions
provider, so access groups, config deployments, /v1/systemone and /v1/decisions
accept it, and the Admin UI can add it. A release without its own cost map row
is priced as eu-jeff-latest instead of $0
* fix(eu_jev): use the pass-through deployment's api_base and accept provider-prefixed models
* fix(eu_jev): declare /v1/systemone on the eu/jev cost map entries so the generated endpoint table keeps it
* fix(eu_jev): take pass-through credentials only from proxy-wide deployments
The eu/jev route now ignores team-scoped deployments when it picks the
upstream key and base, so one team's deployment can no longer redirect
another team's calls, and a base ending in /v1 is not doubled
* fix(eu_jev): let a configured pass-through at /eu_jev/v1/systemone keep serving it
Scope the mapped pass-through and gateway allowlist entries to the decision
route instead of the whole /eu_jev prefix, so custom routes under /eu_jev keep
their auth, metering, and gateway behavior from before this PR
* fix(eu_jev): treat /eu_jev/v1/systemone as a provider route only while the built-in route serves it
Drop the decision path from the mapped pass-through list and classify it with one
check that asks the pass-through registry, so an operator's own pass-through at that
path keeps its auth, metering, LLM-route status and litellm_user_api_key handling
from before this PR. The built-in route now yields on a per-request root path and
on any configured method, and the gateway keeps only the exact decision path
* fix(eu_jev): find a configured route at the decision path with or without a trailing slash, and stop a deleted one forwarding
* test(eu_jev): annotate new test variables with Final
* fix(proxy): keep eu/jev pass-through routing stable during updates and method mismatches
Swap an updated pass-through's routes with no await between removal and
re-registration, so the built-in eu/jev route never sees the path as
unconfigured mid-update. The built-in eu/jev handler no longer takes a
configured route's target, and the built-in route also yields on a
method-only match, so GET and PUT on the no-slash form of a slash route
redirect again.
* test(pass-through): annotate the route-swap test's locals as Final
* fix(pass-through): keep an updated auth route's LLM API route checks on the updating worker
* fix(eu_jev): leave a null model out of the upstream decision request
A JSON model of null is limited and counted as eu-jeff-latest, so send the
request upstream without a model field, the way an omitted model goes, since
eu/jev answers a null model with 400 Unknown model
* fix(pass-through): keep route checks on a path another auth endpoint still serves after an update
Forward a non-string eu/jev model unchanged so eu/jev rejects it, and drop only a null or missing one
* test(eu_jev): keep the non-string model test in the eu/jev route test file so a CI shard runs it
* refactor(eu_jev): build the scoped pass-through body as a new dict instead of updating it in place
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(ui): pin the Workflows page copy with characterization tests
Locks the header, table, filters drawer, detail drawer, relative time and duration text on the Workflows page so the upcoming move onto i18next can be checked against it without editing any test
* feat(ui): add i18next plumbing and move the Workflows page copy onto it
Adds an English-only i18next instance with typed keys from src/i18n/locales/en.json, wraps the root layout in I18nProvider, and loads the instance in the test setup. Every string the Workflows page authors now renders through t(), while backend values such as run status, roles, event types and metadata keys stay raw. The rendered English is unchanged
* fix(guardrails)!: make lakera_v2 on_flagged block reject PII-only detections, add mask
on_flagged block now returns 400 when Lakera flags only PII, instead of
masking it and forwarding the request. on_flagged mask keeps the old
behavior: mask PII-only detections and block anything else.
* test(guardrails): cover lakera_v2 on_flagged on every endpoint, hook and a Lakera outage
* test(guardrails): type the lakera_v2 on_flagged unit test helpers
* feat(proxy): reject chat routes on models declared non-chat via model_info.supported_endpoints
A model whose every deployment sets a non-chat model_info.mode and a supported_endpoints list with no chat endpoint now answers /v1/chat/completions, /v1/messages and /v1/responses with a 400 before any provider call. Models without both fields keep today's behavior.
* fix(proxy): let fallbacks answer and cover more chat routes in the non-chat model check
The check now steps aside when a request, key, team or router fallback would answer, skips such models in the rate-limit fallback loop, ignores mode and supported_endpoints copied from the cost map by old Admin UI saves, treats a blank mode as unset, and also covers /v1/completions, the Responses WebSocket, Gemini generateContent and /v1beta/interactions
* fix(proxy): only step aside for fallbacks that can answer chat, gate the Responses WebSocket after alias resolution, and compare declarations to the loaded catalog
* fix(proxy): gate the Responses WebSocket with the router after aliases resolve, and ignore client-set internal fallback flags
* fix(proxy): walk fallback groups breadth first and release reservations on a rejected WebSocket
Each fallback group is looked up once however often the request repeats it, and a WebSocket closed by the endpoint gate runs the failure hook so rate-limit reservations are released
* fix(proxy): walk fallback layers without recursion and gate the compact Responses route
The fallback walk now folds one layer at a time, so the recursion check passes, and /v1/responses/compact on a model declared non-chat gets the same 400 as the other chat-shaped routes
* fix(proxy): keep chat on rows whose cost map entry serves chat
A row that carries the cost map key of an entry that serves chat counts as declaring nothing, so a catalog edit that adds a chat path never turns old Admin UI copies into 400s
* fix(proxy): check chat sent to a deployment id or backend model name
* fix(proxy): check a model the read-through loads from the database
* fix(proxy): step aside for fallbacks the check cannot follow and wildcards that serve a backend name
A fallback target with no deployments of its own, a client routing stamp, and a wildcard that does not match the requested backend name no longer turn a working request into a 400
* fix(proxy): ignore a client model_group that is not a string when building rate-limit headers
A client metadata.model_group sent as an int or a list crashed the error handler with a 500 once the embedding-only check rejected the call before routing. The header helper now treats a non-string stamp as no model group. Drops the pattern clause in the backend-name lookup, since the group lookup already returns wildcard matches. Adds the third audit round's routing-stamp cells.
* fix(proxy): keep a non-string client model_group out of spend rows
* fix(proxy): validate standard fallbacks before the typed fallback-group lookup
* feat(router): opt in to non-chat deployment filtering
* fix(router): skip unresolved local rate-limit fallbacks
* fix(router): ignore caller-provided endpoint selector values
* fix(router): normalize declared chat endpoint prefixes
* test(router): cover preflight error-value behavior
* test(router): derive replay metadata defaults from model info
* fix(router): preserve fallback aliases and check synchronous text calls
* test(routing): rewrite the WebSocket model with a pre-call callback, since main no longer runs guardrails at socket admission
* test(routing): keep the endpoint gate's owned proxies on the worker's database
* fix(router): keep the endpoint gate on model_info patches that omit it and on same-group stream retries
A partial PATCH of model_info reset a stored enforce_supported_endpoints true back to false, because the
field defaulted to False and the update merges model_dump(exclude_none=True). The field now defaults to None
so an omitted flag keeps the stored value
The same-group retry of a stream dropped before content called the provider without the endpoint selection
context, so it could land on an enforced embedding-only deployment of the same group. It now runs inside
endpoint_selection like every other router attempt
* fix(router): probe rate-limit fallbacks through get_routable_deployments so the router surface does not grow
Main's router fence lets code outside the router use only listed Router members and
router_utils modules. The proxy's fallback probe now calls the listed
get_routable_deployments with the request's call type, and the gate's 400 moves to
litellm.exceptions as EndpointNotSupportedByModelError. It stays separate from main's
UnsupportedEndpointError, which the router retries on the group's other deployments,
while the gate refuses the whole group
* test(responses): cover the WebSocket error frame for a model that refuses the endpoint
* fix(proxy): let the rate-limit fallback probe try a fallback this worker has not loaded yet
The probe now skips a fallback only when the endpoint check removed every deployment it has, so a model added on another worker still reaches route_request and its database lookup
* fix(proxy): keep key router settings off the Responses WebSocket, read unrooted or single-string endpoint lists, and add status to the refusal frame
* fix(router): warn when enforce_supported_endpoints has no usable list and match chat endpoints regardless of case, trailing slash, or Gemini path
* test(router): type the load-warning test's caplog and router
* test(routing): match the endpoint gate's integration tests to base fallback behavior on unloaded fallbacks and the Responses WebSocket
---------
Co-authored-by: Mateo <mateo@Mateos-MBP-2.localdomain>
* feat(vector_stores): add Amazon OpenSearch as a vector store provider
* feat(ui): add Amazon OpenSearch to the vector store provider dropdown
* feat(vector_stores): add OpenSearch k-NN search as a vector store provider
* test(vector_stores): hold the OpenSearch provider tests alone for the first review
The provider source comes back in the next commits on this PR
* ci(tests): run the OpenSearch vector store unit tests
* fix(vector_stores): sign on.aws and FIPS OpenSearch hosts and reject missing AWS credentials before embedding
* fix(vector_stores): regenerate provider endpoint support for opensearch
* fix(vector_stores): keep opensearch retrieve, update, and delete off the index and refuse half-set AWS keys and per-account endpoints
* fix(vector_stores): return only scalar opensearch attributes and keep credential source output out of 401s
* test(vector_stores): inject a failing opensearch credential source instead of running a shell
* fix(vector_stores): sign opensearch for the endpoint's region, refuse list, add opensearch_index_name, honor drop_params
* fix(vector_stores): refuse a caller-chosen opensearch_index_name on search and rag query
* fix(vector_stores): refuse opensearch_index_name only on OpenSearch stores, keep the old 400 text
Other providers drop the key again as they did before OpenSearch support, and the four embedding keys keep their original error sentence.
* fix(vector_stores): refuse a caller index on slash-form OpenSearch stores
* ci: run the dashboard API types check on every change that can alter the OpenAPI spec
* chore: shorten the spec_pattern comment to a pointer at the CI filter
* chore: drop the spec_pattern comment
* docs: name the full ui-api-types trigger set in the make check header and dashboard AGENTS.md
* test: check make check and CI pick the same files for the API-type sync
* ci: run the Terraform provider endpoint audit on enterprise route changes
* test: keep the Terraform trigger test in the mapped workflow test file
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(proxy): require a key for the Claude Code marketplace when configured and keep exception text out of its 500s
* feat(proxy): general_settings.claude_code_marketplace_auth_required requires a LiteLLM key for the Claude Code marketplace
When true, GET /claude-code/marketplace.json answers 401 without a key in Authorization,
x-litellm-api-key, the litellm_key_header_name header, or ?key=, and GET /public/skill_hub
returns an empty list. Both routes' 500 bodies carry a fixed message instead of the exception text.
* fix(proxy): refuse ?key= and send WWW-Authenticate when the Claude Code marketplace requires a key
* fix(proxy): close the marketplace key gate's outage, quoted-setting, and dashboard gaps
Refuse the database-outage fallback identity with a 401, read a quoted
"true" as on and an unreadable value as on, keep the skill hub's 500
detail a fixed string, and have the dashboard setup tab show a settings
snippet with a key header when the marketplace requires a key
* fix(proxy): keep the no-database 500, validate the marketplace key setting at boot, and show the key snippet until the probe answers
* fix(claude-code): read the marketplace key setting from the UI config and answer database outages with 503
* fix(claude-code): keep the no-database hint on skill_hub and warn that the remove step uninstalls plugins
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(responses): resume background response streams with stream and starting_after
Retrieving a response with stream=true and starting_after=N now returns the
replayed event stream from OpenAI and Azure instead of one JSON object, on the
SDK (get_responses / aget_responses) and on the proxy's GET /v1/responses/{id}
Providers that cannot replay a stored stream drop the params under drop_params
and raise UnsupportedParamsError without it. Malformed values answer 400 with
the param named. A replay that reaches its terminal event settles the
background response's bill once and passes the provider's usage through
without estimating one. An empty replay (cursor at the end) ends normally
* fix(responses): encrypt replayed stream ids and resume Azure AI Foundry streams
A replayed background stream now gets the same response id encryption as a
created one, so the key that made the response can reuse the id. Azure AI
Foundry deployments resume streams like Azure ones, and the chat-bridged
Responses iterator closes its wrapped stream instead of raising.
* fix(responses): drop malformed resume params under drop_params and keep JSON answers to a streamed retrieve
A bad stream flag drops to a plain retrieve and a bad cursor to a full replay when drop_params is on, and a cursor without stream is ignored even when malformed. A streamed retrieve that the upstream answers with JSON returns the response object instead of an empty stream. get_responses forwards only the include keys from extra_query.
* fix(responses): give proxy stream resumes the deployment stream_timeout and document retrieve query params
* fix(responses): keep deployment stream off retrieves and reject oversized cursors
* fix(responses): keep router default stream off plain retrieves
* test(responses): black box coverage for resume refusals, drop_params, stream timeout and billing
* test(responses): replay a real stream in the billing test and type the fake wrapper chunks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat: add BigModel provider with native Chat, Responses, and Messages APIs
Co-authored-by: Bytechoreographer <Bytechoreographer@users.noreply.github.com>
* test(providers): make BigModel unit tests a package
Co-authored-by: Bytechoreographer <Bytechoreographer@users.noreply.github.com>
* docs(providers): clarify BigModel parameter interface
Co-authored-by: Bytechoreographer <Bytechoreographer@users.noreply.github.com>
* fix(bigmodel): keep the Coding Plan base, surface Responses auth errors, and scope routes
Chat Completions keeps a GLM Coding Plan api_base. Responses raises the status from
BigModel's success=false body instead of returning an empty 200, and a JSON body
answering a stream request is replayed as a one-shot stream. Messages streams keep
ping and [DONE] lines. bigmodel leaves openai_compatible_providers so speech, image,
and transcription routes send it no request
* fix(responses): replay a JSON body on a stream request only when it has a Content-Length
A gateway that labels a chunked SSE stream as application/json kept streaming before
the replay, so only a sized body is read in full
* fix(bigmodel): scope the JSON stream replay to BigModel, map its business error codes, and echo requested tools
* fix(bigmodel): forward Codex's include and parallel_tool_calls, keep the OpenAI org header off BigModel, and pass unchanged Messages stream events through
* fix(bigmodel): map disabled thinking to low effort, keep Messages temperature, raise context window and content policy errors, and default parallel_tool_calls to true
* fix(bigmodel): keep the caller's reasoning_effort when Messages rewrites disabled thinking
* fix(bigmodel): keep context-window and content-policy errors on streamed Responses replays
---------
Co-authored-by: Bytechoreographer <Bytechoreographer@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(router): reset_time picks when deployment rpd and tpd daily limits start over
reset_time is "HH:MM" in UTC next to rpd/tpd in litellm_params or model_info. The daily cache keys are dated by now minus that offset and the daily 429 retry-after counts to the next reset, so a free tier whose provider quota resets at 08:00 UTC comes back at 08:00 instead of midnight. Without it the window stays the UTC calendar day.
* chore(ui): regenerate schema.d.ts with reset_time
* refactor(router): rpd, tpd and reset_time are read from litellm_params only
* feat(router): rename reset_time to daily_reset_time, drop the request-sent value, accept only HH:MM
* chore(ui): regenerate schema.d.ts for daily_reset_time
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat: add StreamLake provider metadata
* fix(streamlake): complete provider discovery and test request flows
* test(streamlake): assert cost and tool-param invariants instead of pinning vendor prices
* fix(streamlake): accept tools and reasoning on endpoint ids and match the current price page
Endpoint ids have no cost-map row, so tools and reasoning_effort were rejected
before reaching StreamLake. The provider now declares both capabilities, and
named non-reasoning rows opt out explicitly.
The rows now follow StreamLake's 2026-09-17 price page: drop the KAT-Coder and
Qwen3 rows it no longer serves, add the served models that had no price, drop
account-level rpm and tpm, and fill missing context windows. StreamLake is also
registered for streamlake/* wildcards and its messages and responses support.
* fix(streamlake): regenerate the endpoint matrix and type the new tests
The backup endpoint matrix advertised chat only while the root map listed
Messages and Responses, so /public/endpoints hid both. Regenerating with
scripts/generate_provider_endpoints_support.py makes the two files match.
Types the StreamLake tests and sets the Add Model test's API key with
fireEvent.change, per the dashboard test guide.
* fix(streamlake): drop the retired max_tokens field from the StreamLake cost-map rows
* fix(streamlake): price the four served models StreamLake publishes rates for and add the Qwen3.8 cache prices
* fix(streamlake): send token caps as max_completion_tokens and mark image models
StreamLake ignores max_tokens on kimi-k2.6, kimi-k2.7-code, qwen3.6-27b and
qwen3.6-35b-a3b, while every served model honors max_completion_tokens, so
the JSON provider now maps max_tokens to max_completion_tokens instead of the
reverse. Rows whose models read an image in a live probe set supports_vision,
and the Qwen3.8 rows drop the explicit cache write price, which this route
never uses since cache_control is dropped
* fix(streamlake): keep cache_control marks, price explicit cache reads, and use API input limits
StreamLake reads cache_control, so the provider keeps the marks through a preserve_cache_control flag. Qwen3.8 explicit cache hits now use their own read price, rows with a cache read price declare prompt caching, three models StreamLake serves without reasoning drop reasoning_effort, and max_input_tokens follows the limits the API reports.
* fix(utils): warn when a custom_provider_map name matches a built-in provider
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(router): add deployment rpd and tpd daily limits to enforce_model_rate_limits
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): route rate limit cache increments through typed helpers
* chore(ui): regenerate schema.d.ts for deployment rpd and tpd fields
* test(types): add rpd and tpd to the owned litellm_params inventory
* fix(router): run daily rpd/tpd checks after the minute checks and increment rpm and rpd in one cache trip
* fix(router): flatten the async rpm reject branch for the ruff strict gate
* fix(router): hand rpm and rpd back on any rate limit reject
A request the rpm, rpd or tpd check rejects no longer keeps its request counts, so retries against a deployment whose rpd is spent cannot fill its rpm and block the in-group fallover with the minute 429.
* test(router): a minute-rejected request no longer keeps its rpm count
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(skills): add a litellm-setup agent skill
* fix(skills): run the quickstart for Docker with Postgres in litellm-setup
Use scripts/quickstart.sh for the Docker plus Postgres path instead of a
hand-rolled Postgres container, keep uv with the user's own Postgres, load
.env as data instead of sourcing it, pin the uv Python range, chmod an
existing .env, list bedrock_converse and vertex_ai rows in the cost map
lookup, ask before editing the shell profile, and stop the health wait
when the proxy exits
* fix(skills): guide empty Postgres, docker sudo, and key fixes in litellm-setup
Ask for a new empty database on the uv path and DIRECT_URL behind a pooler,
add the baseline error to troubleshooting, and drop inherited database
settings in the loader line. Check python3 and lsof, explain Docker's
permission denied, give quickstart commands by the folder's path, and fix
provider keys by recreating the container or deleting the quickstart model.
Drop the structure-only test.
* fix(skills): keep shell profiles untouched when litellm-setup installs uv
The uv installer edits shell profiles unless UV_NO_MODIFY_PATH is set, so the skill now sets it and puts ~/.local/bin on PATH for later commands. It also says litellm --setup runs in a separate terminal and how to retire the shared master-key UI login
* fix(skills): offer litellm --setup only on the uv path
The Docker paths install no host litellm CLI, and the old wording also passed the skill's own step number on to the user
* fix(skills): ask before using an env provider key, print model names config.yaml accepts, probe the port with curl
* fix(skills): state the env key default in the question, chmod the wizard config, keep uv's bin dir on PATH
* fix(skills): keep litellm-setup on its own .env and install when the shell has others
* fix(skills): read the setup wizard's config without printing keys and hand off the exact restart command
* fix(skills): hide the database-access dev skill from end-user skill installs
* fix(skills): give the litellm --setup path a restart command that keeps its config
* docs(skills): point litellm --setup users to the master key in litellm_config.yaml
* fix(skills): hide every non-end-user skill, use Azure deployment names, keep the restart block out of the user's shell
* test(skills): find non-end-user skills in-process instead of through git
* docs(skills): price Azure deployments with base_model and put uv on PATH in the start block
* feat(ui): let the per-model budget field take a model pattern or access group
* feat(proxy): let model_max_budget entries name a model family or access group, shared across replicas
* fix(proxy): open each shared model budget window once for every replica, and match patterns in linear time
Without Redis, the first replica to charge an entry in a new window now opens it in the window spend table and
the others join it, so no flush replaces spend another replica already added. Budget patterns match by splitting
on `*` instead of a regular expression, a refused request names the entry that refused it, and the key form no
longer offers a typed pattern another row already budgets.
* fix(proxy): read the shared budget window from the writer, and keep concurrent first charges on one replica
Opening a shared window now reads the row from the writer, since a lagging read replica could hand a replica its
own start and let a flush replace the others' spend. A charge that waited on the open no longer resets a local
window another charge on the same replica opened meanwhile. Window tests use fixed timestamps.
* fix(proxy): stop a non proxy admin from lifting a team model cap with a looser group or pattern entry
Group and pattern entries now take precedence over other entries, so a team update by anyone but a proxy admin
must keep each new or changed entry within every existing cap it can take precedence over. A pattern can match a
provider-prefixed or wildcard-routed name, so any two patterns count as overlapping, and a model or group entry
counts as outranking every pattern.
* fix(proxy): narrow model budget pattern overlap, keep team caps under key patterns, and accept only typed patterns in the UI
* fix(proxy): keep model family budgets through database blips, team models and key entries
A failed window open no longer restarts the window at this replica's clock, so it cannot wipe shared spend. Spend charged meanwhile stays local and joins the shared window once the database answers
A key entry named after a model replaces only the team entry of that name, so team pattern and group caps still apply. Team-scoped models are gated and charged under their public name, and group lookups see team deployments
The team update guard compares caps by window length and only ranks patterns that can match the same name, including Bedrock vendor patterns behind a bedrock wildcard route
* fix(proxy): refuse team entries that take precedence over a capped model_max_budget entry
A non-proxy-admin could add a carve-out (an exact model, a narrower pattern or
a group) at or under the family cap, and its own counter then multiplied the
team's spend on that family. Any new entry that takes precedence over a capped
entry now needs a proxy admin, and group members include the team's own public
model names so team deployments cannot hide a group overlap.
* test(router): cover team_public_model_names_of
* docs(proxy): name pattern and access group keys in model_max_budget docstrings
* fix(proxy): name the entry holding the cap in the team model budget raise refusal
* fix(proxy): check precedence when a team edit makes an unenforced model_max_budget entry take effect
* fix(proxy): let an org admin give a legacy model_max_budget row a budget_duration at the same or a lower cap
* fix(proxy): keep a legacy model_max_budget row's stored cap when an org admin gives it a budget_duration
* feat(ui): add per-server MCP tool catalog query hook
One query per server keyed by server_id, so the toolset editor can load
tools for many servers in parallel and a failing server surfaces as an
error on its own row instead of an empty list. (LIT-10012)
* feat(ui): add toolset tool search and bulk selection helpers
Pure functions for the toolset editor: filter a server's tools by a
search query, select/deselect many tools without duplicates or dropping
other selections, and compute tri-state selection. (LIT-10012)
* feat(ui): search tools and bulk select in the toolset editor
The toolset editor search now matches tool names and descriptions as
well as server names. Typing fetches every server's tools in parallel
through the per-server query hook, auto-expands matching servers and
hides servers that loaded with no match. Tool rows become checkboxes,
each server header gets a tri-state select-all over its visible tools,
and a bar offers select/deselect all matching tools. Bulk actions only
touch visible tools, so selections on other servers are kept on save.
(LIT-10012)
* feat(ui): clear all and flag unconfirmed tools in the toolset panel
Adds a Clear all action to the Your Toolset panel and labels selected
tools whose server is gone, unreachable, or no longer lists the tool.
Those tools stay selected and are still saved, so an outage never
silently drops part of a toolset. (LIT-10012)
* test(ui): cover toolset editor tool search and bulk selection
Integration tests that drive the real toolset modal against stubbed
networking and assert the saved tools payload: cross-server search by
tool name and description, select/deselect all matching, server
select-all, single-row toggles, Clear all, and that tools on unreachable
or deleted servers survive a save. (LIT-10012)
* fix(ui): pointer cursor on the toolset server select-all checkbox
Base UI renders the checkbox as a span, which the global button cursor
rule does not cover, so the server header select-all showed the default
arrow. Use a pointer, and not-allowed while it is disabled. (LIT-10012)
* fix(ui): stop toolset editor background fan-out and keep tools on failed refetch
The toolset modal stays mounted when closed, so a finished search kept
every server's tool query enabled and refocusing the window refetched
all of them once stale. Fetch only while the editor is open, require
both the live and debounced search before fanning out, and disable
refetch on window focus for these queries. A failed refetch now keeps
the tools that already loaded instead of flagging the server
unreachable, and tools without a name are dropped. Trim comments that
restated the code. (LIT-10012)
* fix(ui): clear toolset search on close and only keep non-empty tool lists on failed refetch
Reopening the editor after a search rendered once with the old query,
which fetched every stale server; clear the search whenever the editor
closes. A failed refetch now keeps the previous list only when it has
tools, so a server that loaded none and then fails reports the error.
(LIT-10012)
* fix(ui): reset toolset editor state on close and avoid mutating selection sets
Closing the editor now also clears expanded servers, so reopening it cannot
refetch them for one render. Build the selection and expanded-server sets
without in-place mutation, drop a comment that restated the code, and use
screen queries in the new tests to stay within the prefer-screen-queries
lint budget. (LIT-10012)
* feat(daily-activity): read guardrail spend per key from the daily guardrail usage table
Adds DailyActivityRepository.guardrail_spend, which sums LiteLLM_DailyGuardrailUsageUnits.cost by date, guardrail and key. A guardrail row counts only when its key has a spend row on that day under the same scope filters as the rest of the daily activity response, so user, key and date filters apply to guardrail cost too
* feat(daily-activity): add a per-guardrail spend breakdown to /user/daily/activity/aggregated
Each day now has breakdown.guardrails[<guardrail_id>] with metrics.spend and an api_key_breakdown carrying each key's spend and metadata. Guardrail entries only report spend, since that is the one metric stored per key. The breakdown is on the user endpoint only, is empty when a model filter is set because guardrail cost is not stored per model, and lists only keys within api_key_limit while the guardrail total keeps every key. spend and budgets are unchanged
Resolves LIT-9785
* feat(daily-activity): add the guardrail breakdown to team and organization activity, with guardrail names
/team/daily/activity/aggregated and /organization/daily/activity/aggregated now return breakdown.guardrails the same way the user endpoint does, since each key belongs to one team and one organization. Each guardrail entry also carries metadata.guardrail_name, read from LiteLLM_GuardrailsTable for guardrails created through the UI or API and falling back to the breakdown key for config.yaml guardrails, whose key is already their name
* fix(daily-activity): keep daily activity up when the guardrail breakdown query fails, and treat an empty model filter as none
A database error from the guardrail spend query now logs a warning and returns the response without guardrails, where it used to fail the whole aggregated response with a 500. An empty model= no longer drops the guardrail breakdown, matching how the rest of the endpoint already ignores it. The daily activity golden files and fixtures gain the empty guardrails key every breakdown now carries
* refactor(daily-activity): fold guardrail breakdown in a worker thread and drop docstrings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(daily-activity): report untracked guardrail units next to spend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(files): cover owner-scoped file listing under require_managed_files
* fix(files): list only the caller's managed files on every list route under require_managed_files
* fix(files): narrow owner-scoped managed file lists to the named provider or model and page with after_id and before_id
* fix(files): narrow scoped lists the way uploads route, and refuse lists that cannot be scoped
A model name now matches every deployment that serves it, wildcards and
aliases included, and a provider matches deployments by the provider
uploads resolve, so anthropic/* and unprefixed Claude models count. The
OpenAI and Azure lists also return the caller's pass-through uploads.
An unknown provider and different after and after_id cursors answer 400,
and a list with no matching deployment skips the database read.
* fix(files): match stored files by model name and keep flag-off lists as on main
Under require_managed_files, a narrowed list now matches a file's model
mappings by deployment id or model name, so files LiteLLM stores in its own
database show on their model and provider lists. The Anthropic list defaults
to 20 rows when no limit is sent, after_id, before_id and the cursor 400s
apply only under the flag, and an admin keeps the provider list when the
managed files hook is not loaded. List errors are returned as values and
mapped to HTTP errors in one place.
* fix(files): keep admin provider lists and index the owner-scoped walk
Under require_managed_files, an admin keeps the provider list as on main, and
only a non-admin list reads the caller's own and team managed files. A stored
file now also shows on the provider its model name routes to through a
wildcard, with exact model names first, and a list routed to a model takes the
default page size of the provider serving it. A new index on created_by and
created_at keeps each chunk of the walk an index read, and the model check
runs before the file object is parsed.
* fix(files): filter provider and model lists in the query
A non-admin list scoped to a provider or model now asks Postgres only for the
caller's rows that map to one of that scope's deployments or models, or that
live in file storage, instead of reading every owned row and filtering them in
Python. A caller with many files on other providers no longer waits for the
whole set to be read. Team members' files are matched in the same query.
* test(files): type the fake raw query in the files list tests
* refactor(proxy): list managed files through one typed database handle
* fix(files): list alias and routing-group storage uploads on their provider
AriaAvatar pointed at /ui/assets/aria-agent.png, a path that only exists when
the proxy serves the static export under /ui. The dashboard dev server and the
split-chart nginx image 404 it, so the header Aria button and every other
AriaAvatar showed a broken image. Import the PNG statically like the provider
logos (#34125) so the bundle always serves it under /_next/static/media
Replaces the hand-rolled kbd and code key chips across the dashboard with
the shared Kbd component from components/ui/kbd, so every shortcut hint
renders the same way
* feat(providers): add ZeroGPU as an OpenAI-compatible provider
Registers zerogpu through the JSON provider registry so zerogpu/<model> routes to
https://api.zerogpu.ai/v1/chat/completions with ZEROGPU_API_KEY and an optional ZEROGPU_API_BASE.
Adds LlmProviders.ZEROGPU, the constants.py provider and endpoint lists, cost map entries for the
10 ZeroGPU text generation models, the endpoint matrix, the Add Model form fields, the README row
and the dashboard provider entry
* feat(zerogpu): add capability flags verified against the live API
Live calls to https://api.zerogpu.ai/v1/chat/completions confirmed tools, tool_choice and json_schema
response_format on 8 of the 10 catalog text models, cached token reporting on deepseek-v4.1-flash,
gpt-4.1-mini, gpt-5.4-nano and gpt-5.6-luna, and reasoning_effort on gpt-5.6-luna. LFM2.5-1.2B-Instruct
answers stream requests with plain JSON instead of SSE and LFM2.5-1.2B-Thinking returns 500 on every
request, so both stay out of the cost map until the API serves them correctly
* feat(zerogpu): price LFM2.5-1.2B-Instruct
Adds zerogpu/LFM2.5-1.2B-Instruct to the cost map with the catalog prices and context window. It
gets no capability flags because the live API ignores tools, response_format and
max_completion_tokens for this model, so LiteLLM drops tool params for it instead of sending them.
LFM2.5-1.2B-Thinking stays out of the cost map
* fix(zerogpu): reject transcription requests before calling the upstream API
ZeroGPU only serves chat completions, so drop it from the OpenAI transcription transport set. A zerogpu transcription request now fails locally instead of hitting an /audio/transcriptions endpoint that does not exist
* test(zerogpu): cover the responses and messages bridges
Both endpoints reach ZeroGPU through chat completions, so assert the request each one sends upstream and the reply it returns
* refactor(zerogpu): move zerogpu out of the provider list tails
New upstream providers append to the end of openai_compatible_endpoints, openai_compatible_providers and providers.json, so an entry at the tail conflicts with every one of them. Placing zerogpu next to long-stable entries keeps those merges clean. No behavior change
* fix(zerogpu): price cached input for qwen3-30b-a3b-fp8
The ZeroGPU model catalog (https://docs.zerogpu.ai/docs/model-catalog, checked 2026-10-09) lists cached input for qwen3-30b-a3b-fp8 at $0.05 per 1M tokens. Without cache_read_input_token_cost, cached tokens were billed at the full $0.10 input rate
* revert(zerogpu): leave the audio transcription provider set unchanged
Per review, keep OPENAI_AUDIO_TRANSCRIPTION_PROVIDERS as upstream defines it. This PR only covers chat completions, and ZeroGPU now serves /audio/transcriptions anyway, so the local refusal and its test are removed
* fix(zerogpu): add max_tokens so trim_messages can size ZeroGPU history
trim_messages reads max_tokens as the fallback for max_input_tokens and evaluates it eagerly, so without it the lookup raised, got swallowed, and the full history came back untrimmed. max_tokens matches each model's maxTokens in ZeroGPU's models API (https://api-dashboard.zerogpu.ai/api/models, checked 2026-10-10)
* test(zerogpu): record mocked requests on a recorder object
The mock handler appended to a list the caller passed in, which AGENTS.md
forbids. A recorder now owns the captured requests as a tuple
* fix(zerogpu): cap max_tokens at each model's output limit
Each cap is the max_output_tokens ZeroGPU stores for the model in its production models table (read 2026-10-20), the same column its /v1/models serves as max_length. That gives 65536 for deepseek-v4.1-flash, 32768 for gpt-4.1-mini and gpt-oss-120b, 128000 for gpt-5.4-nano and gpt-5.6-luna, 8192 for LFM2.5-1.2B-Instruct and llama-3.1-8b-instruct-fast, and 16384 for qwen3-30b-a3b-fp8. glm-5.3-flash is stored as 32786, which reads as a typo for 32768, so that row uses 32768
trim_messages sizes history from max_input_tokens, so lowering max_tokens leaves it unaffected
* fix(zerogpu): flag prompt caching on glm-5.3-flash
Repeating the same ~4.3K-token prompt against ZeroGPU on 2026-10-20 returned cached_tokens of 4320 on some glm-5.3-flash calls. gpt-oss-120b and llama-3.1-8b-instruct-fast returned 0 on all nine calls, and qwen3-30b-a3b-fp8 never returned cached_tokens, so they stay unflagged. Their cached prices stay as ZeroGPU publishes them, so any cache hit is still billed at the right rate
* fix(zerogpu): set gpt-5.4-nano context window to 272000
ZeroGPU accepts a 271001-token gpt-5.4-nano prompt and rejects 273000 tokens as over the context window (checked 2026-10-21), so max_input_tokens is 272000 rather than the 400000 its catalog lists
* fix(zerogpu): list zerogpu models for zerogpu/* wildcards
zerogpu had no models_by_provider entry, so get_provider_models("zerogpu") returned None and a zerogpu/* wildcard listed no models. zerogpu_models is now filled from the cost map rows like openinfer_models and registered in models_by_provider
* fix(zerogpu): flag gpt-5.4-nano as a reasoning model
ZeroGPU honors reasoning_effort on gpt-5.4-nano (checked 2026-10-21). Without it the model spends 0 reasoning tokens, and with low or high it spends 32 or more. LiteLLM only treated it as a reasoning model by falling back to OpenAI's bare gpt-5.4-nano row, so get_model_info and /model/info reported supports_reasoning as null. The row now carries the flag itself
* fix(zerogpu): flag deepseek, glm, gpt-oss and qwen3 as reasoning models
ZeroGPU's /v1/models marks these four as reasoning models and accepts reasoning_effort on all of them (checked 2026-10-21), but LiteLLM rejected the param with UnsupportedParamsError because their rows had no supports_reasoning flag. glm reports reasoning_tokens under reasoning_effort, and gpt-oss and qwen3 spend hidden reasoning tokens on every reply. The gpt-oss test case now expects reasoning_effort to reach ZeroGPU
* fix(zerogpu): set gpt-oss-120b context window to 128000
ZeroGPU routes gpt-oss-120b across several upstream backends, and at least one of them returns 413 past a 128000-token window, so max_input_tokens uses that as the safe floor. The backends reached on 2026-10-21 accepted up to about 131K prompt tokens, so this only gives up a little headroom on the larger ones
* feat(zerogpu): call ZeroGPU's Responses API natively
ZeroGPU serves /v1/responses for all nine catalog models, so LiteLLM no longer bridges Responses requests through chat completions. Native calls return ZeroGPU's reasoning output items for glm, gpt-oss and qwen3, which the bridge dropped. Checked live on 2026-10-22 through litellm.responses for every model, non-streaming and streaming, plus a gpt-oss tool call. LFM2.5-1.2B-Instruct streams correctly now that ZeroGPU's text_generation lane emits Responses SSE
* fix(zerogpu): stop inferring the zerogpu provider from api_base alone
Listing https://api.zerogpu.ai/v1 in openai_compatible_endpoints rerouted bare model names on that api_base from openai to zerogpu. A bare gpt-5.6-luna call with verbosity used to return 200 and started raising UnsupportedParamsError. Drop the entry and its api_base-alone test, so zerogpu is only selected through the zerogpu/ prefix
* fix(zerogpu): price gpt-6-luna, llama-3.2-3b, LFM2.5-1.2B-Thinking and qwen3.6
A zerogpu/* wildcard served these four chat models with no cost-map row, so their calls logged $0 spend. Add rows at ZeroGPU's catalog prices from https://api-dashboard.zerogpu.ai/api/models, with context windows and output caps from ZeroGPU's models table. LFM2.5-1.2B-Thinking has no cached price, like LFM2.5-1.2B-Instruct. gpt-6-luna is flagged supports_reasoning because LiteLLM already routes gpt-6 names as reasoning models. Checked live on 2026-10-28, all four now report nonzero response_cost
* fix(zerogpu): drop the retired max_tokens field from the zerogpu rows
Upstream #48471 retired max_tokens from the cost map in favor of max_input_tokens and max_output_tokens, and test_no_row_carries_the_retired_max_tokens_field failed on all 13 zerogpu rows. Every row already carries the same value in max_output_tokens, so nothing is lost. The zerogpu limits test now checks that each row's output limit fits its context window
* chore(zerogpu): regenerate provider endpoint support for zerogpu
Upstream #48619 generates provider_endpoints_support.json from the cost map and checks it in CI, and the hand-written zerogpu entry no longer matched. Regenerated with scripts/generate_provider_endpoints_support.py. Only the zerogpu entry changed, and it now matches what the generator emits for other JSON-registry providers such as openinfer
* fix(zerogpu): flag tools, json_schema and prompt caching on gpt-6-luna
ZeroGPU serves function calling, tool_choice, json_schema response_format and prompt caching on gpt-6-luna, so set supports_function_calling, supports_tool_choice, supports_response_schema and supports_prompt_caching on its row in both cost maps. Checked live on 2026-10-28, a weather tool call came back with the right arguments, json_schema output validated, and repeated prompts reported 3260 of 3263 tokens cached
* fix(zerogpu): set qwen3.6-35b-a3b context to 262144 and flag its tools and reasoning
ZeroGPU accepts up to 262,144 input tokens on qwen3.6-35b-a3b and serves tools, tool_choice, json_schema response_format and reasoning_effort on it. Raise max_input_tokens from 16384 and set supports_function_calling, supports_tool_choice, supports_response_schema and supports_reasoning in both cost maps. Before this, LiteLLM rejected tools and reasoning_effort for the model. Checked live on 2026-10-28, a 209,905-token prompt succeeded, an oversized one was refused with a 262144-token limit, and tool calls, json_schema output and reasoning_effort all worked
* fix(zerogpu): flag vision on gpt-6-luna and qwen3.6-35b-a3b
ZeroGPU reads image inputs on gpt-6-luna and qwen3.6-35b-a3b, so set supports_vision on both rows in both cost maps. Checked live on 2026-10-28, both models named the animal in an image_url photo, and the image showed up in their prompt token counts
* fix(gemini): send Google's current Managed Agents fields on create and forward agent_config on interactions
* fix(gemini): accept id on managed agent get, delete, and list_versions
* fix(gemini): drop unsupported agent fields under drop_params and keep agent_config keyword-only
A model interaction drops agent_config under drop_params, agent create drops wrongly typed optional fields under drop_params and otherwise answers 400 naming the field the caller sent, and agent_config on interactions create is keyword-only so positional response settings stay in place
* fix(gemini): drop a non-object agent_config under drop_params and name the id alias the caller sent
* fix(interactions): reject agent_config on bridged providers and drop a typeless one under drop_params
Bridged providers answer 400 for agent_config without drop_params and drop it with drop_params. A Gemini agent run drops an agent_config without a string type under drop_params. The Gemini agents wire test expects Google's current upstream field names.
* fix(interactions): honor additional_drop_params for agent_config on bridged providers
* fix(agents): drop a clashing instructions alias under drop_params
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(proxy): log one line per token count call
Each count call on /v1/messages/count_tokens, /utils/token_counter, /v1/responses/input_tokens and the Gemini countTokens routes now logs one INFO line with the litellm call id, the requested model, the model the count ran against, the count, the count source and the status code, so it joins the inference call that sends the same x-litellm-call-id
* test(google_endpoints): give the count test's fake request a headers list
* fix(proxy): keep token count log lines one per call and trusted in JSON logs, and log Bedrock native counts
Quote log values that carry whitespace, quotes, equals, or control characters so a model name can't split or forge a line. Let JsonFormatter record extras win over a dict parsed from the message. Log the Bedrock native count-tokens route through the same line.
* fix(proxy): escape braces in token count log values, log the Bedrock model AWS counts, and skip the line when there is nothing to count
* fix(proxy): log the deployment id the count routed to on every token count line
* fix(proxy): log count requests rejected before counting with the status they answer
A count route that rejects a request before the shared count function runs now logs the same token count line with its 400. An unsupported content block on the Messages count route becomes its 400 inside the logged wrapper, so its line says 400 instead of 500. A count that ends in a BaseException other than cancellation logs a 500.
* fix(proxy): compute the Bedrock counted model id only after a successful count
* revert(logging): keep the JSON log formatter's field order as on main
The count line already escapes braces in its values, so the formatter never parses a dict from it. Changing the order for every logger is a separate decision.
* test(ui): cover Responses API reasoning events in the playground
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show reasoning for current Responses API reasoning events
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): prefill the key edit Organization picker from org_id
The Virtual Keys table hands KeyEditView rows from /key/list, which
serializes the organization as org_id, while /key/info calls the same
field organization_id. The edit form read only organization_id, so on a
key opened from the list the Organization picker fell back to its "All
Organizations" placeholder even though the key belongs to an org.
Read both field names through one keyOrganizationId helper, used by the
form value mapper, the edit view's selected organization state, and the
two places in key_info_view that already hand-rolled the same fallback.
* fix(ui): keep the key edit team filter in sync with the organization field
* fix(ui): drop the leftover organization setter calls in the key edit team handler
The previous commit derived the organization from the form and removed the
useState behind it, but handleTeamChange still called the deleted setter, so
the dashboard build failed to type check and picking a team with an
organization threw at runtime.
The form.setValue calls next to them already carry the organization, so the
setter calls only had to go. Adds a test that picks a team with an
organization and asserts the form adopts and submits it.
* fix(ui): drop the list row's stale org_id when a key save returns its organization
* fix(ui): send the key's organization only when the edit form changes it
An untouched save from the key edit form now leaves organization_id out
of the /key/update body, the same way the form already drops unchanged
gated fields. This stops a save from writing a team's fallback
organization onto a key that has none, and keeps the audit log free of
no-op organization updates. The organization tests now use typed inputs
* fix(ui): open team page keys with their own organization in the edit form
* fix(ui): keep every team in the key form's team list for users who cannot change the organization
---------
Co-authored-by: mateo-berri <edgetonmas@gmail.com>
* fix(ui): omit blank expiry from key edit and regenerate payloads
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: credit #45048 author
Co-authored-by: Thomas Scholtz <thomas@quantum-sicarius.za.net>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(ui): format regenerate key payload builder
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): compare regenerate payloads by fields, not JSON order
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Thomas Scholtz <thomas@quantum-sicarius.za.net>
XecGuard AI and Airia AI join the third-party names the SI swap leaves alone, so
their Guardrail Garden cards stop reading "XecGuard SI" and "Airia SI"
The copyable agent instructions go to Claude Code and Codex, so they keep AI the
same way the compliance prompts do
