* fix(proxy): list models from team access groups
* 修复 PR #32599 的类型检查并补齐模型列表回退测试
* fix(proxy): keep a key's own model list narrow when adding team access group models
A key with its own model list was shown every model from its team's access
groups on /v1/models, /v1/model/info and /model_group/info, though auth
denies those calls. Team access group models now widen a key's list only when
the key inherits the team's models. /v1/model/info reuses the shared listing
helper without a catch-all, and route-level tests replace the mock-only ones
* fix(proxy): skip the list build in the unrestricted model info check and test every listing route
The unrestricted check for /v1/model/info only needs to know whether the key
and team lists are empty, so it no longer builds and expands the model list a
second time. The route test now covers /v1/models and /model_group/info next
to both model info routes
* fix(proxy): scope model info for personal and agent keys with no model list
* fix(proxy): keep config keys' team access groups and own-team team_id scope in model listing
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(cost): add tier 1 claude messages stream twins and recapture drifted usage shapes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): freeze the login throttle clock so the retry-after assertion is deterministic
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(jwt): honor end_user_id_jwt_field on the JWT-to-virtual-key mapped path
A JWT mapped to a virtual key dropped the end-user claim, so spend rows had
no end user and customer budgets, limits and region never applied. The mapped
token now carries the claim, and it takes precedence over the body user and
the x-litellm-end-user-id header, which remain the fallback when the claim is
missing or empty.
* test(jwt): budget-check the body or header end user when a mapped JWT has no end-user claim
* fix(jwt): replace the annotation offsets with a targeted pyright suppression to keep LIT001 flat
* fix(vertex_ai): route Model Garden models through the generateContent adapter
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(vertex_ai): annotate generateContent route test parameters
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(vertex_ai): type litellm_params in get_vertex_ai_model_route
* fix(vertex_ai): bill and log Model Garden generateContent calls like chat completions
Streamed Model Garden calls through the generateContent adapter now request
stream usage where the deployment supports it, keep cached-token details, and
look up the provider-relative cost key, so they bill like the non-stream call.
The adapter forwards router metadata, passes upstream errors through with their
status, countTokens falls back to local counting for non-Gemini routes, and the
native Gemini config is limited to the Gemini routes.
* fix(google_genai): keep adapter stream usage, admin opt-out, single failure count, and base_model routing
Streamed generateContent through the chat adapter now ends with the upstream usage, honors
always_include_stream_usage false and the caller's stream_options, counts one failure per
failed call, and routes Vertex deployments by their base_model like chat does
* fix(google_genai): ask for stream usage on every adapter stream so Claude streams end with usage
* test(google_genai): expect the trailing usage chunk on bridged OpenAI streams
The OpenAI bridge test now sends the usage chunk OpenAI returns for include_usage and expects it translated into usageMetadata. The response_format support helper goes back to its single-purpose form since its only other caller is gone.
* fix(google_genai): end adapter streams with one STOP event carrying usage, and keep upstream usage after the finish chunk
* fix(google_genai): send the held STOP event when no usage arrives, keep a caller's stream_options, and route countTokens like generateContent
* test(integration): cover the moved Vertex routes, adapter stream usage, the shared iterator and usage settings over the wire
* test(integration): add cache-hit twins for the stream usage setting and non-stream cells for every iterator member
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix: align token counting with inference context
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: preserve Vertex Gemini count message shapes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: forward Google countTokens tools in shapes every provider counter accepts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: add package marker for Vertex count-token unit tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(vertex_ai): ignore deployment system_instruction when counting tokens
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(count_tokens): forward system and tools on Google AI Studio, keep strict on Anthropic tools
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(count_tokens): ruff format gemini count handler
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(count_tokens): return Mapping from count request body builders
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(count_tokens): truncate long Anthropic tool names on Vertex count, update stale strict test expectations
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(count_tokens): simplify Gemini count context and tests
Put the Vertex Gemini count branch back in the same shape as main and the partner model branch, keeping only the system and tools context, the provider error mapping and the gemini tokenizer default
Share one error response helper between the Vertex and AI Studio callers, drop the unread kind fields and the extra validation of map_function output, and say Gemini instead of Vertex AI in the error that AI Studio users also see
Tighten the tests by removing a stray parametrize on a helper, sharing the SDK count args, asserting the exact forwarded system and tools, and folding the Anthropic tool shape tests into one table
* fix(count_tokens): reuse inference tool translation for Vertex Gemini count
Vertex and AI Studio count now convert tools with the same functions inference uses, translate_anthropic_tools_to_openai and the Responses tool converter, before VertexGeminiConfig.map_function. Unnamed and Anthropic hosted tools now count the same declarations inference sends instead of being dropped
Tools that cannot be converted, such as a non list tools value, a non object tool or a schema that is not a JSON object, now return a 400 before any request instead of being counted without them
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: keep model capability flags out of provider requests
A supports_* flag or another model map field set on a deployment's litellm_params was forwarded
to the provider as a request field, so Bedrock answered 400 Extra inputs are not permitted on
chat traffic and on the background health check alike. The shared owned-kwarg predicate now
treats every ProviderSpecificModelInfo field as litellm-owned.
Builds on the health-check-only approach in #40952 by leilei3167.
* fix: treat every supports_ model map flag as a capability field and warn when one sits in litellm_params
* fix(router): skip declared litellm params in the capability flag warning and warn for runtime-added deployments
* fix(router): strip model capability flags from the router's deployments instead of every request
* fix(router): keep capability flags out of health probes and read them from the loaded catalog
* perf(router): read the catalog's supports_ field set once per cost map load
Cache the loaded catalog's field names when the cost map snapshot is taken so the capability-flag check stops walking every catalog entry per deployment, and add integration cells that drive the flagged and control deployments through chat, streaming, messages, responses, embeddings, health and model management against a vendor that rejects the flags
* fix(vertex_ai): return 400 with file line number for malformed batch JSONL row
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(vertex_ai): keep yield outside the JSON decode guard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(http_handler): surface BaseLLMException raised while streaming a request body
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(http_handler): only follow __cause__ when unwrapping streamed body errors
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(http_handler): avoid recursive streamed error unwrapping
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(vertex_ai): return a 400 naming the line for wrong-shape and non-UTF-8 batch rows
Keep main's error path for a directly raised provider error and only unwrap
one found behind a transport error. Add the parser column to invalid JSON
errors, and cover the real aiohttp chain in the _handle_error test
* fix(vertex_ai): report every row conversion error as a 400 naming its line
A message with no role, a non-string role, a bad temperature, or a litellm
BadRequestError raised while converting a row still returned a 500, and the
async upload lost the 400 for litellm's own exceptions. The stream now turns
any error from one row's conversion into a 400 naming that line.
_handle_error only unwraps an httpx.TransportError, which is what the aiohttp
transport raises around a failing request body, so other callers keep their
own status. A file that starts with a UTF-8 byte order mark now uploads, and
provider messages start with 'Batch input file line N' like the proxy check.
* fix(vertex_ai): reject null or missing batch bodies and unparseable rows with a 400 naming the line
A chat row whose body is missing or not an object now raises a 400 naming
its line instead of uploading a blank prompt, and a non-object body on
line 1 no longer crashes the GCS object name lookup. Rows too deeply nested
or too long to parse return a 400 from both the SDK and the proxy's own
batch file check. JSON error columns now count leading whitespace.
* fix(vertex_ai): keep batch rows on their file line through router and guardrail rewrites
The Router model rewrite and batch guardrails now keep blank lines and blank out dropped records, so a provider error naming a line still points into the caller's file. The rewrite strips a leading BOM per line, and a non-string body.model on line 1 is reported as a 400 naming the line
* test(integration): cover Vertex batch row line errors across routes, rewrites, SDK and worker chaos
* fix(router): keep blank lines only for Vertex batch uploads and leave guardrail rewrites compact
* test(integration): score the whole OpenAI upload after a guardrail drop and cover a mixed OpenAI and Vertex batch upload
---------
Co-authored-by: jesus <jesus@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(proxy): scope /spend/tags aggregates to the caller's own and permitted-team logs
Non-admin callers previously received the proxy-wide per-tag spend
aggregation. The endpoint now resolves the same read scope as
/spend/logs/ui and /spend/logs/session/ui, and the query honors the
start_date/end_date parameters it previously accepted but ignored.
* fix(proxy): use Annotated auth dependency on /spend/tags and sync dashboard API types
* fix(proxy): bind /spend/tags dates as UTC datetimes, not session-timezone strings
Raw YYYY-MM-DD strings cast to timestamptz are read in the database
session timezone, shifting the day window by that offset. Parse them
to UTC-aware datetimes first, like every sibling spend query in this
file, and reject malformed dates with a 400 before they reach the
database.
* fix(auth): return a generic 401 for an unknown key
The wrong-key 401 shipped the internal verification table name and the
submitted key's full SHA-256 hash to any unauthenticated caller. Both now
go to the proxy's own log and the response carries only "Invalid API key
provided."
The Admin UI detected a dead session by string-matching that message, so it
now reads the structured error type instead.
* Update ui/litellm-dashboard/src/lib/http/client.ts
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
* fix(auth): log the missing key hash where every resolve path passes
Review catch: JWT key-mapping misses resolve through IdentityStore without
touching the main key path, so logging the hash there left those callers with
nothing but the generic 401. Log it at the store's raise site instead, which
every resolve path shares, and drop the now-duplicate log.
* test(auth): inject a key-less client instead of patching the DB fetch
The store test tripped TQ008 by patching _fetch_key_object_from_db_with_reconnect.
Hand IdentityStore a client whose get_data returns no row instead, so the miss is
driven through the real lookup rather than around it.
* fix(auth): log the unknown key's hash once on the auth failure line
The per-lookup warnings in get_key_object and IdentityStore logged whatever value
they were handed, which on the failure-logging path is the raw key a caller sent
(a provider key included), let a newline in ?key= forge log lines, and fired on
lookups that expect a miss. The hash now rides KeyNotFoundError onto the single
existing auth failure log line next to the requester IP, with control characters
escaped. Also drops the new comments and updates skipped tests that quoted the
old message.
* test(auth): add the blank lines ruff format expects before _InFlightCountingPrisma
* test(integration): expect the generic wrong-key 401 in the management lifecycle tests
* fix(auth): keep the operator wording on the unknown-key log line and drop the redundant 401 rewrite
* test(auth): cover the generic unknown-key 401 across routes, proxies, and a worker kill
* test(auth): cover the realtime upgrade refusing an unknown key without echoing it
---------
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test: inject clocks and wait on conditions in three wall-clock-flaky unit tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: keep generic API logger tests on 3.10 and cover automatic sending
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: pin router usage minute in shared rate-limit header test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: stabilize three wall-clock-sensitive unit tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): cap cold schema validation workers at the CPUs the proxy can use
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: inject the cache clock and wait on lock contention instead of 1s deadlines
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(mcp): wait on listing events instead of 1s deadlines in the discovery drain test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo <mateo@berri.ai>
* fix(proxy): resolve model aliases to their target deployment for listing metadata
A model alias (a key's `aliases`, a team's `model_aliases`, or the router's
`model_group_alias`) is rewritten at request time and owns no deployment row,
so `get_model_listing_info` returns None for it and the alias name is not a
cost-map key either. Both metadata candidate sources come up empty and the
listing entry carries no `mode`, `max_input_tokens`, or `max_output_tokens`,
while the deployment it routes to reports them on the same response.
Resolve a listed alias to the deployment it routes to for the metadata lookup,
keeping the alias as the response id. Maps are applied in order, each against
the result of the last, mirroring how `litellm_pre_call_utils` applies the team
map and then the key map, so a chained alias reports the deployment the caller
actually reaches rather than an intermediate one. An alias shadowing a
same-named deployment follows the alias, because the request rewrite does too.
Fixes the model_aliases half of #40553.
* fix(proxy): look listed aliases up through the shared alias chain
Drop the separate alias resolver and retarget a listed alias in
alias_listing_entries and alias_target instead, so /v1/models and
/v1/models/{id} report the model /chat/completions rewrites it to, and
only when the caller can list that model
* fix(proxy): keep deployed listed models' own rows and resolve team alias targets at list and retrieval
* fix(proxy): resolve router_settings aliases, keep alias labels, and skip paused targets in model listing
* fix(proxy): leave an alias to an undiscoverable model bare in model listing
A team alias whose target has discoverable: false showed the hidden model's mode and limits on GET /v1/models and retrieved it by alias. Undiscoverable groups now count as hidden for alias resolution, as on main, while admins still see them.
* fix(proxy): leave an alias to a model a listing callback hides bare in model listing
* fix(proxy): keep aliases bare for models a listing callback never judged or another team owns
---------
Co-authored-by: haydster7 <haydster7@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
/public/providers/fields re-read and re-parsed the bundled 85K
provider_create_fields.json on every request, and /public/agents/fields did the
same for that file plus agent_create_fields.json. Both are unauthenticated
public endpoints, and both did the work inside an `async def`, so the read and
the parse ran on the event loop and serialized against every other in-flight
request. Measured against the bundled files, that is roughly 0.40 ms of blocking
work per request for the provider endpoint and 0.45 ms for the agent endpoint
Both now memoize with `lru_cache`, which is what `/public/endpoints` in the same
module already does through its `_cached_endpoints` global, so each file is read
once per process rather than once per request
The agent endpoint mutated the dicts it had just loaded, appending inherited
provider credential fields and popping `inherit_credentials_from_provider`. That
is only safe while the data is re-read every time, so caching it meant making the
merge pure: `_agent_with_inherited_credentials` builds a new mapping per agent,
and the cached value is the merged result rather than the raw file contents.
Responses are byte identical before and after, checked on both endpoints against
a live proxy
The bundled records are typed as Mapping[str, object] and narrowed with
isinstance at the point of use, rather than typing.Any, which ruff-strict bans
Side effects: 3 of the repo's ASYNC230 violations go away, and basedpyright
errors in the file drop from 63 to 52
Rebased onto main after /public/providers/fields started validating the bundled
JSON through a pydantic adapter on every request. That validation now runs once
and the validated records are cached; the agent endpoint keeps merging the raw
provider records, so both responses stay byte identical to main.
Co-authored-by: Alok Sharma <alok44170@gmail.com>
* feat(ui): status filter + sort and provider-aware search on Models & Endpoints
The models table mixes active and paused deployments with no way to tell
them apart beyond the pause toggle, and the drawer only filters by public
model name and access group.
Backend (/v2/model/info):
- new optional `blocked` query param to filter by routing status
- new `blocked` sort field (asc = active first); the existing `status`
field keeps sorting by config-vs-DB source
- `search` now also matches litellm_params.model, so typing a provider
or upstream model id (e.g. "nvidia_nim", "openai/gpt-4") finds
deployments whose public name doesn't mention them
Frontend (Models & Endpoints):
- visible Status column with Active/Paused badges, sortable
(active first on asc)
- Status filter (All/Active/Paused) in the Filters drawer, wired to the
new server param through URL state
The routing-status filter ignores non-bool sentinels the same way
exclude_auto_routers does, so direct calls bypassing FastAPI keep their
no-filter behavior.
* fix(ui): inverted routing-status filter mapping and compact Status column
Selecting "Active" in the Filters drawer sent blocked=true to the server,
showing the paused deployments instead of the active ones (and vice
versa). The mapping now lives in routingStatusToBlocked() with unit
tests, and the Status column is narrower so the row actions stay in
view.
* fix(ui): pin the Actions column to the right on the models table
With the new Status column the table overflows on narrower screens and
the pause/resume switches (last column) scroll out of view. Pinning the
Actions column keeps them visible; uses the existing DataTable
meta.pinned support (same pattern as MemberTable).
* refactor: readable db search where-condition and ruff-format clean
* fix: apply routing-status filter inside the search flow and regen API types
Greptile flagged (P1): with search + a status filter combined, the
bounded DB fetch ran before status filtering, so matches of the other
status could consume the page budget and leave short pages / stale
totals.
- extract _matches_routing_status() (sentinel-safe) and apply it to the
router-side matches inside _apply_search_filter_to_models
- push the status condition into the DB where clause of
_fetch_db_models_for_search, so rows of the other status never count
against the bounded fetch
- endpoint passes blocked through to the search flow; tests cover the
combined search+status case and the generated where clause
- regenerate ui/litellm-dashboard/src/lib/http/schema.d.ts for the new
query param
- update useModels.test.ts expectations for the new trailing argument
* fix(proxy): keep the flat DB where shape for single-condition searches
The upstream CI test
test_apply_search_filter_honours_exact_model_name_in_db_query pins the
Prisma where shape: exact model_name stays a top-level key, and plain
substring search keeps {model_name: {contains, mode}} at the top level.
Only genuinely multi-condition queries (status filter and/or router-id
exclusion) now wrap in AND.
* fix(proxy): avoid reassigning a Final variable in the db where builder
basedpyright's reportGeneralTypeIssues budget counts the reassignment of
the Final-annotated db_where_condition; express it as a single
conditional expression instead.
* chore(deps): bump pypdf to 6.19.0 to clear osv-scan
pypdf 6.16.2 has 8 High CVEs (GHSA-5jq2-8x83-x246 and aliases) fixed in
6.17.0-6.19.0; bump past all of them
* Revert "chore(deps): bump pypdf to 6.19.0 to clear osv-scan"
This reverts commit 0e2528123db7ab23304ae20f6a4c0961d0e2d073.
* fix(proxy): keep LIT002 budget flat in the model search where-builder
Extract the dynamic Prisma where-condition fragments into module-level
helpers so every dict starts on a short line, split the litellm_params
fallback lookup into a Final local, and annotate suppressions on the
construct start lines the checker attributes
* chore(tests): repoint unbounded-IN baseline at the extracted where-builder helper
The bounded-router-id filter moved from _fetch_db_models_for_search into
_not_in_router_ids, so the old baseline key is stale and the new helper
site is the finding to record
* fix(proxy): avoid searching encrypted deployment model names
* refactor(ui): keep model routing status as a filter only
* docs(api): regenerate model info sorting description
* fix(api): remove remaining paused status sort references
---------
Co-authored-by: thementalcoding <312162298+thementalcoding@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
LiteLLM_AuditLogs masked before_value and updated_values with
SensitiveDataMasker.mask_dict, which stringifies every non-native leaf, so a
null in the change was stored as the string "None" (or "" under a key-named
field). Walk audit values with mask_credentials_in_payload instead, injecting
the existing {"key"} masker so the masked format and scope stay the same.
* fix(proxy): gzip responses over 16 KiB off the event loop
GZipBufferedResponseMiddleware compressed every body under 1 MiB inline.
gzip.compress holds the event loop for its whole run, and its cost follows
how well the body compresses: an 824 KB base64 embeddings response (100
inputs, the OpenAI SDK default encoding) takes about 165 ms, so every other
request on the worker waits that long, and with several in flight the
compressions run back to back.
Lower OFF_LOOP_SIZE_BYTES to 16 KiB so only bodies that compress in well
under a millisecond stay on the loop. zlib releases the GIL, so the loop
keeps serving while a worker thread compresses.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(proxy): pin the gzip off-loop boundary at exactly OFF_LOOP_SIZE_BYTES
A body of exactly OFF_LOOP_SIZE_BYTES compresses in a worker thread, as the
`<` in _compress implies. Suggested in review.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(proxy): pin an embeddings-sized gzip body off the event loop and drop the source comment
The off-loop test was relative to OFF_LOOP_SIZE_BYTES, so it still passed with the old
1 MiB line. An 824 KB body, the size of the 100-input base64 embeddings response in the
issue, now has to compress in a worker thread. The recorder no longer mutates a list, and
the threshold rationale stays in the commit history instead of a source comment.
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(typesafe): support Jev 1.14 images, box questions, and long-context headers
* fix(cost): drop the typesafe/jev-1.14.0 entry main already prices
* fix(typesafe): send images only with the images header and box questions only to models that answer them
TypeSafe takes assets and input_image parts only when the request carries
TypeSafe-Danger-Zone-Images: true, so without it they follow drop_params like
any other provider that cannot take images. Box support now comes from a
supports_box_questions cost-map flag set on jev-1.14.0 and jev-preview, so
jev-1.13.0 and jev-latest refuse or drop a box question before the call. A
malformed provider_specific_header answers 400 instead of 500.
* fix(typesafe): allow null box confidence and validate asset references and base64
* fix(typesafe): gate box questions on the images header, drop unreadable assets under drop_params, forward opt-in headers on the pass-through
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
LiteLLM_AuditLogs parses before_value and updated_values as JSON but only re-serialized dicts, so a list change reached Prisma as a Python list and the row write failed. Every deployment warning ignore, snooze and restore hit this. Mask and re-serialize any JSON value instead
* fix(responses bridge): keep defer_loading on function tools sent to the Responses API
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(responses bridge): give the defer_loading casts a cast-ok reason
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(responses bridge): build defer_loading outside the tool loop so Final stays valid
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(responses bridge): round-trip tool_search_call and tool_search_output items between turns
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(responses bridge): read replayed tool-search and namespace fields through typed helpers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(responses bridge): keep tool_search items on a turn with no message or tool call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(responses bridge): keep every reasoning item on a search-only turn
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(responses bridge): leave content unset on a search-only turn so replay adds no empty message
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(responses bridge): replay tool search items only with tool_search, count replayed tools in the allowlist, keep namespace off guardrails
* fix(responses bridge): honor drop_params for defer_loading and route tool_search to Responses
Under drop_params, defer_loading is dropped when no tool_search tool is sent,
so the 400 it would cause upstream goes away as before. A non-bool value answers
400 without drop_params and is dropped with it. A null tool-level value falls
back to the nested function value. A request carrying a tool_search tool now
bridges to Responses on OpenAI and Azure endpoints even with reasoning_effort
none, since chat completions rejects the tool_search type.
* fix(responses bridge): replay tool search items in output order and count replayed tools in tool policy
Store the ordered output prefix through the last tool search item, reasoning included, so a
second reasoning item no longer makes OpenAI reject the replay. The shared chat-to-Responses
converter now replays tool search items and namespace only when the bridge opts in, so Noma,
the complexity router and guardrail payloads keep their shape. The tool policy guardrail and
the allowlist count replayed tools only when the request can send them.
* fix(responses bridge): forward defer_loading only beside a tool_search tool
defer_loading only means something with tool_search, and backends without it
reject the key (xAI answers 403). A true value with no tool_search tool now
answers 400 naming drop_params and is dropped under drop_params; a false value
there is a no-op and is dropped.
* fix(tool_loop): keep tool search items and namespace between rounds
* fix(responses bridge): check replayed tool search loads when a deployment adds tool_search
* fix(chat): drop litellm response fields before strict OpenAI-compatible providers
A chat reply from the Responses bridge carries provider_specific_fields on the
message and on each tool call, plus reasoning_items. Groq, Fireworks, Mistral
and Together reject unknown fields, so sending that reply back to them as
history failed with a 400 or 422. Strip them in the shared helper and use it
in those providers.
* fix(caching): keep reasoning_items and tool search items on response cache hits
A cached bridge response lost reasoning_items on a non-streaming hit and lost
both reasoning_items and provider_specific_fields (tool_search_items) on a
streamed hit, so the next turn could not replay the search or the reasoning
around it
* test(integration): cover tool search item replay on the chat to Responses bridge
Proxy, SDK, authorization and chaos cells for returning and replaying
tool_search_call and tool_search_output items and the tool call namespace
* test(integration): add response cache twins for the tool search guardrail cells
Each guardrail cell now has a twin where the answer is already in the Redis response cache: a key opted out of global guardrails caches a replay that loads a policy-blocked tool, the same replay without guardrails is cached before block mode sees it, rewrite mode is not served the unrewritten conversation's cached answer, and an allowed replay is served from the cache.
* fix(responses bridge): forward defer_loading beside a client-executed tool_search
The bridge now replays a client tool_search_call and the client's tool_search_output through provider_specific_fields.tool_search_items, so a deferred function beside a client tool_search keeps defer_loading like it does beside a hosted one
* test(responses bridge): pin the client tool_search turn and its replay, streamed and not
* fix(responses bridge): strip nested namespace for strict providers and keep streamed reasoning_items on text replies
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(responses bridge): keep dropping defer_loading beside a client-executed tool_search
Forwarding defer_loading beside a client tool_search made GPT stop with a
search call the chat client has to answer by hand, so a plain question that
used to get tool calls got none. Keep main's behavior of dropping it there.
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(router): keep the prompt caching affinity pin alive as long as the prefix's cache_control ttl
A prefix marked cache_control ttl 1h lost its pinned deployment after the fixed 300 s, so the next request went to another deployment and paid a cold cache. The pin TTL now follows the longest ephemeral breakpoint in the cacheable prefix, tools included: 3600 s for a 1h breakpoint and 300 s otherwise.
* fix(router): give prompt cache pins their own tier and honour any declared ttl
Pins now live in a DualCache of their own instead of the router's 200-slot
in-memory tier, so an hour-long pin no longer evicts cooldowns and rate windows.
The pin lifetime follows any <n>s|m|h cache_control ttl, floored at 300 s, not
only the literal "1h".
* fix(router): hash the pin ttl off the event loop and flush prompt cache pins with the router cache
Compute the cache_control ttl scan through offload_token_count beside the
prefix hash, clear the dedicated pin cache in Router.flush_cache, and point
the order-fallback and priority-group pin tests at the pin cache they read
* fix(router): pin 1h prefixes only on 1h-capable models, cap the pin at 1h, and give pins 10k in-memory slots
* fix(router): match the 1h pin gate to what Bedrock forwards
Bedrock deployments now decide the 1h pin with is_claude_4_5_on_bedrock,
the same check that decides whether the 1h ttl reaches Bedrock, so an
application inference profile ARN keeps a 5 minute pin whatever its
base_model prices
* test(router): give the anthropic_messages pin fixture its provider
* fix(router): let each pin write set its own lifetime and require a positive 1h price
In a group mixing 1h and 5m deployments the in-memory pin kept the lifetime of its first write, so a 5m deployment could hold an hour-long pin. Each write now clears the in-memory entry first, matching Redis. The Bedrock gate uses supports_bedrock_prompt_cache_ttl after #46058, and a zero 1h price no longer counts as 1h support.
* test(router): integration cells for the one-hour prompt cache pin lifetime
Cover every surface the pin lifetime touches: chat and /v1/messages through
both SDKs, the responses bridge, response-cache hits, each provider family's
1h gate, malformed and duplicated ttl values, rewrites of one prefix, mixed
and cross-group routing, concurrency, every Redis attach path, the in-memory
slot pressure without Redis reads, a Redis outage and a killed worker
* fix(caching): bound the in-memory expiry heap and count prompt cache pins in the memory debug routes
* test(integration): cover prompt cache pins in the memory debug routes and the bounded expiration heap
* fix(caching): serialize in-memory cache writes so a heap rebuild cannot lose another thread's expiry entry
* fix(caching): keep the in-memory cache bounded when entries have no usable expiry
* test(caching): cover the response cache bound when cached entries have no usable ttl
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(cli): run group subcommands in the lite shell and default to the logged-in gateway
Typing a command group alone in the lite shell printed 'Usage: lite lite',
the next subcommand was unknown, quoted arguments were split on spaces, and
a bare lite connected to localhost:4000 even after lite login to a gateway.
The shell now names the group in usage text, keeps the group as the prompt's
scope so its subcommands run next, accepts a leading 'lite', parses lines
with shlex, and a bare lite falls back to the server the stored lite login
credential was issued for before localhost:4000.
* refactor(cli): walk the shell's group path without recursion
* fix(cli): keep backslashes and compact JSON in the lite shell and pair the login server with its key
* fix(cli): keep the login server to the bare lite shell and accept lite options and = quoting at its prompt
* fix(cli): keep the shell's server for a lite options line and name the right login in the keychain notice
* fix(cli): keep compact JSON with a trailing = intact in the shell and pin options lines to the shell's server over LITELLM_PROXY_URL
* feat(proxy): general_settings.disable_requester_ip_address_logging keeps caller IP out of spend logs
When the setting is on, spend log rows store requester_ip_address as null in
the column, the metadata and the stored request (which also drops client-IP
forwarding headers), and IP addresses in stored error messages are masked, so
the Logs page shows no caller IP. The setting is read the way the settings API
validates it. Capture is unchanged, so guardrails, logging callbacks and
metrics still receive the IP. Toggleable from the Admin UI General Settings.
Resolves LIT-7442
* fix(proxy): mask IPv6 addresses that end or start with :: across eight groups
The masking pattern allowed seven colons, so valid forms like 1:2:3:4:5:6:7::
and ::1:2:3:4:5:6:7 stayed in stored error text with
disable_requester_ip_address_logging on. Also lists the request metadata walker
in the recursion check, since it only walks the tree the spend log sanitizer
already returned.
* fix(proxy): mask IPs in every stored header value and after labels or underscores
With disable_requester_ip_address_logging on, client IPs carried in headers outside the forwarding list (do-connecting-ip, incap-client-ip, custom names) and addresses written as host:2001:db8::2, peer_2001:db8::2, socket_192.0.2.1 or an unbracketed IPv6 with a port were still stored in spend logs.
* fix(proxy): mask request header IPs before long values are truncated
A header value longer than the spend log string limit kept the part of an IP that fell before the truncation cut. Request headers in metadata and requester_metadata are now masked before the body is truncated, like error text already was. IPv4 followed by a dotted port (192.0.2.1.443) is masked again.
* fix(proxy): mask the IP piece a header truncation cut leaves instead of pre-masking headers
Masking header values before truncation scanned the whole client value,
including text the cut drops, and walked hook-made header cycles. Mask after
truncation instead, then mask the hex and digit run touching each side of the
truncation marker when it is short enough to be part of an IP address. Every
nested headers mapping gets the same treatment.
* docs(proxy): say the requester IP setting masks IPs in other stored header values
* fix(proxy): mask stored error text after truncation and harden the cut-piece pass
Error messages and tracebacks were regex-scanned in full before the DB cap,
which costs seconds on multi-megabyte provider errors. Mask them after the
cap like header values, with the same pass over the run beside each
truncation marker. That pass now handles every marker, skips long hex tokens
so an address beside one is still masked, keeps at most one address length,
and also masks colon-only IPv6 groups.
* fix(proxy): drop the requester IP from the originating request an auto router classifier row stores
* fix(proxy): drop the requester IP from the classifier input an auto router classifier row stores
* refactor(proxy): mask stored IPs with the content filter's ipv4 and ipv6 patterns
The requester IP setting now reuses the content filter guardrail's prebuilt
ipv4 and ipv6 patterns instead of its own address regexes, so spend logs mask
what that guardrail masks. Addresses glued to a label or underscore stay as
the content filter leaves them.
* fix(proxy): mask stored IPs with the spend-log matcher again
The content filter's ipv4 and ipv6 patterns left labelled IPv6 (host:2001:db8::2),
unbracketed IPv6 with a port and IPv4 after an underscore in stored error text and
header values, and masked parts of text like std::io::Error. Restore the
address-validating matcher and its regression cases.
This reverts commit 79537a8aa5.
---------
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* feat(proxy): expose trace ID response header
* style(otel): format trace correlation middleware
* test(otel): cover websocket trace middleware
* fix(otel): disable correlation on setup failure
* refactor(otel): inject instrumentation setup
* fix(proxy): drop a relayed upstream traceresponse when the gateway sends its own
* fix(proxy): preserve upstream traceresponse without gateway injection
* fix(proxy): replace relayed streaming traceresponse
* test(otel): cover trace response fallback paths
* test(proxy): cover the streaming pass-through relay when the gateway propagator is off
* fix(proxy): keep a relayed upstream traceresponse and stamp the call id on the server span
The gateway's response propagator now leaves a response that already carries an upstream traceresponse untouched, so pass-through, file content and plugin proxy routes return one header and untraced pass-through routes keep the upstream's. The per-helper strips are gone. The propagator also sets litellm.call_id on the server span from the response's x-litellm-call-id.
* fix(proxy): add traceresponse only to sampled HTTP responses, tuple headers and unhandled 500s included
* fix(proxy): write only traceresponse and link a relayed header from a restarted upstream trace
* fix(proxy): send traceresponse only from the proxy app and expose it through CORS only with OTel V2
* test(proxy): check the CORS expose list contains the UI headers instead of pinning its identity
* test(integration): check traceresponse names the gateway server span across routes, errors, pass-through and chaos
* test(integration): read the cache-hit spend row by response id and assert only the traceresponse expose entry
---------
Co-authored-by: adityajoy-1902 <bittu19jain@gmail.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(proxy): let proxy admins open team member defaults to team admins
* fix(proxy): keep a team admin's never-resets duration on a team's first member budget
* test(proxy): pin the existing-row and stored never-resets cases for member defaults
* feat(proxy): let proxy admins open soft_budget to team admins
* feat(proxy): let proxy admins open team metadata to team admins, keeping proxy settings with proxy admins
* test(ui): name expected objects in team admin edit access tests
Keeps local/no-large-inline-object-arg within its eslint budget
* fix(proxy): compare team metadata without recursion and count key presence
A proxy-controlled metadata key that appears or disappears with a null value
is now a change, and the JSON comparison no longer recurses
* fix(proxy): compare team metadata numbers exactly
Numbers are compared as decimals, so integers past float precision and close
floats are no longer treated as equal
* feat(guardrails): add Airia guardrail integration
Adds `airia` as a built-in guardrail provider. The hook sends the prompt under `pre_call` and the
model's response under `post_call` to the Airia AI Gateway, which answers allow, block, or redacted
content; the only configuration is `api_base`, `api_key`, and an optional `timeout`.
`during_call` is not offered: it runs concurrently with the model call, so a block could land after
the prompt has already reached the provider.
A BLOCKED verdict, or any action this version does not recognise, raises with
`blocked_content=True`; a transport error, a non-2xx, or a response body that is not a JSON object
raises with `blocked_content=False`, so callers can tell "could not evaluate" from "evaluated and
blocked" while both still fail closed. On GUARDRAIL_INTERVENED a copy of the inputs is returned with
every rewritten field substituted; an intervention carrying no applicable rewrite, a rewrite of the
wrong shape, or a rewrite whose item count does not match what was sent (which would misalign the
positional write-back downstream) blocks instead of letting the original through.
Streamed responses are moderated whole and then emitted redacted: the hook opts into the unified
hook's `incremental_diff` mode (the default `block_only` drops rewrites) with
`streaming_end_of_stream_only`, so a redaction can never span transform rounds and underflow. Covered
by a test that drives the real streaming iterator hook end to end and checks the assembled client
output, not just the guardrail's own config attributes.
The HTTP client is dependency-injected (an `async_handler` constructor parameter, defaulting to a
real client) rather than mutated post-construction, so tests inject a fake client instead of
monkeypatching an instance method.
Known limitation, not fixable from this guardrail alone: LiteLLM's unified guardrail framework
yields a tool-call chunk to the client immediately during `incremental_diff` streaming, inspecting
the assembled tool calls against the guardrail's verdict only once the stream ends
(`_run_incremental_transform_stream` in `unified_guardrail.py`). A blocked tool call can therefore
reach an executing client before the terminal error arrives. Switching to `block_only` with
`streaming_buffer_until_moderated` does not close this without a regression: that flag has no effect
under `incremental_diff` (the mode branches away before it is read), and `block_only` itself replays
the original, unredacted chunks on release, silently dropping any REDACT verdict on streamed text.
Relatedly, LiteLLM's OpenAI chat completions guardrail translation skips `post_call` guardrails
entirely for a response with no text content, so a tool-call-only response bypasses this guardrail
(and every other post-call guardrail) even outside streaming. Both are shared-framework behavior,
not specific to Airia; fixing either is out of scope for an isolated single-guardrail contribution.
* fix(guardrails): fail closed on an Airia rewrite for a field that was not sent
Bugbot finding on #40784. An INTERVENED verdict carrying `structured_messages: []` with
`texts` omitted passed `_rewritten` unchanged: an empty list has the same length as an
omitted field, so it counted as an applied rewrite while the `texts` that actually carried
the content went through untouched. post_call sends only `texts`, so this was reachable.
A rewrite for a field the request did not send is now a mismatch, same as a wrong length,
and the verdict blocks. Regression test covers both the empty and the non-empty case.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(guardrails): fail closed on streamed non-chat routes the Airia rewrite cannot reach
Veria finding on #40784. incremental_diff, the mode that carries a rewrite into the stream,
exists only for OpenAI chat completions; UnifiedLLMGuardrails sends every other streamed route
(/v1/responses, /v1/messages) to block_only, which streams the original text live and drops
rewrites. So with post_call on, a streamed Responses or Anthropic call could receive text that
Airia then blocked or redacted.
Two changes, both confined to this guardrail:
- streaming_buffer_until_moderated is set, so on the block_only fallback the framework holds
every chunk until the end-of-stream verdict instead of releasing it live. The flag is never
read on the incremental_diff path, so streamed chat completions are unchanged.
- A REDACT verdict on a streamed response whose route cannot carry a rewrite is enforced as a
block, since block_only would otherwise replay the original. The route check mirrors
_resolve_transform_call_type so the two cannot disagree; an unresolvable route fails closed.
Request-side rewrites and non-streamed calls are untouched.
The end-to-end test drives the real UnifiedLLMGuardrails iterator over an Anthropic
/v1/messages SSE stream and asserts not one original chunk is released for either verdict.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(guardrails): keep the Airia stream test's pytest.raises block to one statement
CI's test-tree ruff run flags PT012 on the non-chat streaming test: the `async for` that
collected released chunks sat inside `pytest.raises`, so a raise from the loop body would be
indistinguishable from the one the test is about. The loop now lives in a `_drain_into` helper
and the `with` block awaits it alone. Same assertion, same partial-leak capture.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* refactor(guardrails): keep the Airia guardrail within the per-rule basedpyright ceilings
scripts/type_check_gate.py now gates every rule on its merge-base count (#40193), and the
guardrail added sixteen strict-rule errors it previously absorbed under the global budget.
- The parsed gateway response goes through `_as_object` once, so `action`, `blocked_reason`
and the rewrite fields are read from `dict[str, object]` rather than `dict[Unknown, Unknown]`;
`blocked_reason` is only used as the message when it is a non-empty string.
- `get_config_model` returns the concrete `type[AiriaGuardrailConfigModel]`, and the config model
carries the same pyright ignore as its siblings for the unparameterised base.
- The remaining untyped upstream members (the decorator, the client factory, `super().__init__`'s
kwargs, the handler's `post`) are ignored per line with the reason, as the Wingback guardrail
does. The registry reads `guardrail["guardrail_name"]`, a required TypedDict key.
- Tests reach the injected mock and the handler timeout through typed helpers instead of the
real handler's partially typed attributes.
No behaviour change: 46/46 tests, ruff default and strict, and both lint gates pass.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(guardrails): clear the Airia guardrail's type-discipline gate violations
Drop the mutable-ok comments that suppress nothing (LIT013), give the two
deferred-import noqa lines a reason (LIT003), and annotate the parsed JSON
object as Mapping instead of dict (LIT001).
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* fix(responses): propagate cache_control through responses->chat completion bridge
User/assistant messages, tool call outputs, and function calls built from
Responses API input items were dropping cache_control, breaking prompt
caching for Bedrock/Anthropic when routed through the litellm_completion
transform. Also propagate cache_control onto the last content element of
user/assistant messages, since Anthropic/Bedrock read it from the content
block rather than (or in addition to) the message level.
Adds the missing cache_control field to ChatCompletionToolMessage and
ChatCompletionResponseMessage TypedDicts to keep the assignments type-safe.
* chore(ui): regenerate schema.d.ts for ChatCompletionToolMessage.cache_control
Adding cache_control to the ChatCompletionToolMessage TypedDict changed
the proxy's OpenAPI spec; regenerated via `npm run gen:api`.
* fix(responses): don't overwrite existing content-block cache_control; add list-content test
Per Greptile review on the PR:
- The message-level cache_control was unconditionally overwriting the
last content block's cache_control when that block already carried
its own (preserved by _transform_responses_api_content_to_chat_completion_content).
Now only propagates onto the last block when it doesn't already have one.
- Added the missing test coverage for the content-block propagation path
with list content, which no existing test exercised (all used plain
string content, so the isinstance(content, list) branch was never hit).
* style(responses): extract cache_control into a named variable, trim comment
Per style review: the message-building branch read cache_control 3
different ways in 5 lines (.get() twice, then [] indexing once), unlike
the sibling tool-call-output/function-call functions this PR touches,
which each extract it into a named variable up front
(output_cache_control, function_call_cache_control). Matched that
convention here too. Also trimmed the 3-line inline comment to one
line, matching the file's usual comment length for similarly subtle
behavior.
* chore: refresh lazy OpenAPI snapshot
* fix: type cache control propagation
* fix: place cache control type suppressions
* style: format cache control type conversion
* refactor(responses): build bridged cache_control messages in one shot
* fix(responses): keep cache_control marks on bridged tool calls and within the provider cap
A function_call item's mark now rides on the tool call the bridge builds, an assistant text
item merged behind a function_call keeps its own mark, a developer item keeps its mark when
it is mapped to a system message, and a previous_response_id replay yields its stale marks
to the newest ones so the provider cap holds. The Responses layer counts the marks on
function_call and function_call_output items before it places an ordinal injection point,
and the chat pass replaces that count with its own instead of adding to it.
* fix(responses): count each cache_control mark once and keep the bridge within the provider's limits
List content carries an item-level cache_control on its last block only, so the
provider sees one cache point and the breakpoint census counts one. Marked function
tools and SDK-typed input items are handled by the Responses-layer reserve, assistant
text merged into a function call turn keeps its mark on its own block, and the
previous_response_id replay budget drops marks per turn, newest kept, instead of
stripping every older turn once one overshoots.
The request's own marks now fall under the same budget, newest kept, so five marked
items no longer answer the provider's 400. A cache_control value that is not an
ephemeral mark with an optional ttl is left off the bridged message instead of
reaching the provider as sent, and a developer message maps to the system role with
its content, name, and cache_control only, so unknown keys never reach a strict
provider.
* fix(responses): send item cache_control only to cache-point targets and follow drop_params
Item marks reach the provider only when the target turns them into cache
points, and every other target keeps main's behavior of stripping them.
Without drop_params a mark the provider would refuse reaches it as sent,
so the client gets the same 400 as on /v1/chat/completions. With
drop_params a malformed mark is dropped and only the newest 4 are kept.
The shared input-to-messages helper and the developer mapping in
base_utils go back to main, and a marked developer item reaches a
cache-point target as a system message in the bridge.
* fix(responses): budget top-level marks, follow drop_params on Nova tool marks, and match chat-keeping providers
* fix(responses): validate replayed marks and count input tokens without item marks
* fix(responses): leave unknown Bedrock ARNs unmarked under drop_params, match OpenRouter's list, and reserve only forwarded marks
* test(responses): cover item cache_control parity per chat route, drop_params trimming, and replay budgets
* fix(responses): keep item marks off inspection paths and match each chat route's keep rule
Only the request the bridge sends gets item marks now. Guardrails, token
counting, rate limiting, and input_tokens read the input with no item marks,
as on main, so an off-spec mark cannot fail a guardrail call. Tool message
and tool call marks are typed inside the bridge, leaving the shared chat
types and guardrail schemas as on main.
Databricks keeps marks only for Claude, Snowflake uses its chat route's
Claude check, and every Anthropic deployment keeps them whatever its model
name. The injection census skips SDK-typed tools, and the owned-proxy test
expects the marked messages on /v1/responses.
* test(responses): cover guardrail reads without item marks, Snowflake name parity, and SDK-typed tools in the census
* fix(responses): leave a non-list input untouched when stripping item marks
* test(responses): cover string, list, object and tuple input with and without item marks kept
Each input shape now answers the same whether the unmapped Bedrock ARN keeps item marks or strips them under drop_params, through the proxy and the SDK. Object input also runs through the OpenAI-compatible chat bridge, /v1/responses/input_tokens and the project input-token limiter
* test(cache): derive owned-proxy cell timeouts from the harness boot budget
The capped-burst cells and the uncapped response-cache cells boot their own two-worker proxy, and a fixed 240 or 360 second timeout cut them during boot on a slow runner before the harness readiness budget could decide. They now carry the same derived budget as the sibling chaos and guardrail files, with four boot and stop budgets for the restart cell that boots twice
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Cancelling a task mid-request closed the query engine socket, the engine
handed the still-busy pool connection back, the next transaction start
queued its BEGIN behind the orphaned statement, and when that waiter was
cancelled too the engine panicked on its dropped reply channel
(interactive_transactions/actors.rs) and exited under the shutdown spend
flush, losing daily user and team spend for requests that had returned 200.
Every engine request now runs in its own task behind asyncio.shield so only
the caller is cancelled, and a transaction that opens after its caller left
is rolled back.
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(anthropic): keep refusal stop_reason and stop_details on the logged response
* fix(anthropic): keep the refusal stop-fields boundary tolerant of a non-dict upstream body
* test(anthropic): use Anthropic's documented refusal example as the unit-test fixture
* fix(streaming): keep provider fields on async stream cache hits and received_model_id on synthesized finish chunks
* fix(caching): replay cached completion_tokens_details on stream cache hits and skip non-object cached provider fields
* test(integration): cover Anthropic-family refusal stop_details on the wire, the cache and under chaos
* fix(caching): skip a cached completion_tokens_details that fails validation on stream replay
* fix(anthropic): keep refusal stop fields on a JSON-mode non-stream response
Also adds the round 3 audit cells for JSON-mode refusals, the proxy response serialization, and wrong-typed cached usage details on both stream replays
* test(anthropic): expect content_filter on streamed JSON tool refusals after main's stream change
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(proxy): let proxy admins open team member defaults to team admins
* fix(proxy): keep a team admin's never-resets duration on a team's first member budget
* test(proxy): pin the existing-row and stored never-resets cases for member defaults
* fix(proxy): evict the cached user on every proxy for every /user/update and /user/bulk_update
Previously the cached user row was only evicted and broadcast when the
update touched max_budget, model_max_budget, tpm_limit, rpm_limit or
metadata, so a models or user_role change kept being served by peer
proxies until the cache TTL expired
* test(proxy): cover user cache eviction past the first bulk invalidation batch
* fix(config): treat blank numeric env values as unset instead of crashing at import
A numeric knob exported as an empty string, which is what a Helm value: "" or an
emptied .env line produces, raised ValueError inside litellm/constants.py and the
proxy crash-looped before serving traffic. Every int(os.getenv(...)) and
float(os.getenv(...)) in constants.py now goes through get_env_int / get_env_float,
which treat blank as unset and fall back to the default with a warning on garbage.
* fix(guardrails): treat blank litellm_params values as unset so the guardrail still loads
A guardrail whose litellm_params read os.environ/X resolves X to "" when the
variable is exported but empty. Pydantic rejected "" for bool and typed fields
such as default_on and fail_on_error, the boot log reported the guardrail as
skipped and the proxy came up without it. A before-validator on BaseLitellmParams
now drops blank string values so each field takes its default, and the
default_on shortcut in LitellmParams treats a blank value as False.
* fix(config): warn once about an unparsable numeric env value so logging cannot recurse
The LiteLLM loggers' filter reads MAX_STRING_LENGTH_STDOUT_LOG and MAX_BASE64_LENGTH_STDOUT_LOG through get_env_int on every record. Warning about a garbage value from inside that read re-entered the filter, so every log call raised RecursionError. The value is now marked as reported before the warning is logged, which also stops per-record readers from repeating it. get_env_int_or_none and the new get_env_float_or_none share the same parser
* fix(config): treat blank values as unset for the remaining import-time numeric knobs
LITELLM_MAX_STREAMING_DURATION_SECONDS, REALTIME_WEBSOCKET_MAX_MESSAGE_SIZE_BYTES, a whitespace CLI_JWT_EXPIRATION_HOURS, LITELLM_CONFIG_PARAM_CACHE_TTL_SECONDS, the agent health check timeouts and the Prisma reconnect knobs still crashed the proxy at boot when blank. A blank or unparsable REQUEST_TIMEOUT no longer counts as explicitly set, so chat completion() keeps its 600s fallback
* fix(guardrails): map blank litellm_params to the field default so PATCH can still clear a value
Dropping blank keys before validation made PATCH /guardrails/{id} with api_base "" a silent no-op. Blank values of optional fields now resolve to the field default and stay in the set fields, so the PATCH clears the stored value. Required fields and unknown keys keep the value as given
* test: cover proxy numeric knobs in-process and expect no warning for a blank Straiker api_version
The three proxy import-time numeric knobs move into litellm/constants.py so the existing in-process blank-env test covers them, replacing the subprocess test. A blank api_version now means unset, so the Straiker test asserts the key-prefix version with no warning.
* fix(guardrails): keep LitellmParams kwargs typed without adding argument-type errors
---------
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
* fix(vllm): write spend log rows for /vllm passthrough calls to router models
The vLLM passthrough config never built a logged response for relayed calls, so stream and non-stream calls through /vllm with a router model wrote no spend row and never charged the key. vLLM now shares the OpenAI-shaped relay logging that Azure already had, moved into an openai_like helper both configs call.
* ci: run the vllm passthrough unit tests in the llms-providers shard
* fix(proxy): log /vllm text completions and ask router models for stream usage
A chat stream relayed to a router model without stream_options now gets
include_usage added on the way out, and the usage chunk vLLM appends is
removed before the client sees it, so the spend row carries vLLM's own
token counts instead of a local estimate. /vllm/completions calls now
write a row too, stream and non-stream.
* fix(vllm): keep usage, fingerprints, and disconnect rows correct on /vllm streams
Inject and strip stream usage on /vllm/completions too, hold every finish
frame for the fingerprint when n > 1, pass a forced-usage server's stream
through untouched, stop turning continuous_usage_stats alone into per-chunk
usage, close the upstream when the client hangs up so the row is written,
and assemble text choices with one sort instead of a scan per choice
* fix(vllm): keep n>1 stream order when holding finish chunks for the fingerprint
A held finish chunk now goes out as soon as any other chunk arrives, so a
choice that finishes while another still streams keeps its place. Only the
finish chunks right before the usage chunk get vLLM's system_fingerprint.
* fix(vllm): keep caller stream_options and log both success handlers
Forward a stream_options the proxy cannot read, or a non-boolean
include_usage, to vLLM as the caller sent it. Only a false
always_include_stream_usage changes the /vllm relay, so clients get the
same stream bytes under every setting.
Record the passthrough origin when logging starts so the sync and async
success handlers both parse the relayed body, hand callbacks a parsed
text completion, and read relayed messages and prompt separately so a
stray field no longer drops the local count.
Add the audit's integration tests for the relay, the settings, and the
Azure spend row.
* test(vllm): expect the caller's stream under always_include_stream_usage true
* fix(vllm): relay long SSE frames in linear time
* test(vllm): time the relay's usage and finish_reason prefilter on one large frame
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(proxy): add deployment warning endpoints with instance-wide ignore and snooze
GET /health/deployment_warnings returns the active deployment warnings
(env credential login, license expiry, no Redis on multiple workers,
detailed debug logging) with a severity and ignore state, for proxy admins
and proxy admin viewers. PATCH /health/deployment_warnings/{warning_id}
lets a proxy admin ignore a warning forever or snooze it for up to 30 days,
or restore it. Ignores live in one LiteLLM_UISettings row, apply to every
admin, and are written to the config audit log. Expired snoozes drop out
on read, and LITELLM_DISABLE_NO_REDIS_WARNING is reported as an env var
ignore. /health/readiness/details is unchanged
* fix(proxy): keep readiness Redis short-circuit ahead of the env lookup and harden ignore storage
Readiness details again answers from Redis before reading
LITELLM_DISABLE_NO_REDIS_WARNING, so proxies with Redis and a secret
manager do not fetch it on every poll. Stored ignores require timezone
aware datetimes and are read from the writer before a rewrite
* fix(proxy): keep deployment warning ignores readable in the config audit log
The audit payload keyed ignores by warning id, so the secret matcher
redacted the env_credential_login entry and the audit row lost who
snoozed it and until when. Log a list of id and ignore entries instead
* feat(dashscope): add Wan 3.0, Wan 2.7 and HappyHorse video generation via /v1/videos
DashScopeVideoConfig maps litellm's OpenAI Videos API onto Alibaba Model
Studio's async task API: create hits
/api/v1/services/aigc/video-generation/video-synthesis and status and content
poll /api/v1/tasks/{task_id}. It is registered for dashscope and for the
qwencloud and qwen_ai_platform brand aliases, following the same
get_dashscope_family_*_config pattern image generation uses. Unlike the
Sora-shaped providers the request body is nested, so prompt and media go under
input and everything else goes under parameters. X-DashScope-Async is always
sent, because DashScope rejects the call outright without it
OpenAI params translate to DashScope's contract. seconds becomes duration, and
size yields both a ratio and a named resolution tier (1280x720 gives 16:9 plus
720P), since sending only the ratio leaves the request on the default 1080P and
silently bills at that rate. input_reference becomes a single media entry,
base64 encoded into a data URI for file inputs. Which media type a model takes
(first_frame or, on the reference-to-video models, reference_image) is read
from provider_specific_entry.dashscope_video_reference_input via
get_model_info rather than hardcoded; models without it predate the media
array and get the flat img_url field. A caller-supplied media array always
wins, as that is the only way to express multi-role combinations
The create call is the only billed call, so it bills what DashScope will
deliver: the requested duration and tier, or the documented 5 second and 1080P
defaults when they are omitted. Smart duration (duration -1) lets the service
pick the length after that call, so it is rejected rather than recorded at
zero, with a 400. Task creation and task lookup both report failures as a 200 carrying
top-level code and message with no output, so that shape is raised instead of
surfacing as a queued video with an empty id. Status responses re-encode the
deployment model from the polled id, so follow-up polls and downloads route to
the same deployment. Timestamps parse as UTC+8. DashScope publishes no list,
delete or remix endpoint, so those raise rather than guess at a URL
The end-to-end tests drive the real avideo_generation() entrypoint through an
httpx MockTransport and assert on the JSON body httpx actually encoded. That is
what catches a mapper that drops input_reference before the request is built,
or a body fragment json cannot serialize, neither of which a test calling the
transform directly would see
Both model_prices maps gain wan3.0-video, wan3.0-video-prime, wan2.7-t2v,
wan2.7-i2v, wan2.7-r2v and the HappyHorse 1.0 and 1.1 t2v, i2v and r2v models,
for dashscope and the qwencloud and qwen_ai_platform aliases, priced per output
second by resolution tier from Model Studio's USD rate cards. dashscope and
qwen_ai_platform default to the Beijing host and use its rate card; qwencloud
defaults to the international host and uses Singapore's. The three
*_API_BASE_VIDEO overrides are documented in BerriAI/litellm-docs#1920
Co-authored-by: AaronHowell <237895480@qq.com>
* fix(dashscope): fail loudly on video responses that cannot be read
Addresses the review on the video generation PR. A body that is not the
documented task shape, an output block missing task_id or task_status, an
undocumented task_status and a missing output all raise DashScopeVideoError
now, instead of defaulting into a queued video with an empty id that callers
poll forever. A JSONDecodeError from a gateway response no longer escapes as
a generic connection error.
task_id and task_status are required on _TaskOutput, and a missing output
raises even when the body carries no code or message.
Download failures move into the 4xx range, since a 5xx has the proxy retry
them: 409 while the task is still running, 400 for a task that failed, 404
for an id that expired. The status a provider reported is kept when it
reported one, so an in-body failure that arrives on a 200 is billed as the
400 it is rather than as a retryable connection error.
characters and image are only dropped when drop_params is set; the params
that are rebuilt into the body or consumed by the caller (model, prompt,
user, extra_*) stay silently skipped, since the proxy stamps user from the
caller's end-user header.
Co-authored-by: AaronHowell <237895480@qq.com>
* fix(dashscope): clear the lint gates after merging main
* fix(dashscope): accept file tuples and paths as video input_reference
video_generation() types input_reference as FileTypes, which includes
(filename, content[, content_type[, headers]]) tuples and pathlib paths,
but the DashScope encoder only read bytes and file objects, so those
image-to-video calls raised before reaching DashScope. Bare strings stay
URLs and are never opened as paths.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(dashscope): snap size to a supported ratio and read file tuples and paths
* test(dashscope): assert whole mapped values and bill against hand-written rates
The tier pricing test read its expected rate from the same map the cost
calculator uses, so a wrong rate passed. Billing is now checked against
fixed rates driven through the status response usage, while the map test
only asserts every supported tier has its own rate. Size mapping tests
compare the full mapped dict, and the single-test wiring class is a
plain function.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(videos): download content through the caller's client and stop retrying dead tasks
Two Cursor Bugbot findings on the DashScope video PR.
The DashScope artifact lives on OSS, so transform_video_content_response makes a
second request. It built its own cached client, discarding an injected client and
its ssl_verify. Thread the client through the content response the way
transform_video_status_retrieve_response already does, and use it in the three
providers that fetch from a second URL (DashScope, RunwayML, fal.ai). The
remaining providers accept it and read the response body, matching how they
already accept an unused client on status retrieve.
A FAILED or CANCELED task reported without a code or message fell past the
in-flight check into the 502 branch, so the proxy retried a task that can never
produce a video. Raise 400 for a terminal status instead.
* fix(dashscope): answer 404, not 409, when a Wan video is downloaded before it finishes
A download that lands while the task is still PENDING or RUNNING answered 409. That
reads as a client error, which was the intent, but litellm's retry predicate counts 409
as retryable, so the proxy polled DashScope twice more before giving up and then counted
the caller's own early download against the model's error rate
404 is what the OpenAI videos API documents for content that is not ready, and it is the
one status in that family the retry predicate leaves alone. The rest of the mapping is
unchanged: 400 for a task the provider reported as failed, 404 for an expired task id,
502 when a succeeded task carries no artifact
The download tests now drive avideo_content and assert the caller sees NotFoundError
rather than an internal server error
* fix(dashscope): route a video poll back to the model group that created the task
The create response encoded the upstream model name into the returned id. Both the
dashscope and qwen_ai_platform prefixes serve the same model names from their own hosts,
so a proxy with a deployment per region had deployments that only the model group told
apart. Encoding the bare model name resolved a poll to whichever deployment the router
lists first, which sent it to the other region's host with that region's key and a task
id that region never issued. That is the reroute the earlier proof records as the
dashscope and qwen_ai_platform poll ids coming back tagged qwencloud
The id now carries the model group the create was routed to, which is what the proxy
resolves a poll back to, and falls back to the model name for a direct SDK call that has
no router
The regression test drives a real router with one deployment per region, under a
different prefix and under the same one, and asserts the poll carries the key of the
region that created the task
* fix(dashscope): read the routed model group from litellm_metadata
The router stamps the model group it picked into litellm_metadata. metadata only
carries a copy when the caller sent no metadata of its own, and every proxy request
sends some, so a proxied create still encoded the bare model name and the poll
resolved to whichever region's deployment the router lists first
The regression test now also creates the video with caller metadata, which fails
without this change
* fix(dashscope): reject unbillable video requests and keep fal and runway on their own clients
* fix(dashscope): show the caller why a video task is gone or not ready instead of the bare upstream body
* fix(dashscope): check video request values before the billed create and honor per-call drop_params
Type check documented DashScope parameters so a bad seed answers 400 instead of a billed failed task,
pass the call's and deployment's drop_params into the video mapping, answer 400 for an expired task id so
the deployment is not cooled down, read DASHSCOPE_API_BASE for video, keep polling SUSPENDED tasks, and
refuse to map size onto models litellm has no price for.
* fix(dashscope): carry the requested model name in video ids so alias and wildcard keys can poll
A key scoped to an alias or to a name a wildcard serves could create a
video and then got a 403 on the poll and the download, because the id
carried the group that served the call (wan3, qwencloud/*) and the key
was never given that name. The id now carries the name the router was
asked for on that hop, and the router resolves an alias, routing group
or wildcard name in an id back to itself when no deployment name or
provider model matches it first
* fix(dashscope): keep a routing group's video polls on the member that created the task
A routing group spans several model groups, so an id carrying the group's name
load-balanced its polls, and some left for another region's host and key with a
task id that region never issued. The id now carries the member that served the
task when the request named a routing group, and keeps the requested name for a
model group, an alias, or a wildcard match.
* fix(dashscope): carry the requested and served model in video ids so group-scoped keys poll in the creating region
* fix(proxy): pin a video's served group only after key and team aliases resolve
* test(integration): audit DashScope video create, poll, download, routing and chaos paths
* fix(proxy): keep catch-all video polls on their provider and count served polls against the requested group
* test(routing): expect a forged served group to be dropped from the polled video id
* test(integration): cover rate-limited routing group polls and catch-all polls of DashScope videos
---------
Co-authored-by: AaronHowell <237895480@qq.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(proxy): build configured logging callbacks at boot so a broken callback fails startup instead of the first request
Startup already built every configured callbacks, success_callback and
failure_callback entry, but through the per-request helper that swallows
every construction error. A callback whose SDK could not be imported (for
example langfuse outside the supported version range) stayed a string, so
the first request on each worker rebuilt it inside function_setup and
returned a 500.
Startup now builds them with a variant that re-raises ImportError naming
the callback, which fails boot. Other construction errors keep the old
non-blocking behavior, and callbacks added at runtime keep the lenient path.
* refactor(proxy): build the configured callback replacements in one pass
Collect the startup-built callbacks with a generator and a frozen mapping
instead of a mutable dict filled in a loop, and drop a docstring that
restated the function body
---------
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
* feat(ui): add decisions and classifier providers to the Add Model form
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(cost): use literal upstream decisions paths
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: drop duplicate TYPESAFE provider fields entry superseded by main's TypeSafe
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(ui): drop stray blank line in provider_info_helpers test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): type the OSS decisions provider fields test
* fix(decisions): keep Laya and Bespoke env keys on their configured server
Route laya and bespoke Decisions deployments through the same oss_connection
and validate_oss_model rules the pass-through and the classifier router use,
so LAYA_API_KEY and BESPOKE_API_KEY only go to LAYA_API_BASE and BESPOKE_API_BASE,
an api_base with credentials, a query, or a non-HTTP scheme answers 400, and
an unknown checkpoint name answers 400 before any upstream call
* fix(decisions): send the Laya and Bespoke env key to a form-added model on the env server
A deployment added through Add Model always carries an API base, so the
env key only reached deployments with no base. The env key now goes to a
deployment whose base matches LAYA_API_BASE or BESPOKE_API_BASE, an empty
deployment key counts as no key, and the Bespoke placeholder is the name
Ollama serves.
* test(decisions): cover the env-base classifier case in the complexity router test
* fix(decisions): match the env server with or without /v1 and point Ollama users at bespoke/nimble
* fix(decisions): call /v1/systemone once from classifiers and pass-through on a /v1 base
* fix(decisions): strip /v1 from the API base path only, so a host named v1 still works
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(mcp): separate ID-JAG leg-2 client identity, support public resource clients
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(mcp): format ID-JAG credential changes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(mcp): satisfy lint gates and regenerate API snapshot for ID-JAG resource client
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(mcp): require a confidential ID-JAG resource client, drop none auth method
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(mcp): drop formatting churn from ID-JAG resource client tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: retrigger after flaky test_mcp_tool_search timeout
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: rerun against main after duplicate gemini-4-argon price keys were fixed
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(decisions): send images to Cloudflare Clef
OpenAI-format input_image parts and the System One images field now reach Cloudflare clef and clef-flash in its images field, with remote URLs downloaded and inlined first. Providers that cannot take images drop them under drop_params and answer 400 otherwise.
* fix(decisions): follow drop_params for images and use the call timeout for image downloads
* fix(decisions): drop a malformed images entry on its own under drop_params
* fix(decisions): refuse images next to input whatever drop_params says
* fix(decisions): bound clef image fetches, honor drop_params for images, and log caller URLs
Refuse more than 4 images for Cloudflare clef before any download, keep the
caller's timeout as one deadline across image fetch retries, accept images
next to input under drop_params, validate image strings and objects, and pass
the caller's image URLs (not inlined bytes) to logging callbacks.
* fix(decisions): check input_image URLs and clef image sizes
* fix(decisions): accept data URL media type parameters and check clef image types
* fix(decisions): count clef images after dropping unsupported types and flag clef vision
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Non-admin callers previously received deployment-wide cache activity,
error breakdowns, every key alias on the proxy, and could filter the
aggregates by any other tenant's key alias. All four queries now take
the caller's read scope as bind parameters, which also shrinks the
key-alias and model filter options to the caller's own visibility.
The three-way scope resolution (admin / internal user / fallback) now
lives in resolve_spend_log_read_scope, shared with /spend/tags and
/spend/logs/session/ui instead of being duplicated per endpoint.
* feat(claude-code): emit installationPreference in marketplace.json entries
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(claude-code): satisfy marketplace CI checks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(claude-code): qualify marketplace entry as read only
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(claude-code): suppress typed marketplace assembly warning
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(claude-code): restrict installation_preference to the documented values
Accept only available, auto_install, or required so a typo returns 422 instead of being silently dropped, and cover the PUT full-replace clearing behavior with a test
* fix(claude-code): echo installation_preference on the public skill hub
* refactor(claude-code): read installation_preference with one membership check and cover hand-edited rows
* test(integration): audit installation_preference on the Claude Code marketplace
* test(integration): kill a serving worker in the marketplace chaos cell
* test(integration): give the marketplace worker-kill cell the two-boot timeout
* fix(proxy): keep installationPreference on the MarketplacePluginEntry model
* fix(proxy): refuse self-installing preferences off sha256 archives, accept the camelCase key, and echo the preference on POST and PUT
* fix(claude-code): serve installation_preference only on archive plugins and refuse both key spellings
Claude Code 2.1.75 and older reject a marketplace.json once any plugin carries installationPreference,
and every Claude Code release before archive support rejects a catalog with an archive plugin. Requiring
an archive source for every value, available included, keeps the field off catalogs those clients can read.
A request that sends installation_preference and installationPreference together now gets a 422.
* fix(claude-code): read the preference spelling that carries a value when the other is null
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
