Commit graph

898 commits

Author SHA1 Message Date
Mateo Wang
6b7c77b7d3
fix(http): restore the 15s aiohttp keep-alive default in the Python and Rust clients (#46355)
* fix: lower aiohttp keepalive timeout

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(http): restore the 15s keep-alive default and apply it to the Rust client

15s is aiohttp's own default and what every release before v1.78.0 used. The Rust HTTP client reads the same AIOHTTP_KEEPALIVE_TIMEOUT variable, so its default moves with it.

* test(integration): cover upstream connections dropped after the keep-alive idle window

---------

Co-authored-by: nate <nate@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 10:50:11 -07:00
Rick
4f3cb48dc1
feat(dashscope): add Wan 3.0, Wan 2.7 and HappyHorse video generation via /v1/videos (#43706)
* feat(dashscope): add Wan 3.0, Wan 2.7 and HappyHorse video generation via /v1/videos

DashScopeVideoConfig maps litellm's OpenAI Videos API onto Alibaba Model
Studio's async task API: create hits
/api/v1/services/aigc/video-generation/video-synthesis and status and content
poll /api/v1/tasks/{task_id}. It is registered for dashscope and for the
qwencloud and qwen_ai_platform brand aliases, following the same
get_dashscope_family_*_config pattern image generation uses. Unlike the
Sora-shaped providers the request body is nested, so prompt and media go under
input and everything else goes under parameters. X-DashScope-Async is always
sent, because DashScope rejects the call outright without it

OpenAI params translate to DashScope's contract. seconds becomes duration, and
size yields both a ratio and a named resolution tier (1280x720 gives 16:9 plus
720P), since sending only the ratio leaves the request on the default 1080P and
silently bills at that rate. input_reference becomes a single media entry,
base64 encoded into a data URI for file inputs. Which media type a model takes
(first_frame or, on the reference-to-video models, reference_image) is read
from provider_specific_entry.dashscope_video_reference_input via
get_model_info rather than hardcoded; models without it predate the media
array and get the flat img_url field. A caller-supplied media array always
wins, as that is the only way to express multi-role combinations

The create call is the only billed call, so it bills what DashScope will
deliver: the requested duration and tier, or the documented 5 second and 1080P
defaults when they are omitted. Smart duration (duration -1) lets the service
pick the length after that call, so it is rejected rather than recorded at
zero, with a 400. Task creation and task lookup both report failures as a 200 carrying
top-level code and message with no output, so that shape is raised instead of
surfacing as a queued video with an empty id. Status responses re-encode the
deployment model from the polled id, so follow-up polls and downloads route to
the same deployment. Timestamps parse as UTC+8. DashScope publishes no list,
delete or remix endpoint, so those raise rather than guess at a URL

The end-to-end tests drive the real avideo_generation() entrypoint through an
httpx MockTransport and assert on the JSON body httpx actually encoded. That is
what catches a mapper that drops input_reference before the request is built,
or a body fragment json cannot serialize, neither of which a test calling the
transform directly would see

Both model_prices maps gain wan3.0-video, wan3.0-video-prime, wan2.7-t2v,
wan2.7-i2v, wan2.7-r2v and the HappyHorse 1.0 and 1.1 t2v, i2v and r2v models,
for dashscope and the qwencloud and qwen_ai_platform aliases, priced per output
second by resolution tier from Model Studio's USD rate cards. dashscope and
qwen_ai_platform default to the Beijing host and use its rate card; qwencloud
defaults to the international host and uses Singapore's. The three
*_API_BASE_VIDEO overrides are documented in BerriAI/litellm-docs#1920

Co-authored-by: AaronHowell <237895480@qq.com>

* fix(dashscope): fail loudly on video responses that cannot be read

Addresses the review on the video generation PR. A body that is not the
documented task shape, an output block missing task_id or task_status, an
undocumented task_status and a missing output all raise DashScopeVideoError
now, instead of defaulting into a queued video with an empty id that callers
poll forever. A JSONDecodeError from a gateway response no longer escapes as
a generic connection error.

task_id and task_status are required on _TaskOutput, and a missing output
raises even when the body carries no code or message.

Download failures move into the 4xx range, since a 5xx has the proxy retry
them: 409 while the task is still running, 400 for a task that failed, 404
for an id that expired. The status a provider reported is kept when it
reported one, so an in-body failure that arrives on a 200 is billed as the
400 it is rather than as a retryable connection error.

characters and image are only dropped when drop_params is set; the params
that are rebuilt into the body or consumed by the caller (model, prompt,
user, extra_*) stay silently skipped, since the proxy stamps user from the
caller's end-user header.

Co-authored-by: AaronHowell <237895480@qq.com>

* fix(dashscope): clear the lint gates after merging main

* fix(dashscope): accept file tuples and paths as video input_reference

video_generation() types input_reference as FileTypes, which includes
(filename, content[, content_type[, headers]]) tuples and pathlib paths,
but the DashScope encoder only read bytes and file objects, so those
image-to-video calls raised before reaching DashScope. Bare strings stay
URLs and are never opened as paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(dashscope): snap size to a supported ratio and read file tuples and paths

* test(dashscope): assert whole mapped values and bill against hand-written rates

The tier pricing test read its expected rate from the same map the cost
calculator uses, so a wrong rate passed. Billing is now checked against
fixed rates driven through the status response usage, while the map test
only asserts every supported tier has its own rate. Size mapping tests
compare the full mapped dict, and the single-test wiring class is a
plain function.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(videos): download content through the caller's client and stop retrying dead tasks

Two Cursor Bugbot findings on the DashScope video PR.

The DashScope artifact lives on OSS, so transform_video_content_response makes a
second request. It built its own cached client, discarding an injected client and
its ssl_verify. Thread the client through the content response the way
transform_video_status_retrieve_response already does, and use it in the three
providers that fetch from a second URL (DashScope, RunwayML, fal.ai). The
remaining providers accept it and read the response body, matching how they
already accept an unused client on status retrieve.

A FAILED or CANCELED task reported without a code or message fell past the
in-flight check into the 502 branch, so the proxy retried a task that can never
produce a video. Raise 400 for a terminal status instead.

* fix(dashscope): answer 404, not 409, when a Wan video is downloaded before it finishes

A download that lands while the task is still PENDING or RUNNING answered 409. That
reads as a client error, which was the intent, but litellm's retry predicate counts 409
as retryable, so the proxy polled DashScope twice more before giving up and then counted
the caller's own early download against the model's error rate

404 is what the OpenAI videos API documents for content that is not ready, and it is the
one status in that family the retry predicate leaves alone. The rest of the mapping is
unchanged: 400 for a task the provider reported as failed, 404 for an expired task id,
502 when a succeeded task carries no artifact

The download tests now drive avideo_content and assert the caller sees NotFoundError
rather than an internal server error

* fix(dashscope): route a video poll back to the model group that created the task

The create response encoded the upstream model name into the returned id. Both the
dashscope and qwen_ai_platform prefixes serve the same model names from their own hosts,
so a proxy with a deployment per region had deployments that only the model group told
apart. Encoding the bare model name resolved a poll to whichever deployment the router
lists first, which sent it to the other region's host with that region's key and a task
id that region never issued. That is the reroute the earlier proof records as the
dashscope and qwen_ai_platform poll ids coming back tagged qwencloud

The id now carries the model group the create was routed to, which is what the proxy
resolves a poll back to, and falls back to the model name for a direct SDK call that has
no router

The regression test drives a real router with one deployment per region, under a
different prefix and under the same one, and asserts the poll carries the key of the
region that created the task

* fix(dashscope): read the routed model group from litellm_metadata

The router stamps the model group it picked into litellm_metadata. metadata only
carries a copy when the caller sent no metadata of its own, and every proxy request
sends some, so a proxied create still encoded the bare model name and the poll
resolved to whichever region's deployment the router lists first

The regression test now also creates the video with caller metadata, which fails
without this change

* fix(dashscope): reject unbillable video requests and keep fal and runway on their own clients

* fix(dashscope): show the caller why a video task is gone or not ready instead of the bare upstream body

* fix(dashscope): check video request values before the billed create and honor per-call drop_params

Type check documented DashScope parameters so a bad seed answers 400 instead of a billed failed task,
pass the call's and deployment's drop_params into the video mapping, answer 400 for an expired task id so
the deployment is not cooled down, read DASHSCOPE_API_BASE for video, keep polling SUSPENDED tasks, and
refuse to map size onto models litellm has no price for.

* fix(dashscope): carry the requested model name in video ids so alias and wildcard keys can poll

A key scoped to an alias or to a name a wildcard serves could create a
video and then got a 403 on the poll and the download, because the id
carried the group that served the call (wan3, qwencloud/*) and the key
was never given that name. The id now carries the name the router was
asked for on that hop, and the router resolves an alias, routing group
or wildcard name in an id back to itself when no deployment name or
provider model matches it first

* fix(dashscope): keep a routing group's video polls on the member that created the task

A routing group spans several model groups, so an id carrying the group's name
load-balanced its polls, and some left for another region's host and key with a
task id that region never issued. The id now carries the member that served the
task when the request named a routing group, and keeps the requested name for a
model group, an alias, or a wildcard match.

* fix(dashscope): carry the requested and served model in video ids so group-scoped keys poll in the creating region

* fix(proxy): pin a video's served group only after key and team aliases resolve

* test(integration): audit DashScope video create, poll, download, routing and chaos paths

* fix(proxy): keep catch-all video polls on their provider and count served polls against the requested group

* test(routing): expect a forged served group to be dropped from the polled video id

* test(integration): cover rate-limited routing group polls and catch-all polls of DashScope videos

---------

Co-authored-by: AaronHowell <237895480@qq.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 10:10:07 -07:00
yucheng-berri
d17cd947b9
fix(sso)!: reject empty identities before session creation (#44098) 2026-10-14 09:10:58 -07:00
ryan-crabbe-berri
12cce6c372
fix(anthropic): check Vertex forced-tool support on the real model, skip non-string user ids, drop speed where unsupported (#46337)
* fix(anthropic): check Vertex and Bedrock forced-tool support on the real model

The Vertex and Bedrock invoke Claude transforms swapped the model for a
claude-3-sonnet stand-in before calling the shared Anthropic mapping so
response_format would take the tool-based path. Every model-keyed check
in that mapping then ran against the stand-in, so the synthetic json
tool was forced on models whose map entry says forced tool use is
unsupported and the request 400ed upstream. Bedrock patched the one
symptom with a backstop, Vertex had nothing.

The shared mapping now asks a uses_native_structured_output hook for the
structured-output decision and the two overrides answer it with their
provider-scoped lookup, so the real model reaches every check and the
stand-in, the restore dance and the Bedrock backstop go away.

* fix(anthropic): skip non-string metadata user ids instead of raising

_valid_user_id ran re.match straight on litellm_params metadata.user_id,
so an integer id raised TypeError and the proxy answered 500. The user
param path already skips non-strings, so the validator now does the
same for any caller: a non-string id is simply not forwarded.

* fix(utils): drop speed on every provider without fast mode

speed rides into get_optional_params as a provider-specific kwarg, so the
only gate for it lived in the Anthropic transform and Bedrock converse,
OpenAI, Azure and the extra_body legs forwarded it to a 400. The gate now
runs once in get_optional_params before provider-specific kwargs are
copied, so every provider drops it with drop_params or rejects it with a
clean 400, and a direct Anthropic fast-mode model keeps it.

* fix(utils): gate speed only under drop_params and keep it for litellm_proxy

Raising on speed when drop_params is off would turn requests that tolerant OpenAI-compatible backends accept into 400s, and dropping it for litellm_proxy would disable fast mode on a downstream proxy that supports it

* test(anthropic): derive the forced-tool regression model from the cost map and drop new docstrings

* test(utils): pin that speed is untouched without drop_params and annotate new test locals

* test(integration): lock the Vertex and Bedrock invoke json tool to the real model's forced-tool support

Vertex sends tool_choice only when the model accepts forced tools, reasoning_effort maps the same with and without response_format, and Bedrock invoke keeps sending the json tool unforced
2026-10-14 08:56:08 -07:00
Mateo Wang
d81a5719ad
fix(cache): keep cache hits free of unset fields, and follow intentional behavior changes in CircleCI suites (#46359)
* test(integration): follow count_tokens error surfacing and decisions pricing in CircleCI suites

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): restore Responses input_file PDF pricing coverage

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): keep integration AGENTS guide unchanged

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(cache): preserve provider field presence on cache hits

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): expect credential display name in response

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): align OpenInference refusal field expectations

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): expect Akto request ingestion

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(mcp): expect missing tool error wording

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): expect no refusal field in cursor chat wire

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 08:56:05 -07:00
devin-ai-integration[bot]
a93fee9f6c
feat(decisions): add Laya and Bespoke providers to the Add Model form (#45260)
* feat(ui): add decisions and classifier providers to the Add Model form

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(cost): use literal upstream decisions paths

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: drop duplicate TYPESAFE provider fields entry superseded by main's TypeSafe

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(ui): drop stray blank line in provider_info_helpers test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): type the OSS decisions provider fields test

* fix(decisions): keep Laya and Bespoke env keys on their configured server

Route laya and bespoke Decisions deployments through the same oss_connection
and validate_oss_model rules the pass-through and the classifier router use,
so LAYA_API_KEY and BESPOKE_API_KEY only go to LAYA_API_BASE and BESPOKE_API_BASE,
an api_base with credentials, a query, or a non-HTTP scheme answers 400, and
an unknown checkpoint name answers 400 before any upstream call

* fix(decisions): send the Laya and Bespoke env key to a form-added model on the env server

A deployment added through Add Model always carries an API base, so the
env key only reached deployments with no base. The env key now goes to a
deployment whose base matches LAYA_API_BASE or BESPOKE_API_BASE, an empty
deployment key counts as no key, and the Bespoke placeholder is the name
Ollama serves.

* test(decisions): cover the env-base classifier case in the complexity router test

* fix(decisions): match the env server with or without /v1 and point Ollama users at bespoke/nimble

* fix(decisions): call /v1/systemone once from classifiers and pass-through on a /v1 base

* fix(decisions): strip /v1 from the API base path only, so a host named v1 still works

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-14 08:11:11 -07:00
devin-ai-integration[bot]
aa0279e0a2
feat(anthropic): generate auto mode safeguard_results with a classifier model on the /v1/messages chat bridge (#45600)
* feat(anthropic): generate auto mode safeguard_results with a classifier model on the /v1/messages chat bridge

* fix(anthropic): check caller limits for the safeguards classifier, mark truncated tool calls, and keep pre-compaction context

* fix(anthropic): log classifier spend under the caller's key hash and read list-shaped classifier replies

* feat(anthropic): generate safeguard_results on the Responses bridge and native non-Claude /v1/messages providers

* fix(messages): keep safeguards requests on Python under the Rust opt-in

* fix(anthropic): keep non-ASCII text unescaped in the safeguards classifier prompt

* test(anthropic): build the text-parts classifier reply without openai's construct path

Message.model_construct goes through openai's construct_type, which caches
discriminated unions in a weak-key map. On Python 3.12 a PEP 604 union cannot be
weak-referenced, so the fixture raised inside the fake classifier and the test
read an unavailable verdict. model_copy sets the list content without that path.

* fix(anthropic): scan only new bytes for SSE frame ends on safeguarded native streams

* fix(anthropic): answer safeguards on web search short circuits and widen the classifier reply cap

* fix(anthropic): answer unsupported when the proxy serves no deployment for the safeguards classifier

* fix(anthropic): keep a backend's stream verdicts and join split SSE data lines before stamping

* fix(anthropic): map only the router's no-healthy-deployments refusal to unsupported

* fix(anthropic): answer unavailable for the turn on a transient caller denial and let the classifier deployment's timeout apply

* fix(anthropic): keep the user's whole opening message in the classifier transcript past its budget

* fix(anthropic): run the proxy's own access checks for the safeguards classifier call

The classifier and compaction sub-calls now pass through the same model access
checks as the parent request (key, team, team member, user, project, end user,
managed agent grants), with budgets left to the per-model budget check so
nothing is reserved. They also carry the parent's logging params and headers,
so a redaction header or per-request callback params apply to their own rows.

Only per-model RPM limits count against a sub-call now, since a key-wide or
team-wide RPM window is already spent by the parent request. The MCP
auto-execute loop classifies only its final response, a chat-bridge stream
without safeguards yields frames directly, and a classifier provider failure
logs one warning line instead of a traceback

* fix(anthropic): keep safeguards sub-calls off the parent's session routing and carry a body trace id

The classifier and compaction summary calls no longer get the parent's proxy_server_request, so a Claude Code session bound to a router no longer pulls them off their configured model. Redaction still reads the headers carried in metadata.

Their access checks now use the parent's resolved litellm_trace_id when the parent metadata has no trace_id, so a team that requires a trace id accepts a sub-call whose parent sent it in the body.

* fix(anthropic): keep the disable-callbacks header on safeguards sub-calls

* fix(proxy): charge per-model budgets from litellm_metadata when metadata holds only routing keys

* test(integration): add safeguard_results cells for the bridges, native routes, caller checks, and classifier failures

* fix(proxy): check the caller's budgets on the classifier call and keep failed access reads retryable

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-14 13:09:11 +00:00
devin-ai-integration[bot]
2f3ccbe7c7
feat(decisions)!: send images to Cloudflare Clef through its images field (#45734)
* feat(decisions): send images to Cloudflare Clef

OpenAI-format input_image parts and the System One images field now reach Cloudflare clef and clef-flash in its images field, with remote URLs downloaded and inlined first. Providers that cannot take images drop them under drop_params and answer 400 otherwise.

* fix(decisions): follow drop_params for images and use the call timeout for image downloads

* fix(decisions): drop a malformed images entry on its own under drop_params

* fix(decisions): refuse images next to input whatever drop_params says

* fix(decisions): bound clef image fetches, honor drop_params for images, and log caller URLs

Refuse more than 4 images for Cloudflare clef before any download, keep the
caller's timeout as one deadline across image fetch retries, accept images
next to input under drop_params, validate image strings and objects, and pass
the caller's image URLs (not inlined bytes) to logging callbacks.

* fix(decisions): check input_image URLs and clef image sizes

* fix(decisions): accept data URL media type parameters and check clef image types

* fix(decisions): count clef images after dropping unsupported types and flag clef vision

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-14 12:14:48 +00:00
mrinal-berri
1aee1a21ed
fix(databricks): send a single plain text block as a string for non-Claude models (#45686)
* fix(databricks): send a single plain text block as a string for non-Claude models

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(databricks): simplify single-text-block helper

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* docs(databricks): clarify json_object validation wording

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(databricks): integration cells for single text block content shape

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(databricks): rename list-content cells to stays_list

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(responses): expect plain string content for bridged Databricks messages

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(databricks): cover the customer's Unity Catalog model from #45617 and #45618

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(databricks): drop casts from _collapse_single_text_block

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(databricks): assert the whole outgoing messages list in the collapse unit test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(databricks): cover system joins, fallback, SDK clients, and outage burst for one-block content

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-14 09:22:02 +00:00
devin-ai-integration[bot]
b22a0bd9e5
test(integration): cover fine_tuning endpoint contracts (#44697)
* test(integration): cover fine_tuning endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): prove fine_tuning ownership hook, header spelling and drain order

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(fine_tuning): assert unified job id and retrieve ownership through route-granted keys

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(fine_tuning): assert managed file and job ownership rows in Postgres

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(fine_tuning): cover validation_file ownership, default retrieve ownership and full job objects

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 08:31:51 +00:00
ryan-crabbe-berri
b37eac29f0
fix(anthropic): cache tool result screenshots on the /v1/messages to Responses bridge (#46324)
* fix(anthropic): put a tool_result cache breakpoint on its hoisted images on the Responses bridge

A marked tool_result whose images get moved into a trailing user message now carries
the breakpoint on the last moved image instead of the function_call_output, so the
prefix OpenAI caches includes the images

* test(anthropic): cover a marked tool_result with two screenshots on the Responses bridge

* fix(anthropic): keep every tool result screenshot inside the breakpoint that covers it on the Responses bridge

A marked tool_result's screenshots now go right after its output, so earlier unmarked screenshots land inside the breakpoint and later tool outputs stay after it. A cache_control mark on a block inside a tool_result is read too.

* fix(anthropic): cover only the screenshots up to a mark inside a tool result, and drop raw breakpoints under drop_params

A cache mark on a text block inside a tool result now covers that result only up to the marked block, the way
Anthropic caches it, so screenshots after the marked text stay outside the breakpoint. A mark on the tool result
itself still covers every screenshot in it

A deployment that reads no prompt cache breakpoints now drops raw prompt_cache_breakpoint marks, top-level and
inside tool results, when drop_params is on, and still forwards them as sent without it

* fix(anthropic): keep raw breakpoints under drop_params for models that take them

* docs(anthropic): say a mark inside a tool result covers its whole text and documents

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
2026-10-14 08:12:11 +00:00
Oliver Jensen
14e820ef7c
fix(proxy): scope /global/activity/cache_hits analytics to the caller's own and permitted-team logs (#46210)
Non-admin callers previously received deployment-wide cache activity,
error breakdowns, every key alias on the proxy, and could filter the
aggregates by any other tenant's key alias. All four queries now take
the caller's read scope as bind parameters, which also shrinks the
key-alias and model filter options to the caller's own visibility.

The three-way scope resolution (admin / internal user / fallback) now
lives in resolve_spend_log_read_scope, shared with /spend/tags and
/spend/logs/session/ui instead of being duplicated per endpoint.
2026-10-14 10:01:21 +02:00
devin-ai-integration[bot]
46a389c60b
feat(claude-code): emit installationPreference in marketplace.json entries (#41430)
* feat(claude-code): emit installationPreference in marketplace.json entries

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(claude-code): satisfy marketplace CI checks

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(claude-code): qualify marketplace entry as read only

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(claude-code): suppress typed marketplace assembly warning

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(claude-code): restrict installation_preference to the documented values

Accept only available, auto_install, or required so a typo returns 422 instead of being silently dropped, and cover the PUT full-replace clearing behavior with a test

* fix(claude-code): echo installation_preference on the public skill hub

* refactor(claude-code): read installation_preference with one membership check and cover hand-edited rows

* test(integration): audit installation_preference on the Claude Code marketplace

* test(integration): kill a serving worker in the marketplace chaos cell

* test(integration): give the marketplace worker-kill cell the two-boot timeout

* fix(proxy): keep installationPreference on the MarketplacePluginEntry model

* fix(proxy): refuse self-installing preferences off sha256 archives, accept the camelCase key, and echo the preference on POST and PUT

* fix(claude-code): serve installation_preference only on archive plugins and refuse both key spellings

Claude Code 2.1.75 and older reject a marketplace.json once any plugin carries installationPreference,
and every Claude Code release before archive support rejects a catalog with an archive plugin. Requiring
an archive source for every value, available included, keeps the field off catalogs those clients can read.
A request that sends installation_preference and installationPreference together now gets a 422.

* fix(claude-code): read the preference spelling that carries a value when the other is null

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-14 07:59:25 +00:00
devin-ai-integration[bot]
cfa98e2f40
test(integration): cover pass_through endpoint contracts (#44690)
* test(integration): cover pass_through endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): run configured endpoint contracts on one proxy worker

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): prove pass-through streams, SDK spellings, endpoint renames and bedrock stream bodies

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover pass-through wire behavior

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover Vertex and Gemini passthrough authentication

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* Prove pass-through runtime method, credential-less Anthropic, stream barriers and Vertex strip layers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(pass_through): gate unquoted Bedrock stream targets and assert configured endpoint part types and methods after rename

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): assert pass-through query pairs so duplicate params fail

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 07:38:17 +00:00
devin-ai-integration[bot]
feb92d3255
test(integration): cover orgs_users_customers endpoint contracts (#44725) 2026-10-14 07:38:06 +00:00
devin-ai-integration[bot]
5f7de8d6ac
test(integration): cover keys_auth endpoint contracts (#44674)
* test(integration): cover keys_auth endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover hashed, path and token spellings for keys_auth contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): strengthen JWT mapping lifecycle coverage

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover reset_spend over plaintext and hashed path spellings with reset_to 0 and 0.02

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): drop Final in reset loop and use drain-then-fresh-refusal in warmed-peer delete proof

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): disable model info refresh in JWT lifecycle rig

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): drop unused discovery constant

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover cache invalidation and key paths

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo <mateo@berri.ai>
2026-10-14 07:12:28 +00:00
devin-ai-integration[bot]
1047a450bf
test(integration): pin assistants and evals endpoint contracts (#44734)
* test(integration): cover assistants_evals endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(tests): replace exception assertions flagged by ruff

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(openai): type assistant run options

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(unit): cover assistant and eval contract paths

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(openai): keep assistant run options unvalidated

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin assistants and evals wire contracts as BUG skips

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover every eval route, full responses, image parts and full SDK stream payloads

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): relay upstream eval run model, send reasoning_effort, cover SDK root alias, done frame and client disconnect

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): name reasoning_effort in the run BUG skip

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): use a non-null marker thread and name the eval run model restamp in its BUG skip

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): mark disconnect cases with a messages.create the proxy relays on main

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(assistants_evals): cover thread message shapes, raw run options, header leaks and /v1 eval bases

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): expect the OpenAI /cancel path for eval run cancel

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 07:08:35 +00:00
devin-ai-integration[bot]
27209b92b3
test(integration): cover mcp_runtime endpoint contracts (#44671)
* test(integration): cover mcp_runtime endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(integration): ruff format and import order in mcp runtime tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): assert out-of-path calls and untrusted forwarded headers are refused

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): move the untrusted forwarded-header discovery case into its own test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): hash the token row lookup, widen spend polling, and union open servers after listing

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): tighten mcp runtime contracts per review (sdk client, toolset scope, forwarded trust, callback html, session tools/call)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): assert the sdk unknown-path error without pinning the sdk's exception wrappers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover trailing-path and group csv mcp paths, rfc 9207 iss-less and error-iss callbacks, and id_token exchange

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover lite logout spelling, team-mapped and unmapped IdP subjects

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover per-server mcp-session resource and root /token spelling

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): unpack the sealed client code in the step-up tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 07:00:38 +00:00
devin-ai-integration[bot]
595cf066cd
test(integration): cover config_utils endpoint contracts (#44737)
* test(integration): cover config_utils endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): prove config_utils contracts on the peer, Vault AppRole and cert logins, and admin-view checks

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): assert no Vault traffic after config override delete

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): prove the Vault cert login presents the client certificate over TLS

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover the blank plugin_key dashboard edit and the env-only CyberArk readback

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(tests): ruff-sort imports and reformat config_utils integration tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): check plaintext Vault traffic in the cert phase with one drain

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): probe CyberArk runtime state after DELETE with a secret-resolving chat

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): barrier the Langfuse absence check on the shared export channel

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): send the dashboard's blanked CyberArk field and check row, readback and runtime env

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 07:00:23 +00:00
devin-ai-integration[bot]
5261a2cc06
test(integration): cover responses endpoint contracts (#44733)
* test(integration): cover responses endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): tighten responses contract tests per review

Isolate team-only response-id ownership with a same-user different-team key,
send every denial through the OpenAI SDK on all three aliases, make the
previous_response_id affinity check deterministic, assert the whole cancel
response, prove the Cursor stream text, -fast suffixes and include_obfuscation
strip, add SDK compact cases, restore router pre-call checks after the
affinity test, and inline the root-level responses_test_support helpers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): wait for every proxy worker before routing to new deployments

Restore the cross-worker convergence wait dropped when the shared helper was
inlined, so the affinity and retrieve-routing tests cannot pass before the
alternative deployment is loaded on a multi-worker gateway

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover Responses ownership and wire contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): address responses contract review

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): expect the provider model in the conversation passthrough body

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): expand responses endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): restore streamed response retrieval

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): wait for proxy workers after deployment setup

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): track readiness per proxy worker

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 06:59:15 +00:00
devin-ai-integration[bot]
8822e086ec
test(integration): cover budgets_spend endpoint contracts (#44736)
* test(integration): cover budgets_spend endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover budgets_spend endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover budgets_spend endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover budgets_spend endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): harden budgets_spend contracts per review

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): make spend report window midnight safe and drop redundant comments

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin organization spend report scope

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): close budgets and spend review findings

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): drop streamed cost header absence assertion

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(spend): cover streamed spend on linked budget tiers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(spend): wait for async spend logs

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 06:58:22 +00:00
devin-ai-integration[bot]
60ee966885
test(integration): cover anthropic_messages endpoint contracts (#44704)
* test(integration): cover anthropic_messages endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): close anthropic_messages review gaps

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): prove azure beta filtering and the safeguards allowlist

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): compare the forwarded anthropic-beta token-for-token and pin the truncated stream bug

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: strengthen endpoint wire contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: follow count_tokens upstream error relay from main

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: follow device-code refresh token response from main

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 06:57:54 +00:00
devin-ai-integration[bot]
0ca9f8185c
test(integration): cover files_batches endpoint contracts (#44703)
* test(integration): cover files_batches endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): split raw cloud-storage delete test by caller

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): drop mutable polling state in content download test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): dispatch request spellings without rebinding

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): import assert_never from typing_extensions for python 3.10

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): tighten files_batches wire and access assertions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): tidy files_batches access tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover repeated target models, upload expiry, batch aliases

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): prove files streaming path, SDK model denial and admin GCS delete

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): give the streaming head the wire barrier's full budget

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: strengthen files and batches wire coverage

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: drop duplicate parametrize from the model-routed upload helper

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: pass the SDK base url in the skipped repeated-plain upload test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): allow batch output metadata lookup

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: avoid draining file content requests before wait

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 06:57:41 +00:00
devin-ai-integration[bot]
e226016aee
test(integration): cover health_routing endpoint contracts (#44744)
* test(integration): cover health_routing endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): tighten health_routing contracts per review

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): close health_routing review gaps

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(integration): format observed routing fallback cells

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): close health_routing final review findings

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): resolve health_routing review findings

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 06:34:22 +00:00
devin-ai-integration[bot]
0f0c1bc80b
test(integration): cover teams endpoint contracts (#44705)
* test(integration): cover teams endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): address review on teams contract tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): drain shared upstream observations at fixture setup

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): prove team block, member and bulk permission contracts at runtime

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): assert email admin adds and whole membership rows for team members

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): drive key update grant and revoke, Z-suffixed temp budget expiry, link LIT-9257

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): close review gaps in teams endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): harden team member endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 06:29:20 +00:00
ryan-crabbe-berri
aa879f77a9
feat(proxy): let team admins create keys for team members behind an opt-in setting (#45870)
* feat(proxy): let team admins create keys for team members behind an opt-in setting

A team admin calling POST /key/generate with a team they administer and the
user_id of another member of that team got a 403, and the dashboard only
offered the "Another User" owner option to proxy admins

Add a create_member_keys entry to team_admin_editable_team_fields. When a
proxy admin turns it on, a team admin can create a team key owned by another
member of a team they administer, through the API and through the Create Key
form. It stays off by default, so nothing changes until a proxy admin opts in

With the setting off, the 403 a team admin gets for this call now names the
setting to enable instead of saying they can only create keys for themselves

* fix(ui): order the member owner picker after the team and drop the viewer from it

For a team admin the owner picker depends on the team, so it now sits under the Team field, lists only the other members of that team, and the Team field says why it is required

* fix(proxy): keep keys a team admin creates for a member from carrying more than LLM access

A key authenticates as its owner, so with create_member_keys on a team admin could create a key owned by a proxy admin who sits in their team and act as a proxy admin with it

The member now has to hold the internal_user or internal_user_viewer role, and the key is always an llm_api key, so the team admin never holds a secret that reaches management routes. Asking for a management or read_only key for a member is refused

The unit tests now check the permission decision directly instead of replacing generate_key_helper_fn, and the saved key is checked against a real proxy in the team admin gate matrix

* style(proxy): sort the team admin field permission imports

* refactor(proxy): force the member key type on the request instead of merging a second dict

* test(proxy): drive member key creation through the key generation helper

The pure helpers were covered but the wiring in _common_key_generation_helper was not. Dropping the owner role lookup, the refusal call, or the user_id handed to common_key_access_checks left every unit test green

* fix(proxy): skip the owner role lookup when create_member_keys is off

The owner role was read before the verdict checked the setting, so with the setting off a failing user table read surfaced as a 500 instead of the 403 a team admin got before, and every such request paid an extra writer read

---------

Co-authored-by: Yuneng Jiang <yuneng@berri.ai>
2026-10-14 06:25:58 +00:00
devin-ai-integration[bot]
148ca66ecb
test(integration): cover prompts_tags_caching endpoint contracts (#44686)
* test(integration): cover prompts_tags_caching endpoint contracts

* test(integration): assert prompt lifecycle requests per environment

* test(integration): pin non-latest prompt versions, label/path spellings and tag header/key spellings

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover production PATCH, streamed and root /responses prompts, and memory create scope

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): ignore proxy model discovery probes in prompt upstream doubles

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): prove tag update, peer prompt resync, memory team rows and Responses prompt spellings at runtime

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover memory metadata and key_prefix listing, multi-tag spellings for tag cache invalidation

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo <mateo@berri.ai>
2026-10-14 04:10:31 +00:00
devin-ai-integration[bot]
02f56c6319
test(integration): cover guardrails_policies endpoint contracts (#44722)
Some checks are pending
unit / core-utils (push) Blocked by required conditions
unit / endpoints (push) Blocked by required conditions
unit / enterprise-package (push) Blocked by required conditions
unit / enterprise-repositories-secrets (push) Blocked by required conditions
unit / integrations (push) Blocked by required conditions
unit / llms-anthropic (push) Blocked by required conditions
unit / llms-bedrock (push) Blocked by required conditions
unit / llms-openai-meta (push) Blocked by required conditions
unit / llms-providers (push) Blocked by required conditions
unit / llms-vertex-ai (push) Blocked by required conditions
unit / mcp-elicitation (push) Blocked by required conditions
unit / mcp-server (push) Blocked by required conditions
unit / proxy-auth (push) Blocked by required conditions
unit / proxy-endpoints (push) Blocked by required conditions
unit / proxy-extras (push) Blocked by required conditions
unit / proxy-feature-endpoints (push) Blocked by required conditions
unit / proxy-hooks-client (push) Blocked by required conditions
unit / ui-unit (push) Waiting to run
unit / docs (push) Waiting to run
unit / helm (push) Waiting to run
unit / OpenAI 2.20.0 compatibility (push) Waiting to run
unit / OpenAI 3.0.0 compatibility (push) Waiting to run
unit / OpenAI 3.25.0 compatibility (push) Waiting to run
unit / redis-py 5.3.1 compatibility (push) Blocked by required conditions
unit / redis-py 6.4.0 compatibility (push) Blocked by required conditions
unit / redis-py 7.4.1 compatibility (push) Blocked by required conditions
unit / redis-py 8.0.1 compatibility (push) Blocked by required conditions
unit / coverage (push) Blocked by required conditions
unit / unit passed (push) Blocked by required conditions
GitHub Actions Security Analysis / zizmor (push) Waiting to run
* test(integration): cover guardrails_policies endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): let the admin submissions list carry other rows

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): close guardrails policies review gaps

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover guardrail auth and policy attachments

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): tidy guardrail policy test helpers

* test(integration): keep plain non-admin submission refusal live

* test: expand policy attachment integration coverage

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): accept the intended 403 for plain submission keys

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin multi-tag header and spend row tags for policy tag scope

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): drop the duplicate pydantic import left by the main merge

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Devin <noreply@devin.ai>
2026-10-14 04:07:26 +00:00
devin-ai-integration[bot]
c9a1d9b026
test(integration): cover small_inference endpoint contracts (#44654)
* test(integration): cover small_inference endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): assert full search tool listing for scoped keys

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): keep moderation and search tool peers stateless

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover filename-agnostic OCR uploads, list queries, object rerank documents and search tool authorization

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): build search tool and moderation proxy configs without mutating the loaded yaml

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover the /ocr alias, model search spelling and Cohere v2 max_tokens_per_doc, clean up non-admin search tool rows

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover octet-stream OCR uploads and BUG-skip the Python requests boolean spelling

* test(integration): pin Exa passthrough of native fields and caller contents

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Mateo Wang <277851410+mateo-berri@users.noreply.github.com>
2026-10-14 04:06:15 +00:00
devin-ai-integration[bot]
32f0d680d2
test(integration): cover google_genai endpoint contracts (#44680)
* test(integration): cover google_genai endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): tighten google_genai contract assertions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover non-admin agent api_base guard and exactly-once spend

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): expand Google GenAI route contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover adapter responseJsonSchema, tool-loop history, /models alias and anthropic stream

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin raw SSE id/retry framing and agent list/versions key guard

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): strengthen Gemini endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 04:01:35 +00:00
devin-ai-integration[bot]
903f04dd46
test(integration): cover vector_stores endpoint contracts (#44739)
* test(integration): cover vector_stores endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): strengthen vector store route contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): update vector store route contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): drop owner-update BUG skip for admin-only vector store routes

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(vector_stores): cover the managed-file fallback, owner delete and per-request retries

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): probe vector store worker readiness by deployment id

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin vector store retry header and create routings

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(integration): ruff format vector store managed file tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 03:59:08 +00:00
devin-ai-integration[bot]
0a4ab269d7
test(integration): cover mcp_management endpoint contracts (#44687)
* test(integration): cover mcp_management endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): require server_not_found for pending and rejected submissions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin per-worker submission reloads, toolset and alias-collision spellings, raw key overrides

Assert the approving and rejecting worker immediately and only wait on the peer, add an outsider key and exact upstream calls, list by toolset_name, cover a config alias shadowing a DB server_name, send key_hash as a raw sk- key, and add BUG skips for API disallowed_tools and overrides on keys or teams without object_permission

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin XFF trusted-peer, load balancer chains and sibling refusal body

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): enforce output_policy trust chain, JSON-RPC XFF views, MCP tool-policy BUG pins

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(mcp): force a registry reload during the pending window, pin the admin default listing and the whole tools/call body

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 03:55:50 +00:00
devin-ai-integration[bot]
17c860293e
test(integration): cover realtime endpoint contracts (#44689)
* test(integration): cover realtime endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): reject duplicate multipart parts and cover the transcription_sessions alias

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): drop the type ignore and check the aliased transcription response

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin realtime multipart part types, replayed transcription session, undeclared session fields and budget refusal path

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): move the refusal path note next to the expected message

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): assert realtime refusal status before parsing and pin the replayed transcription session whole

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): sort the realtime webrtc test imports

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pick the transcription mint and token with one Final each

* test(integration): pin realtime transcription mint, replay, and redeem contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(integration): ruff format the realtime webrtc test

* test(integration): drop the redundant comment on the realtime SDK calls test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): type the realtime client secret refusals table

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(realtime): tighten endpoint contract assertions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 03:49:34 +00:00
devin-ai-integration[bot]
90a503d98c
test(integration): cover videos_containers endpoint contracts (#44691)
* test(integration): cover videos_containers endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): end the first container page on an ownerless id

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): assert container upload parts, SDK video extend, bare video ids, cost from map

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): catch a stale container list cache, cover video edit JSON, bracketed video refs and object input_reference

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin container optional fields, video client extras and the /videos alias on the wire

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin multipart character id decoding on video create as a BUG skip

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 03:49:01 +00:00
devin-ai-integration[bot]
d3cc77def4
test(integration): cover chat_completions endpoint contracts (#44688)
* test(integration): cover chat_completions endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin per-call prompts and both n=2 chat choices

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover mixed path-model auth, string guardrail rejection and typed text-completion forms

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): drop the prompt-spellings cache workaround and cover Azure SDK text completions on the deployments alias

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 03:45:38 +00:00
devin-ai-integration[bot]
11e91f464a
test(integration): cover scim endpoint contracts (#44656)
* test(integration): cover scim endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): assert scim admin key access and required active

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover scim cold-cache reactivation, entra membership ops, username filter and external id

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): assert full SCIM responses and denied writes

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin full SCIM resources and group filters

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): deny non-admin SCIM user deletes

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 03:41:54 +00:00
devin-ai-integration[bot]
8a2f706ab7
test(integration): cover audio endpoint contracts (#44699)
* test(integration): cover audio endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): close audio SDK clients after each test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): avoid list mutation in audio tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): keep exact upstream call check in model resolution test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover audio translations, bracket spellings, streaming and scoped alias keys

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin audio metadata parse sites, exact cost and speech bug pins

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): drop inline comment on audio wire assertion

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): strengthen audio wire and authorization contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 03:41:20 +00:00
devin-ai-integration[bot]
35124f3b49
test(integration): cover images endpoint contracts (#44653)
* test(integration): cover images endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): add SDK-driven image edit contract

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover image route aliases, repeated mask[] and variations route

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover AzureOpenAI images.edit alias and extra_body on gpt-image generation

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 03:40:29 +00:00
devin-ai-integration[bot]
117684cce5
test(integration): cover agents_a2a endpoint contracts (#44650)
* test(integration): cover agents_a2a endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin agent daily spend to the logged request day and assert cleanups

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): expect agent daily activity newest day first

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover A2A 1.0 task aliases, agent id headers, declared access groups and per-token spend

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): assert full A2A stream, header, alias, task and spend contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): assert forwarded a2a-version header on A2A sends

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): release A2A stream gate only after a full SSE frame

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin A2A trace id, stream headers, denied alias and task methods

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin A2A card protocol version

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): assert the whole accepted push config response

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 03:40:07 +00:00
devin-ai-integration[bot]
1c0f722aae
test(integration): cover embeddings endpoint contracts (#44660)
* test(integration): cover embeddings endpoint contracts

* test(integration): cover azure, vllm, nebius token arrays and flag bedrock token-array bugs

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin path-over-body model routing for azure embeddings

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin embeddings base64 relay, flat token arrays and api-key-only auth

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo <mateo@berri.ai>
2026-10-14 03:38:12 +00:00
devin-ai-integration[bot]
013c440597
test(integration): cover model_management endpoint contracts (#44684)
* test(integration): cover model_management endpoint contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin the exact model access refusal after a rename

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): wait for allowlist convergence in the rename test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): give the rename test lint suppression its reason

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): close reviewer gaps on model management contracts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): share the blocked serving assertions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): strengthen authorization and configuration regression tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): converge spend readbacks, ignore model discovery, cover slash credential names

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin the dashboard credential edit body round-trip

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-14 03:36:47 +00:00
yucheng-berri
b581ebc405
fix(prometheus): skip custom metadata labels a metric already carries (#46267)
* fix(prometheus): skip custom metadata labels a metric already carries

A custom metadata label whose sanitized name matches one of a metric's
built-in labels was appended a second time, so every .labels() call on
that metric raised ValueError. With custom_prometheus_metadata_labels:
[project_alias], the per-project per-model rate limit gauges raised on
every request from a project key with per-model limits, and the rest of
the success event (latency, deployment success, total requests) was
skipped.

* test(prometheus): cover shared custom labels on every endpoint, failure, metadata and outage burst

* test(prometheus): drain the shared upstream at the start of each integration test

* test(prometheus): assert the forwarded route, model and stream per endpoint cell

---------

Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
2026-10-13 18:20:52 -07:00
devin-ai-integration[bot]
c581c85559
fix(vertex_ai): unwrap tool-emulated json_schema envelopes and send output_config.format for Claude (#45871)
* fix(vertex_ai): unwrap tool-emulated json_schema envelopes and send output_config.format for Claude

Vertex AI Claude deployments that emulate response_format through a forced
tool sometimes nest the schema object under an extra key such as
`parameters` or `input`, and that envelope reached message.content with a
200. The converter now unwraps exactly one nested object that matches the
schema, leaving a payload that is itself schema-shaped alone.

Native structured-output rows sent the deprecated top-level `output_format`,
which Vertex rejects with a 400. The Vertex request now carries it as
`output_config.format`, with an explicit caller value winning.

* fix(anthropic): unwrap tool-emulated json_schema envelopes on streams and tool-name wrappers

* test(anthropic): expect the json tool description through a shared constant

* fix(anthropic): drop the json tool description that makes sonnet-5 omit required keys

* fix(anthropic): keep schema-accepted answers whole and decide held streams at the second key

An answer the json_schema accepts is returned as is before any envelope rule runs, the tool-name
wrapper is unwrapped only when the schema rejects it, a malformed schema turns the unwrap off, a
streamed answer whose first key is a schema property named name or tool_name waits for its second
key before streaming, the held fragments go through a StringIO, and a prefix that stays undecided
past 4 KB is held to the end

* test(anthropic): keep the stream family cells to providers whose stream iterator gets the schema

* fix(anthropic): unwrap json tool envelopes by structure instead of schema validation

The unwrap now reads only the schema's top-level property names. A lone
undeclared key holding an object, alone or beside a name equal to the json
tool's own name, is the envelope, and the unwrap repeats while the inner
object shares no key with the schema. jsonschema and referencing are no
longer imported on the request path. The stream holds only an answer that
opens with the tool's name or with an undeclared key holding an object, and a
schema without top-level properties streams as it arrives.

* fix(anthropic): walk json tool envelopes iteratively

* fix(anthropic): keep json tool arguments too deep to decode instead of raising

* fix(anthropic): unwrap json tool envelopes beside undeclared string labels

* fix(anthropic): keep json answers whole on schemas that allow extra keys and stream held text as sent

A schema that allows undeclared keys and requires none now keeps an undeclared wrapper key on both the stream and non-stream paths. When no envelope is found, the held stream text is now sent exactly as the model wrote it. On Vertex /v1/messages, output_format now wins over output_config.format when both are set, which matches the Anthropic adapter.

* test(anthropic): cover label envelopes, dynamic-key schemas, as-written held streams and output_format precedence

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-14 00:12:19 +00:00
Kerry Lu
ccdca3d39c
fix(cost_calculator): honour base_model when the deployment provider row has no rates, with a live-proxy integration test (#46318)
* fix(cost_calculator): price an explicit base_model from its own row when the deployment provider row has no rates

A chatgpt/ deployment with model_info.base_model recorded zero spend because the
base model was looked up under the chatgpt provider, which resolves to the
deliberately unpriced chatgpt/ row. When the deployment provider's row for the
base model declares no rates, price it under the provider of the base model's
own row instead. Deployments whose provider row declares rates are unchanged.

(cherry picked from commit 42d9a8693e)

* test(integration): bill a chatgpt base_model deployment from the priced row on a live proxy

Co-authored-by: sla55er <stiliyanmarkov@gmail.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: sla55er <stiliyanmarkov@gmail.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-13 17:10:10 -07:00
ryan-crabbe-berri
365c4885de
fix(openai): carry cache_control to prompt_cache_breakpoint on chat completions to gpt-5.6 (#46101)
* fix(openai): carry cache_control to prompt_cache_breakpoint on chat completions to gpt-5.6

A cache_control mark on a system, developer, user or tool message sent to /v1/chat/completions now reaches an OpenAI deployment that reads prompt_cache_breakpoint (gpt-5.6+ on api.openai.com, or one opted in through prompt_cache_options) as a prompt_cache_breakpoint on the same content part, instead of being stripped. Marks on assistant turns and tool definitions are still dropped, and deployments that do not read breakpoints keep today's behavior.

* test(openai): rewrite chat prompt_cache_breakpoint wire test as sent-vs-received literals

Each test now shows the request the caller sends and the exact body OpenAI receives as plain JSON, with one helper that returns the upstream body. Drops the per-test marker plumbing and the spend-log check, which did not test the cache mark translation.

* test(openai): cover a marked chat message with null content on the breakpoint path

A system, developer, user or tool message that carries cache_control but whose content is neither a string nor a list of parts is handed to the existing strip untouched instead of raising inside the transform. The new case pins that the request still goes out with the mark stripped and no breakpoint added.

* refactor(openai): reuse the cache control hook's breakpoint placement on chat completions

The chat transform had its own copy of how a message-level cache_control lands on a part for OpenAI's prompt_cache_breakpoint. It now calls one hook helper that the injection path also uses, so which roles and part types can carry a breakpoint is decided in one place. A message-level mark on list content now lands on the last part OpenAI accepts a breakpoint on, matching injection.

* test(e2e): cover cache_control reaching gpt-5.6 explicit prompt caching on chat completions

Four live cases through the proxy: a marked system part, a marked string system message, a marked user part and a marked tool result. Each must write the cache on the first call and read it on the second.

* fix(openai): parse content parts instead of a banned TypeGuard when renaming cache marks

The strict ruff gate bans typing.TypeGuard as unverified narrowing. The part rename now validates an accepted part with pydantic before handing it to the shared rename helper.
2026-10-13 15:45:49 -07:00
ryan-crabbe-berri
7bd108139f
fix(openai): carry cache_control to prompt_cache_breakpoint on /v1/responses to gpt-5.6 (#46105)
* fix(openai): carry cache_control to prompt_cache_breakpoint on /v1/responses to gpt-5.6

Native /v1/responses used to strip every Anthropic cache_control mark, so a
gpt-5.6 deployment in explicit mode never wrote a cache. Marks on system,
developer and user parts and on function_call_output now become
prompt_cache_breakpoint on the same block when the deployment reads
breakpoints, and are stripped exactly as before everywhere else

* fix(openai): read str providers and base_url in the prompt cache breakpoint gate

Responses configs generated from a provider JSON file, and openai_like
itself, name their provider as a plain str, so .value crashed every native
Responses request through them. The gate also only read api_base, while
responses() keeps a caller's base_url as an extra field on the params, so an
injected mark on a custom host was rewritten to a breakpoint

* test(openai): cover the pass-through branches of the prompt cache breakpoint swap

A part type OpenAI does not cache keeps no breakpoint and an item mark skips
it, and an item or content of a shape OpenAI will reject reaches the strip
untouched instead of failing inside the proxy

* fix(prompt_caching): keep a client-written prompt_cache_breakpoint when a cache_control mark lands on the same block
2026-10-13 15:23:07 -07:00
mrinal-berri
93b29c3977
fix(proxy): broadcast team cache invalidation on team writes (#46113)
* fix(proxy): broadcast team cache invalidation on team writes

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): cover cross-instance team cache invalidation

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-13 16:25:22 -04:00
devin-ai-integration[bot]
74e5dc9747
fix(proxy): keep provider refusal, logprobs and system_fingerprint in chat completions (#44788)
* fix(proxy): keep provider refusal, logprobs and system_fingerprint in chat completions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): handle missing chat completion roles

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(types): restore throwaway Usage skip in ModelResponse

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): handle explicit fields on custom response dumps

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: update refusal expectations for LIT-9235

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(lint): suppress type discipline violations with reasons

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style: ruff format proxy utils

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(types): keep basedpyright counts at merge-base for azure dump and ModelResponse

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(types): keep an explicit null system_fingerprint passed to ModelResponse

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-13 13:17:40 -07:00
Kerry Lu
d7b58126a7
feat(ui): source Home What's new from day 0 support blog posts (#46260)
* feat(ui): source Home What's new from day 0 support blog posts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(ui): Home cards read card_title and card_summary from blog tags with provider logo

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-13 13:11:59 -07:00
moyai-devin-berriai[bot]
0b3fcc197d
fix(logging): redact nested callback input and provider response fields (#45923)
* fix(logging): redact nested callback input and provider response fields

* fix(logging): keep shared request body for per-callback redaction and drop hidden provider fields

The deprecated per-callback message_logging=False path ran perform_redaction on
the shared model_call_details, which now also redacted the nested
proxy_server_request body. The spend log Request column and
previous_response_id replay read that body, so one opted-out callback blanked
them for everyone. That path now keeps the shared body and hidden params, and
redact_streaming_responses_for_custom_logger hands the opted-out callback its
own redacted litellm_params.

Redacted litellm_params also clear provider_specific_fields copied into
metadata.hidden_params (Anthropic thinking text), in litellm_params and in the
nested request body, and the nested body redacts system and instructions.

* fix(logging): keep replayed instructions and router usage marker under redaction

The shared request body keeps system and instructions again, so spend logs
store them and previous_response_id follow-ups replay the real instructions.
Per-callback copies still redact them.

Provider-specific fields in metadata.hidden_params are cleared in place, so
the metadata dict keeps its identity and the router's counted-usage marker
stays visible to deployment_callback_on_success (no double tpm/rpm count).

* fix(logging): never copy shared metadata dicts during redaction

The shared redaction pass now leaves litellm_params metadata dicts as they
are, so a hidden_params write from the other success handler between the
in-place clear and the copy can no longer move the router usage marker to a
stale dict. The key-presence predicate change is reverted as unneeded.

* refactor(logging): redact nested request body and provider fields with the existing message rules

Run perform_redaction on a copy of proxy_server_request.body, the same call the
spend log already makes, and redact reasoning inside provider_specific_fields the
way the message's own reasoning_content and thinking_blocks are redacted, instead
of clearing every provider field. Opaque fields such as Gemini thought_signatures
stay, and the metadata dicts and streaming objects are handled as on main.

---------

Co-authored-by: moyai-devin-berriai[bot] <336287033+moyai-devin-berriai[bot]@users.noreply.github.com>
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
2026-10-13 12:25:56 -07:00