* fix(anthropic): put a tool_result cache breakpoint on its hoisted images on the Responses bridge
A marked tool_result whose images get moved into a trailing user message now carries
the breakpoint on the last moved image instead of the function_call_output, so the
prefix OpenAI caches includes the images
* test(anthropic): cover a marked tool_result with two screenshots on the Responses bridge
* fix(anthropic): keep every tool result screenshot inside the breakpoint that covers it on the Responses bridge
A marked tool_result's screenshots now go right after its output, so earlier unmarked screenshots land inside the breakpoint and later tool outputs stay after it. A cache_control mark on a block inside a tool_result is read too.
* fix(anthropic): cover only the screenshots up to a mark inside a tool result, and drop raw breakpoints under drop_params
A cache mark on a text block inside a tool result now covers that result only up to the marked block, the way
Anthropic caches it, so screenshots after the marked text stay outside the breakpoint. A mark on the tool result
itself still covers every screenshot in it
A deployment that reads no prompt cache breakpoints now drops raw prompt_cache_breakpoint marks, top-level and
inside tool results, when drop_params is on, and still forwards them as sent without it
* fix(anthropic): keep raw breakpoints under drop_params for models that take them
* docs(anthropic): say a mark inside a tool result covers its whole text and documents
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
Non-admin callers previously received deployment-wide cache activity,
error breakdowns, every key alias on the proxy, and could filter the
aggregates by any other tenant's key alias. All four queries now take
the caller's read scope as bind parameters, which also shrinks the
key-alias and model filter options to the caller's own visibility.
The three-way scope resolution (admin / internal user / fallback) now
lives in resolve_spend_log_read_scope, shared with /spend/tags and
/spend/logs/session/ui instead of being duplicated per endpoint.
* feat(claude-code): emit installationPreference in marketplace.json entries
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(claude-code): satisfy marketplace CI checks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(claude-code): qualify marketplace entry as read only
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(claude-code): suppress typed marketplace assembly warning
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(claude-code): restrict installation_preference to the documented values
Accept only available, auto_install, or required so a typo returns 422 instead of being silently dropped, and cover the PUT full-replace clearing behavior with a test
* fix(claude-code): echo installation_preference on the public skill hub
* refactor(claude-code): read installation_preference with one membership check and cover hand-edited rows
* test(integration): audit installation_preference on the Claude Code marketplace
* test(integration): kill a serving worker in the marketplace chaos cell
* test(integration): give the marketplace worker-kill cell the two-boot timeout
* fix(proxy): keep installationPreference on the MarketplacePluginEntry model
* fix(proxy): refuse self-installing preferences off sha256 archives, accept the camelCase key, and echo the preference on POST and PUT
* fix(claude-code): serve installation_preference only on archive plugins and refuse both key spellings
Claude Code 2.1.75 and older reject a marketplace.json once any plugin carries installationPreference,
and every Claude Code release before archive support rejects a catalog with an archive plugin. Requiring
an archive source for every value, available included, keeps the field off catalogs those clients can read.
A request that sends installation_preference and installationPreference together now gets a 422.
* fix(claude-code): read the preference spelling that carries a value when the other is null
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(integration): cover keys_auth endpoint contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover hashed, path and token spellings for keys_auth contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): strengthen JWT mapping lifecycle coverage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover reset_spend over plaintext and hashed path spellings with reset_to 0 and 0.02
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): drop Final in reset loop and use drain-then-fresh-refusal in warmed-peer delete proof
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): disable model info refresh in JWT lifecycle rig
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): drop unused discovery constant
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover cache invalidation and key paths
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo <mateo@berri.ai>
* test(integration): cover assistants_evals endpoint contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(tests): replace exception assertions flagged by ruff
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(openai): type assistant run options
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(unit): cover assistant and eval contract paths
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(openai): keep assistant run options unvalidated
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): pin assistants and evals wire contracts as BUG skips
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover every eval route, full responses, image parts and full SDK stream payloads
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): relay upstream eval run model, send reasoning_effort, cover SDK root alias, done frame and client disconnect
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): name reasoning_effort in the run BUG skip
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): use a non-null marker thread and name the eval run model restamp in its BUG skip
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): mark disconnect cases with a messages.create the proxy relays on main
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(assistants_evals): cover thread message shapes, raw run options, header leaks and /v1 eval bases
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): expect the OpenAI /cancel path for eval run cancel
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover mcp_runtime endpoint contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(integration): ruff format and import order in mcp runtime tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): assert out-of-path calls and untrusted forwarded headers are refused
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): move the untrusted forwarded-header discovery case into its own test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): hash the token row lookup, widen spend polling, and union open servers after listing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): tighten mcp runtime contracts per review (sdk client, toolset scope, forwarded trust, callback html, session tools/call)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): assert the sdk unknown-path error without pinning the sdk's exception wrappers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover trailing-path and group csv mcp paths, rfc 9207 iss-less and error-iss callbacks, and id_token exchange
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover lite logout spelling, team-mapped and unmapped IdP subjects
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover per-server mcp-session resource and root /token spelling
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): unpack the sealed client code in the step-up tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover config_utils endpoint contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): prove config_utils contracts on the peer, Vault AppRole and cert logins, and admin-view checks
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): assert no Vault traffic after config override delete
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): prove the Vault cert login presents the client certificate over TLS
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover the blank plugin_key dashboard edit and the env-only CyberArk readback
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(tests): ruff-sort imports and reformat config_utils integration tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): check plaintext Vault traffic in the cert phase with one drain
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): probe CyberArk runtime state after DELETE with a secret-resolving chat
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): barrier the Langfuse absence check on the shared export channel
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): send the dashboard's blanked CyberArk field and check row, readback and runtime env
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover responses endpoint contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): tighten responses contract tests per review
Isolate team-only response-id ownership with a same-user different-team key,
send every denial through the OpenAI SDK on all three aliases, make the
previous_response_id affinity check deterministic, assert the whole cancel
response, prove the Cursor stream text, -fast suffixes and include_obfuscation
strip, add SDK compact cases, restore router pre-call checks after the
affinity test, and inline the root-level responses_test_support helpers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): wait for every proxy worker before routing to new deployments
Restore the cross-worker convergence wait dropped when the shared helper was
inlined, so the affinity and retrieve-routing tests cannot pass before the
alternative deployment is loaded on a multi-worker gateway
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover Responses ownership and wire contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): address responses contract review
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): expect the provider model in the conversation passthrough body
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): expand responses endpoint contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): restore streamed response retrieval
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): wait for proxy workers after deployment setup
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): track readiness per proxy worker
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover files_batches endpoint contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): split raw cloud-storage delete test by caller
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): drop mutable polling state in content download test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): dispatch request spellings without rebinding
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): import assert_never from typing_extensions for python 3.10
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): tighten files_batches wire and access assertions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): tidy files_batches access tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover repeated target models, upload expiry, batch aliases
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): prove files streaming path, SDK model denial and admin GCS delete
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): give the streaming head the wire barrier's full budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: strengthen files and batches wire coverage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: drop duplicate parametrize from the model-routed upload helper
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: pass the SDK base url in the skipped repeated-plain upload test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): allow batch output metadata lookup
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: avoid draining file content requests before wait
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover teams endpoint contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): address review on teams contract tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): drain shared upstream observations at fixture setup
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): prove team block, member and bulk permission contracts at runtime
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): assert email admin adds and whole membership rows for team members
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): drive key update grant and revoke, Z-suffixed temp budget expiry, link LIT-9257
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): close review gaps in teams endpoint contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): harden team member endpoint contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): let team admins create keys for team members behind an opt-in setting
A team admin calling POST /key/generate with a team they administer and the
user_id of another member of that team got a 403, and the dashboard only
offered the "Another User" owner option to proxy admins
Add a create_member_keys entry to team_admin_editable_team_fields. When a
proxy admin turns it on, a team admin can create a team key owned by another
member of a team they administer, through the API and through the Create Key
form. It stays off by default, so nothing changes until a proxy admin opts in
With the setting off, the 403 a team admin gets for this call now names the
setting to enable instead of saying they can only create keys for themselves
* fix(ui): order the member owner picker after the team and drop the viewer from it
For a team admin the owner picker depends on the team, so it now sits under the Team field, lists only the other members of that team, and the Team field says why it is required
* fix(proxy): keep keys a team admin creates for a member from carrying more than LLM access
A key authenticates as its owner, so with create_member_keys on a team admin could create a key owned by a proxy admin who sits in their team and act as a proxy admin with it
The member now has to hold the internal_user or internal_user_viewer role, and the key is always an llm_api key, so the team admin never holds a secret that reaches management routes. Asking for a management or read_only key for a member is refused
The unit tests now check the permission decision directly instead of replacing generate_key_helper_fn, and the saved key is checked against a real proxy in the team admin gate matrix
* style(proxy): sort the team admin field permission imports
* refactor(proxy): force the member key type on the request instead of merging a second dict
* test(proxy): drive member key creation through the key generation helper
The pure helpers were covered but the wiring in _common_key_generation_helper was not. Dropping the owner role lookup, the refusal call, or the user_id handed to common_key_access_checks left every unit test green
* fix(proxy): skip the owner role lookup when create_member_keys is off
The owner role was read before the verdict checked the setting, so with the setting off a failing user table read surfaced as a 500 instead of the 403 a team admin got before, and every such request paid an extra writer read
---------
Co-authored-by: Yuneng Jiang <yuneng@berri.ai>
* test(integration): cover guardrails_policies endpoint contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): let the admin submissions list carry other rows
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): close guardrails policies review gaps
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover guardrail auth and policy attachments
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): tidy guardrail policy test helpers
* test(integration): keep plain non-admin submission refusal live
* test: expand policy attachment integration coverage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): accept the intended 403 for plain submission keys
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): pin multi-tag header and spend row tags for policy tag scope
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): drop the duplicate pydantic import left by the main merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Devin <noreply@devin.ai>
* test(integration): cover vector_stores endpoint contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): strengthen vector store route contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): update vector store route contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): drop owner-update BUG skip for admin-only vector store routes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(vector_stores): cover the managed-file fallback, owner delete and per-request retries
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): probe vector store worker readiness by deployment id
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): pin vector store retry header and create routings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(integration): ruff format vector store managed file tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover mcp_management endpoint contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): require server_not_found for pending and rejected submissions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): pin per-worker submission reloads, toolset and alias-collision spellings, raw key overrides
Assert the approving and rejecting worker immediately and only wait on the peer, add an outsider key and exact upstream calls, list by toolset_name, cover a config alias shadowing a DB server_name, send key_hash as a raw sk- key, and add BUG skips for API disallowed_tools and overrides on keys or teams without object_permission
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): pin XFF trusted-peer, load balancer chains and sibling refusal body
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): enforce output_policy trust chain, JSON-RPC XFF views, MCP tool-policy BUG pins
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(mcp): force a registry reload during the pending window, pin the admin default listing and the whole tools/call body
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover realtime endpoint contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): reject duplicate multipart parts and cover the transcription_sessions alias
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): drop the type ignore and check the aliased transcription response
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): pin realtime multipart part types, replayed transcription session, undeclared session fields and budget refusal path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): move the refusal path note next to the expected message
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): assert realtime refusal status before parsing and pin the replayed transcription session whole
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): sort the realtime webrtc test imports
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): pick the transcription mint and token with one Final each
* test(integration): pin realtime transcription mint, replay, and redeem contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(integration): ruff format the realtime webrtc test
* test(integration): drop the redundant comment on the realtime SDK calls test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): type the realtime client secret refusals table
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(realtime): tighten endpoint contract assertions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover videos_containers endpoint contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): end the first container page on an ownerless id
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): assert container upload parts, SDK video extend, bare video ids, cost from map
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): catch a stale container list cache, cover video edit JSON, bracketed video refs and object input_reference
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): pin container optional fields, video client extras and the /videos alias on the wire
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): pin multipart character id decoding on video create as a BUG skip
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover chat_completions endpoint contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): pin per-call prompts and both n=2 chat choices
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover mixed path-model auth, string guardrail rejection and typed text-completion forms
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): drop the prompt-spellings cache workaround and cover Azure SDK text completions on the deployments alias
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover audio endpoint contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): close audio SDK clients after each test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): avoid list mutation in audio tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): keep exact upstream call check in model resolution test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover audio translations, bracket spellings, streaming and scoped alias keys
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): pin audio metadata parse sites, exact cost and speech bug pins
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): drop inline comment on audio wire assertion
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): strengthen audio wire and authorization contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover agents_a2a endpoint contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): pin agent daily spend to the logged request day and assert cleanups
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): expect agent daily activity newest day first
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover A2A 1.0 task aliases, agent id headers, declared access groups and per-token spend
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): assert full A2A stream, header, alias, task and spend contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): assert forwarded a2a-version header on A2A sends
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): release A2A stream gate only after a full SSE frame
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): pin A2A trace id, stream headers, denied alias and task methods
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): pin A2A card protocol version
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): assert the whole accepted push config response
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover model_management endpoint contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): pin the exact model access refusal after a rename
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): wait for allowlist convergence in the rename test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): give the rename test lint suppression its reason
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): close reviewer gaps on model management contracts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): share the blocked serving assertions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): strengthen authorization and configuration regression tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): converge spend readbacks, ignore model discovery, cover slash credential names
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): pin the dashboard credential edit body round-trip
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(telemetry): count Admin UI page views and tab clicks through the proxy
* fix(ui): share one telemetry settings save across both tabs so a second toggle waits for the first
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(telemetry): let every dashboard role reach the UI event routes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): fail with the denial detail when a dashboard role cannot reach UI event routes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(telemetry): document POST /telemetry/ui_events with a description and a 200 response schema
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): keep the test-quality reason on the line pytest's checker reads
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): pick the telemetry destination on the Telemetry page and warn when none is chosen
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): send the destination along with groups in the telemetry settings hook test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): add a top-level telemetry switch and gate turning telemetry on behind an enterprise modal
The page can only turn telemetry off. Turning it on from the switch, a group
or a first destination opens a modal: enterprise proxies are told to contact
their LiteLLM enterprise representative, others that telemetry is an
enterprise-only feature. Group switches and the destination stay visible but
disabled while telemetry is off.
---------
Co-authored-by: nate <nate@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(telemetry): keep reports in a local table with a persistent instance id and an admin export route
* feat(telemetry): key deployment hashes from a random secret stored in the proxy DB
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(telemetry): retry a locally stored window under the same id so it is never counted twice
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* docs(telemetry): state the overlap and dedupe contract for paging stored reports
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): regenerate dashboard API types for the telemetry reports paging note
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(telemetry): require an explicit LITELLM_TELEMETRY_DESTINATION or saved destination before recording
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): regenerate dashboard API types for the explicit telemetry destination
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(telemetry): register the attempt logger and create its hash secret only once a sink is built
A proxy with a database that never opts in now carries no telemetry callback
and writes no deployment hash secret to LiteLLM_Config.
---------
Co-authored-by: nate <nate@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(telemetry): record proxy requests and provider attempts into the telemetry sink
* fix(telemetry): validate env settings, fix the window and only record the request's own provider calls
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(telemetry): bind the call id from a typed local
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(telemetry): record a finished request into the sink current when it finishes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(telemetry): record every failed provider attempt, including retries and fallbacks on a shared logging object
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(telemetry): report timeouts and connection failures as their own provider status classes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(telemetry): record failed attempts from the per-deployment failure hook instead of a new core logging hook
Reverts the async_log_attempt_failure_event hook added to CustomLogger and
litellm_logging in 3421b38685. The router already calls
async_post_call_failure_deployment_hook once per failed attempt, retries and
fallbacks included, so every failed fallback still gets its own row without
touching the shared logging object.
* fix(telemetry): register the attempt logger only once a telemetry group is turned on
A proxy that never opts in now carries no telemetry callback. Once registered
the logger stays, returning early while every group is off.
---------
Co-authored-by: nate <nate@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(telemetry): add AggregatingSink, fixed-bucket histograms, Report and HttpExporter
* fix(telemetry): add records to rows in place, guard concurrent flushes, cap restored rows and count bad histogram values
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(telemetry): give the shared row helper its lint reasons
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(telemetry): retry the same statuses as the proxy's should_retry (408, 409, 429, 5xx)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(telemetry): cover populated block_types and header_keys in report JSON
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: nate <nate@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(prometheus): skip custom metadata labels a metric already carries
A custom metadata label whose sanitized name matches one of a metric's
built-in labels was appended a second time, so every .labels() call on
that metric raised ValueError. With custom_prometheus_metadata_labels:
[project_alias], the per-project per-model rate limit gauges raised on
every request from a project key with per-model limits, and the rest of
the success event (latency, deployment success, total requests) was
skipped.
* test(prometheus): cover shared custom labels on every endpoint, failure, metadata and outage burst
* test(prometheus): drain the shared upstream at the start of each integration test
* test(prometheus): assert the forwarded route, model and stream per endpoint cell
---------
Co-authored-by: yucheng-berri <293926326+yucheng-berri@users.noreply.github.com>
* feat(telemetry): add litellm.telemetry records, consent groups and sink protocol
* docs(telemetry): reword TelemetryGroup docstring and drop module docstring
* fix(telemetry): add timeout and connection_error status classes, check relative imports in the package boundary test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: nate <nate@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(vertex_ai): unwrap tool-emulated json_schema envelopes and send output_config.format for Claude
Vertex AI Claude deployments that emulate response_format through a forced
tool sometimes nest the schema object under an extra key such as
`parameters` or `input`, and that envelope reached message.content with a
200. The converter now unwraps exactly one nested object that matches the
schema, leaving a payload that is itself schema-shaped alone.
Native structured-output rows sent the deprecated top-level `output_format`,
which Vertex rejects with a 400. The Vertex request now carries it as
`output_config.format`, with an explicit caller value winning.
* fix(anthropic): unwrap tool-emulated json_schema envelopes on streams and tool-name wrappers
* test(anthropic): expect the json tool description through a shared constant
* fix(anthropic): drop the json tool description that makes sonnet-5 omit required keys
* fix(anthropic): keep schema-accepted answers whole and decide held streams at the second key
An answer the json_schema accepts is returned as is before any envelope rule runs, the tool-name
wrapper is unwrapped only when the schema rejects it, a malformed schema turns the unwrap off, a
streamed answer whose first key is a schema property named name or tool_name waits for its second
key before streaming, the held fragments go through a StringIO, and a prefix that stays undecided
past 4 KB is held to the end
* test(anthropic): keep the stream family cells to providers whose stream iterator gets the schema
* fix(anthropic): unwrap json tool envelopes by structure instead of schema validation
The unwrap now reads only the schema's top-level property names. A lone
undeclared key holding an object, alone or beside a name equal to the json
tool's own name, is the envelope, and the unwrap repeats while the inner
object shares no key with the schema. jsonschema and referencing are no
longer imported on the request path. The stream holds only an answer that
opens with the tool's name or with an undeclared key holding an object, and a
schema without top-level properties streams as it arrives.
* fix(anthropic): walk json tool envelopes iteratively
* fix(anthropic): keep json tool arguments too deep to decode instead of raising
* fix(anthropic): unwrap json tool envelopes beside undeclared string labels
* fix(anthropic): keep json answers whole on schemas that allow extra keys and stream held text as sent
A schema that allows undeclared keys and requires none now keeps an undeclared wrapper key on both the stream and non-stream paths. When no envelope is found, the held stream text is now sent exactly as the model wrote it. On Vertex /v1/messages, output_format now wins over output_config.format when both are set, which matches the Anthropic adapter.
* test(anthropic): cover label envelopes, dynamic-key schemas, as-written held streams and output_format precedence
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(cost_calculator): price an explicit base_model from its own row when the deployment provider row has no rates
A chatgpt/ deployment with model_info.base_model recorded zero spend because the
base model was looked up under the chatgpt provider, which resolves to the
deliberately unpriced chatgpt/ row. When the deployment provider's row for the
base model declares no rates, price it under the provider of the base model's
own row instead. Deployments whose provider row declares rates are unchanged.
(cherry picked from commit 42d9a8693e)
* test(integration): bill a chatgpt base_model deployment from the priced row on a live proxy
Co-authored-by: sla55er <stiliyanmarkov@gmail.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: sla55er <stiliyanmarkov@gmail.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(openai): carry cache_control to prompt_cache_breakpoint on chat completions to gpt-5.6
A cache_control mark on a system, developer, user or tool message sent to /v1/chat/completions now reaches an OpenAI deployment that reads prompt_cache_breakpoint (gpt-5.6+ on api.openai.com, or one opted in through prompt_cache_options) as a prompt_cache_breakpoint on the same content part, instead of being stripped. Marks on assistant turns and tool definitions are still dropped, and deployments that do not read breakpoints keep today's behavior.
* test(openai): rewrite chat prompt_cache_breakpoint wire test as sent-vs-received literals
Each test now shows the request the caller sends and the exact body OpenAI receives as plain JSON, with one helper that returns the upstream body. Drops the per-test marker plumbing and the spend-log check, which did not test the cache mark translation.
* test(openai): cover a marked chat message with null content on the breakpoint path
A system, developer, user or tool message that carries cache_control but whose content is neither a string nor a list of parts is handed to the existing strip untouched instead of raising inside the transform. The new case pins that the request still goes out with the mark stripped and no breakpoint added.
* refactor(openai): reuse the cache control hook's breakpoint placement on chat completions
The chat transform had its own copy of how a message-level cache_control lands on a part for OpenAI's prompt_cache_breakpoint. It now calls one hook helper that the injection path also uses, so which roles and part types can carry a breakpoint is decided in one place. A message-level mark on list content now lands on the last part OpenAI accepts a breakpoint on, matching injection.
* test(e2e): cover cache_control reaching gpt-5.6 explicit prompt caching on chat completions
Four live cases through the proxy: a marked system part, a marked string system message, a marked user part and a marked tool result. Each must write the cache on the first call and read it on the second.
* fix(openai): parse content parts instead of a banned TypeGuard when renaming cache marks
The strict ruff gate bans typing.TypeGuard as unverified narrowing. The part rename now validates an accepted part with pydantic before handing it to the shared rename helper.
* chore(typing): validate provider payloads at the boundary to clear 464 basedpyright Any errors
Replaces untyped reads of decoded JSON in 48 provider, guardrail and proxy
modules with one pydantic validation at the point the payload enters the
module, so every value read afterwards carries a real type.
reportAny 6111 -> 5663 and reportExplicitAny 1416 -> 1400. No basedpyright,
ruff-strict or type-discipline rule is higher repo-wide or in any single
file, and no cast, ignore or suppression marker was added.
* chore(typing): replace 47 Any annotations that every caller already satisfies with object
Each annotation here is a parameter or return type whose value is only
stored, compared, logged or forwarded, so object describes it exactly and
no function body changes. A site was kept only if no basedpyright,
ruff-strict or type-discipline rule rose in any file that references it.
reportExplicitAny 1400 -> 1353 and reportAny 5663 -> 5655.
* chore(typing): type upstream responses and loose annotations across providers, proxy and integrations
Second pass of the Any reduction. Decoded upstream JSON is validated once at
the provider boundary and read through typed views afterwards, annotations
that every caller already satisfies move from Any to a concrete type, and the
nine search transformations validate each result item on its own so error
text no longer carries the item index.
No cast, no type: ignore, no suppression marker and no budget file edit
* ci(unit): run the brave, google_pse, nlp_cloud and opensandbox tests in the providers shard
* fix(openai): carry cache_control to prompt_cache_breakpoint on /v1/responses to gpt-5.6
Native /v1/responses used to strip every Anthropic cache_control mark, so a
gpt-5.6 deployment in explicit mode never wrote a cache. Marks on system,
developer and user parts and on function_call_output now become
prompt_cache_breakpoint on the same block when the deployment reads
breakpoints, and are stripped exactly as before everywhere else
* fix(openai): read str providers and base_url in the prompt cache breakpoint gate
Responses configs generated from a provider JSON file, and openai_like
itself, name their provider as a plain str, so .value crashed every native
Responses request through them. The gate also only read api_base, while
responses() keeps a caller's base_url as an extra field on the params, so an
injected mark on a custom host was rewritten to a breakpoint
* test(openai): cover the pass-through branches of the prompt cache breakpoint swap
A part type OpenAI does not cache keeps no breakpoint and an item mark skips
it, and an item or content of a shape OpenAI will reject reaches the strip
untouched instead of failing inside the proxy
* fix(prompt_caching): keep a client-written prompt_cache_breakpoint when a cache_control mark lands on the same block
A JSON-RPC tools/call on /{server_name}/mcp (or /mcp with x-mcp-servers naming one server) that uses the upstream's bare tool name answered "Tool not found" on every proxy process that had not yet listed that server, because the per-process tool-name map stays empty for a server whose upstream only lists tools with the caller's own credentials, and mcp SDK 2 no longer lists before an uncached call. call_mcp_tool now hands the single scoped server to execute_mcp_tool as requested_server_id, so the existing REST contract lists the server with the caller's headers and routes the bare name to it, and a bare name shared by two servers goes to the one the path names
Co-authored-by: aayushbaluni <73417844+aayushbaluni@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): broadcast team cache invalidation on team writes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): cover cross-instance team cache invalidation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
