* test(integration): drop the contracts.json manifest and the covers requirement
Groups live as a GROUPS literal in run.py, the browser expectations move next to the
browser tests, and the runner fails only on pytest failure, collection errors or a
selected file that collects zero tests. The covers marker stays registered for the
existing tests but is no longer checked. The mcp directory gets its own group
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci(integration): run mcp as its own shard with xdist and a peer proxy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci(integration): INTEGRATION_COVERAGE=1 runs the proxy under coverage for the MCP modules
The mcp shard sets it. The proxy and its peer start under coverage run in parallel mode,
get SIGTERM after the tests so coverage flushes, and the combined text and HTML reports
land in the suite results that CircleCI already stores as artifacts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci(integration): let the test proxy flush coverage when uvicorn re-raises SIGTERM
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): add SSE, stdio, scripted, OpenAPI and OAuth 2.1 MCP peer doubles
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): add MCP transport and access-control matrices
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): add MCP credential and OAuth flow coverage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): add MCP LLM endpoint, accounting, guardrail, resilience and lifecycle coverage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): stop the same-URL grant test from counting a late initialize as a leaked call and satisfy the test-tree lint
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): assert the REST denied-server listing is refused or empty
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): pin the REST denied-server listing to 403 access_denied
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
PATCH /model/{id}/update rejected read-modify-write edits that resent an unchanged but dangling litellm_credential_name. Existence validation now runs only when the requested name differs from the stored one; empty string and non-admin detach rejections are unchanged
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): revoke UI session tokens on logout and password change
Adds POST /session/logout to revoke the presented UI session key server
side (previously logout was client-side only and the key stayed valid
until expiry). Password changes now revoke the user's other UI sessions:
self-change keeps the caller's session, admin reset and onboarding claim
revoke all. The BYOK OAuth cookie auth now re-resolves the embedded key
against the DB so revoked sessions get a 401.
* fix(proxy): satisfy B008 budget and backend allowlist for /session/logout
* refactor(proxy): satisfy type-discipline budget in session_endpoints
* feat(otel): emit gen_ai.conversation.id from the caller's session id on v2 LLM spans
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): keep the caller's header session under missing_session_id: generate and read replayed payload session ids
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): drop only the proxy-minted session id so a caller id on the other metadata key survives
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): keep a replayed session id hidden when it only echoes the payload trace id
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): keep a replayed session id even when the payload trace id fell back to it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): stop reading the replayed payload's session id, the generated marker does not survive replay
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): audit gen_ai.conversation.id on otel v2 spans through a real proxy, sink and postgres
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): keep otel conversation rigs alive for the whole session so shuffled shards do not reboot the proxy per test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(otel): stop the audit rig proxies from probing sibling test peers for model info
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(otel): record accepted OTLP batches in the sink instead of mutating the collector
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(otel): guard the accepted batch deque so snapshots cannot race sink appends
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mrinal <mrinal@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng <yucheng@berri.ai>
* fix(proxy): bound auth cache invalidation publish so a wedged coordination Redis cannot stall user updates
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): resolve publish callable at call time in evict_and_broadcast
The keyword-only default bound publish_auth_cache_invalidation at
function-definition time, so tests patching the module attribute observed
zero calls. Default to None, resolve the real publisher inside the body,
and keep the keyword-shaped cache_key call the existing contract asserts
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): publish auth cache invalidations in the background so a wedged coordination Redis costs handlers nothing
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): cap in-flight auth cache invalidation publishes so a wedge cannot drain the redis pool
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(policy_engine): keep inherited parent guardrails when a child policy condition misses
Attachment applicability now walks the policy inheritance chain, so an attached child whose own condition does not match still contributes the guardrails of its unconditional ancestors, and a non-default attachment that applies through an ancestor still suppresses default attachments. The resolver continues to skip only the chain members whose own condition fails
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(policy_engine): skip a policy's pipeline when its own condition misses
resolve_pipelines_for_context returned the pipeline of a matched policy without evaluating its own condition, so a condition-missing child admitted by the chain-aware matcher still ran its pipeline. It now mirrors resolve_policy_guardrails and drops the pipeline when the policy's own condition does not match.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(policy_engine): property test that chain matching only widens to applicable ancestors
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(policy_engine): log policies admitted only through an inherited ancestor
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(policy_engine): log ancestor admissions once per attachment scan
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): saving echoed model_info never persists cost map pricing as a deployment override (Pylon #6870)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): budget_duration change on /budget/update recomputes budget_reset_at (Pylon #6913)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): count_tokens on a budgeted key reserves no budget and a later completion still succeeds (Pylon #6966)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): /cost/estimate reports configured prices for a deployment absent from the cost map (Pylon #7014)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cache the team member default budget in Redis as JSON (Pylon #7180)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): aggregated team daily activity reports whole-range team spend in one page (Pylon #7224)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): failed daily user rollup commits are retried so spend report and daily activity agree (Pylon #7268)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): CLI session token without org_id is charged to and capped by the team organization budget (Pylon #7291)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): gemini passthrough success releases its budget reservation from the spend counter (Pylon #7295)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): batch retrieval spend row sums reasoning tokens and counts output and error file failures (Pylon #7341)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): uncostable batches retire from the cost poll page so newer batches are costed (Pylon #7342)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): charge a team member added without any budget on its membership row (Pylon #7363)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): failed dispatched requests keep estimated input tokens in spend logs (Pylon #7519)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock passthrough converse guardrail ignores tool definitions (Pylon #7524)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): explicit null budget_duration on /team/new is not replaced by default_team_params (Pylon #7536)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): PATCH /organization/update with a null limit clears it (Pylon #7577)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): ultrafast service_tier bills ultrafast rates without leaking pricing fields upstream (Pylon #7587)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): keep the selected model in the response and spend log for an Azure Model Router alias (Pylon #7636)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): disconnected Bedrock /v1/messages stream still bills terminal usage (Pylon #7685)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): databricks cached prompt tokens bill at cache rates (Pylon #7738)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): completed batch spend row records reasoning tokens and error file failures (Pylon #7928)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bill OCR annotation pages at annotation_cost_per_page (Pylon #7958)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): in-flight count tokens request reserves no key budget so a completion still reaches the provider (Pylon #7307)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): fail-closed key rejects known estimate over remaining budget before provider (Pylon #7691)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): streamed /v1/responses success callbacks keep provider response headers (Pylon #7775)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Revert "test(integration): fail-closed key rejects known estimate over remaining budget before provider (Pylon #7691)"
This reverts commit 910348be7a.
* test(integration): reconcile contracts manifest for bundled regression tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): isolate proxy config writes in bundled regression tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): address review feedback on budget reset bounds and callback batch accumulation
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): avoid rebinding the cache identity accumulator
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): assert forwarded messages per cache identity call
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): make budget reset and team default tests deterministic
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(vector_stores): keep config-defined vector stores listed and read-only
Vector stores declared in config.yaml were purged from the in-memory registry by /vector_store/list because the database was treated as the only source of truth. Config-defined stores now carry is_config=True, stay in the list beside database rows, are never overwritten or evicted by database state, and reject /vector_store/new, /vector_store/update and /vector_store/delete with 400. The Admin UI renders them read-only
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show vector store source and read-only state for config-defined stores
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): audit config-owned vector stores across list, writes, search, authz, peers and redis outage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): show a visible read-only hint in the config vector store actions menu
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): Bedrock batch files upload completions and responses records as user messages (Pylon #6882)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): client Anthropic OAuth token never replaces Bedrock SigV4 authorization (Pylon #6888)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bridge /v1/messages and /v1/responses streams through empty-choices chunks (Pylon #6992)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): prepend azure content-filter metadata chunk to the messages stream (Pylon #6992)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): fireworks routers/ slug reaches the provider as accounts/fireworks/routers/<id> (Pylon #7030)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock hidden thinking tokens are not reported as text tokens (Pylon #7067)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): azure_ai FLUX.2-flex image generation targets the flex provider path with the BFL body (Pylon #7092)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): keep Databricks streaming usage and cache reads in the client stream and spend log (Pylon #7094)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): openai-compatible image edits forward provider-specific form fields to the backend (Pylon #7122)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): replayed intercepted web search turn reaches Bedrock as text through /v1/messages (Pylon #7181)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): streamed web search turn capped by max_agentic_loops ends the turn with snippets and ordered blocks (Pylon #7230)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock rerank keeps forwarded client headers out of the SigV4 signature (Pylon #7284)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): azure_ai rerank authenticates with an Entra token when no api key is set (Pylon #7303)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): perplexity stream with cost breakdown object completes and bills total_cost (Pylon #7331)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): azure_ai strips Anthropic message fields before the Foundry request (Pylon #7336)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): capped intercepted web search ends the turn without an internal tool_use block (Pylon #7378)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): Bedrock passthrough converse-stream keeps event-stream content-type (Pylon #7482)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): v1/messages success exposes v3 priority rate limit headers (Pylon #7532)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): config deployment dropped by a stale boot cost map is restored after reload (Pylon #7564)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): Bedrock Mantle project id reaches the provider as anthropic-workspace-id (Pylon #7583)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): prefixed Opus 4.8 reasoning_effort reaches Bedrock as adaptive thinking (Pylon #7586)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): dashscope chat forwards reasoning_effort to the provider (Pylon #7606)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): failing stream logging callback still releases the max_parallel_requests slot (Pylon #7608)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): gen 5 Claude Bedrock Invoke tool search sends the Bedrock beta field (Pylon #7642)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): databricks ai gateway api_base requests OAuth token from workspace origin (Pylon #7724)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): deepseek vision image content list reaches the provider unchanged (Pylon #7729)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): Bedrock Mantle context overflow surfaces as 400 prompt is too long (Pylon #7732)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): Codex history items reach Bedrock Mantle as supported Responses input types (Pylon #7783)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): chat over responses deployment returns finish_reason length when output tokens run out (Pylon #7784)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock_mantle rewrites Codex history items before the Responses wire (Pylon #7812)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): advisor sub-call on /v1/messages uses the configured advisor deployment (Pylon #7828)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): tencent thinking reaches the provider body instead of failing the request (Pylon #7834)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): xAI chat web search reaches /v1/responses with instructions and nested filters (Pylon #7835)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): send Bedrock Converse config blocks once at top level (Pylon #7839)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock converse sends gpt-5 reasoning_effort as reasoning.effort (Pylon #7850)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock cohere.embed-english-v3 embeddings accept encoding_format and dimensions (Pylon #7963)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): streamed chat completions emit SSE keepalive pings while the upstream is silent before its first token (Pylon #7987)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(integration): format the TTFT keepalive regression test (Pylon #7987)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): openai chat drops tool_choice when the request has no tools (Pylon #8022)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): stream whose first chunk has no choices falls back and bills the fallback (Pylon #8006)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Revert "test(integration): Bedrock Mantle project id reaches the provider as anthropic-workspace-id (Pylon #7583)"
This reverts commit 864b65811f.
* test(integration): reconcile contracts manifest for bundled regression tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): isolate proxy config writes in bundled regression tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): address review feedback on keepalive, cost map reload and websearch order
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(pricing): correct cached-token fields on realtime cost-map entries
azure/gpt-realtime-2 was the only member of the gpt-realtime-2 family priced
on one side of its cached-audio meter. Azure publishes that meter as
"gpt-realtime-2 Audio cd inp Gl 1M Tokens" at 0.4 per 1M and charges the
same rate for the write that populates the cache and the read that hits it,
so cache_creation_input_audio_token_cost lands at 4e-07, matching
azure/gpt-realtime-2.1, azure/gpt-realtime-2.1-mini and the openai
gpt-realtime-2 entry. No cost path reads that field yet, so this corrects
what get_model_info reports rather than what anything bills.
The gemini Live entries go the other way. Google's Vertex context-caching
page publishes separate supported-model lists for implicit and explicit
caching, and no Live or native-audio model is in either one. Its pricing
page prints N/A in both cached-input columns for every Gemini 2.5 Flash
Live API row, where plain 2.5 Flash and 2.5 Flash-Lite both carry real
cached prices, and the Vertex model card for the family marks context
caching not supported outright. Vertex never reports cachedContentTokenCount
on a Live session either, including for a byte-identical 7,021-token prefix
replayed across sessions minutes apart, which is well past the 2,048-token
minimum the same page sets for the Gemini 2 family.
So the 7.5e-08 on the two preview siblings priced something the provider does
not sell, and supports_prompt_caching on all three claimed a capability the
model does not have. The rate comes out. The flag is set to false rather than
removed, because get_model_info maps an absent key to None, and None is how
this map spells "nobody checked" across the 2,788 entries that omit it, where
false records the vendor's documented no. Both readers of the flag gate on
`is True`, so nothing bills or behaves differently either way.
Only the cached fields change on the two 09-2025 preview entries. Their
source field points at the Gemini API pricing page rather than the Vertex
one, so they describe a different surface with its own published limits, and
their context windows are left alone rather than assumed to match the Vertex
model card that drives the GA entry.
Tests cover all three halves: the family invariant that a cached audio read
implies an equal cached audio write, a cached count on a Live entry leaving
the bill at the fresh-input total instead of adding the old 7.5e-08, and
supports_prompt_caching answering false for all three entries while still
answering true for 2.5 Flash, so the false cannot be a swallowed lookup
error.
* fix(cost): correct gemini-live-2.5-flash-native-audio limits and capabilities
Google's model card for model ID gemini-live-2.5-flash-native-audio gives a
128K context window and 64K maximum output tokens, and marks structured
output, context caching and URL context as not supported. Its modality list
is text in and out, image in, audio in and out, and video in, with no
document input of any kind.
The entry advertised a 1M context window, an off-by-one 65535 output cap, and
three capability flags the vendor marks unsupported. Context caching is the
fourth and is handled in the cached-fields change alongside its two preview
siblings.
Both the bare id and vertex_ai/gemini-live-2.5-flash-native-audio resolve to
this single entry, so the test drives the corrected values through both.
* test(integration): cover live preview cached tokens billed at the fresh rate
Co-authored-by: Marty Sullivan <marty@martysullivan.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(cost): cite dated sources for Live entry pins and drop restating docstrings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Marty Sullivan <marty@martysullivan.com>
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(e2e): tolerate provider-side flakes on five full-suite cells
Mistral OCR retries a provider-relayed 429 with backoff, the Vertex vision
probe turns reasoning off so its 32 tokens go to the answer, the Vertex
cache cell spaces eight never-seen prefixes 15s apart around Google's
nondeterministic minimum-token rejection and prices the cached tokens
instead of prompt_tokens, and the Azure content-policy cell resends the
jailbreak prompt while Azure skips its filter
* test(e2e): shorten the new helper docstrings
* test(e2e): accept a relayed provider 429 on the rust OCR cells
The gateway already retries a provider 429 three times per call and the
Mistral key is shared across pipelines, so a throttle can hold across all
four attempts of the OCR cell. After the bounded retries the cell now
accepts the gateway's faithful relay of the provider's 429 (throttling_error,
code 429) as its second expected outcome; the gateway's own 429 and any
other error still fail the cell at once.
* test(e2e): drop the harness unit tests, the live cells cover the helpers
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(bedrock): send json_schema as a forced tool on Claude Opus 4.7 and 4.8 Converse
Bedrock rejects outputConfig.textFormat on Opus 4.7 and 4.8 with
"output_config.format: Extra inputs are not permitted", and the AWS
model cards list structured outputs as not supported for both, so
their cost-map entries no longer claim supports_native_structured_output
and json_schema requests fall back to the json_tool_call tool.
Fixes#27846
* test(bedrock): assert Opus 4.7 and 4.8 inline the schema on Invoke, move the native case to Sonnet 4.6
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(bedrock): stream /v1/messages Invoke bytes through instead of holding them in a 1024-byte chunker
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(bedrock): apply ruff format to invoke messages stream passthrough
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(bedrock): drop drive-by reformat of existing invoke messages tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(bedrock): collect streamed chunks into a tuple in passthrough regression test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(bedrock): give the passthrough regression test a 10s first-chunk budget
* test(bedrock): type the eventstream frame helper's payload as Mapping[str, object]
* test(bedrock): take the gated byte stream's chunks as an immutable Sequence
---------
Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* chore(e2e): move the compat-matrix populator from a GCE VM to a Render cron job
The daily Claude Code compatibility-matrix job ran as a systemd timer on
the litellm-compatibility-matrix-populator VM in the vertex-check GCP
project. Replace that with a Render Docker cron job built from a new
Dockerfile in tests/e2e/claude_code/cron_vm: pinned and checksummed
debian base, gh, uv, and Claude Code CLI, a non-root populator user, and
run_daily.sh as the entrypoint. run_daily.sh now clones a fresh blobless
checkout per run (Render cron disks are ephemeral), reads the publish PAT
from the github-token secret file under CREDENTIALS_DIRECTORY, and its
comments no longer describe systemd. The .service and .timer units are
gone; README.md and the env example describe the Render service, its
secret files, and the local docker build instead.
* docs(e2e): name the plan and trigger route Render's cron-job API accepts
Render answers a bare 404 for the legacy pro_max plan name on a cron job
(4c-16g is the same 4 CPU / 16 GB size) and the manual trigger route is
/v1/cron-jobs, not /v1/cronjobs.
* fix(e2e): install the published litellm wheel instead of building the tag from source
The tag builds a Rust extension through maturin, which needs a C and Rust
toolchain the cron image does not carry, so the first Render run failed at
uv sync with "linker cc not found". Sync the locked dependencies with
--no-install-project, install the PyPI wheel (what users run) with
--no-build, and pass --no-sync to every uv run so uv never puts the source
build back.
* fix(e2e): keep the SKIP_PUBLISH matrix where a Render run can read it
The validation run wrote the matrix into the image checkout, which nobody
can read once the container exits. Save it under HOME and print it at the
end of the log instead.
* fix(e2e): let the stale compat-matrix PR sweep see past the newest 100 docs PRs
The docs repo has a few hundred open PRs, so a 100-item list never
reached the week-old compat-matrix PR and the sweep left it open on
every run.
* docs(e2e): say the Render cron needs a manual deploy after each merge
Pushes never started a deploy during setup because Render only hears
about them through its GitHub app, which the org does not have, so the
README now carries the deploy command and the wait-for-live rule
* ci: build the compat-matrix cron image on pull requests
The CI coverage gate requires every Dockerfile to be built by a job, and
building this one on each PR that touches it also catches a broken pin
or checksum before Render does
* fix(e2e): shim the whole tests/e2e tree into the compat-matrix worktree
The five-file helper allowlist missed fixture_mode, which e2e_config now
imports, so the first Render run died at conftest load with
ModuleNotFoundError. Copy the image's whole tests/e2e tree instead and
keep pytest from loading the EKS-harness conftest with --confcutdir
* fix(e2e): scope the compat-matrix sweep to the publishing account's own PRs
The stale-PR sweep selected every open docs PR whose head branch starts
with compat-matrix/, so a contributor's fork PR under that name would
have been closed once a newer matrix PR existed. The sweep now resolves
the publishing login from the token and only closes same-repo PRs that
account opened
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix: repair seven regressions caught by CircleCI on main
- vertex_ai: stop treating fine-tuned endpoint ids (numeric or
vertex_ai/gemini/<id>) and gemma models as Gemini 3+, which injected
temperature=1.0 and Gemini 3 thinking config into their requests (#42465)
- cost: price Azure DALL-E 3 from its azure/<quality>/<size>/dall-e-3 rows;
it only worked through the OpenAI rows that #42435 removed
- bedrock: stream bedrock/invoke/moonshot through an OpenAI-shaped chunk
decoder; the generic decoder dropped every chunk, which the
supports_response_schema flag from #42338 un-skipped in CI
- proxy: keep the public model_group on pre-routing rejections so the Usage
page groups them under the model name, not the deployment (#41077)
- cost map: mirror the base rows' capability flags onto Bedrock regional and
cross-region copies (#42254 and later syncs)
- whitelist the new regional Bedrock rows from #42543 and #42588 for the
converse routing check, following the existing regional-row convention
* fix(model-prices): mirror capability flags onto ap-southeast-3 bedrock rows
* refactor(bedrock): tighten types on the moonshot stream decoder and its tests
The pinned base's /etc/apk/world locks glibc-2.44=2.44-r1, so the apk upgrade in the runtime stage cannot move it. The new digest ships 2.44-r6 on amd64 and arm64
* test(utils): raise the post-success hook error from a guardrail in the failure-hook regression
Since #42535 a plain logger raising inside async_post_call_success_deployment_hook
is logged and the completed request returns, so the regression added by #36657 for
"a post-success error never reaches async_post_call_failure_deployment_hook" failed
with DID NOT RAISE on every main run once #42603 revived the misc unit shard. The
raising callback is now a CustomGuardrail, the one kind of callback whose
post-success raise still propagates, which keeps the original assertions intact
* test(utils): type the guardrail's success-hook request_data as a Mapping
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(logger): add shared Rust diagnostics and Python logging bridge
* feat(logger): dispatch diagnostic processing through Rust
* chore: regenerate Cargo.lock after rebase
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: allowlist bounded logging tree walkers in recursive detector
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* perf(logger): skip decoding plain access arguments
* test(logger): skip embedded-python logger test when litellm deps are absent
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style: cargo fmt
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: expect NativeDiagnosticProcessor in the native public surface
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(stub): export NativeDiagnosticProcessor via __new__ in _native.pyi
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(tracing): rename logger crate and document host sink contract
* test(logger): cover exc, stack, and nested extras in the diagnostic filter
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(logger): keep rendered redacted line when template scan flags a key pattern
The blanket REDACTED for a changed msg/color template discarded lines
whose rendered form was already redacted by the same pipeline, e.g.
'password=%s' became 'REDACTED' instead of 'password=REDACTED'. Only
fall back to REDACTED when the rendered form did not change either,
which is where interpolation can mangle the key pattern the scrub
would otherwise see.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci(rust): install python deps so the logger bridge test runs
The end-to-end bridge test skipped silently when litellm's Python deps
were absent. uv sync --no-install-project installs them without a
maturin build, and PYTHONPATH makes them visible to the embedded
interpreter
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
#42435 removed the OpenAI dall-e-3 cost-map row that
test_dalle_3_azure_cost_tracking pinned, and Azure retired DALL-E 3
inference on 2026-03-04, so the test now mirrors a real Azure
gpt-image-2 response and derives the expected cost from the cost map.
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(ci): close open pull requests superseded by a merged fix on their linked issue
* fix(ci): replace the mixed-anchor release-line regex with a plain predicate
* fix(ci): recover a half-done close, page linked pull requests, trust only the workflow's marker
* fix(ci): close superseded pull requests whatever closed the issue and fail the job when the script throws
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
