mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-26 01:12:21 +00:00
19462 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
devin-ai-integration[bot]
|
170eb7fb95
|
feat(rust-bridge): extend native dispatch foundation to chat completions, responses, and messages (#42805)
* feat(rust-bridge): declare native chat completions and responses bindings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(dispatch): cover chat completions and messages dispatch Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(rust-bridge): keep secret manager stub formatting unchanged Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(rust-bridge): match stub parameter names and exports to the native surface Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(rust-bridge): name declining entrypoint parameters and export embeddings in the stub Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(rust-bridge): cover embeddings bindings in the route matrix Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(rust-bridge): keep secret manager stub formatting unchanged Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Yujong Lee <yujong@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
49d0ece934
|
fix(mcp): forward caller bearer on REST oauth_delegate tool calls (#42787)
* fix(mcp): forward caller bearer on REST oauth_delegate tool calls Co-Authored-By: bot_apk <apk@cognition.ai> * fix(mcp): only forward caller bearer on REST for client-forwarded-token servers Co-Authored-By: bot_apk <apk@cognition.ai> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: bot_apk <apk@cognition.ai> |
||
|
devin-ai-integration[bot]
|
dc14e76147
|
fix(mcp): return 401 challenge for REST token-exchange tool calls without a subject token (#42782)
* fix(mcp): return 401 challenge for REST token-exchange tool calls without a subject token Co-Authored-By: bot_apk <apk@cognition.ai> * fix(mcp): keep tool_server_mismatch when server_id disagrees with the tool prefix Co-Authored-By: bot_apk <apk@cognition.ai> * test(mcp): type the token-exchange challenge test helpers Co-Authored-By: bot_apk <apk@cognition.ai> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: bot_apk <apk@cognition.ai> |
||
|
devin-ai-integration[bot]
|
2b3a7f7f8e
|
feat(embeddings): add native dispatch foundation (#42799)
* feat(embeddings): add native dispatch foundation * ci: cover embeddings dispatch tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style: format embeddings dispatch Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Yujong Lee <yujong@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
02d1e2c579
|
feat(cache): add a guarded native response-cache resolver foundation (#42769)
* feat(cache): resolve configured backend for native inference * fix(cache): reuse the resolved native runtime only while its facade guard matches Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(cache): decline native inference when the resolved runtime no longer matches its facade Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Yujong Lee <yujong@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
35225709ed
|
test(integration): cover customer reported cache key, cache_control, bedrock request id, responses schema, scim and tag budget contracts (#42785)
* test(integration): cover prompt cache key, system cache_control, bedrock request id, responses schema, scim and tag budget contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): drop invented instructions shape and unused imports Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): register scim placeholder cleanup before asserting the patch Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: kerry <kerry@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
5db8543817
|
fix(anthropic): preserve MCP tool results in the non-Anthropic Messages bridge (#42783)
The tool_result user message built by the /v1/messages MCP loop used tuple content, which the Messages to Chat Completions adapter silently dropped, so non-Anthropic models re-requested the tool until the iteration cap or the provider rejected the follow-up. Emit list content so the existing tool_result branch translates it into a role tool message keyed by tool_call_id. Resolves LIT-8474 Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: bot_apk <apk@cognition.ai> |
||
|
PhimmStraiker
|
e0af9917a1
|
feat(guardrails): straiker guardrail speaks the v3 platform API (/api/v3/detect) (#41880)
* feat(guardrails): speak the Straiker v3 platform API (/api/v3/detect)
The Straiker guardrail posted a webhook envelope to /api/v1/detect/webhook.
The v3 platform exposes /api/v3/detect instead, and its integration keys
(sk_agt_…) are rejected by the v1 route with an empty 401, so a tenant on
the v3 platform could not run this guardrail at all. Measured on a
customer gateway on 2026-09-17 after they rotated to a v3 key.
v3 parses the gateway's own traffic server-side, the same contract as
Straiker's unified Kong plugin. So on v3 the guardrail relays: the
request phase posts the provider body LiteLLM received (Anthropic
Messages or OpenAI chat), the response phase posts
{straiker_phase, sse, model, request}, the answer beside the request it
answers, and Straiker derives prompt, answer, agent and archetype. Both
phases also carry the flat prompt / app_response pair: a gateway-mode
integration key scores only the flat pair and an api-mode key only the
relayed body, each ignoring the other, so one payload serves whichever
key the console issued and it is one turn either way (measured on tenant
123, both key modes, 2026-09-18).
- api_version: "v1" | "v3", unset follows the key prefix, so a v3 key
needs no extra configuration. Explicit override still wins.
- The relayed body is an allowlist of provider fields. The hook sees the
client body merged with proxy state: `deployment` carries the resolved
provider credential and `proxy_server_request` the client's own
Authorization header. Neither travels. Identity survives as the
metadata subset Straiker's LiteLLM adapter reads.
- Identity never sends a proxy placeholder. `default_user_id` and the
master-key alias were being forwarded as a user and became the
session's identity on the platform.
- Headers: x-tool: litellm (ingress), x-straiker-phase, x-straiker-user,
and x-claude-code-session-id forwarded when the client sent it.
- Verdict: hookSpecificOutput.permissionDecision on the gateway envelope,
`action` on the flat one; block on block/deny, and on a non-empty
blocked_by as a backstop. A detect-mode control reads NONE.
- An error status from Straiker is now a webhook failure. LiteLLM's HTTP
client raises on any non-2xx and the retry loop caught only connection
errors, so a 401 or 503 from Straiker escaped the guardrail as an
exception and was relayed raw to the client, bypassing fail_open /
fail_closed. Retryable statuses retry; the rest are final.
- v1 is unchanged: same envelope, same X-Straiker-Webhook-Format header.
Tests: 15 new, fixtures from the request dict a hook sees on 1.98.0 and
the verdict envelopes the v3 platform returned on 2026-09-18. Each fix
was mutation-checked (handling removed, the test fails). Live: the same
eight-case battery (chat, /v1/messages, streaming, tool call; benign,
injection, PII) passes on a gateway-mode and an api-mode key, blocks at
pre_call with the tenant's block message, and lands under the declared
agent with the end user attributed.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* feat(guardrails): name the agent per application on v3 (x-s6r-agent)
One integration key can front several applications. Straiker enumerates them
as separate agents when the turn names one, which is what the unified Kong
plugin sends as x-s6r-agent. Without it every application on a gateway
collapses onto a single agent.
- Forwards a client-supplied x-s6r-agent.
- New `agent_ref` config names one agent for a route when the client sends
nothing. The client wins, matching Kong's precedence.
- Neither set: no header, and the platform derives the agent from the traffic.
Verified live on tenant 123 against an integration whose connector is
`gateway`: three distinct values minted three observed agents, and a turn
with no hint derived one from the traffic shape. An integration whose
connector is `custom-agent` declares its agent, so every turn attributes to
that one agent and the hint is ignored (agent_ref_source: attested).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs(guardrails): which v3 shape is scored depends on the connector, not the key mode
The earlier comment said a gateway-mode key scores only the flat pair. Re-measured
on tenant 123 across all three integration types with one injection prompt:
custom-agent connector (Add Agent) raw body ignored flat prompt scored
gateway connector raw body scored flat prompt scored
api mode raw body scored flat prompt ignored
Behaviour unchanged: the payload already carries both shapes, which is why it works
on every type. Comment only.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(guardrails): send exactly what the unified Kong plugin sends on v3
The v3 platform parses the gateway's traffic itself and derives agent,
archetype and identity from it. The earlier commits added to the relayed
body (a flat prompt / app_response pair, source, user_name) and to the
headers (x-tool, x-straiker-phase, x-straiker-user). None of that is in
the Kong v0.12 contract, and traffic through this guardrail was not
classifying by shape the way the same traffic through Kong does. Match
Kong byte for byte and leave classification to the platform.
Request phase: the provider body, plus session_id and
original.processed.Meta.user. Response phase: {straiker_phase, sse,
model, request} plus the same two. No flat fields, no phase or user
headers, no x-tool.
Session id follows Kong's precedence: the client's x-claude-code-session-id,
then the session LiteLLM resolved, then an md5 of system prompt + first
message so a conversation that states no session still groups across its
replays.
Routing hints complete the Kong set: x-s6r-agent (client header, else
`agent_ref`), and new `client` (x-s6r-client) and `format_hint`
(x-s6r-format) config, both optional.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(guardrails): sort imports in the v3 session test
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(straiker): send a streamed Messages answer back in the Messages shape on v3
On a streamed /v1/messages call the proxy rebuilds the answer as a chat
completion before the post-call hook runs, and that is what the plugin put in
the response envelope's sse field. Straiker's coding-agent reader parses a
Messages answer, so a Claude Code turn relayed this way came back
coding_agent/claude with no session and zero events scored: the model's tool
calls were never screened on the response phase. Captured live on 2026-09-18
against tenant 123, a real Claude Code Bash tool call through the proxy.
The proxy's own Anthropic adapter turns the rebuilt answer back into a Messages
response when the call arrived on the anthropic_messages route, which is what a
transport relay forwards. Chat completions calls keep the chat completion shape
and a buffered Messages answer is relayed untouched.
The regression test's fixture is the chat completion the proxy actually built
for that captured turn. After the fix the same turn scores on the response
phase (session resolved, one event, the Bash tool_use block present).
* style(straiker): ruff format the v3 guardrail and its tests
* refactor(straiker): one attempt per call in the webhook retry loop
The HTTPStatusError branch added for v3 duplicated the non-200 branch and put
_post_webhook over the strict complexity ceiling. One attempt is now its own
method that returns the verdict or a failure marked retryable, and the loop only
decides whether to try again. Behaviour is unchanged: retryable statuses and
transport errors retry, everything else is final.
* fix(straiker): name Claude Code's client and agent on v3 so its session lands under one coding agent
Straiker types a gateway turn as a coding agent from the "You are Claude Code"
preamble, which only the main agent turns carry. Claude Code's title and
topic-detection sidecars have their own system prompts, so they resolved by
shape as autonomous, and because they share the session id with the main turns
the whole session was filed under Autonomous rather than under a coding agent.
Kong does not hit this because its plugin config names the client and agent on
every call.
The User-Agent (claude-cli/...) is on every call including the sidecars, so the
plugin now reads it and sends x-s6r-client: claude plus, when the route names no
agent, x-s6r-agent: "Claude (LiteLLM)". A client-supplied x-s6r-agent or the
agent_ref config still wins. Verified live on tenant 123: a real Claude Code
session now lands as one coding_agent labelled "Claude (LiteLLM)" with its turns
scored, where before it split across Autonomous.
Identity: the key's own user (email then id) now outranks the end user the
request named. LiteLLM resolves Claude Code's hashed metadata.user_id as the end
user when nothing better is set, so a per-user key was being shadowed by a
session token. The key is the authenticated principal, the way a Kong consumer
is, so it wins; the request end user is the fallback.
* refactor(straiker): build the v3 request, envelope and headers as frozen mappings
The v3 builders seeded dicts and grew them, which the type-discipline gate
counts as mutable accumulators. Each is now one expression over a tuple of
pairs, frozen with MappingProxyType, and the JSON encoder unwraps a frozen
mapping through a default. The session seed and the verdict parser no longer
rebind locals. The wire is unchanged: 36 live calls through the proxy on this
commit carry the same fields, shapes, headers and identities as before, with
no mappingproxy text in any body.
* fix(straiker): satisfy basedpyright on the v3 builders
The frozen-mapping refactor left a shadowed headers local, a Mapping handed to
an HTTP client that takes a dict, an unguarded optional response, a turn id
typed object, and a redundant isinstance on already-typed texts. No behaviour
change: 4 live calls (chat, Messages, Bedrock, injection) return 200 with the
expected verdicts on this commit.
* fix(straiker): type the v3 config fields at the initializer and keep the verbose log as JSON
The four v3 routing fields (api_version, agent_ref, client, format_hint)
travelled through the untyped kwargs passthrough, which basedpyright counts
against the budget. They are now validated through a small Pydantic model at
the initializer and passed by name.
The verbose log serialized the frozen payload with default=str, which printed
a Python repr instead of JSON once the builders returned MappingProxyType.
Every serializer now unwraps a frozen mapping first. A test asserts the logged
payload parses as JSON and carries the identity; mutating the log site back to
default=str fails it.
* fix(straiker): address review findings on the v3 relay
Text completions relay their prompt: `prompt`, `suffix`, `echo` and `best_of`
join the provider allowlist, so /v1/completions traffic is screened.
The route's `agent_ref` now outranks the caller's `x-s6r-agent` header. The
header is caller-supplied, and letting it beat a pinned route would let any key
file its traffic under another application's agent and controls. On a route
that names nothing the header still names the application, which is how
several applications enumerate behind one key.
Credentials inside `tools` and `mcp_servers` (an OpenAI `mcp` tool's `headers`,
Anthropic's `authorization_token`) are replaced with `[redacted]` before the
body leaves the proxy, on both phases and in the verbose log. Detection reads
tool names, descriptions and schemas, never these.
A 200 whose body is valid JSON but not an object now reports an invalid
schema and follows the failure policy instead of raising out of the hook.
Comments that restated a constant are gone. Tests cover each change and the
failure paths (unreadable error body, client exceptions, missing response,
unmodellable request, session seeds from Anthropic block shapes); every fix
fails its test when reverted.
* fix(straiker): scrub tool credentials one level deep, without recursion
* fix(straiker): scrub only the fields that carry a credential, never a schema
The credential set is now the three fields that actually hold one on a tools
or mcp_servers entry (headers, authorization, authorization_token), read one
level deep. A function tool whose parameter schema defines a token, headers or
api_key property is relayed exactly as sent; a test pins that, and fails
against the recursive version.
* test(straiker): use example.com identities; drop a comment that restated its branch
* fix(straiker): present a legacy completion as the chat exchange it is
Straiker scores chat on both phases of a gateway turn but has no reader for a
text_completion answer: the request phase of a /v1/completions call was
scored and the response phase was refused with 501, whether or not the call
named an agent. A completion is one user turn and one assistant turn, so both
phases now present that exchange: the prompt becomes the single user message
and the TextCompletionResponse becomes a chat completion. Measured through the
proxy on this commit, both phases return 200 and score, and the derived
session is shared between them.
The derived session seed accepts the tuple the conversion produces; the test
pins the session on both phases and fails against the list-only check. The
unreachable "parsed is None" branch is folded into the failure branch, and a
malformed tools value is shown to relay as sent.
* fix(straiker): screen a completions prompt as the text the model receives
LiteLLM's /v1/completions accepts a string, a list of strings, a list of
token ids or a list of token-id lists, and decodes token ids with the
text-davinci-003 tokenizer before calling the model. The relay now renders
the prompt the same way, one user message per prompt, so a pre-tokenized
prompt is screened as the text it stands for rather than as digit strings.
A prompt in a shape this cannot render (empty, mixed, or with no tokenizer
available) is relayed untouched instead of being replaced with something
else. Tests cover all four accepted shapes and six unrenderable ones.
* fix(straiker): seed the derived session on the preamble and the first user turn
An OpenAI chat body carries its system prompt as messages[0], and the derived
session seeded on the Anthropic `system` field plus messages[0] with no role
check. For that shape the seed was the system prompt twice and the first user
turn never counted, so every unnamed conversation behind one system prompt
collapsed into one Straiker session. The seed now takes the preamble from
wherever the API puts it (`system`, `instructions`, or a leading system or
developer message) and the first message with role `user`, else a Responses
`input` string, else `prompt`. Two conversations sharing a system prompt are
two sessions again; a replayed conversation stays one.
* fix(straiker): seed the derived session on every text block of the first turn
A user turn that opens with an image or a document block and carries its
text later seeded the session on an empty string, so two different
conversations under the same preamble shared one Straiker session. Read
every text block of the turn instead of only the first block. A plain
string or a single text block seeds exactly as before.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(straiker): cover the tokenizer fallback, a textless first turn and Responses instructions
Three branches of the v3 relay had no test: a token-id prompt relayed as
sent when the tokenizer cannot be fetched, a first user turn with no text
seeding the session on the preamble alone, and a Responses API body
seeding on its instructions and first input turn. Each test fails when
its branch is mutated.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(straiker): seed the derived session on the principal as well as the conversation
Straiker skips turns it has already scored for a session. The derived
session hashed the system prompt and the first user turn alone, so two
users who opened a conversation with the same words shared one session,
and the second user's copy of an attack came back as a replay: unscored
and allowed. Measured live on 2026-09-20: the first user's SSN turn was
blocked (`social_security_number`, scored=2), the second user's identical
turn was allowed (`controls: []`, replayed=2).
The principal now joins the seed. Explicit session ids, the Claude Code
header and LiteLLM's own session are unchanged.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(straiker): derive the session id with sha256 and drop comments that restated constants
The derived session now hashes the principal, and CodeQL flags MD5 over an
identity as a weak hash on sensitive data. SHA-256 truncated to the same
32 hex characters keeps the id shape. Comments that only labelled the
allowlist groups or restated a constant are removed; the two that explain
a non-obvious choice stay.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(straiker): keep a blocked conversation blocked when it is replayed
Straiker de-duplicates turns it has already scored per session and
answers a replay `allow`, whatever the first verdict was. A client that
resends a blocked request, or grows the conversation past the blocked
turn, was let through: measured on 2026-09-20, `block` then `allow,
events_replayed=2` for the same session and body, and Claude Code's
automatic retry after the 400 turned a blocked poisoned-file read into
a pass.
The guardrail now remembers, per session, a fingerprint of every
conversation it blocked (a bounded, day-long in-memory cache) and blocks
a request that repeats or extends one without asking again. A different
session with the same words is a new conversation and is scored afresh.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(straiker): scope the block memory by session or principal, never by content alone
A request with no derivable session keyed the replay memory on the
conversation fingerprint alone, so one caller's block could answer
another caller's identical request. The memory is now scoped by the
session, else by the principal, and a request with neither is not
remembered at all.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(straiker): remember only a block that names a control, never one that comes from state
The replay memory kept every block, including one the platform returns
because a kill switch is engaged (`action: block` with `blocked_by: []`).
An administrator lifting the kill switch then left the conversation
refused by the remembered copy: measured on 2026-09-21, traffic stayed
blocked after `POST /inventory/agents/{id}/restore` returned `engaged:
false`.
The same words are the same attack tomorrow, so a control-named block is
still worth remembering; state is not ours to cache. The parsed verdict
now carries `blocked_by` so the two can be told apart.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Phimmasone Phonpaseuth <PhimmStraiker@users.noreply.github.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
|
||
|
devin-ai-integration[bot]
|
59eb943a6e
|
test(rust): model the blocking OCR hook as a guardrail so its raise propagates (#42775)
Co-authored-by: Yujong Lee <yujong@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
9fd25b2228
|
fix(ui): keep per-user MCP credentials updatable and clearable after setup (#42652)
* fix(ui): keep per-user MCP credentials updatable and clearable after setup Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(ui): keep card keyboard activation off nested credential buttons Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(ui): confirm before clearing saved per-user MCP credentials Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(ui): reset the clear confirmation when the credentials modal closes Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(mcp): cover per-user env var edge paths in browser and integration contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): wait for the peer process to grant the key before listing its MCP tools Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(e2e): drop the mutable removed flag from the deleted-server browser contract Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: ryan <ryan@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
1c289e5ecd
|
fix(prices): add baseten/zai-org/GLM-5.3-Fast pricing (#42764)
* fix(prices): add baseten/zai-org/GLM-5.3-Fast pricing with cost tracking e2e Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(e2e): assert message instead of comment on breakdown row Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(baseten): drop the live e2e cost tracking test Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: kerry <kerry@berri.ai> |
||
|
devin-ai-integration[bot]
|
c2b388ebe6
|
fix(bedrock): honour stream_chunk_size in Invoke streaming (#42686) | ||
|
devin-ai-integration[bot]
|
efb93e62f8
|
fix(completion_extras): forward non-enum reasoning_effort through the Responses bridge instead of dropping it (#42452) | ||
|
devin-ai-integration[bot]
|
b0407ad33e
|
ci: add merge smoke checks workflow with loopback-only harness and 11 curated cases (#42709)
* ci: add dashboard and core smoke checks across supported Python versions Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ci: tighten merge smoke harness and keep mapped test diffs additive Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ci: terminate proxy on readiness timeout and use contextlib.suppress in teardown Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: yuneng <yuneng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
e73f949fbb
|
fix(params): stop stream_chunk_size reaching provider request bodies (#42664)
* fix(params): carry stream_chunk_size through litellm_params instead of provider params * test(integration): fence stream_chunk_size out of every provider request body * test(bedrock): type parametrized stream chunk test params * test(integration): drop the contracts manifest resurrected by the main merge * test(bedrock): type the stream_chunk_size test helpers * test(params): finish AGENTS.md typing pass on stream_chunk_size tests * test(integration): drop the covers marker from the stream_chunk_size wire test --------- Co-authored-by: shrey kharbanda <shreshth@berri.ai> |
||
|
devin-ai-integration[bot]
|
514bc181d6
|
test(integration): regression tests for July cost tracking, budgeting and spend bugs (#42694)
* test(integration): streamed Bedrock Messages usage cost equals the recorded spend (Pylon #6667) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): echoed cost-map model info is not persisted as deployment overrides (Pylon #6844) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): reset sweep runs on one pod per tick while replicas share the lease (Pylon #6521) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): bedrock post-call guardrail scans streamed Anthropic Messages tool use without 500 (Pylon #6503) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): realtime cached audio tokens bill at the audio cache-read rate (Pylon #6704) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): legacy GET /spend/logs returns at most the 10000 most recent rows and flags truncation (Pylon #6752) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): itemize Responses API cache write tokens as cache creation cost (Pylon #6454) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): migration entrypoint deploys pending migrations before proxy startup (Pylon #6649) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): opted-in team keys stop at the owner's personal budget (Pylon #6641) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): guardrail information stays in the spend log when the caller sends metadata on /v1/messages (Pylon #6614) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): key model allowlist is enforced on Bedrock passthrough routes (Pylon #6419) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): JWT mapped key backfills a null user email from token claims (Pylon #6266) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): scheduled budget reset recovers from a transient DB transport failure (Pylon #6582) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): team key lists models granted through a team access group (Pylon #6044) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): JWT subject without team claim lands in the configured default team (Pylon #5895) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): end-user spend lands for a key without user_id when the auth cache is Redis (Pylon #6021) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): stale-low redis counter still blocks team member over budget (Pylon #5824) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): prompt-carrying spend rows are written in byte-bounded statements (Pylon #6083) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): logs UI session_total_spend sums every round of a multi-round session (Pylon #5928) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): config.yaml guardrails are served by the guardrail usage detail and overview (Pylon #5813) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): plain chat request skips the object permission lookup (Pylon #5965) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): register july accounting regression contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): isolate cost map override clear on owned proxy Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): make reset lease claim and db relay refusal deterministic Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): poll pg_stat settle, bound unbanned relay refusals, clear reset lease on teardown Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): bound relay refusals so the budget sweep can reconnect Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: kerry <kerry@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
a286ebf42e
|
test(integration): regression tests for July provider translation, routing and streaming bugs (#42693)
* test(integration): optional Anthropic tool properties stay optional on the OpenAI Responses wire (Pylon #6619) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): Bedrock InvokeModel count-suffixed cache usage fields are reported and charged (Pylon #6708) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): anthropic messages honors the deployment request timeout (Pylon #6505) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): drop client_metadata before the Bedrock Converse body reaches the provider (Pylon #6645) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): repeat Bedrock requests under one session name assume the role once (Pylon #6681) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): messages stream keeps include_usage off the Responses wire with always_include_stream_usage (Pylon #6466) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): clamp sub-16 max_tokens to the Responses API floor instead of 400 (Pylon #6539) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): forwarded client x- headers reach the provider on /v1/responses (Pylon #6565) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): Anthropic messages stop_sequences reach OpenAI-compatible providers as stop (Pylon #6536) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): Codex namespace tools reach a chat upstream flattened and round-trip through /v1/responses (Pylon #6409) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): keep Claude 4.6 legacy thinking budget_tokens on /v1/messages (Pylon #6727) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): nvidia nim ranking keeps image passages and applies top_n without sending top_k (Pylon #6401) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): tpm-only model rejects priority traffic once recorded tokens reach the model tpm (Pylon #6344) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): reasoning-only chunks open an Anthropic thinking block at index zero on /v1/messages streams (Pylon #6337) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): file content streams to the client before the upstream finishes sending (Pylon #6315) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): agent whose card lives only at agentCard/v1.0 is reached with bearer auth (Pylon #6249) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): vertex batch create returns a batch when outputInfo is null (Pylon #6374) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): fireworks session id is sent as x-session-affinity and cached tokens land in spend log metadata (Pylon #6220) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): advisor sub-call failure does not cool down the executor deployment (Pylon #6212) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): Gemini /v1/messages cache_control creates cachedContent with Anthropic ttl (Pylon #6221) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): bedrock_mantle max_output_tokens below 16 is clamped before reaching Mantle (Pylon #6262) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): missing thinking signature 400 on /v1/messages retries without thinking blocks (Pylon #6222) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): format Gemini messages cache_control wire test (Pylon #6221) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): rebuilt shared aiohttp session keeps the configured keepalive timeout (Pylon #6387) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): sagemaker_chat signs the inference component header and sends hf_model_name as the body model (Pylon #6187) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): Bedrock Converse DeepSeek drops Anthropic thinking and sends V3 reasoning_effort raw (Pylon #6149) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): concurrent team model TPM requests are reserved before the provider call (Pylon #6075) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): /v1/messages honors the configured timeout against a stalled upstream (Pylon #6025) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): Codex additional_tools input items reach Bedrock Mantle as top-level tools (Pylon #6012) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): advisor api_base without api_key never sends the proxy Anthropic key to the caller host (Pylon #6226) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): rerank responses carry call id, latency and cost headers (Pylon #5981) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): parse the outbound Anthropic body with the typed JSON adapter (Pylon #6025) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): Bedrock Knowledge Base search forwards userContext to the Retrieve body (Pylon #5991) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): Marengo 3.0 text embeddings reach Bedrock nested under inputType (Pylon #5949) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): sub-16 max_tokens over a responses deployment reaches OpenAI as 16 (Pylon #6008) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): midturn system correction reaches the OpenAI Responses wire via /v1/messages (Pylon #6449) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): concurrent requests over a key tpm limit are rejected before reaching the provider (Pylon #5737) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): chat to responses bridge keeps deployment AWS credentials for Bedrock Mantle SigV4 (Pylon #5870) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): vertex gemini stream split across many fragments completes without stalling the proxy (Pylon #5838) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): bedrock mantle /v1/messages stream keeps stream true and relays SSE events (Pylon #5596) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): large chat payloads are released from worker memory after the request ends (Pylon #5920) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): streaming success logs v3 rate limit remaining values for callbacks (Pylon #5767) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): a database created search tool backs Anthropic web search interception (Pylon #5669) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): register july provider regression contracts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): make july provider regression tests deterministic under cache and worker sharing Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): drop order-fragile worker memory probe pending a real retention regression check Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): apply ruff import sorting and formatting Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): drop stale contract entry and pass question to advisor executor Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): use tiktoken-backed executor model in advisor tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: kerry <kerry@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
21530d887b
|
feat(gemini): add gemini-3.8-flash-tts and gemini-3.8-flash-lite-tts prices (#42752)
* feat(gemini): add gemini-3.8-flash-tts and gemini-3.8-flash-lite-tts prices Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(gemini): bill tiered TTS output through output_cost_per_token tiers Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: kerry <kerry@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
cddc53464e
|
test: deflake fuzzy picker, breached-password HIBP, and MCP stdio timeout tests (rolling deflake 2026-09-22) (#42125)
* test(autoroute): wait for a valid fuzzy selection index and cancel the prompt on driver failure Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(autoroute): read the fuzzy selection through the public InquirerPy property Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): inject the HIBP client into change_password so the breached-password test never touches the network Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(mcp): only use the 200ms read timeout in the silent mode of the transport completion test Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): record HIBP requests so the ordering test asserts no lookup happened Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ci(codeql): filter the weak-sensitive-data-hashing false positive on the HIBP k-anonymity lookup Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
19556952d9
|
feat(secrets): route secret resolution through native Rust backends (#42619)
* fix(secrets): verify provider API request and payload contracts * wip * fix(secrets): unify backend reads and route secret resolution * feat(secrets): bind built-in managers to retained Rust backends * refactor(secrets): centralize catalog dispatch and native binding * test(secrets): split provider integration tests * refactor(secrets): enforce cache and rotation contracts * test(secrets): stub parent packages in failing resolver fixture Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(secrets): pass manager settings through the interop boundary Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(secrets): align cloud KMS auth and harden provider reads * ci(rust): raise native wheel size gate to 40 MB for secrets backends Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): treat unset google kms flag as disabled like the old loader Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(secrets): preserve certificate credentials and disabled KMS flags * test(secrets): cover certificate validation and bounded auth retries * test(secrets): cover Python dispatch without the native extension * test(proxy): skip legacy secret manager cases when the optional SDK is missing Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(secrets): port Python parity tests and preserve provider behavior * fix(secrets): store the captured native config without setattr to satisfy the strict lint budget Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(secrets): preserve missing Azure manager values Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(secrets): pin typed values and recovery failure precedence * refactor(secrets): organize provider internals and behavioral test suites * refactor(secrets): simplify recovery and isolate Python compatibility * fix(secrets): distinguish Azure callback absence from HTTP not found * fix(secrets): preserve Python AWS read results at the bridge * fix(secrets): route public reads through the native catalog bridge * fix(secrets): keep JSON selection outside the bridge Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(secrets): preserve provider JSON reads at the bridge Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(secrets): preserve Python primary JSON semantics Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(secrets): preserve CyberArk mutation behavior through the native bridge * docs(secrets): record public API replacement gaps * refactor(secrets): share Vault write payload preparation * feat(secrets): route Vault mutations through the native bridge * fix(secrets): preserve typed Vault rotation failures * refactor(secrets): move Python dispatch into bridge * refactor(secrets): move CyberArk Python policy into bridge * refactor(secrets): move Vault Python policy into bridge * test(secrets): assert Vault rotation request paths * fix(secrets): keep bridge JSON interop centralized Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Yujong Lee <yujong@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
6b642f3648
|
feat(cost-map): add Azure Foundry pricing for gpt-6-sol and gpt-6-luna (#42747)
* feat(cost-map): add Azure Foundry pricing for gpt-6-sol and gpt-6-luna Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(cost-map): give azure/eu gpt-6-sol and gpt-6-luna full model metadata Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: kerry <kerry@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
48050d9646
|
chore(ci): drop litellm_internal_staging and litellm_oss_staging references, main is the only trunk (#42745) | ||
|
devin-ai-integration[bot]
|
e26a6450c8
|
test(integration): add MCP gateway coverage wave 1 with a dedicated mcp shard and proxy coverage artifact (#42711)
* test(integration): drop the contracts.json manifest and the covers requirement Groups live as a GROUPS literal in run.py, the browser expectations move next to the browser tests, and the runner fails only on pytest failure, collection errors or a selected file that collects zero tests. The covers marker stays registered for the existing tests but is no longer checked. The mcp directory gets its own group Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ci(integration): run mcp as its own shard with xdist and a peer proxy Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ci(integration): INTEGRATION_COVERAGE=1 runs the proxy under coverage for the MCP modules The mcp shard sets it. The proxy and its peer start under coverage run in parallel mode, get SIGTERM after the tests so coverage flushes, and the combined text and HTML reports land in the suite results that CircleCI already stores as artifacts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ci(integration): let the test proxy flush coverage when uvicorn re-raises SIGTERM Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): add SSE, stdio, scripted, OpenAPI and OAuth 2.1 MCP peer doubles Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): add MCP transport and access-control matrices Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): add MCP credential and OAuth flow coverage Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): add MCP LLM endpoint, accounting, guardrail, resilience and lifecycle coverage Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): stop the same-URL grant test from counting a late initialize as a leaked call and satisfy the test-tree lint Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): assert the REST denied-server listing is refused or empty Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): pin the REST denied-server listing to 403 access_denied Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: yuneng <yuneng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
5028f9ec59
|
fix(proxy): validate model credential name only when it changes (#42701)
PATCH /model/{id}/update rejected read-modify-write edits that resent an unchanged but dangling litellm_credential_name. Existence validation now runs only when the requested name differs from the stored one; empty string and non-admin detach rejections are unchanged
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
||
|
Oliver Jensen
|
bc3b5b1d5b
|
fix(proxy): revoke UI session tokens on logout and password change (#42463)
* fix(proxy): revoke UI session tokens on logout and password change Adds POST /session/logout to revoke the presented UI session key server side (previously logout was client-side only and the key stayed valid until expiry). Password changes now revoke the user's other UI sessions: self-change keeps the caller's session, admin reset and onboarding claim revoke all. The BYOK OAuth cookie auth now re-resolves the embedded key against the DB so revoked sessions get a 401. * fix(proxy): satisfy B008 budget and backend allowlist for /session/logout * refactor(proxy): satisfy type-discipline budget in session_endpoints |
||
|
devin-ai-integration[bot]
|
a3196907e4
|
feat(otel): emit gen_ai.conversation.id from the caller's session id on v2 LLM spans (#42486)
* feat(otel): emit gen_ai.conversation.id from the caller's session id on v2 LLM spans Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(otel): keep the caller's header session under missing_session_id: generate and read replayed payload session ids Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(otel): drop only the proxy-minted session id so a caller id on the other metadata key survives Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(otel): keep a replayed session id hidden when it only echoes the payload trace id Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(otel): keep a replayed session id even when the payload trace id fell back to it Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(otel): stop reading the replayed payload's session id, the generated marker does not survive replay Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): audit gen_ai.conversation.id on otel v2 spans through a real proxy, sink and postgres Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): keep otel conversation rigs alive for the whole session so shuffled shards do not reboot the proxy per test Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(otel): stop the audit rig proxies from probing sibling test peers for model info Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(otel): record accepted OTLP batches in the sink instead of mutating the collector Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(otel): guard the accepted batch deque so snapshots cannot race sink appends Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: mrinal <mrinal@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: yucheng <yucheng@berri.ai> |
||
|
devin-ai-integration[bot]
|
40ec84caa2
|
fix(proxy): publish auth cache invalidations in the background so a wedged coordination Redis cannot stall user updates (#42534)
* fix(proxy): bound auth cache invalidation publish so a wedged coordination Redis cannot stall user updates Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): resolve publish callable at call time in evict_and_broadcast The keyword-only default bound publish_auth_cache_invalidation at function-definition time, so tests patching the module attribute observed zero calls. Default to None, resolve the real publisher inside the body, and keep the keyword-shaped cache_key call the existing contract asserts Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): publish auth cache invalidations in the background so a wedged coordination Redis costs handlers nothing Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): cap in-flight auth cache invalidation publishes so a wedge cannot drain the redis pool Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: yucheng <yucheng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
320ad73f56
|
fix(policy_engine): keep inherited parent guardrails when a child policy condition misses (#42548)
* fix(policy_engine): keep inherited parent guardrails when a child policy condition misses Attachment applicability now walks the policy inheritance chain, so an attached child whose own condition does not match still contributes the guardrails of its unconditional ancestors, and a non-default attachment that applies through an ancestor still suppresses default attachments. The resolver continues to skip only the chain members whose own condition fails Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(policy_engine): skip a policy's pipeline when its own condition misses resolve_pipelines_for_context returned the pipeline of a matched policy without evaluating its own condition, so a condition-missing child admitted by the chain-aware matcher still ran its pipeline. It now mirrors resolve_policy_guardrails and drops the pipeline when the policy's own condition does not match. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(policy_engine): property test that chain matching only widens to applicable ancestors Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat(policy_engine): log policies admitted only through an inherited ancestor Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(policy_engine): log ancestor admissions once per attachment scan Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: yucheng <yucheng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
24a05b29a6
|
test(integration): allow skipped nodes and drop the shard cap (#42687)
Co-authored-by: yuneng <yuneng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
65468219db
|
test(integration): regression tests for August cost tracking and budgeting bugs (#42622)
Some checks are pending
LiteLLM Rust / rust-lint (push) Waiting to run
LiteLLM Rust / rust-test (push) Waiting to run
LiteLLM Rust / rust-wheel (push) Waiting to run
Terraform Modules / fmt, validate, test (aws) (push) Waiting to run
Terraform Modules / fmt, validate, test (gcp) (push) Waiting to run
Terraform Provider / gofmt, vet, build, test (push) Waiting to run
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Waiting to run
* test(integration): saving echoed model_info never persists cost map pricing as a deployment override (Pylon #6870)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): budget_duration change on /budget/update recomputes budget_reset_at (Pylon #6913)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): count_tokens on a budgeted key reserves no budget and a later completion still succeeds (Pylon #6966)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): /cost/estimate reports configured prices for a deployment absent from the cost map (Pylon #7014)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cache the team member default budget in Redis as JSON (Pylon #7180)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): aggregated team daily activity reports whole-range team spend in one page (Pylon #7224)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): failed daily user rollup commits are retried so spend report and daily activity agree (Pylon #7268)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): CLI session token without org_id is charged to and capped by the team organization budget (Pylon #7291)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): gemini passthrough success releases its budget reservation from the spend counter (Pylon #7295)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): batch retrieval spend row sums reasoning tokens and counts output and error file failures (Pylon #7341)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): uncostable batches retire from the cost poll page so newer batches are costed (Pylon #7342)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): charge a team member added without any budget on its membership row (Pylon #7363)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): failed dispatched requests keep estimated input tokens in spend logs (Pylon #7519)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock passthrough converse guardrail ignores tool definitions (Pylon #7524)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): explicit null budget_duration on /team/new is not replaced by default_team_params (Pylon #7536)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): PATCH /organization/update with a null limit clears it (Pylon #7577)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): ultrafast service_tier bills ultrafast rates without leaking pricing fields upstream (Pylon #7587)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): keep the selected model in the response and spend log for an Azure Model Router alias (Pylon #7636)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): disconnected Bedrock /v1/messages stream still bills terminal usage (Pylon #7685)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): databricks cached prompt tokens bill at cache rates (Pylon #7738)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): completed batch spend row records reasoning tokens and error file failures (Pylon #7928)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bill OCR annotation pages at annotation_cost_per_page (Pylon #7958)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): in-flight count tokens request reserves no key budget so a completion still reaches the provider (Pylon #7307)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): fail-closed key rejects known estimate over remaining budget before provider (Pylon #7691)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): streamed /v1/responses success callbacks keep provider response headers (Pylon #7775)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Revert "test(integration): fail-closed key rejects known estimate over remaining budget before provider (Pylon #7691)"
This reverts commit
|
||
|
devin-ai-integration[bot]
|
f44052d87b
|
fix(vector_stores): keep config-defined vector stores listed and read-only (#42574)
* fix(vector_stores): keep config-defined vector stores listed and read-only Vector stores declared in config.yaml were purged from the in-memory registry by /vector_store/list because the database was treated as the only source of truth. Config-defined stores now carry is_config=True, stay in the list beside database rows, are never overwritten or evicted by database state, and reject /vector_store/new, /vector_store/update and /vector_store/delete with 400. The Admin UI renders them read-only Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(ui): show vector store source and read-only state for config-defined stores Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): audit config-owned vector stores across list, writes, search, authz, peers and redis outage Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(ui): show a visible read-only hint in the config vector store actions menu Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: ryan <ryan@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
5c0b374f0a
|
test(integration): regression tests for August provider translation and streaming bugs (#42621)
* test(integration): Bedrock batch files upload completions and responses records as user messages (Pylon #6882)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): client Anthropic OAuth token never replaces Bedrock SigV4 authorization (Pylon #6888)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bridge /v1/messages and /v1/responses streams through empty-choices chunks (Pylon #6992)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): prepend azure content-filter metadata chunk to the messages stream (Pylon #6992)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): fireworks routers/ slug reaches the provider as accounts/fireworks/routers/<id> (Pylon #7030)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock hidden thinking tokens are not reported as text tokens (Pylon #7067)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): azure_ai FLUX.2-flex image generation targets the flex provider path with the BFL body (Pylon #7092)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): keep Databricks streaming usage and cache reads in the client stream and spend log (Pylon #7094)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): openai-compatible image edits forward provider-specific form fields to the backend (Pylon #7122)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): replayed intercepted web search turn reaches Bedrock as text through /v1/messages (Pylon #7181)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): streamed web search turn capped by max_agentic_loops ends the turn with snippets and ordered blocks (Pylon #7230)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock rerank keeps forwarded client headers out of the SigV4 signature (Pylon #7284)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): azure_ai rerank authenticates with an Entra token when no api key is set (Pylon #7303)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): perplexity stream with cost breakdown object completes and bills total_cost (Pylon #7331)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): azure_ai strips Anthropic message fields before the Foundry request (Pylon #7336)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): capped intercepted web search ends the turn without an internal tool_use block (Pylon #7378)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): Bedrock passthrough converse-stream keeps event-stream content-type (Pylon #7482)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): v1/messages success exposes v3 priority rate limit headers (Pylon #7532)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): config deployment dropped by a stale boot cost map is restored after reload (Pylon #7564)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): Bedrock Mantle project id reaches the provider as anthropic-workspace-id (Pylon #7583)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): prefixed Opus 4.8 reasoning_effort reaches Bedrock as adaptive thinking (Pylon #7586)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): dashscope chat forwards reasoning_effort to the provider (Pylon #7606)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): failing stream logging callback still releases the max_parallel_requests slot (Pylon #7608)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): gen 5 Claude Bedrock Invoke tool search sends the Bedrock beta field (Pylon #7642)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): databricks ai gateway api_base requests OAuth token from workspace origin (Pylon #7724)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): deepseek vision image content list reaches the provider unchanged (Pylon #7729)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): Bedrock Mantle context overflow surfaces as 400 prompt is too long (Pylon #7732)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): Codex history items reach Bedrock Mantle as supported Responses input types (Pylon #7783)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): chat over responses deployment returns finish_reason length when output tokens run out (Pylon #7784)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock_mantle rewrites Codex history items before the Responses wire (Pylon #7812)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): advisor sub-call on /v1/messages uses the configured advisor deployment (Pylon #7828)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): tencent thinking reaches the provider body instead of failing the request (Pylon #7834)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): xAI chat web search reaches /v1/responses with instructions and nested filters (Pylon #7835)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): send Bedrock Converse config blocks once at top level (Pylon #7839)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock converse sends gpt-5 reasoning_effort as reasoning.effort (Pylon #7850)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bedrock cohere.embed-english-v3 embeddings accept encoding_format and dimensions (Pylon #7963)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): streamed chat completions emit SSE keepalive pings while the upstream is silent before its first token (Pylon #7987)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(integration): format the TTFT keepalive regression test (Pylon #7987)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): openai chat drops tool_choice when the request has no tools (Pylon #8022)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): stream whose first chunk has no choices falls back and bills the fallback (Pylon #8006)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Revert "test(integration): Bedrock Mantle project id reaches the provider as anthropic-workspace-id (Pylon #7583)"
This reverts commit
|
||
|
devin-ai-integration[bot]
|
3eb7e45615
|
fix(pricing): drop the unpublished cached rate from the Gemini Live preview entries (#42651)
* fix(pricing): correct cached-token fields on realtime cost-map entries azure/gpt-realtime-2 was the only member of the gpt-realtime-2 family priced on one side of its cached-audio meter. Azure publishes that meter as "gpt-realtime-2 Audio cd inp Gl 1M Tokens" at 0.4 per 1M and charges the same rate for the write that populates the cache and the read that hits it, so cache_creation_input_audio_token_cost lands at 4e-07, matching azure/gpt-realtime-2.1, azure/gpt-realtime-2.1-mini and the openai gpt-realtime-2 entry. No cost path reads that field yet, so this corrects what get_model_info reports rather than what anything bills. The gemini Live entries go the other way. Google's Vertex context-caching page publishes separate supported-model lists for implicit and explicit caching, and no Live or native-audio model is in either one. Its pricing page prints N/A in both cached-input columns for every Gemini 2.5 Flash Live API row, where plain 2.5 Flash and 2.5 Flash-Lite both carry real cached prices, and the Vertex model card for the family marks context caching not supported outright. Vertex never reports cachedContentTokenCount on a Live session either, including for a byte-identical 7,021-token prefix replayed across sessions minutes apart, which is well past the 2,048-token minimum the same page sets for the Gemini 2 family. So the 7.5e-08 on the two preview siblings priced something the provider does not sell, and supports_prompt_caching on all three claimed a capability the model does not have. The rate comes out. The flag is set to false rather than removed, because get_model_info maps an absent key to None, and None is how this map spells "nobody checked" across the 2,788 entries that omit it, where false records the vendor's documented no. Both readers of the flag gate on `is True`, so nothing bills or behaves differently either way. Only the cached fields change on the two 09-2025 preview entries. Their source field points at the Gemini API pricing page rather than the Vertex one, so they describe a different surface with its own published limits, and their context windows are left alone rather than assumed to match the Vertex model card that drives the GA entry. Tests cover all three halves: the family invariant that a cached audio read implies an equal cached audio write, a cached count on a Live entry leaving the bill at the fresh-input total instead of adding the old 7.5e-08, and supports_prompt_caching answering false for all three entries while still answering true for 2.5 Flash, so the false cannot be a swallowed lookup error. * fix(cost): correct gemini-live-2.5-flash-native-audio limits and capabilities Google's model card for model ID gemini-live-2.5-flash-native-audio gives a 128K context window and 64K maximum output tokens, and marks structured output, context caching and URL context as not supported. Its modality list is text in and out, image in, audio in and out, and video in, with no document input of any kind. The entry advertised a 1M context window, an off-by-one 65535 output cap, and three capability flags the vendor marks unsupported. Context caching is the fourth and is handled in the cached-fields change alongside its two preview siblings. Both the bare id and vertex_ai/gemini-live-2.5-flash-native-audio resolve to this single entry, so the test drives the corrected values through both. * test(integration): cover live preview cached tokens billed at the fresh rate Co-authored-by: Marty Sullivan <marty@martysullivan.com> Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(cost): cite dated sources for Live entry pins and drop restating docstrings Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Marty Sullivan <marty@martysullivan.com> Co-authored-by: kerry <kerry@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
berriai-litellm-provider-info-sync[bot]
|
5dfaa8d620
|
chore(prices): sync AWS Bedrock prices and sources from the AWS price list (#42632)
* chore(prices): sync AWS Bedrock prices: 3 models [sync failed: AWS Bedrock] anthropic.claude-opus-5-5: supports_audio_input, supports_response_schema global.anthropic.claude-opus-5-5: supports_audio_input, supports_response_schema us.anthropic.claude-opus-5-5: supports_audio_input, supports_response_schema * chore(prices): sync AWS Bedrock prices: 32 models ai21.j2-mid-v1: ai21.j2-ultra-v1: ai21.jamba-1-5-large-v1:0: ai21.jamba-1-5-mini-v1:0: ai21.jamba-instruct-v1:0: au.anthropic.claude-opus-4-7: au.anthropic.claude-opus-4-8: au.anthropic.claude-opus-5: au.anthropic.claude-sonnet-4-6: au.anthropic.claude-sonnet-5: cohere.command-light-text-v14: cohere.command-text-v14: input_cost_per_token cohere.embed-english-v3: cohere.embed-multilingual-v3: cohere.embed-v4:0: eu.anthropic.claude-fable-5: eu.anthropic.claude-opus-4-7: eu.anthropic.claude-opus-4-8: eu.anthropic.claude-opus-5: eu.anthropic.claude-sonnet-4-6: eu.anthropic.claude-sonnet-5: jp.anthropic.claude-opus-4-7: cache_creation_input_token_cost_above_1hr jp.anthropic.claude-opus-4-8: jp.anthropic.claude-opus-5: jp.anthropic.claude-sonnet-4-6: jp.anthropic.claude-sonnet-5: meta.llama2-13b-chat-v1: meta.llama2-70b-chat-v1: us.writer.palmyra-x4-v1:0: us.writer.palmyra-x5-v1:0: writer.palmyra-x4-v1:0: writer.palmyra-x5-v1:0: Price-Sync: litellm-providers * fix(bedrock): correct eu.anthropic.claude-opus-4-5 regional prices from the AWS price list Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore(prices): sync AWS Bedrock prices: 30 models anthropic.claude-haiku-4-5-20251001-v1:0: anthropic.claude-opus-4-5-20251101-v1:0: anthropic.claude-opus-4-6-v1: anthropic.claude-sonnet-4-20250514-v1:0: anthropic.claude-sonnet-4-5-20250929-v1:0: apac.anthropic.claude-sonnet-4-20250514-v1:0: au.anthropic.claude-haiku-4-5-20251001-v1:0: au.anthropic.claude-opus-4-6-v1: au.anthropic.claude-sonnet-4-5-20250929-v1:0: eu.anthropic.claude-haiku-4-5-20251001-v1:0: eu.anthropic.claude-opus-4-5-20251101-v1:0: eu.anthropic.claude-opus-4-6-v1: eu.anthropic.claude-sonnet-4-20250514-v1:0: eu.anthropic.claude-sonnet-4-5-20250929-v1:0: global.anthropic.claude-haiku-4-5-20251001-v1:0: global.anthropic.claude-opus-4-5-20251101-v1:0: global.anthropic.claude-opus-4-6-v1: global.anthropic.claude-sonnet-4-20250514-v1:0: global.anthropic.claude-sonnet-4-5-20250929-v1:0: jp.anthropic.claude-haiku-4-5-20251001-v1:0: jp.anthropic.claude-sonnet-4-5-20250929-v1:0: mistral.voxtral-mini-3b-2507: mistral.voxtral-small-24b-2507: us-gov.anthropic.claude-sonnet-4-5-20250929-v1:0: us.anthropic.claude-haiku-4-5-20251001-v1:0: us.anthropic.claude-opus-4-1-20250805-v1:0: us.anthropic.claude-opus-4-5-20251101-v1:0: us.anthropic.claude-opus-4-6-v1: us.anthropic.claude-sonnet-4-20250514-v1:0: us.anthropic.claude-sonnet-4-5-20250929-v1:0: Price-Sync: litellm-providers * fix(bedrock): keep Claude Opus 5.5 response schema support as a maintainer ruled in #42626 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(bedrock): stop pinning the jp Opus 4.7 cache field absence in the ttl fallback test Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * chore(prices): sync AWS Bedrock prices: 3 models anthropic.claude-opus-5-5: supports_audio_input, supports_response_schema global.anthropic.claude-opus-5-5: supports_audio_input, supports_response_schema us.anthropic.claude-opus-5-5: supports_audio_input, supports_response_schema Price-Sync: litellm-providers * fix(bedrock): keep Claude Opus 5.5 response schema support per the #42626 ruling, reverting the cron restack Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: berriai-litellm-provider-info-sync[bot] <328147090+berriai-litellm-provider-info-sync[bot]@users.noreply.github.com> Co-authored-by: kerry <kerry@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
41ca4656cd
|
test(e2e): tolerate provider-side flakes on five full-suite cells (#42628)
* test(e2e): tolerate provider-side flakes on five full-suite cells Mistral OCR retries a provider-relayed 429 with backoff, the Vertex vision probe turns reasoning off so its 32 tokens go to the answer, the Vertex cache cell spaces eight never-seen prefixes 15s apart around Google's nondeterministic minimum-token rejection and prices the cached tokens instead of prompt_tokens, and the Azure content-policy cell resends the jailbreak prompt while Azure skips its filter * test(e2e): shorten the new helper docstrings * test(e2e): accept a relayed provider 429 on the rust OCR cells The gateway already retries a provider 429 three times per call and the Mistral key is shared across pipelines, so a throttle can hold across all four attempts of the OCR cell. After the bounded retries the cell now accepts the gateway's faithful relay of the provider's 429 (throttling_error, code 429) as its second expected outcome; the gateway's own 429 and any other error still fail the cell at once. * test(e2e): drop the harness unit tests, the live cells cover the helpers --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
5c24802fbd
|
fix(bedrock): send json_schema as a forced tool on Claude Opus 4.7 and 4.8 Converse (#42644)
* fix(bedrock): send json_schema as a forced tool on Claude Opus 4.7 and 4.8 Converse Bedrock rejects outputConfig.textFormat on Opus 4.7 and 4.8 with "output_config.format: Extra inputs are not permitted", and the AWS model cards list structured outputs as not supported for both, so their cost-map entries no longer claim supports_native_structured_output and json_schema requests fall back to the json_tool_call tool. Fixes #27846 * test(bedrock): assert Opus 4.7 and 4.8 inline the schema on Invoke, move the native case to Sonnet 4.6 --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
975bd28549
|
fix(bedrock): stream /v1/messages Invoke bytes through instead of holding them in a 1024-byte chunker (#42607)
* fix(bedrock): stream /v1/messages Invoke bytes through instead of holding them in a 1024-byte chunker Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style(bedrock): apply ruff format to invoke messages stream passthrough Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style(bedrock): drop drive-by reformat of existing invoke messages tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(bedrock): collect streamed chunks into a tuple in passthrough regression test Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(bedrock): give the passthrough regression test a 10s first-chunk budget * test(bedrock): type the eventstream frame helper's payload as Mapping[str, object] * test(bedrock): take the gated byte stream's chunks as an immutable Sequence --------- Co-authored-by: mateo <mateo@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
1bc7d34a84
|
chore(e2e): move the compat-matrix populator from a GCE VM to a Render cron job (#42608)
* chore(e2e): move the compat-matrix populator from a GCE VM to a Render cron job The daily Claude Code compatibility-matrix job ran as a systemd timer on the litellm-compatibility-matrix-populator VM in the vertex-check GCP project. Replace that with a Render Docker cron job built from a new Dockerfile in tests/e2e/claude_code/cron_vm: pinned and checksummed debian base, gh, uv, and Claude Code CLI, a non-root populator user, and run_daily.sh as the entrypoint. run_daily.sh now clones a fresh blobless checkout per run (Render cron disks are ephemeral), reads the publish PAT from the github-token secret file under CREDENTIALS_DIRECTORY, and its comments no longer describe systemd. The .service and .timer units are gone; README.md and the env example describe the Render service, its secret files, and the local docker build instead. * docs(e2e): name the plan and trigger route Render's cron-job API accepts Render answers a bare 404 for the legacy pro_max plan name on a cron job (4c-16g is the same 4 CPU / 16 GB size) and the manual trigger route is /v1/cron-jobs, not /v1/cronjobs. * fix(e2e): install the published litellm wheel instead of building the tag from source The tag builds a Rust extension through maturin, which needs a C and Rust toolchain the cron image does not carry, so the first Render run failed at uv sync with "linker cc not found". Sync the locked dependencies with --no-install-project, install the PyPI wheel (what users run) with --no-build, and pass --no-sync to every uv run so uv never puts the source build back. * fix(e2e): keep the SKIP_PUBLISH matrix where a Render run can read it The validation run wrote the matrix into the image checkout, which nobody can read once the container exits. Save it under HOME and print it at the end of the log instead. * fix(e2e): let the stale compat-matrix PR sweep see past the newest 100 docs PRs The docs repo has a few hundred open PRs, so a 100-item list never reached the week-old compat-matrix PR and the sweep left it open on every run. * docs(e2e): say the Render cron needs a manual deploy after each merge Pushes never started a deploy during setup because Render only hears about them through its GitHub app, which the org does not have, so the README now carries the deploy command and the wait-for-live rule * ci: build the compat-matrix cron image on pull requests The CI coverage gate requires every Dockerfile to be built by a job, and building this one on each PR that touches it also catches a broken pin or checksum before Render does * fix(e2e): shim the whole tests/e2e tree into the compat-matrix worktree The five-file helper allowlist missed fixture_mode, which e2e_config now imports, so the first Render run died at conftest load with ModuleNotFoundError. Copy the image's whole tests/e2e tree instead and keep pytest from loading the EKS-harness conftest with --confcutdir * fix(e2e): scope the compat-matrix sweep to the publishing account's own PRs The stale-PR sweep selected every open docs PR whose head branch starts with compat-matrix/, so a contributor's fork PR under that name would have been closed once a newer matrix PR existed. The sweep now resolves the publishing login from the token and only closes same-repo PRs that account opened --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
yuneng-jiang
|
b395bfefdd
|
fix: repair seven regressions caught by CircleCI on main (#42640)
* fix: repair seven regressions caught by CircleCI on main - vertex_ai: stop treating fine-tuned endpoint ids (numeric or vertex_ai/gemini/<id>) and gemma models as Gemini 3+, which injected temperature=1.0 and Gemini 3 thinking config into their requests (#42465) - cost: price Azure DALL-E 3 from its azure/<quality>/<size>/dall-e-3 rows; it only worked through the OpenAI rows that #42435 removed - bedrock: stream bedrock/invoke/moonshot through an OpenAI-shaped chunk decoder; the generic decoder dropped every chunk, which the supports_response_schema flag from #42338 un-skipped in CI - proxy: keep the public model_group on pre-routing rejections so the Usage page groups them under the model name, not the deployment (#41077) - cost map: mirror the base rows' capability flags onto Bedrock regional and cross-region copies (#42254 and later syncs) - whitelist the new regional Bedrock rows from #42543 and #42588 for the converse routing check, following the existing regional-row convention * fix(model-prices): mirror capability flags onto ap-southeast-3 bedrock rows * refactor(bedrock): tighten types on the moonshot stream decoder and its tests |
||
|
devin-ai-integration[bot]
|
5fa1257b7c
|
test(utils): raise the post-success hook error from a guardrail in the failure-hook regression (#42646)
* test(utils): raise the post-success hook error from a guardrail in the failure-hook regression Since #42535 a plain logger raising inside async_post_call_success_deployment_hook is logged and the completed request returns, so the regression added by #36657 for "a post-success error never reaches async_post_call_failure_deployment_hook" failed with DID NOT RAISE on every main run once #42603 revived the misc unit shard. The raising callback is now a CustomGuardrail, the one kind of callback whose post-success raise still propagates, which keeps the original assertions intact * test(utils): type the guardrail's success-hook request_data as a Mapping --------- Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
Emerson Gomes
|
30004f5f05
|
fix(bedrock): drop unsupported sampling params on converse reasoning models (#39834)
* fix(bedrock): drop unsupported sampling params on converse reasoning models * test(bedrock): resolve duplicate import after rebase |
||
|
devin-ai-integration[bot]
|
b0ac23d385
|
feat(logger): dispatch Python logging through the Rust diagnostics processor (#42616)
* feat(logger): add shared Rust diagnostics and Python logging bridge * feat(logger): dispatch diagnostic processing through Rust * chore: regenerate Cargo.lock after rebase Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: allowlist bounded logging tree walkers in recursive detector Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * perf(logger): skip decoding plain access arguments * test(logger): skip embedded-python logger test when litellm deps are absent Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style: cargo fmt Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test: expect NativeDiagnosticProcessor in the native public surface Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(stub): export NativeDiagnosticProcessor via __new__ in _native.pyi Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * refactor(tracing): rename logger crate and document host sink contract * test(logger): cover exc, stack, and nested extras in the diagnostic filter Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(logger): keep rendered redacted line when template scan flags a key pattern The blanket REDACTED for a changed msg/color template discarded lines whose rendered form was already redacted by the same pipeline, e.g. 'password=%s' became 'REDACTED' instead of 'password=REDACTED'. Only fall back to REDACTED when the rendered form did not change either, which is where interpolation can mangle the key pattern the scrub would otherwise see. Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * ci(rust): install python deps so the logger bridge test runs The end-to-end bridge test skipped silently when litellm's Python deps were absent. uv sync --no-install-project installs them without a maturin build, and PYTHONPATH makes them visible to the embedded interpreter Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Yujong Lee <yujong@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
2d2b7e8fa0
|
test(cost): repoint the Azure image cost test at gpt-image-2 (#42631)
#42435 removed the OpenAI dall-e-3 cost-map row that test_dalle_3_azure_cost_tracking pinned, and Azure retired DALL-E 3 inference on 2026-03-04, so the test now mirrors a real Azure gpt-image-2 response and derives the expected cost from the cost map. Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
327515a3ba
|
fix(mcp): preserve credential authority in DCR bridge authentication (#42563)
* fix(mcp): admit dcr_bridge envelope alongside an explicit litellm credential and mint under jwt principals Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * style(mcp): suppress LIT002 on concrete dict header payloads Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(mcp): mint bridge envelope for jwt mapped to a key without a user_id Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(mcp): mint and admit bridge envelopes under the master key Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(mcp): bind mapped JWT envelopes to stored key tokens * fix(mcp): preserve master envelope scope enforcement * fix(mcp): reject bridge minting that loses JWT restrictions --------- Co-authored-by: joshua <joshua@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com> |
||
|
moe-berri
|
2157351004
|
feat(ui): simplify auto-router setup and clarify feature limits (#42625)
* feat(ui): simplify auto-router setup and clarify feature limits * fix(ui): validate auto-router drafts before saving * fix: keep auto-router allowances consistent after deletes and refreshes |
||
|
yujonglee
|
630c4624f6
|
test(e2e): add secret manager lanes for HashiCorp Vault and CyberArk Conjur (#42503)
* test(e2e): add a HashiCorp Vault secret manager lane key_management_system had no end-to-end coverage: the Rust crates and the Python unit tests all run against mocked managers. This adds a secret_manager suite that drives a proxy configured with hashicorp_vault against a real Vault. The tests seed a fresh secret name per test with the runner's OPENAI_API_KEY and register a deployment pointing at os.environ/<name>. The proxy's env never holds that name, so get_secret's os.environ fallback cannot mask a broken manager, and a bogus value in Vault must come back as the provider's 401. Virtual keys are checked written to and removed from Vault under prefix_for_stored_virtual_keys. The setting is global to the proxy, so the lane has its own config and the secret_manager_vault opt-in marker, and stays out of the per-PR selector. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): make the secret manager suite backend-agnostic One marker and opt-in (secret_manager / E2E_SECRET_MANAGER=<system>) pick the backend from secret_backends.BACKENDS. The tests reach the manager through a SecretStore protocol, and each backend contributes a secret_store_<system>.py module, a registry entry, and gateway/secret_manager_<system>_ci_config.yml. requires_capability deselects tests a backend cannot support (CyberArk does not delete), and test_secret_backends.py checks every lane config against its backend without a live stack. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(e2e): add a CyberArk Conjur secret manager lane Adds cyberark as the second secret_manager backend: a Conjur store over its REST API (policy-declared variables, raw-text values, policy-patch teardown), its lane config, and a registry entry without deletes_stored_keys, since the proxy's CyberArk delete answers not_supported and Conjur keeps the key. secret_manager/backend.sh up|down <system> boots any backend in Docker and writes proxy.env and tests.env, so every lane runs the same way; the registry test checks the script boots exactly the registered backends. e2e_http gains send_text_external for APIs that speak raw text rather than JSON. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(e2e): give the secret manager suite a client with .proxy and address review The shared resources fixture reads client.proxy, so a bare ProxyClient errored every live test at setup. backend.sh now writes its env under a per-user directory with umask 077, the markerless unit tests are gone per tests/e2e/AGENTS.md, and routine comments are trimmed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
devin-ai-integration[bot]
|
944f44d82b
|
fix(utils): isolate callback errors in async_post_call_success_deployment_hook (#42535)
* fix(utils): isolate callback errors in async_post_call_success_deployment_hook A callback that raises inside async_post_call_success_deployment_hook no longer fails the completed request. The exception is logged with the callback class and call_type, the response stays as it was, and later callbacks still run. Guardrail callbacks are exempt because raising is how a post-call guardrail blocks Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(utils): drop unrelated ruff autofixes from test_utils Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(utils): drop fastapi import from guardrail propagation regression Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(utils): cover every success deployment hook call type with a raising hook Parametrize the unit regression over video, embedding, responses, image, rerank, transcription, chat and anthropic messages responses and assert the failure log names the callback and call type. Run the integration test through a real proxy for /v1/chat/completions, /v1/embeddings, /v1/responses and /v1/videos Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(integration): move raising success hook cases into the existing callback delivery file Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: yucheng <yucheng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
a80379baf8
|
fix(proxy): keep the in-flight daily spend batch when shutdown cancels the flush (#42593)
* fix(proxy): keep the in-flight daily spend batch when shutdown cancels the flush A daily spend batch drained from the in-memory queue was dropped for good when the scheduler tick was cancelled by shutdown, because asyncio.CancelledError bypasses the except Exception requeue. The flush now requeues the drained rows on cancellation and re-raises, and each daily batch upsert runs in an interactive transaction so a statement that already reached Postgres is rolled back with the cancel instead of committing behind the requeue Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): requeue the cancelled daily spend batch before its rollback returns Behind a lock the rollback of the cancelled interactive transaction only returns once the blocked statement does, which is after the shutdown flush has already run. The commit now runs as a shielded task so the cancelled tick requeues the batch at once and lets the rollback finish in the background. The final flush then finds the rows and writes them exactly once Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): give the recording db a transaction seam for the bulk upsert tests Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(proxy): route the mocked daily tag spend upsert through the transaction seam Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(proxy): restore the drained Redis tag batch when shutdown cancels its commit Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: yucheng <yucheng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |
||
|
devin-ai-integration[bot]
|
21a2d828df
|
ci(test-unit): drop dead misc shard paths and skip missing paths with a warning (#42603)
Eight directories the misc shard named moved to tests/unit on 2026-09-20, and one
missing path makes pytest-xdist collect [0 items] for the whole shard, which the
exit-5 tolerance turned into a green required check running nothing. The shared
Run tests step now drops a path that does not exist with a :⚠️: and runs
pytest over the rest, keeping option tokens verbatim.
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
|
||
|
devin-ai-integration[bot]
|
8ee6bab529
|
fix(bedrock): treat blank AWS_S3_* env vars as unset for batch jobs (#42528)
* test(e2e): pin bedrock batch create with blank S3 env vars Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix(bedrock): treat blank S3 env vars as unset for batch jobs Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(e2e): trim blank S3 env gateway config Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(e2e): register blank_s3_env capability and clean gateway tempdir Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * test(e2e): move blank S3 env batch test to its own module Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: yucheng <yucheng@berri.ai> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> |