* fix(vertex_ai): keep batch output_file_id null until Vertex reports outputInfo
Vertex only sets outputInfo.gcsOutputDirectory once a batch job has written
output. Falling back to outputConfig's outputUriPrefix named the per-model
directory shared by every batch of the deployment, an object that never
exists, so the proxy minted a managed file for it under the first key and
every other key's file calls on that id were 403s
* fix(vertex_ai): treat a null gcsOutputDirectory as no output file yet
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Prisma types a raw array parameter from the first batch a connection sees. After a flush in
which every member cost was a whole number (a free model), the connection's cached statement
expected int8[] and every later fractional batch on it failed with "improper binary format in
array element", so member spend silently stopped landing while team spend kept rising.
The rows now travel as one JSON document unpacked by jsonb_to_recordset with the column types
declared in SQL, so Postgres types the numbers and the batch shape no longer matters.
* feat(proxy): add uncapped server-side team usage export route
GET /team/daily/activity/export answers the same scoping as
/team/daily/activity/aggregated with one unbounded rollup query, so keys
past USAGE_TOP_API_KEYS_LIMIT are included. Supports daily,
daily_with_keys, daily_with_users and daily_with_models export types as
CSV (default) or JSON. The PTU flat-cost sentinel stays in the plain
daily rollup and is excluded from the keyed and per-model exports
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): export team usage server-side when the key list was truncated
When the aggregated spend response reports api_key truncation, EntityUsage
passes a serverExport into the export modal that downloads CSV or JSON
from GET /team/daily/activity/export instead of building the file from
the truncated on-screen data. apiClient gains a responseType option so
the download can arrive as a Blob, and truncation no longer blocks the
export button
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): cover team usage export types, sentinel handling and scope
Unit tests pin the uncapped key rollup past USAGE_TOP_API_KEYS_LIMIT,
PTU sentinel inclusion in the daily rollup and exclusion elsewhere, the
per-user fold, and the CSV column layout. Integration tests exercise the
route against a live proxy, including member scope denial
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): tidy team usage export route
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): use membership test for export type branch (PLR1714)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(ui): format exportBlockedReason test with prettier
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): satisfy type-discipline gate in team usage export
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): pass export rows as a sequence to the response model
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy-behavior): cover team usage export in the daily activity scope matrix
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): carry PTU flat cost and escape formulas in team usage export
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): keep the truncation export block on surfaces without a server export
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): drop redundant comments in team export call and modal test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): audit cells for team usage export
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): tighten team usage export audit cells
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): type the export params tuple and fold user keys in one pass
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(ui): bring entity usage export helpers under eslint budgets
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(ui): prettier-format UsagePageView after merge
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): route unmapped openai family model ids to converse
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(bedrock): rename the e2e openai family backend constant
global.openai.gpt-6-sol has a cost-map row now, so the constant no longer
names an unmapped model
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(logging): pass provider response headers to callbacks on every endpoint
Custom callbacks only received kwargs["response_headers"] for chat
completions. Responses, image generation and edit, speech, and
transcription calls either never recorded the provider's headers or
recorded them in one place and not the other.
Every handler now records the provider's httpx headers on the response's
hidden params as "headers" (raw) and "additional_headers" (processed,
with LiteLLM's own entries winning on a clash), and the logging object
derives model_call_details["response_headers"] from those hidden params
before cost calculation on the non-stream and both streaming success
paths, keeping a handler-set value authoritative. Binary speech responses
expose their hidden params to the standard logging payload, and the sync
OpenAI transcription request always fetches the raw response.
* test(images): point the legacy image and speech fakes at the raw response surface
Image generation now goes through the SDK's raw response so the provider headers can be read, and the speech binary response now carries hidden params. The unit fakes in the image generation, xinference, proxy provider, image edit, Vertex speech, and otel suites still pinned the old call surface and the old "no hidden params" assertion, so they read an uncalled mock or a fake response without headers.
* test(images): drop the rewritten mock comments and the generated edit PNGs
* test(images): move the llm-span test's image fake to the raw response surface
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat(vertex): native batch JSONL passthrough with cost tracking
Add a per-request `passthrough=true` multipart field on `POST /v1/files`
(and the same kwarg on `litellm.create_file`) that uploads a native
Vertex AI batch JSONL to the deployment's GCS bucket unchanged, so rows
using `googleSearch` and other Gemini-only features run as written and
the output, `groundingMetadata` included, comes back untouched.
Passthrough is sticky through the GCS object path
(`litellm-vertex-files/passthrough/...`), so batch create and output
retrieval inherit it without new state. Native output rows are costed
from their `usageMetadata` with the deployment's model and model_info,
in the polling and retrieve paths and for the existing global
`disable_vertex_batch_output_transformation` flag, which billed $0
before.
The proxy requires the target to resolve to vertex_ai deployments only,
refuses `passthrough` with a non-batch purpose, a non-default
`target_storage`, or pre-call guardrails, and validates native rows on
`request` instead of the OpenAI batch keys.
* refactor(vertex): keep native batch row pricing inside the Vertex adapter
Moves native Vertex batch row detection, response parsing, and per-row
pricing from litellm/batches/batch_utils.py into
litellm/llms/vertex_ai/batches/transformation.py, so batch_utils only
aggregates the rows it gets back. Adds tests/test_litellm/files to the
misc unit shard so the new test directory is claimed by a shard.
* fix(files): say what a passthrough batch upload takes when a row is not native
The missing-key 400 listed bare key names, so an OpenAI-shaped row under
passthrough=true read "Each line must be a JSON object with keys request".
The batch line shape now carries its own hint, and the passthrough one says
a passthrough upload takes native Vertex batch rows with a request key
* fix(batches): bill native Vertex embedding batch rows on the native cost path
A native Vertex output row whose response holds an embedding was validated as a
generateContent response, so the documented tokenCount-only shape counted as a failed
row. Price embedding rows from their own usage (promptTokenCount, else tokenCount) with
the helper the transformed embeddings path already used, and drop the prompt-details
helper nothing calls anymore.
* fix(batches): keep modality batch rates on native Vertex embedding rows
An embedding row that carries usageMetadata was billed from promptTokenCount alone, so
its promptTokensDetails no longer reached the audio, image, and video batch rates the
way it did before the native cost path. Run every row with usageMetadata through the
Gemini usage parser and keep the flat tokenCount fallback for embedding rows without it.
* fix(batches): price native Vertex batch rows by modelVersion under a wildcard deployment
A `vertex_ai/*` deployment hands the batch cost path `*` as the deployment model, which
no cost map resolves, so every native (passthrough or flag-on) row was billed at $0. A
wildcard deployment model now defers to the row's own `modelVersion`, the way the
transformed path already prices by the row's `model`.
Also moves the native passthrough tests under tests/test_litellm, the tree codecov
reads, and covers the raw upload chunking, the embedding output translation, the
unpriceable-row path, and the flag-on dispatch.
* fix(batches): keep explicit deployment prices for native Vertex rows without a modelVersion
Under a wildcard deployment a native batch row that carries no modelVersion (an embedding
row, or a generateContent row Vertex returned without one) was billed at $0 even when the
deployment's model_info sets explicit batch prices, because the cost calculator was never
called. The row now falls back to the wildcard name, which the cost calculator prices from
the explicit model_info, and only a row with neither a modelVersion nor a deployment model
is billed at $0 with the warning
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(prometheus): add model_group label to deployment request and rate limit metrics
litellm_deployment_total_requests, litellm_deployment_success_responses,
litellm_deployment_failure_responses, litellm_deployment_tpm_limit and
litellm_deployment_rpm_limit had no way to identify which model_group a
pooled deployment belongs to, only requested_model, litellm_model_name and
model_id, none of which name the alias a model_name resolves through when
it fans out to more than one deployment.
model_group was already resolved onto enum_values for every request in
async_log_success_event, so this is a label-list addition for the metrics
built directly from that enum_values (the two request counters). The
failure counter builds its own UserAPIKeyLabelValues locally and had a
model_group variable already in scope that it never passed through, and
the tpm/rpm limit gauges are set from a helper that took no model_group
parameter at all even though its only caller already had it on
enum_values. Both now thread the value through.
* test(prometheus): expect model_group in deployment success/total request labels
test_set_llm_deployment_success_metrics asserts the exact label set passed
to litellm_deployment_success_responses.labels() and
litellm_deployment_total_requests.labels(), which now includes model_group
since it was added to those metrics' label list.
* fix(prometheus): bound model_group on deployment failure metrics
On a pre-routing reject (no deployment selected), model_group is
caller-supplied via litellm_params.metadata and was passed through
unbounded, letting an unrecognized value mint unlimited label series on
litellm_deployment_failure_responses / litellm_deployment_total_requests.
Bound it with the same _bounded_requested_model_label used for
requested_model on this path. When a deployment is actually selected,
model_group is router-resolved and passed through as-is.
Also documents the model_group parameter on
_set_deployment_tpm_rpm_limit_metrics and the bounding behavior on
set_llm_deployment_failure_metrics.
---------
Co-authored-by: ahamedshaik16 <24526479+ahamedshaik16@users.noreply.github.com>
* test(rust): encode anthropic response serialization shape as rstest cases
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* feat(rust): shape Anthropic Messages requests natively
The Rust Messages route only relayed the body. It now runs the request shaping
the Python handler does for the direct Anthropic provider: history sanitizers
(empty blocks, tool ids, replayed web search results, provider_specific_fields,
encrypted reasoning, advisor blocks), reasoning_effort and adaptive/legacy
thinking translation against the model's capability flags, the sampling and
speed gates under drop_params, the metadata allowlist, additional_drop_params,
reasoning auto summary, OAuth and ANTHROPIC_AUTH_TOKEN credentials,
provider_specific_header merging and anthropic-beta injection. Capability flags
and LiteLLM settings reach Rust through route_host.shaping(). A request the
route rejects before the call now maps to BadRequestError instead of
APIConnectionError
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* test(rust): port Anthropic Messages shaping tests and pin comment contracts as cases
Every Python unit test that exercises the ported shaping for the direct
Anthropic provider now has a named rstest counterpart, and every comment that
stated a behavior contract is deleted in favor of a case that pins it. Measured
with cargo-mutants over the touched files, all viable mutants are caught
Porting the tests surfaced parity gaps, fixed here to match Python: every
casing of a forwarded anthropic-beta header is merged, replayed web search
results are rewritten from their own block (an empty result keeps its slot and
a server_tool_use with a non-string query stays), an empty output_config.effort
falls back to medium, speed and reasoning effort errors quote values the way
Python does, additional_drop_params apply after metadata validation and the
auto summary and never touch model or messages, and a non-string
metadata.user_id is rejected before the call
* fix(rust): resolve Messages credentials through the secret source and scope headers by resolved provider
The native Messages route read ANTHROPIC_API_KEY, ANTHROPIC_AUTH_TOKEN and the
base URL straight from the process environment, so a key or base held in a
configured secret manager was never found. Each provider config now declares
its secret names and the route resolves them through the same SecretSource the
OCR route uses, with the Python bridge passing in litellm's configured manager
provider_specific_header entries were scoped by the explicit
custom_llm_provider only, falling back to anthropic, so an azure_ai/ model
lost its azure_ai scoped headers. Scoping now happens in the route after the
provider is resolved from the model, as Python's handler does
The Azure config now adds the same anthropic-beta feature headers Python's
Azure route adds, and the metadata allowlist, reasoning auto summary and
history sanitizers move from the core route into the llms crate, mirroring
their home in Python's messages handler
* test(rust): escape the dot in the metadata.user_id match pattern
* refactor(rust-bridge): project Messages capabilities without mutable dicts
The capability flags and effort tiers were built as dict comprehensions,
which the type-discipline gate counts as mutable construction, and the
asdict call carried a mutable-ok suppression that suppressed nothing. The
flags are now passed one by one and the effort tiers are a frozen dataclass,
which asdict projects to the same map the native side reads
---------
Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* test(guardrails): run the cache-hit redis outage test on the shared owned_redis helper
test_redis_outage_keeps_serving_in_memory_hits (#42780) spawned `redis-server`
straight from PATH. The CircleCI integration machine has no redis-server
binary, so the test died with FileNotFoundError before reaching the proxy.
Use integration._support.redis_process.owned_redis, which runs the local
binary when there is one and otherwise the job's redis-cache container, and
take the outage with its stop()/start() pair the way test_redis_recovery
already does. The assertions are unchanged.
* test(guardrails): describe the owned_redis stop as an outage, not a kill
#42791 switched the MCP management endpoints from create_mcp_server to
create_mcp_server_if_identifier_free, which keeps the same arguments and
returns the created row on success. Two tests in
tests/store_model_in_db_tests/test_mcp_servers.py still patched the old name,
so mock.patch raised AttributeError before the tests ran and
proxy_store_model_in_db_tests has been red on main since.
test_malformed_bodies_missing_users_and_foreign_servers_are_rejected used
`body` as a comprehension variable and then called the module-level `body()`
helper a few lines later. CPython 3.12.2, which the CircleCI integration job
runs, compiles that later call as a local read, so the test raised
UnboundLocalError on every integration-mcp run since #42652. Newer 3.12
patch releases and 3.13 compile it as a global read, which is why it passes
locally.
Renaming the comprehension variable makes both reads unambiguous.
The new CircleCI tests pipeline (#42773) runs tests/unit under pytest-cov on
CPython 3.12.2, where coverage traces every line through sys.settrace. The two
tracemalloc peak comparisons in test_vertex_ai_files_streaming.py drive 8000-row
payloads through both pipelines and slow from ~10s to over 3 minutes under that
tracer, so both hit the 90s pytest-timeout on every run.
Mark them no_cover so pytest-cov pauses tracing for just these two. Their
assertions are unchanged and every other test in the file still reports coverage.
#42463 made Logout revoke the dashboard session key on the server. Both
logout specs ran on the shared ADMIN_STORAGE_PATH session that globalSetup
mints once, so clicking Logout revoked the key every later admin spec reuses.
The CircleCI run is serial, and from the auth/ folder on, every admin-session
spec failed with "Invalid proxy server token passed" (80 failures, up from 7)
while the internal-user, internal-viewer and team-admin specs kept passing.
Each logout spec now starts from an empty storage state and logs in through
the login page, so the session it revokes is its own. The login steps live in
a shared logInThroughLoginPage helper next to the other onboarding helpers.
* fix(ui): pass is_proxy_admin for proxy admins on the models page team drill-in
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): drop explanatory comments from the models page team drill-in tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): exclude view-only admins from is_proxy_admin on the models page team drill-in
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ui): add browser integration contract for the team guardrail kill switch on the models page
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
test_ollama_image returned the request's images list as the mocked `response`
field and read it back from message content. Since #42838 the completion
transform validates `response` as a string, so the list is dropped and the test
fails with "string index out of range". Ollama always sends a string there.
The mock now returns a real string reply and the test asserts on the images the
transform actually sent, which is what it was checking all along.
Non-streaming responses served by the Rust core already carry
x-litellm-rust: true through _hidden_params.additional_headers, which the
SDK exposes and the gateway renders as a response header. Native streams
did not, because the lifecycle Stream and SyncStream objects had nowhere
to hold hidden params and the marker writer skips objects without them.
Give both stream classes the same _hidden_params bag every other litellm
response has, so the existing marker attaches without wrapping the stream
or changing its identity.
Co-authored-by: Yujong Lee <yujong@berri.ai>
* refactor(types): replace Any with proven types in 13 files
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(types): drop unused executor import from utils type-checking block
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(realtime): keep reserved-key filtering on azure realtime health params
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(realtime): pin reserved-key filtering in azure realtime health auth params
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(realtime): exercise the real azure header builder in the reserved-key test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): fail parked DB lookups at a deadline and flip readiness while they stall
Under a load burst with a slow authentication database every request parked inside
the pod with no deadline while /health/readiness kept answering 200 (its own ping
gets a fresh connection), so the load balancer kept sending traffic until the pod hit
its memory limit, and the parked requests completed against the provider minutes
after every client had hung up
Every pre-request read (key, team, user, end user, budget, membership, organization,
object permission, jwt mapping, project, proxy budget, spend counter reseed) now runs
under one deadline, PROXY_DB_LOOKUP_DEADLINE_SECONDS (default 10 s). A lookup that
hits it fails the request with the existing 503 "authentication database is
temporarily unreachable" answer, honours allow_requests_on_db_unavailable, and never
triggers the transport reconnect (the transport is fine, the query is slow), which is
what turned the repro's stall into "too many clients". Writes stay unbounded
A deadline hit marks the pod stalled for PROXY_DB_LOOKUP_STALL_WINDOW_SECONDS
(default 30 s, 0 disables), during which /health/readiness answers 503 with
"db": "stalled" behind the same fail-open gate, so the pod leaves rotation before it
fills its memory. The existing litellm_in_flight_requests gauge already exposes the
parked set on /metrics
The deadline is enforced on the wall clock: bounded_db_lookup waits on the lookup
task with asyncio.wait and raises DBLookupDeadlineExceeded when the deadline passes
even if the lookup absorbs its cancellation, where asyncio.wait_for on 3.12+ would
sit on the cancelled task for as long as it takes
The failure spend-log row no longer re-runs the key and team lookups when the
failure itself is a database connection or deadline error, so a request that hit
the deadline is answered after one deadline instead of two
* fix(proxy): bound the spend counter gate wait and narrow the stalled lookup shortcut
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep the global spend lookup on the prisma client handle
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): list key and team model aliases in GET /v1/models
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep alias listing helpers within the type discipline budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): cover alias rows on GET /v1/models and /v1/models/{id}
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): apply team then key aliases like chat completions and keep the alias as the retrieved id
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): apply key aliases twice like chat completions and skip only malformed alias entries
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): apply the global model_alias_map between the key alias passes like chat completions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): list only the caller's own aliases and never rewrite a listed model id on retrieval
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(proxy): ruff format model_info alias lookup
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): hide undiscoverable names from model retrieval so an alias named like one resolves to its target
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep undiscoverable models retrievable by id while excluding them from the alias guard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): pass an immutable name sequence into the model_info alias guard
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): type the model list alias test helpers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): annotate the new alias listing test fixtures and helpers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): edge-case matrices for malformed token limits and callback_settings shapes
Extends the integration suite so two classes of issues found by gauntlet
reviews are caught end to end against a real proxy:
- non-numeric or odd model_info token limits (from /model/new and from
config YAML) must be listed as absent on /v1/models, /models,
/v1/models/{id} and /model/info, keep sibling models listed, and still
serve chat
- every callback_settings shape (top level and per consumer) must let
the proxy boot, register the configured callbacks and serve chat
Four product bugs on main surfaced by the matrices are recorded as
BUG skips per the suite convention: chat 500 and /model_group/info 500
on non-numeric token limits, a startup crash on a non-object
callback_settings, and otel silently dropped on a non-object
callback_settings.otel
* test(integration): pin the exact coerced value for numeric-edge token limits
Addresses review feedback: the numeric-edge matrix only asserted
'int or absent'. It now asserts the listed value for each case on
/v1/models, /models and /v1/models/{id}, which also lets the listing
helper drop its optional-expectation branch.
* fix(s3_v2): bound concurrent S3 uploads per flush and add opt-in JSONL batch files
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(s3_v2): keep failed uploads queued, parse env-backed flags, add integration coverage
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(s3_v2): type test helpers and honor constructor bound when config value is null
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(s3): annotate required casts for the type-discipline gate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(s3_v2): keep tenant prefixes, stable retries and cold storage safety in batch file mode
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(s3_v2): cover root-level batch file keys for codecov patch target
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(s3_v2): audit matrix across chat, messages and responses surfaces with sink faults
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(s3_v2): read sink objects under the lock in the audit cells
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(s3_v2): rebind the retry queue instead of slicing in place
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(s3_v2): ignore stray non-POST requests in the surface upstream
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(s3_v2): keep fake upload state on the fake client instead of nonlocal counters
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng <yucheng@berri.ai>
* test(integration): add read-replica routing harness
* refactor(integration): hoist the maintenance url imports
* fix(integration): keep per-test databases and the witness sequence readable under replica roles
* fix(integration): opt bespoke database and pool tests out of the injected read replica
* test(integration): commit recorded replica routing expectations
* fix(integration): judge routing by role containment so shrinking role sets do not fail
* fix(integration): run the pool-limit shutdown choreography on the superuser database url
* ci(integration): add the mcp group to the replica matrix
* fix(integration): judge routing by exact role sets with a named either-role allowlist
* test(integration): drop containment-era routing expectations for re-recording
* chore(integration): drop docstrings from the replica harness scripts
* docs(integration): describe exact routing matching and the either-role list
* test(integration): record exact replica routing expectations
* test(integration): allow the SELECT 1 health probe on either role
* test(integration): replace committed routing expectations with an on-demand base-vs-head parity run
* test(integration): fix parity env scope, readme wording, and seed-deterministic serialization test
* test(integration): make the sorted-role serialization test deterministic in-process
* test(integration): swap all product code in parity runs and pin role gains
* ci(integration): force tracked-file removal before parity checkout
---------
Co-authored-by: yuneng <yuneng@berri.ai>
* test(integration): reproduce passthrough upstream error body missing from logs and spend row
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(passthrough): log upstream 4xx/5xx error bodies and carry them into the failure hook
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(error_normalization): let the passthrough prefix win over upstream body text
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(passthrough): honor message redaction for upstream error bodies
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(passthrough): bound the upstream error body read and sanitize it before logging
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(passthrough): use the Sequence import directly in the allowed-routes cast
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(passthrough): rechunk the upstream error stream so the preview read stays bounded
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): audit matrix for passthrough upstream error visibility
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(passthrough): drop the restating docstring on the upstream failure logger
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): drop the retired covers markers from the passthrough error tests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(passthrough): keep the upstream status when the error body peek fails
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(passthrough): cover the relay aclose in the mid-read failure test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(passthrough): relay decoded partial body on mid-read failure
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): do not requeue a daily spend batch whose commit already left for postgres
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): settle an interrupted daily spend commit from the shutdown flush instead of blocking the cancelled tick
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): burst two workers and SIGTERM during daily spend COMMIT, expect exactly once
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): keep deployment labels on cache-hit post_call rejections
A post-call failure on a response served from the litellm cache set no first_api_call_start_time, so the failure hook flagged it as rejected before routing and dropped the model_id and provider labels
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): read the cache hit from caching_details in the failure hook
model_call_details[cache_hit] is stamped inside the enqueued success handler, so a post-call failure can observe it too early; logging_obj.caching_details is set synchronously before the cached response returns
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover cache-hit guardrail reject deployment labels across endpoints, modes and chaos
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): bound the worker-kill reject count by in-flight losses
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): assert provider and model labels on the cache-hit regression test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(logging): bound the exceeded budget regex so a crafted error message cannot stall the proxy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): audit normalized_error clustering on long messages with a real two worker proxy
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): poll for the budget denial and correlate upstream 503 bursts by request marker
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(logging): replace the bounded exceeded budget regex with a linear scan that keeps the original semantics
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): compare upstream error wording against the decoded message
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): tolerate a reaped worker while listing proxy children
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(presidio): stream non-Anthropic raw SSE through the post_call hook unbuffered
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(presidio): keep the pytest.raises block to a single await
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(presidio): move raw SSE format check into a helper to keep hook complexity flat
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(presidio): fold the raw SSE format check into the existing bytes branch to stay within the complexity budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(presidio): decide raw SSE stream shape on a complete first frame, not a transport fragment
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cover presidio post_call streaming for native gemini passthrough and anthropic messages
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(presidio): cap first SSE frame coalescing at 64 KiB so an unterminated first event cannot buffer unbounded
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(presidio): name raw SSE passthrough in the skipped output masking warning
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(fireworks_ai): route firerouter short names and bill pass-through legs at the routed model's rates
fireworks_ai/firerouter and fireworks_ai/firerouter/<slug> resolve to
accounts/fireworks/routers/... instead of a models/ path, and the cost
calculator falls back to the routed model's own catalog entry before the
Fireworks size buckets so a Claude leg is no longer priced at $0
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(fireworks_ai): bill routed legs under the routed model's own provider
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(fireworks_ai): require the k suffix when parsing tiered input fields
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): assert /v1/models reports max_input_tokens and max_output_tokens
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): pin published gpt-4o-mini limits instead of reading the cost map in-test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): name the gpt-4o-mini deployment explicitly in the /v1/models test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): cite the source of the pinned gpt-4o-mini limits
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Adds two integration tests to tests/integration/spend/test_tag_budget_enforcement.py
test_key_tag_rpm_limit_rejects_the_second_request_carrying_that_tag proves
a key metadata tag_rpm_limit of 1 rejects the second request carrying that
tag with 429 while a request carrying a different tag still passes
test_tag_budget_duration_resets_spend_and_unblocks_the_tag boots an owned
proxy with a 2 to 3 second budget rescheduler, creates a tag with
max_budget 0.0001 and budget_duration 5s, observes the spend block, then
observes the tag serving again once ResetBudgetJob zeroes the tag spend
Mutation evidence
get_key_tag_rpm_limit forced to return None: the second tagged request
returned 200 instead of 429, test red
_queue_budget_linked_resets for uow.tags disabled in
_commit_budget_cascade_once: the tag stayed blocked at 422 for the full
70 second recovery window, test red
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(model_prices): registry audit 2026-09-23, in-region Bedrock Claude and OpenRouter price fixes
Absorbs #42698
Co-authored-by: coldStoneSoul <shekharpachlore@gmail.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(model_prices): keep registry formatting unchanged
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: use eu.amazon.nova-pro for regional pricing probe after in-region parity
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: lock in-region parity for bare Bedrock Claude ids
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: compare every pricing field for bare Bedrock Claude parity
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(model-catalog): add above_32k cost fields to ModelInfo round-trip
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* Revert "fix(model-catalog): add above_32k cost fields to ModelInfo round-trip"
This reverts commit c71d5a3de3.
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: coldStoneSoul <shekharpachlore@gmail.com>
* fix(proxy): document responses API request and response schemas in openapi
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): namespace colliding openapi defs instead of overwriting existing components
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(proxy): regenerate lazy openapi snapshot and dashboard schema types
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): require model and input in responses schema, document event stream, fix def collision refs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): mark responses request fields readonly required
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): reuse existing OpenAPI components when a $defs entry has the same shape
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ollama): read the JSON thinking field on non-streaming completions
Ollama's /api/generate returns reasoning in a top-level `thinking` field, but
the completion transport only looked for inline <think> tags. reasoning_content
was therefore always null, and a model that spent its whole turn reasoning
returned an empty assistant message with tokens billed.
Port the precedence the ollama_chat transport already uses: the field wins and
inline tags stay the fallback. Applied to both non-streaming paths, including
the JSON-mode text fallback. The two fields are read through a small validated
model rather than off the untyped JSON, so absent and explicitly null
`response` stay distinct exactly as before.
* fix(ollama): keep the thinking field on JSON-mode completions
The first pass read `thinking` for plain replies and for JSON-mode text that
failed to parse, but the three JSON-mode branches that succeed still dropped
it: an empty `response`, a valid JSON object, and a function-call shaped one.
A model that spent its whole turn reasoning under `format: json` therefore
still came back blank with the tokens billed.
Carry the field on all three, type the new test helper's parameters, and cover
the null and malformed `response` fallbacks.
---------
Co-authored-by: Pawan-Shahane <shahanepawan511@gmail.com>
* fix(models): sync openrouter prices from the models API
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(models): allow above_32k_tokens cost fields in price map schema test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): gate disable_global_guardrails on keys and teams to proxy admins
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: cover metadata smuggle with explicit false and UI toggle gating
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(proxy): satisfy PT017 in resend-stored guardrail flag test
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): keep regenerate_key_fn under the C901 ceiling via a guardrail opt-out helper
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): regenerate schema.d.ts for guardrail opt-out docstrings
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): gate disable_global_guardrails on caller-sent metadata, not server defaults
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): audit cells for disable_global_guardrails admin gate
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): restore contracts.json formatting
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): share guardrail opt-out helpers
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): hide the team disable_global_guardrails switch from non proxy admins
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(integration): drop covers markers and bound the slow sink check to the sink delay
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): honor model_info.discoverable on the model listing endpoints
A model_list entry marked model_info: {discoverable: false} is left out of GET /v1/models (OpenAI and Anthropic shapes, scope=expand and wildcard routes included), the list path of GET /v1/model/info and GET /model_group/info for every caller without the admin view, while direct requests naming the model keep routing to it. The field defaults to None so an absent flag reads as discoverable and nothing is persisted or echoed for configs that never set it.
* fix(proxy): hide flagged team models under their public name and cover the scope=expand filter
The discoverability lookup now resolves a listed name with the caller's team context, so a team-scoped deployment marked discoverable: false drops out for that team's keys under its public name instead of failing open. The scope=expand branch is now exercised by a team admin caller, and the OCI secrets test builds a real UserAPIKeyAuth instead of a spec mock that has no pydantic fields.
* perf(proxy): resolve only candidate names in the discoverable filter
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(mcp): reject duplicate MCP server names and aliases
MCP server_name and alias were unchecked at write time, so two servers
could share one tool prefix and tool routing resolved to an arbitrary
winner. Writes now run inside an advisory-locked transaction that
rejects a collision on either column case-insensitively with a 400
naming the colliding identifier, covering create, edit, connector
import and restricted-admin submission. Server reload logs one warning
per identifier already shared in the database.
Co-Authored-By: bot_apk <apk@cognition.ai>
* fix(ui): block duplicate MCP server names and aliases before submit
The create and edit forms now check the normalized name/alias against
the loaded server list (case-insensitive, spaces to underscores, own
row excluded on edit) and show a field error instead of submitting.
Structured proxy error bodies are unwrapped so a 400 no longer renders
as 'Error: [object Object]'.
Co-Authored-By: bot_apk <apk@cognition.ai>
* fix(mcp): check identifier conflicts when an alias is cleared
Clearing an alias drops the tool prefix to the stored server_name, so
that name must go through the conflict check too; an explicit alias:null
is now treated as an identifier write. Also narrows the new db tests to
behavioral assertions instead of pinning prisma where shapes.
Co-Authored-By: bot_apk <apk@cognition.ai>
* fix(mcp): treat an empty alias as a clear in conflict checks
An empty-string alias was written unchecked even though the prefix falls
back to server_name; the update path now treats any falsy alias like a
clear. The edit form likewise compares a cleared alias as empty instead
of re-checking the alias being removed.
Co-Authored-By: bot_apk <apk@cognition.ai>
* test(mcp): cover clearing an alias to an empty string
Co-Authored-By: bot_apk <apk@cognition.ai>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: bot_apk <apk@cognition.ai>
* fix(otel): root post-response service spans in their own trace linked to the request
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(otel): trim service span context docstring
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(straiker): deterministic integration audit of the v3 platform relay
34 cells against a real two-worker proxy, Postgres and Redis with a scripted
provider upstream and a local Straiker sink: v3 allow, block, deny, replay and
killswitch verdicts on chat completions, messages, responses and completions
across the OpenAI and Anthropic SDKs and raw httpx, pre_call, post_call and
logging_only modes, header and identity precedence, credential redaction,
sink outages, malformed verdicts, unauthenticated and unknown-model requests,
management endpoints, the unchanged v1 path, and a mixed burst through a sink
outage, a worker kill and a proxy restart
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(straiker): bind the spend-row pattern inside the outage burst poll
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(straiker): kill a real uvicorn worker and prove detect runs before the unknown-model error
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(straiker): assert the v1 webhook ran on the v1 block cell and check every non-streaming burst spend row
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
