Commit graph

19521 commits

Author SHA1 Message Date
devin-ai-integration[bot]
7faeb15ff3
fix(e2e): skip unpublished npm versions in the Claude Code PR-gate resolver (#43053)
npm keeps an unpublished version's timestamp in the packument's time map but drops it from versions, so the resolver could hand npm install a version it refuses with ETARGET. Only versions still present in versions are candidates now.

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-24 15:17:31 -07:00
devin-ai-integration[bot]
de8aeff6c6
feat(proxy_cli): add --validate_config dry-run flag (#41705)
* feat(proxy_cli): add --validate_config dry-run flag

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(tests): format --validate_config CliRunner calls

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy_cli): run --validate_config before the ollama auto-start

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy_cli): restore file and add ollama validate_config regression test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
2026-09-24 17:07:10 -05:00
devin-ai-integration[bot]
b72a030501
test: take keys out of the legacy proxy, enterprise and mcp unit tests before moving them (#42901)
* ci: fix the litellm-tests unit job with sysmon coverage, an env allowlist and coverage upload on failure

* test: replace key-dependent proxy, enterprise and mcp unit tests with synthetic values and integration and e2e coverage

* test: drop key reads at the legacy proxy, enterprise and mcp paths and wire the gemini pass-through split

* ci: fail the unit shard when circleci tests split errors

* test: drop restating comments from the gemini pass-through split

* ci: exit the unit shard cleanly when circleci tests split assigns it no files

---------

Co-authored-by: yuneng <yuneng@berri.ai>
2026-09-24 14:56:52 -07:00
yuneng-jiang
2be2d68ac3
test(e2e/ui): hide the LiteAdmin button in the shared admin session (#43033)
#42443 pins a floating LiteAdmin button to the bottom-right corner, where it covers the logs page's next-page control. Flip the per-user Hide LiteAdmin switch during global setup so every spec reusing the admin storage state loads with the button hidden.
2026-09-24 14:48:35 -07:00
devin-ai-integration[bot]
86ba4fc16f
feat(compat-matrix): resolve and install the Claude Code CLI per run (#43038)
* feat(compat-matrix): resolve and install the Claude Code CLI per run

* chore(compat-matrix): drop the installer header comment

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-24 14:35:03 -07:00
devin-ai-integration[bot]
fc29fb513c
fix(vertex_ai): keep batch output_file_id null until Vertex reports outputInfo (#43030)
* fix(vertex_ai): keep batch output_file_id null until Vertex reports outputInfo

Vertex only sets outputInfo.gcsOutputDirectory once a batch job has written
output. Falling back to outputConfig's outputUriPrefix named the per-model
directory shared by every batch of the deployment, an object that never
exists, so the proxy minted a managed file for it under the first key and
every other key's file calls on that id were 403s

* fix(vertex_ai): treat a null gcsOutputDirectory as no output file yet

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-24 14:05:31 -07:00
ryan-crabbe-berri
5e4b1b9df0
fix(proxy): pass team member spend rows as jsonb so a $0 flush cannot poison the pool connection (#43029)
Prisma types a raw array parameter from the first batch a connection sees. After a flush in
which every member cost was a whole number (a free model), the connection's cached statement
expected int8[] and every later fractional batch on it failed with "improper binary format in
array element", so member spend silently stopped landing while team spend kept rising.

The rows now travel as one JSON document unpacked by jsonb_to_recordset with the column types
declared in SQL, so Postgres types the numbers and the batch shape no longer matters.
2026-09-24 20:44:56 +00:00
devin-ai-integration[bot]
fc87a06f00
fix(proxy): stop leaking periodic tasks on every DB config reload (#42784)
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 15:42:36 -05:00
devin-ai-integration[bot]
77eccaca78
feat(proxy): server-side Team Usage export beyond the top-N key cap (#42996)
* feat(proxy): add uncapped server-side team usage export route

GET /team/daily/activity/export answers the same scoping as
/team/daily/activity/aggregated with one unbounded rollup query, so keys
past USAGE_TOP_API_KEYS_LIMIT are included. Supports daily,
daily_with_keys, daily_with_users and daily_with_models export types as
CSV (default) or JSON. The PTU flat-cost sentinel stays in the plain
daily rollup and is excluded from the keyed and per-model exports

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(ui): export team usage server-side when the key list was truncated

When the aggregated spend response reports api_key truncation, EntityUsage
passes a serverExport into the export modal that downloads CSV or JSON
from GET /team/daily/activity/export instead of building the file from
the truncated on-screen data. apiClient gains a responseType option so
the download can arrive as a Blob, and truncation no longer blocks the
export button

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): cover team usage export types, sentinel handling and scope

Unit tests pin the uncapped key rollup past USAGE_TOP_API_KEYS_LIMIT,
PTU sentinel inclusion in the daily rollup and exclusion elsewhere, the
per-user fold, and the CSV column layout. Integration tests exercise the
route against a live proxy, including member scope denial

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(proxy): tidy team usage export route

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): use membership test for export type branch (PLR1714)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(ui): format exportBlockedReason test with prettier

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): satisfy type-discipline gate in team usage export

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): pass export rows as a sequence to the response model

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy-behavior): cover team usage export in the daily activity scope matrix

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(proxy): carry PTU flat cost and escape formulas in team usage export

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): keep the truncation export block on surfaces without a server export

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(ui): drop redundant comments in team export call and modal test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): audit cells for team usage export

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): tighten team usage export audit cells

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): type the export params tuple and fold user keys in one pass

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(ui): bring entity usage export helpers under eslint budgets

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(ui): prettier-format UsagePageView after merge

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 15:40:36 -05:00
devin-ai-integration[bot]
1fbd1e9ce9
fix(bedrock): route unmapped openai family model ids to converse (#42713)
* fix(bedrock): route unmapped openai family model ids to converse

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(bedrock): rename the e2e openai family backend constant

global.openai.gpt-6-sol has a cost-map row now, so the constant no longer
names an unmapped model

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-24 13:08:27 -07:00
devin-ai-integration[bot]
c2eb549ee6
feat(usage): search team keys beyond the top-N in the Team usage view (#42857) 2026-09-24 15:04:12 -05:00
devin-ai-integration[bot]
5a8ec13786
feat(usage): search keys beyond the top-N usage subset (#42827) 2026-09-24 15:04:03 -05:00
devin-ai-integration[bot]
1edc4ba580
fix(logging): pass provider response headers to callbacks on every endpoint (#42824)
* fix(logging): pass provider response headers to callbacks on every endpoint

Custom callbacks only received kwargs["response_headers"] for chat
completions. Responses, image generation and edit, speech, and
transcription calls either never recorded the provider's headers or
recorded them in one place and not the other.

Every handler now records the provider's httpx headers on the response's
hidden params as "headers" (raw) and "additional_headers" (processed,
with LiteLLM's own entries winning on a clash), and the logging object
derives model_call_details["response_headers"] from those hidden params
before cost calculation on the non-stream and both streaming success
paths, keeping a handler-set value authoritative. Binary speech responses
expose their hidden params to the standard logging payload, and the sync
OpenAI transcription request always fetches the raw response.

* test(images): point the legacy image and speech fakes at the raw response surface

Image generation now goes through the SDK's raw response so the provider headers can be read, and the speech binary response now carries hidden params. The unit fakes in the image generation, xinference, proxy provider, image edit, Vertex speech, and otel suites still pinned the old call surface and the old "no hidden params" assertion, so they read an uncalled mock or a fake response without headers.

* test(images): drop the rewritten mock comments and the generated edit PNGs

* test(images): move the llm-span test's image fake to the raw response surface

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-24 13:01:12 -07:00
devin-ai-integration[bot]
c9e8a04139
feat(vertex): native batch JSONL passthrough with cost tracking (#42810)
* feat(vertex): native batch JSONL passthrough with cost tracking

Add a per-request `passthrough=true` multipart field on `POST /v1/files`
(and the same kwarg on `litellm.create_file`) that uploads a native
Vertex AI batch JSONL to the deployment's GCS bucket unchanged, so rows
using `googleSearch` and other Gemini-only features run as written and
the output, `groundingMetadata` included, comes back untouched.

Passthrough is sticky through the GCS object path
(`litellm-vertex-files/passthrough/...`), so batch create and output
retrieval inherit it without new state. Native output rows are costed
from their `usageMetadata` with the deployment's model and model_info,
in the polling and retrieve paths and for the existing global
`disable_vertex_batch_output_transformation` flag, which billed $0
before.

The proxy requires the target to resolve to vertex_ai deployments only,
refuses `passthrough` with a non-batch purpose, a non-default
`target_storage`, or pre-call guardrails, and validates native rows on
`request` instead of the OpenAI batch keys.

* refactor(vertex): keep native batch row pricing inside the Vertex adapter

Moves native Vertex batch row detection, response parsing, and per-row
pricing from litellm/batches/batch_utils.py into
litellm/llms/vertex_ai/batches/transformation.py, so batch_utils only
aggregates the rows it gets back. Adds tests/test_litellm/files to the
misc unit shard so the new test directory is claimed by a shard.

* fix(files): say what a passthrough batch upload takes when a row is not native

The missing-key 400 listed bare key names, so an OpenAI-shaped row under
passthrough=true read "Each line must be a JSON object with keys request".
The batch line shape now carries its own hint, and the passthrough one says
a passthrough upload takes native Vertex batch rows with a request key

* fix(batches): bill native Vertex embedding batch rows on the native cost path

A native Vertex output row whose response holds an embedding was validated as a
generateContent response, so the documented tokenCount-only shape counted as a failed
row. Price embedding rows from their own usage (promptTokenCount, else tokenCount) with
the helper the transformed embeddings path already used, and drop the prompt-details
helper nothing calls anymore.

* fix(batches): keep modality batch rates on native Vertex embedding rows

An embedding row that carries usageMetadata was billed from promptTokenCount alone, so
its promptTokensDetails no longer reached the audio, image, and video batch rates the
way it did before the native cost path. Run every row with usageMetadata through the
Gemini usage parser and keep the flat tokenCount fallback for embedding rows without it.

* fix(batches): price native Vertex batch rows by modelVersion under a wildcard deployment

A `vertex_ai/*` deployment hands the batch cost path `*` as the deployment model, which
no cost map resolves, so every native (passthrough or flag-on) row was billed at $0. A
wildcard deployment model now defers to the row's own `modelVersion`, the way the
transformed path already prices by the row's `model`.

Also moves the native passthrough tests under tests/test_litellm, the tree codecov
reads, and covers the raw upload chunking, the embedding output translation, the
unpriceable-row path, and the flag-on dispatch.

* fix(batches): keep explicit deployment prices for native Vertex rows without a modelVersion

Under a wildcard deployment a native batch row that carries no modelVersion (an embedding
row, or a generateContent row Vertex returned without one) was billed at $0 even when the
deployment's model_info sets explicit batch prices, because the cost calculator was never
called. The row now falls back to the wildcard name, which the cost calculator prices from
the explicit model_info, and only a row with neither a modelVersion nor a deployment model
is billed at $0 with the warning

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-24 12:35:34 -07:00
ahamedshaik16
f39a56b004
fix(prometheus): add model_group label to deployment request and rate limit metrics (#42966)
* fix(prometheus): add model_group label to deployment request and rate limit metrics

litellm_deployment_total_requests, litellm_deployment_success_responses,
litellm_deployment_failure_responses, litellm_deployment_tpm_limit and
litellm_deployment_rpm_limit had no way to identify which model_group a
pooled deployment belongs to, only requested_model, litellm_model_name and
model_id, none of which name the alias a model_name resolves through when
it fans out to more than one deployment.

model_group was already resolved onto enum_values for every request in
async_log_success_event, so this is a label-list addition for the metrics
built directly from that enum_values (the two request counters). The
failure counter builds its own UserAPIKeyLabelValues locally and had a
model_group variable already in scope that it never passed through, and
the tpm/rpm limit gauges are set from a helper that took no model_group
parameter at all even though its only caller already had it on
enum_values. Both now thread the value through.

* test(prometheus): expect model_group in deployment success/total request labels

test_set_llm_deployment_success_metrics asserts the exact label set passed
to litellm_deployment_success_responses.labels() and
litellm_deployment_total_requests.labels(), which now includes model_group
since it was added to those metrics' label list.

* fix(prometheus): bound model_group on deployment failure metrics

On a pre-routing reject (no deployment selected), model_group is
caller-supplied via litellm_params.metadata and was passed through
unbounded, letting an unrecognized value mint unlimited label series on
litellm_deployment_failure_responses / litellm_deployment_total_requests.
Bound it with the same _bounded_requested_model_label used for
requested_model on this path. When a deployment is actually selected,
model_group is router-resolved and passed through as-is.

Also documents the model_group parameter on
_set_deployment_tpm_rpm_limit_metrics and the bounding behavior on
set_llm_deployment_failure_metrics.

---------

Co-authored-by: ahamedshaik16 <24526479+ahamedshaik16@users.noreply.github.com>
2026-09-24 12:15:01 -07:00
devin-ai-integration[bot]
bdf854c3ea
feat(rust): shape Anthropic Messages requests natively (#42982)
* test(rust): encode anthropic response serialization shape as rstest cases

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(rust): shape Anthropic Messages requests natively

The Rust Messages route only relayed the body. It now runs the request shaping
the Python handler does for the direct Anthropic provider: history sanitizers
(empty blocks, tool ids, replayed web search results, provider_specific_fields,
encrypted reasoning, advisor blocks), reasoning_effort and adaptive/legacy
thinking translation against the model's capability flags, the sampling and
speed gates under drop_params, the metadata allowlist, additional_drop_params,
reasoning auto summary, OAuth and ANTHROPIC_AUTH_TOKEN credentials,
provider_specific_header merging and anthropic-beta injection. Capability flags
and LiteLLM settings reach Rust through route_host.shaping(). A request the
route rejects before the call now maps to BadRequestError instead of
APIConnectionError

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* test(rust): port Anthropic Messages shaping tests and pin comment contracts as cases

Every Python unit test that exercises the ported shaping for the direct
Anthropic provider now has a named rstest counterpart, and every comment that
stated a behavior contract is deleted in favor of a case that pins it. Measured
with cargo-mutants over the touched files, all viable mutants are caught

Porting the tests surfaced parity gaps, fixed here to match Python: every
casing of a forwarded anthropic-beta header is merged, replayed web search
results are rewritten from their own block (an empty result keeps its slot and
a server_tool_use with a non-string query stays), an empty output_config.effort
falls back to medium, speed and reasoning effort errors quote values the way
Python does, additional_drop_params apply after metadata validation and the
auto summary and never touch model or messages, and a non-string
metadata.user_id is rejected before the call

* fix(rust): resolve Messages credentials through the secret source and scope headers by resolved provider

The native Messages route read ANTHROPIC_API_KEY, ANTHROPIC_AUTH_TOKEN and the
base URL straight from the process environment, so a key or base held in a
configured secret manager was never found. Each provider config now declares
its secret names and the route resolves them through the same SecretSource the
OCR route uses, with the Python bridge passing in litellm's configured manager

provider_specific_header entries were scoped by the explicit
custom_llm_provider only, falling back to anthropic, so an azure_ai/ model
lost its azure_ai scoped headers. Scoping now happens in the route after the
provider is resolved from the model, as Python's handler does

The Azure config now adds the same anthropic-beta feature headers Python's
Azure route adds, and the metadata allowlist, reasoning auto summary and
history sanitizers move from the core route into the llms crate, mirroring
their home in Python's messages handler

* test(rust): escape the dot in the metadata.user_id match pattern

* refactor(rust-bridge): project Messages capabilities without mutable dicts

The capability flags and effort tiers were built as dict comprehensions,
which the type-discipline gate counts as mutable construction, and the
asdict call carried a mutable-ok suppression that suppressed nothing. The
flags are now passed one by one and the effort tiers are a frozen dataclass,
which asdict projects to the same map the native side reads

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 19:08:32 +00:00
yuneng-jiang
759c216366
test(guardrails): run the cache-hit redis outage test on the shared owned_redis helper (#42925)
* test(guardrails): run the cache-hit redis outage test on the shared owned_redis helper

test_redis_outage_keeps_serving_in_memory_hits (#42780) spawned `redis-server`
straight from PATH. The CircleCI integration machine has no redis-server
binary, so the test died with FileNotFoundError before reaching the proxy.

Use integration._support.redis_process.owned_redis, which runs the local
binary when there is one and otherwise the job's redis-cache container, and
take the outage with its stop()/start() pair the way test_redis_recovery
already does. The assertions are unchanged.

* test(guardrails): describe the owned_redis stop as an outage, not a kill
2026-09-24 12:00:10 -07:00
yuneng-jiang
4c68a97c25
test(mcp): patch create_mcp_server_if_identifier_free in the store-model-in-db MCP tests (#42916)
#42791 switched the MCP management endpoints from create_mcp_server to
create_mcp_server_if_identifier_free, which keeps the same arguments and
returns the created row on success. Two tests in
tests/store_model_in_db_tests/test_mcp_servers.py still patched the old name,
so mock.patch raised AttributeError before the tests ran and
proxy_store_model_in_db_tests has been red on main since.
2026-09-24 11:59:52 -07:00
yuneng-jiang
2cbaa4c9b5
test(mcp): stop a comprehension variable from shadowing the body() helper (#42906)
test_malformed_bodies_missing_users_and_foreign_servers_are_rejected used
`body` as a comprehension variable and then called the module-level `body()`
helper a few lines later. CPython 3.12.2, which the CircleCI integration job
runs, compiles that later call as a local read, so the test raised
UnboundLocalError on every integration-mcp run since #42652. Newer 3.12
patch releases and 3.13 compile it as a global read, which is why it passes
locally.

Renaming the comprehension variable makes both reads unambiguous.
2026-09-24 11:56:56 -07:00
yuneng-jiang
f4f1a75a9c
test(vertex_ai): run the files peak-memory guards without coverage tracing (#42914)
The new CircleCI tests pipeline (#42773) runs tests/unit under pytest-cov on
CPython 3.12.2, where coverage traces every line through sys.settrace. The two
tracemalloc peak comparisons in test_vertex_ai_files_streaming.py drive 8000-row
payloads through both pipelines and slow from ~10s to over 3 minutes under that
tracer, so both hit the 90s pytest-timeout on every run.

Mark them no_cover so pytest-cov pauses tracing for just these two. Their
assertions are unchanged and every other test in the file still reports coverage.
2026-09-24 11:56:42 -07:00
yuneng-jiang
259f954f56
test(e2e/ui): give the logout specs their own admin session (#42930)
#42463 made Logout revoke the dashboard session key on the server. Both
logout specs ran on the shared ADMIN_STORAGE_PATH session that globalSetup
mints once, so clicking Logout revoked the key every later admin spec reuses.
The CircleCI run is serial, and from the auth/ folder on, every admin-session
spec failed with "Invalid proxy server token passed" (80 failures, up from 7)
while the internal-user, internal-viewer and team-admin specs kept passing.

Each logout spec now starts from an empty storage state and logs in through
the login page, so the session it revokes is its own. The login steps live in
a shared logInThroughLoginPage helper next to the other onboarding helpers.
2026-09-24 11:55:07 -07:00
devin-ai-integration[bot]
12960f3edf
fix(ui): pass is_proxy_admin for proxy admins on the models page team drill-in (#43003)
* fix(ui): pass is_proxy_admin for proxy admins on the models page team drill-in

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(ui): drop explanatory comments from the models page team drill-in tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): exclude view-only admins from is_proxy_admin on the models page team drill-in

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(ui): add browser integration contract for the team guardrail kill switch on the models page

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 11:54:52 -07:00
yuneng-jiang
0ac435bd01
test(ollama): assert the images sent to Ollama instead of echoing them through response (#42905)
test_ollama_image returned the request's images list as the mocked `response`
field and read it back from message content. Since #42838 the completion
transform validates `response` as a string, so the list is dropped and the test
fails with "string index out of range". Ollama always sends a string there.

The mock now returns a real string reply and the test asserts on the images the
transform actually sent, which is what it was checking all along.
2026-09-24 11:54:26 -07:00
joshua-berri
57eb3ff6b6
fix(mcp): reject origins outside the configured allowlist (#42649)
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
2026-09-24 10:23:30 -07:00
devin-ai-integration[bot]
571ada0b0f
feat(rust_bridge): mark native streams with the x-litellm-rust header (#42758)
Non-streaming responses served by the Rust core already carry
x-litellm-rust: true through _hidden_params.additional_headers, which the
SDK exposes and the gateway renders as a response header. Native streams
did not, because the lifecycle Stream and SyncStream objects had nowhere
to hold hidden params and the marker writer skips objects without them.

Give both stream classes the same _hidden_params bag every other litellm
response has, so the existing marker attaches without wrapping the stream
or changing its identity.

Co-authored-by: Yujong Lee <yujong@berri.ai>
2026-09-24 09:36:25 -07:00
devin-ai-integration[bot]
f0e671f754
refactor(types): replace Any with proven types in 13 files (#42937)
* refactor(types): replace Any with proven types in 13 files

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(types): drop unused executor import from utils type-checking block

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(realtime): keep reserved-key filtering on azure realtime health params

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(realtime): pin reserved-key filtering in azure realtime health auth params

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(realtime): exercise the real azure header builder in the reserved-key test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 09:02:02 -07:00
devin-ai-integration[bot]
e135a199ad
fix(proxy): fail parked DB lookups at a deadline and flip readiness while they stall (#42654)
Some checks failed
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
LiteLLM Rust / rust-wheel (push) Has been cancelled
* fix(proxy): fail parked DB lookups at a deadline and flip readiness while they stall

Under a load burst with a slow authentication database every request parked inside
the pod with no deadline while /health/readiness kept answering 200 (its own ping
gets a fresh connection), so the load balancer kept sending traffic until the pod hit
its memory limit, and the parked requests completed against the provider minutes
after every client had hung up

Every pre-request read (key, team, user, end user, budget, membership, organization,
object permission, jwt mapping, project, proxy budget, spend counter reseed) now runs
under one deadline, PROXY_DB_LOOKUP_DEADLINE_SECONDS (default 10 s). A lookup that
hits it fails the request with the existing 503 "authentication database is
temporarily unreachable" answer, honours allow_requests_on_db_unavailable, and never
triggers the transport reconnect (the transport is fine, the query is slow), which is
what turned the repro's stall into "too many clients". Writes stay unbounded

A deadline hit marks the pod stalled for PROXY_DB_LOOKUP_STALL_WINDOW_SECONDS
(default 30 s, 0 disables), during which /health/readiness answers 503 with
"db": "stalled" behind the same fail-open gate, so the pod leaves rotation before it
fills its memory. The existing litellm_in_flight_requests gauge already exposes the
parked set on /metrics

The deadline is enforced on the wall clock: bounded_db_lookup waits on the lookup
task with asyncio.wait and raises DBLookupDeadlineExceeded when the deadline passes
even if the lookup absorbs its cancellation, where asyncio.wait_for on 3.12+ would
sit on the cancelled task for as long as it takes

The failure spend-log row no longer re-runs the key and team lookups when the
failure itself is a database connection or deadline error, so a request that hit
the deadline is answered after one deadline instead of two

* fix(proxy): bound the spend counter gate wait and narrow the stalled lookup shortcut

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): keep the global spend lookup on the prisma client handle

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 10:09:49 -05:00
devin-ai-integration[bot]
8477fe4108
fix(proxy): list key and team model aliases in GET /v1/models (#42908)
* fix(proxy): list key and team model aliases in GET /v1/models

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): keep alias listing helpers within the type discipline budget

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): cover alias rows on GET /v1/models and /v1/models/{id}

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): apply team then key aliases like chat completions and keep the alias as the retrieved id

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): apply key aliases twice like chat completions and skip only malformed alias entries

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): apply the global model_alias_map between the key alias passes like chat completions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): list only the caller's own aliases and never rewrite a listed model id on retrieval

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(proxy): ruff format model_info alias lookup

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): hide undiscoverable names from model retrieval so an alias named like one resolves to its target

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): keep undiscoverable models retrievable by id while excluding them from the alias guard

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(proxy): pass an immutable name sequence into the model_info alias guard

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): type the model list alias test helpers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): annotate the new alias listing test fixtures and helpers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 06:41:13 -07:00
devin-ai-integration[bot]
2eeb16266b
fix(cost-map): sync vertex-ai rows (gemma 4 maas cache price, chirp_2) (#42942)
* fix(cost-map): sync vertex-ai rows (gemma 4 maas cache price, chirp_2)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(vertex-ai): expect chirp_2 as speech-to-text model now that catalog row exists

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 04:39:09 -07:00
yuneng-jiang
265f874eaf
test(integration): edge-case matrices for malformed token limits and callback_settings shapes (#42895)
* test(integration): edge-case matrices for malformed token limits and callback_settings shapes

Extends the integration suite so two classes of issues found by gauntlet
reviews are caught end to end against a real proxy:

- non-numeric or odd model_info token limits (from /model/new and from
  config YAML) must be listed as absent on /v1/models, /models,
  /v1/models/{id} and /model/info, keep sibling models listed, and still
  serve chat
- every callback_settings shape (top level and per consumer) must let
  the proxy boot, register the configured callbacks and serve chat

Four product bugs on main surfaced by the matrices are recorded as
BUG skips per the suite convention: chat 500 and /model_group/info 500
on non-numeric token limits, a startup crash on a non-object
callback_settings, and otel silently dropped on a non-object
callback_settings.otel

* test(integration): pin the exact coerced value for numeric-edge token limits

Addresses review feedback: the numeric-edge matrix only asserted
'int or absent'. It now asserts the listed value for each case on
/v1/models, /models and /v1/models/{id}, which also lets the listing
helper drop its optional-expectation branch.
2026-09-24 01:05:55 -07:00
devin-ai-integration[bot]
09ebb28473
fix(s3_v2): bound concurrent S3 uploads per flush and add opt-in JSONL batch files (#41258)
* fix(s3_v2): bound concurrent S3 uploads per flush and add opt-in JSONL batch files

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(s3_v2): keep failed uploads queued, parse env-backed flags, add integration coverage

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(s3_v2): type test helpers and honor constructor bound when config value is null

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(s3): annotate required casts for the type-discipline gate

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(s3_v2): keep tenant prefixes, stable retries and cold storage safety in batch file mode

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(s3_v2): cover root-level batch file keys for codecov patch target

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(s3_v2): audit matrix across chat, messages and responses surfaces with sink faults

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(s3_v2): read sink objects under the lock in the audit cells

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(s3_v2): rebind the retry queue instead of slicing in place

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(s3_v2): ignore stray non-POST requests in the surface upstream

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(s3_v2): keep fake upload state on the fake client instead of nonlocal counters

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng <yucheng@berri.ai>
2026-09-24 07:32:00 +00:00
devin-ai-integration[bot]
3fb6f8740b
test(integration): add read-replica routing harness to the CircleCI integration suite (#42692)
* test(integration): add read-replica routing harness

* refactor(integration): hoist the maintenance url imports

* fix(integration): keep per-test databases and the witness sequence readable under replica roles

* fix(integration): opt bespoke database and pool tests out of the injected read replica

* test(integration): commit recorded replica routing expectations

* fix(integration): judge routing by role containment so shrinking role sets do not fail

* fix(integration): run the pool-limit shutdown choreography on the superuser database url

* ci(integration): add the mcp group to the replica matrix

* fix(integration): judge routing by exact role sets with a named either-role allowlist

* test(integration): drop containment-era routing expectations for re-recording

* chore(integration): drop docstrings from the replica harness scripts

* docs(integration): describe exact routing matching and the either-role list

* test(integration): record exact replica routing expectations

* test(integration): allow the SELECT 1 health probe on either role

* test(integration): replace committed routing expectations with an on-demand base-vs-head parity run

* test(integration): fix parity env scope, readme wording, and seed-deterministic serialization test

* test(integration): make the sorted-role serialization test deterministic in-process

* test(integration): swap all product code in parity runs and pin role gains

* ci(integration): force tracked-file removal before parity checkout

---------

Co-authored-by: yuneng <yuneng@berri.ai>
2026-09-24 00:25:03 -07:00
devin-ai-integration[bot]
6dbd65b230
fix(passthrough): log upstream 4xx/5xx error bodies and carry them into the failure hook (#42695)
* test(integration): reproduce passthrough upstream error body missing from logs and spend row

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(passthrough): log upstream 4xx/5xx error bodies and carry them into the failure hook

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(error_normalization): let the passthrough prefix win over upstream body text

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(passthrough): honor message redaction for upstream error bodies

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(passthrough): bound the upstream error body read and sanitize it before logging

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(passthrough): use the Sequence import directly in the allowed-routes cast

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(passthrough): rechunk the upstream error stream so the preview read stays bounded

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): audit matrix for passthrough upstream error visibility

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(passthrough): drop the restating docstring on the upstream failure logger

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): drop the retired covers markers from the passthrough error tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(passthrough): keep the upstream status when the error body peek fails

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(passthrough): cover the relay aclose in the mid-read failure test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(passthrough): relay decoded partial body on mid-read failure

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 23:42:24 -07:00
devin-ai-integration[bot]
093ceb576d
fix(proxy): do not requeue a daily spend batch whose commit already left for postgres (#42786)
* fix(proxy): do not requeue a daily spend batch whose commit already left for postgres

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): settle an interrupted daily spend commit from the shutdown flush instead of blocking the cancelled tick

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): burst two workers and SIGTERM during daily spend COMMIT, expect exactly once

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 22:54:19 -07:00
devin-ai-integration[bot]
1519032d90
fix(proxy): keep deployment labels on cache-hit post_call guardrail rejections (#42780)
* fix(proxy): keep deployment labels on cache-hit post_call rejections

A post-call failure on a response served from the litellm cache set no first_api_call_start_time, so the failure hook flagged it as rejected before routing and dropped the model_id and provider labels

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): read the cache hit from caching_details in the failure hook

model_call_details[cache_hit] is stamped inside the enqueued success handler, so a post-call failure can observe it too early; logging_obj.caching_details is set synchronously before the cached response returns

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover cache-hit guardrail reject deployment labels across endpoints, modes and chaos

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): bound the worker-kill reject count by in-flight losses

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): assert provider and model labels on the cache-hit regression test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 22:48:04 -07:00
devin-ai-integration[bot]
251fdf0308
fix(logging): scan the exceeded budget wording linearly so a crafted error message cannot stall the proxy (#42778)
* fix(logging): bound the exceeded budget regex so a crafted error message cannot stall the proxy

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): audit normalized_error clustering on long messages with a real two worker proxy

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): poll for the budget denial and correlate upstream 503 bursts by request marker

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(logging): replace the bounded exceeded budget regex with a linear scan that keeps the original semantics

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): compare upstream error wording against the decoded message

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): tolerate a reaped worker while listing proxy children

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 22:44:23 -07:00
devin-ai-integration[bot]
b8154bcbc0
fix(presidio): stream non-Anthropic raw SSE through the post_call hook unbuffered (#42777)
* fix(presidio): stream non-Anthropic raw SSE through the post_call hook unbuffered

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(presidio): keep the pytest.raises block to a single await

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(presidio): move raw SSE format check into a helper to keep hook complexity flat

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(presidio): fold the raw SSE format check into the existing bytes branch to stay within the complexity budget

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(presidio): decide raw SSE stream shape on a complete first frame, not a transport fragment

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover presidio post_call streaming for native gemini passthrough and anthropic messages

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(presidio): cap first SSE frame coalescing at 64 KiB so an unterminated first event cannot buffer unbounded

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(presidio): name raw SSE passthrough in the skipped output masking warning

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 22:08:23 -07:00
devin-ai-integration[bot]
7370650d91
fix(model_prices): bedrock bare Claude ids priced at the Global SKU (aws-bedrock sync) (#42875)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 21:27:25 -07:00
devin-ai-integration[bot]
d248cc5914
fix(fireworks_ai): route firerouter short names and bill pass-through legs at the routed model's rates (#42814)
* fix(fireworks_ai): route firerouter short names and bill pass-through legs at the routed model's rates

fireworks_ai/firerouter and fireworks_ai/firerouter/<slug> resolve to
accounts/fireworks/routers/... instead of a models/ path, and the cost
calculator falls back to the routed model's own catalog entry before the
Fireworks size buckets so a Claude leg is no longer priced at $0

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(fireworks_ai): bill routed legs under the routed model's own provider

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(fireworks_ai): require the k suffix when parsing tiered input fields

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 20:53:17 -07:00
devin-ai-integration[bot]
3b715525d3
test(integration): assert /v1/models reports max_input_tokens and max_output_tokens (#42858)
* test(integration): assert /v1/models reports max_input_tokens and max_output_tokens

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin published gpt-4o-mini limits instead of reading the cost map in-test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): name the gpt-4o-mini deployment explicitly in the /v1/models test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cite the source of the pinned gpt-4o-mini limits

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 20:42:27 -07:00
devin-ai-integration[bot]
a3d791f348
test: add tag rpm limit and tag budget reset integration coverage (#42859)
Adds two integration tests to tests/integration/spend/test_tag_budget_enforcement.py

test_key_tag_rpm_limit_rejects_the_second_request_carrying_that_tag proves
a key metadata tag_rpm_limit of 1 rejects the second request carrying that
tag with 429 while a request carrying a different tag still passes

test_tag_budget_duration_resets_spend_and_unblocks_the_tag boots an owned
proxy with a 2 to 3 second budget rescheduler, creates a tag with
max_budget 0.0001 and budget_duration 5s, observes the spend block, then
observes the tag serving again once ResetBudgetJob zeroes the tag spend

Mutation evidence

get_key_tag_rpm_limit forced to return None: the second tagged request
returned 200 instead of 429, test red

_queue_budget_linked_resets for uow.tags disabled in
_commit_budget_cascade_once: the tag stayed blocked at 422 for the full
70 second recovery window, test red

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 02:49:01 +00:00
devin-ai-integration[bot]
a4f69e058c
fix(model_prices): registry audit 2026-09-23, in-region Bedrock Claude prices (#42779)
* fix(model_prices): registry audit 2026-09-23, in-region Bedrock Claude and OpenRouter price fixes

Absorbs #42698

Co-authored-by: coldStoneSoul <shekharpachlore@gmail.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(model_prices): keep registry formatting unchanged

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: use eu.amazon.nova-pro for regional pricing probe after in-region parity

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: lock in-region parity for bare Bedrock Claude ids

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: compare every pricing field for bare Bedrock Claude parity

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(model-catalog): add above_32k cost fields to ModelInfo round-trip

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* Revert "fix(model-catalog): add above_32k cost fields to ModelInfo round-trip"

This reverts commit c71d5a3de3.

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: coldStoneSoul <shekharpachlore@gmail.com>
2026-09-24 02:47:29 +00:00
devin-ai-integration[bot]
6d5e87b71b
test(integration): assert /v1/responses usage reports Anthropic system cache write then read (#42855)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 18:52:06 -07:00
devin-ai-integration[bot]
320b40645c
fix(caching): stamp provider on sync cache-hit logs so responses spend logs record provider (#42830)
* fix(caching): stamp provider on sync cache-hit logs so responses spend logs record provider

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(caching): tighten sync cache-hit provider regression docstring

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(caching): drop redundant docstring on sync cache-hit provider test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 20:49:23 -05:00
devin-ai-integration[bot]
8e74bb0d23
fix(proxy): document request body and response schemas for the Responses API in OpenAPI (#42802)
* fix(proxy): document responses API request and response schemas in openapi

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): namespace colliding openapi defs instead of overwriting existing components

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(proxy): regenerate lazy openapi snapshot and dashboard schema types

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): require model and input in responses schema, document event stream, fix def collision refs

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): mark responses request fields readonly required

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): reuse existing OpenAPI components when a $defs entry has the same shape

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 01:28:34 +00:00
devin-ai-integration[bot]
ca86898633
fix(ollama): read the JSON thinking field on non-streaming completions (#42838)
* fix(ollama): read the JSON thinking field on non-streaming completions

Ollama's /api/generate returns reasoning in a top-level `thinking` field, but
the completion transport only looked for inline <think> tags. reasoning_content
was therefore always null, and a model that spent its whole turn reasoning
returned an empty assistant message with tokens billed.

Port the precedence the ollama_chat transport already uses: the field wins and
inline tags stay the fallback. Applied to both non-streaming paths, including
the JSON-mode text fallback. The two fields are read through a small validated
model rather than off the untyped JSON, so absent and explicitly null
`response` stay distinct exactly as before.

* fix(ollama): keep the thinking field on JSON-mode completions

The first pass read `thinking` for plain replies and for JSON-mode text that
failed to parse, but the three JSON-mode branches that succeed still dropped
it: an empty `response`, a valid JSON object, and a function-call shaped one.
A model that spent its whole turn reasoning under `format: json` therefore
still came back blank with the tokens billed.

Carry the field on all three, type the new test helper's parameters, and cover
the null and malformed `response` fallbacks.

---------

Co-authored-by: Pawan-Shahane <shahanepawan511@gmail.com>
2026-09-23 18:26:02 -07:00
devin-ai-integration[bot]
b2b0b6c17e
fix(models): sync openrouter prices from the models API (#42832)
* fix(models): sync openrouter prices from the models API

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(models): allow above_32k_tokens cost fields in price map schema test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 18:03:09 -07:00
devin-ai-integration[bot]
cad49ee171
fix(proxy): gate disable_global_guardrails on keys and teams to proxy admins (#42699)
* fix(proxy): gate disable_global_guardrails on keys and teams to proxy admins

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: cover metadata smuggle with explicit false and UI toggle gating

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): satisfy PT017 in resend-stored guardrail flag test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(proxy): keep regenerate_key_fn under the C901 ceiling via a guardrail opt-out helper

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(ui): regenerate schema.d.ts for guardrail opt-out docstrings

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): gate disable_global_guardrails on caller-sent metadata, not server defaults

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): audit cells for disable_global_guardrails admin gate

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): restore contracts.json formatting

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): share guardrail opt-out helpers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): hide the team disable_global_guardrails switch from non proxy admins

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): drop covers markers and bound the slow sink check to the sink delay

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 18:03:02 -07:00
devin-ai-integration[bot]
1e5403288c
feat(proxy): honor model_info.discoverable on the model listing endpoints (#42825)
Some checks failed
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests / caching-local (push) Waiting to run
Unit Tests / core-utils (push) Waiting to run
Unit Tests / enterprise-package (push) Waiting to run
Unit Tests / enterprise-routing (push) Waiting to run
Unit Tests / integrations (push) Waiting to run
Unit Tests / All Other Providers (push) Waiting to run
Unit Tests / Vertex AI (push) Waiting to run
Unit Tests / mcp-integration (push) Waiting to run
Unit Tests / misc (push) Waiting to run
Unit Tests / proxy-auth (push) Waiting to run
Unit Tests / proxy-endpoints (push) Waiting to run
Unit Tests / proxy-extras (push) Waiting to run
Unit Tests / proxy-infra (push) Waiting to run
Unit Tests / proxy-server (push) Waiting to run
Unit Tests / responses-caching-types (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
VS Code Extension / vscode-extension (push) Has been cancelled
* feat(proxy): honor model_info.discoverable on the model listing endpoints

A model_list entry marked model_info: {discoverable: false} is left out of GET /v1/models (OpenAI and Anthropic shapes, scope=expand and wildcard routes included), the list path of GET /v1/model/info and GET /model_group/info for every caller without the admin view, while direct requests naming the model keep routing to it. The field defaults to None so an absent flag reads as discoverable and nothing is persisted or echoed for configs that never set it.

* fix(proxy): hide flagged team models under their public name and cover the scope=expand filter

The discoverability lookup now resolves a listed name with the caller's team context, so a team-scoped deployment marked discoverable: false drops out for that team's keys under its public name instead of failing open. The scope=expand branch is now exercised by a team admin caller, and the OCI secrets test builds a real UserAPIKeyAuth instead of a spec mock that has no pydantic fields.

* perf(proxy): resolve only candidate names in the discoverable filter

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-23 17:51:16 -07:00
devin-ai-integration[bot]
ce582affaa
fix(mcp): reject duplicate MCP server names and aliases (#42791)
* fix(mcp): reject duplicate MCP server names and aliases

MCP server_name and alias were unchecked at write time, so two servers
could share one tool prefix and tool routing resolved to an arbitrary
winner. Writes now run inside an advisory-locked transaction that
rejects a collision on either column case-insensitively with a 400
naming the colliding identifier, covering create, edit, connector
import and restricted-admin submission. Server reload logs one warning
per identifier already shared in the database.

Co-Authored-By: bot_apk <apk@cognition.ai>

* fix(ui): block duplicate MCP server names and aliases before submit

The create and edit forms now check the normalized name/alias against
the loaded server list (case-insensitive, spaces to underscores, own
row excluded on edit) and show a field error instead of submitting.
Structured proxy error bodies are unwrapped so a 400 no longer renders
as 'Error: [object Object]'.

Co-Authored-By: bot_apk <apk@cognition.ai>

* fix(mcp): check identifier conflicts when an alias is cleared

Clearing an alias drops the tool prefix to the stored server_name, so
that name must go through the conflict check too; an explicit alias:null
is now treated as an identifier write. Also narrows the new db tests to
behavioral assertions instead of pinning prisma where shapes.

Co-Authored-By: bot_apk <apk@cognition.ai>

* fix(mcp): treat an empty alias as a clear in conflict checks

An empty-string alias was written unchecked even though the prefix falls
back to server_name; the update path now treats any falsy alias like a
clear. The edit form likewise compares a cleared alias as empty instead
of re-checking the alias being removed.

Co-Authored-By: bot_apk <apk@cognition.ai>

* test(mcp): cover clearing an alias to an empty string

Co-Authored-By: bot_apk <apk@cognition.ai>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: bot_apk <apk@cognition.ai>
2026-09-23 17:50:27 -07:00