Commit graph

52998 commits

Author SHA1 Message Date
devin-ai-integration[bot]
cc7aaae748
fix(cost-map): sync openrouter deepseek-v4-pro prices (#42978)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 09:27:11 -07:00
devin-ai-integration[bot]
f0e671f754
refactor(types): replace Any with proven types in 13 files (#42937)
* refactor(types): replace Any with proven types in 13 files

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(types): drop unused executor import from utils type-checking block

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(realtime): keep reserved-key filtering on azure realtime health params

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(realtime): pin reserved-key filtering in azure realtime health auth params

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(realtime): exercise the real azure header builder in the reserved-key test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 09:02:02 -07:00
devin-ai-integration[bot]
991c339946
fix(cost-map): sync openrouter deepseek v4 flash, v4 pro and v4.1 flash prices (#42974)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 08:53:31 -07:00
devin-ai-integration[bot]
cfa2830bde
fix(bedrock): extrapolate global cris pricing for gpt-5.4 and gpt-5.5 (#42971)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 08:30:48 -07:00
devin-ai-integration[bot]
553f0b6ee7
feat(cost-map): sync azure models, add MAI-Image-2.6, deepseek-v4.1-flash, muse-spark-1.3 (#42970)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 08:27:42 -07:00
devin-ai-integration[bot]
58d7cafb97
fix(cost-map): sync openrouter deepseek-v4-pro prices (#42969)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 08:24:27 -07:00
devin-ai-integration[bot]
e135a199ad
fix(proxy): fail parked DB lookups at a deadline and flip readiness while they stall (#42654)
Some checks failed
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
LiteLLM Rust / rust-wheel (push) Has been cancelled
* fix(proxy): fail parked DB lookups at a deadline and flip readiness while they stall

Under a load burst with a slow authentication database every request parked inside
the pod with no deadline while /health/readiness kept answering 200 (its own ping
gets a fresh connection), so the load balancer kept sending traffic until the pod hit
its memory limit, and the parked requests completed against the provider minutes
after every client had hung up

Every pre-request read (key, team, user, end user, budget, membership, organization,
object permission, jwt mapping, project, proxy budget, spend counter reseed) now runs
under one deadline, PROXY_DB_LOOKUP_DEADLINE_SECONDS (default 10 s). A lookup that
hits it fails the request with the existing 503 "authentication database is
temporarily unreachable" answer, honours allow_requests_on_db_unavailable, and never
triggers the transport reconnect (the transport is fine, the query is slow), which is
what turned the repro's stall into "too many clients". Writes stay unbounded

A deadline hit marks the pod stalled for PROXY_DB_LOOKUP_STALL_WINDOW_SECONDS
(default 30 s, 0 disables), during which /health/readiness answers 503 with
"db": "stalled" behind the same fail-open gate, so the pod leaves rotation before it
fills its memory. The existing litellm_in_flight_requests gauge already exposes the
parked set on /metrics

The deadline is enforced on the wall clock: bounded_db_lookup waits on the lookup
task with asyncio.wait and raises DBLookupDeadlineExceeded when the deadline passes
even if the lookup absorbs its cancellation, where asyncio.wait_for on 3.12+ would
sit on the cancelled task for as long as it takes

The failure spend-log row no longer re-runs the key and team lookups when the
failure itself is a database connection or deadline error, so a request that hit
the deadline is answered after one deadline instead of two

* fix(proxy): bound the spend counter gate wait and narrow the stalled lookup shortcut

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): keep the global spend lookup on the prisma client handle

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 10:09:49 -05:00
devin-ai-integration[bot]
b8f3ba03b3
fix(cost-map): sync openrouter deepseek-v4-pro prices (#42964)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 07:57:26 -07:00
devin-ai-integration[bot]
11ed3335c8
fix(cost-map): sync openrouter deepseek-v4-pro prices (#42956)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 07:18:08 -07:00
devin-ai-integration[bot]
77c7a870d5
fix(cost-map): add azure realtime, audio and partner model rows (#42954)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 06:55:14 -07:00
devin-ai-integration[bot]
3acdfda19c
fix(cost-map): add batch prices for vertex gemini-3.8-flash-cyber (#42953)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 06:52:06 -07:00
devin-ai-integration[bot]
785d4974ba
fix(cost-map): sync openrouter prices for deepseek v4 and glm-5.3 (#42952)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 06:49:17 -07:00
devin-ai-integration[bot]
8477fe4108
fix(proxy): list key and team model aliases in GET /v1/models (#42908)
* fix(proxy): list key and team model aliases in GET /v1/models

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): keep alias listing helpers within the type discipline budget

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): cover alias rows on GET /v1/models and /v1/models/{id}

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): apply team then key aliases like chat completions and keep the alias as the retrieved id

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): apply key aliases twice like chat completions and skip only malformed alias entries

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): apply the global model_alias_map between the key alias passes like chat completions

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): list only the caller's own aliases and never rewrite a listed model id on retrieval

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(proxy): ruff format model_info alias lookup

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): hide undiscoverable names from model retrieval so an alias named like one resolves to its target

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): keep undiscoverable models retrievable by id while excluding them from the alias guard

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(proxy): pass an immutable name sequence into the model_info alias guard

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): type the model list alias test helpers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): annotate the new alias listing test fixtures and helpers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 06:41:13 -07:00
devin-ai-integration[bot]
c4b56b6ada
fix(cost-map): update azure gpt-4.1-nano and gpt-4o-2024-05-13 retirement dates (#42947)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 06:15:55 -07:00
devin-ai-integration[bot]
ddc7ee6838
feat(bedrock): add gpt-5.4 and gpt-5.5 us and global inference profile pricing (#42941)
* feat(bedrock): add gpt-5.4 and gpt-5.5 us and global inference profile pricing

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(bedrock): drop supports_max_reasoning_effort from gpt-5.4 and gpt-5.5 rows

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 04:54:10 -07:00
devin-ai-integration[bot]
2eeb16266b
fix(cost-map): sync vertex-ai rows (gemma 4 maas cache price, chirp_2) (#42942)
* fix(cost-map): sync vertex-ai rows (gemma 4 maas cache price, chirp_2)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(vertex-ai): expect chirp_2 as speech-to-text model now that catalog row exists

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 04:39:09 -07:00
devin-ai-integration[bot]
d17c0d7724
refactor: daily fresh tech debt cleanup, rolling PR (#42710)
* refactor: clear fresh tech debt from the last 24 hours (2026-09-05, 2026-09-06)

Drop the TID251 cast import and both cast-ok casts from the refusal
message_delta rebuild by narrowing the TypedDict union on its type literal,
drop the redundant Mapping cast after the isinstance check in
_mapping_field, and type the Lyria predict read-only helpers as
Mapping[str, object] instead of a bare dict with mutable-ok.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor: clear fresh tech debt from the last 24 hours (2026-09-09)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor: clear fresh tech debt from the last 24 hours (2026-09-10)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor: keep the pre-existing cost-estimate comment and usage cost read out of the cleanup

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor: clear fresh tech debt from the last 24 hours (2026-09-13)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor: keep the model info pricing helper out of the cleanup

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor: drop suppressions that no longer suppress anything (2026-09-16)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor: keep the rebind-ok reason inside the line limit

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(anthropic): rebuild the refusal message_delta by spreading the chunk

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor: clear fresh tech debt from the last 24 hours (2026-09-17)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor: clear fresh tech debt from the last 24 hours (2026-09-18)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor: clear fresh tech debt from the last 24 hours (2026-09-19)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor: keep the pre-existing protected-resource return type out of the cleanup

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor: clear fresh tech debt from the last 24 hours (2026-09-20)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor: type fresh getattr, Any, and bare dict debt from 2026-09-22

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(mcp): keep the string guard on tools/list next_cursor

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(vercel_ai_gateway): type the embedding error headers dict

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(techdebt): fix inert suppressions and missing Final in 2026-09-22 changes

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(techdebt): shorten suppression reason to fit line length

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(techdebt): format provider spread so its suppression sits on the literal

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(techdebt): drop the logger extras suppression that LIT013 now flags as inert

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(techdebt): clear fresh suppressions, Any aliases and slop from the 24h window

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(vercel): take a read-only headers mapping in get_error_class

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: mateo <mateo@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 04:17:23 -07:00
devin-ai-integration[bot]
8bbe7edb71
fix(cost-map): add azure deprecation dates for regional gpt-6 rows (#42933)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 03:03:53 -07:00
devin-ai-integration[bot]
b550db1b4f
fix(cost-map): update openrouter kimi-k2.7-code input price (#42932)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 02:58:44 -07:00
devin-ai-integration[bot]
04f3ade124
feat(cost-map): add wandb DeepSeek-V4.1-Flash and gemma-4-26B-A4B-it (#42924)
* feat(cost-map): add wandb DeepSeek-V4.1-Flash and gemma-4-26B-A4B-it

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(cost-map): mark wandb gemma-4-26B-A4B-it as reasoning capable

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 02:07:12 -07:00
devin-ai-integration[bot]
1c8a0ff602
fix(cost-map): add azure deprecation dates for gpt-6 and gpt-realtime-whisper (#42897)
* fix(cost-map): add azure deprecation dates for gpt-6 and gpt-realtime-whisper

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(cost-map): add azure deprecation dates to dated gpt-6 keys

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 01:06:54 -07:00
yuneng-jiang
265f874eaf
test(integration): edge-case matrices for malformed token limits and callback_settings shapes (#42895)
* test(integration): edge-case matrices for malformed token limits and callback_settings shapes

Extends the integration suite so two classes of issues found by gauntlet
reviews are caught end to end against a real proxy:

- non-numeric or odd model_info token limits (from /model/new and from
  config YAML) must be listed as absent on /v1/models, /models,
  /v1/models/{id} and /model/info, keep sibling models listed, and still
  serve chat
- every callback_settings shape (top level and per consumer) must let
  the proxy boot, register the configured callbacks and serve chat

Four product bugs on main surfaced by the matrices are recorded as
BUG skips per the suite convention: chat 500 and /model_group/info 500
on non-numeric token limits, a startup crash on a non-object
callback_settings, and otel silently dropped on a non-object
callback_settings.otel

* test(integration): pin the exact coerced value for numeric-edge token limits

Addresses review feedback: the numeric-edge matrix only asserted
'int or absent'. It now asserts the listed value for each case on
/v1/models, /models and /v1/models/{id}, which also lets the listing
helper drop its optional-expectation branch.
2026-09-24 01:05:55 -07:00
devin-ai-integration[bot]
d11705a24d
fix(cost-map): source for bedrock mantle gpt-5.6 luna, sol, terra and grok-4.6 (#42898)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 01:04:31 -07:00
devin-ai-integration[bot]
09ebb28473
fix(s3_v2): bound concurrent S3 uploads per flush and add opt-in JSONL batch files (#41258)
* fix(s3_v2): bound concurrent S3 uploads per flush and add opt-in JSONL batch files

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(s3_v2): keep failed uploads queued, parse env-backed flags, add integration coverage

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(s3_v2): type test helpers and honor constructor bound when config value is null

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(s3): annotate required casts for the type-discipline gate

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(s3_v2): keep tenant prefixes, stable retries and cold storage safety in batch file mode

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(s3_v2): cover root-level batch file keys for codecov patch target

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(s3_v2): audit matrix across chat, messages and responses surfaces with sink faults

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(s3_v2): read sink objects under the lock in the audit cells

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(s3_v2): rebind the retry queue instead of slicing in place

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(s3_v2): ignore stray non-POST requests in the surface upstream

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(s3_v2): keep fake upload state on the fake client instead of nonlocal counters

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng <yucheng@berri.ai>
2026-09-24 07:32:00 +00:00
devin-ai-integration[bot]
3fb6f8740b
test(integration): add read-replica routing harness to the CircleCI integration suite (#42692)
* test(integration): add read-replica routing harness

* refactor(integration): hoist the maintenance url imports

* fix(integration): keep per-test databases and the witness sequence readable under replica roles

* fix(integration): opt bespoke database and pool tests out of the injected read replica

* test(integration): commit recorded replica routing expectations

* fix(integration): judge routing by role containment so shrinking role sets do not fail

* fix(integration): run the pool-limit shutdown choreography on the superuser database url

* ci(integration): add the mcp group to the replica matrix

* fix(integration): judge routing by exact role sets with a named either-role allowlist

* test(integration): drop containment-era routing expectations for re-recording

* chore(integration): drop docstrings from the replica harness scripts

* docs(integration): describe exact routing matching and the either-role list

* test(integration): record exact replica routing expectations

* test(integration): allow the SELECT 1 health probe on either role

* test(integration): replace committed routing expectations with an on-demand base-vs-head parity run

* test(integration): fix parity env scope, readme wording, and seed-deterministic serialization test

* test(integration): make the sorted-role serialization test deterministic in-process

* test(integration): swap all product code in parity runs and pin role gains

* ci(integration): force tracked-file removal before parity checkout

---------

Co-authored-by: yuneng <yuneng@berri.ai>
2026-09-24 00:25:03 -07:00
devin-ai-integration[bot]
6dbd65b230
fix(passthrough): log upstream 4xx/5xx error bodies and carry them into the failure hook (#42695)
* test(integration): reproduce passthrough upstream error body missing from logs and spend row

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(passthrough): log upstream 4xx/5xx error bodies and carry them into the failure hook

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(error_normalization): let the passthrough prefix win over upstream body text

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(passthrough): honor message redaction for upstream error bodies

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(passthrough): bound the upstream error body read and sanitize it before logging

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(passthrough): use the Sequence import directly in the allowed-routes cast

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(passthrough): rechunk the upstream error stream so the preview read stays bounded

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): audit matrix for passthrough upstream error visibility

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(passthrough): drop the restating docstring on the upstream failure logger

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): drop the retired covers markers from the passthrough error tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(passthrough): keep the upstream status when the error body peek fails

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(passthrough): cover the relay aclose in the mid-read failure test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(passthrough): relay decoded partial body on mid-read failure

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 23:42:24 -07:00
devin-ai-integration[bot]
a5b9b6da4d
feat(cost-map): add vertex_ai/gemini-3.8-live (#42891)
* feat(cost-map): add vertex_ai/gemini-3.8-live

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(cost-map): price vertex_ai/gemini-3.8-live video tokens

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 23:41:10 -07:00
devin-ai-integration[bot]
42d8d08815
fix(cost-map): source and chat completions endpoint for bedrock mantle gpt-5.4 and gpt-5.5 (#42890)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 23:25:24 -07:00
devin-ai-integration[bot]
8f7cea5fcd
fix(cost-map): sync openrouter prices and add fireworks ember-1 (#42889)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 23:21:59 -07:00
devin-ai-integration[bot]
093ceb576d
fix(proxy): do not requeue a daily spend batch whose commit already left for postgres (#42786)
* fix(proxy): do not requeue a daily spend batch whose commit already left for postgres

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): settle an interrupted daily spend commit from the shutdown flush instead of blocking the cancelled tick

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): burst two workers and SIGTERM during daily spend COMMIT, expect exactly once

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 22:54:19 -07:00
devin-ai-integration[bot]
1519032d90
fix(proxy): keep deployment labels on cache-hit post_call guardrail rejections (#42780)
* fix(proxy): keep deployment labels on cache-hit post_call rejections

A post-call failure on a response served from the litellm cache set no first_api_call_start_time, so the failure hook flagged it as rejected before routing and dropped the model_id and provider labels

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): read the cache hit from caching_details in the failure hook

model_call_details[cache_hit] is stamped inside the enqueued success handler, so a post-call failure can observe it too early; logging_obj.caching_details is set synchronously before the cached response returns

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover cache-hit guardrail reject deployment labels across endpoints, modes and chaos

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): bound the worker-kill reject count by in-flight losses

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): assert provider and model labels on the cache-hit regression test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 22:48:04 -07:00
devin-ai-integration[bot]
251fdf0308
fix(logging): scan the exceeded budget wording linearly so a crafted error message cannot stall the proxy (#42778)
* fix(logging): bound the exceeded budget regex so a crafted error message cannot stall the proxy

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): audit normalized_error clustering on long messages with a real two worker proxy

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): poll for the budget denial and correlate upstream 503 bursts by request marker

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(logging): replace the bounded exceeded budget regex with a linear scan that keeps the original semantics

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): compare upstream error wording against the decoded message

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): tolerate a reaped worker while listing proxy children

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 22:44:23 -07:00
devin-ai-integration[bot]
0094cff47a
fix(cost-map): add azure gpt-realtime-mini-2025-10-06 deprecation date (#42885)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 22:38:31 -07:00
devin-ai-integration[bot]
6b1ec4cd3a
fix(models): correct fireworks kimi k3 us pricing to the published rate (#42884)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 22:26:59 -07:00
devin-ai-integration[bot]
dfb61e9b48
fix(cost-map): add azure gpt-realtime-mini deprecation date (#42883)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 22:24:37 -07:00
devin-ai-integration[bot]
001179a636
fix(cost-map): sync vertex-ai deprecation dates from Vertex model lifecycle pages (#42882)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 22:21:05 -07:00
devin-ai-integration[bot]
0fb999b613
fix(cost-map): halve openrouter deepseek-v4-flash-0731 output price (#42881)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 22:18:28 -07:00
devin-ai-integration[bot]
b8154bcbc0
fix(presidio): stream non-Anthropic raw SSE through the post_call hook unbuffered (#42777)
* fix(presidio): stream non-Anthropic raw SSE through the post_call hook unbuffered

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(presidio): keep the pytest.raises block to a single await

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(presidio): move raw SSE format check into a helper to keep hook complexity flat

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(presidio): fold the raw SSE format check into the existing bytes branch to stay within the complexity budget

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(presidio): decide raw SSE stream shape on a complete first frame, not a transport fragment

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cover presidio post_call streaming for native gemini passthrough and anthropic messages

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(presidio): cap first SSE frame coalescing at 64 KiB so an unterminated first event cannot buffer unbounded

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(presidio): name raw SSE passthrough in the skipped output masking warning

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 22:08:23 -07:00
devin-ai-integration[bot]
b21b20ed13
feat(vertex_ai): add gemini-3.8-flash-cyber pricing (#42879)
Some checks failed
LiteLLM Rust / rust-lint (push) Waiting to run
LiteLLM Rust / rust-test (push) Waiting to run
LiteLLM Rust / rust-wheel (push) Waiting to run
Terraform Provider / gofmt, vet, build, test (push) Waiting to run
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Waiting to run
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
* feat(vertex_ai): add gemini-3.8-flash-cyber pricing

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(vertex_ai): mark gemini-3.8-flash-cyber minimal reasoning unsupported

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 21:55:55 -07:00
devin-ai-integration[bot]
9d12c217af
fix(models): correct gemini robotics er 2 preview audio input price (#42877)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 21:42:23 -07:00
devin-ai-integration[bot]
e99c5d30b6
feat(cost-map): add retired azure gpt-5 chat and o1-preview data zone rows (#42878)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 21:32:27 -07:00
devin-ai-integration[bot]
7370650d91
fix(model_prices): bedrock bare Claude ids priced at the Global SKU (aws-bedrock sync) (#42875)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 21:27:25 -07:00
devin-ai-integration[bot]
15a2bd8b28
fix(models): add fireworks 2026-09-25 deprecation dates for glm 5.2, kimi k2.6, kimi k2.7 code, deepseek v4 and muse glimmer serverless rows (#42874)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 21:07:44 -07:00
devin-ai-integration[bot]
871f562f72
fix(models): add openai deprecation date for gpt-5-chat-latest and gpt-5-chat (#42872)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 21:01:10 -07:00
devin-ai-integration[bot]
1f1817ed39
fix(models): add azure gpt-4o-mini-transcribe and gpt-4o-mini-tts deprecation dates (#42873)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 21:00:33 -07:00
devin-ai-integration[bot]
d248cc5914
fix(fireworks_ai): route firerouter short names and bill pass-through legs at the routed model's rates (#42814)
* fix(fireworks_ai): route firerouter short names and bill pass-through legs at the routed model's rates

fireworks_ai/firerouter and fireworks_ai/firerouter/<slug> resolve to
accounts/fireworks/routers/... instead of a models/ path, and the cost
calculator falls back to the routed model's own catalog entry before the
Fireworks size buckets so a Claude leg is no longer priced at $0

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(fireworks_ai): bill routed legs under the routed model's own provider

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(fireworks_ai): require the k suffix when parsing tiered input fields

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 20:53:17 -07:00
devin-ai-integration[bot]
3b715525d3
test(integration): assert /v1/models reports max_input_tokens and max_output_tokens (#42858)
* test(integration): assert /v1/models reports max_input_tokens and max_output_tokens

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): pin published gpt-4o-mini limits instead of reading the cost map in-test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): name the gpt-4o-mini deployment explicitly in the /v1/models test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): cite the source of the pinned gpt-4o-mini limits

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 20:42:27 -07:00
devin-ai-integration[bot]
58e41e697b
chore(vertex_ai): add deprecation dates for retired claude 3 and jamba 1.5 partner models (#42867)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 20:37:43 -07:00
devin-ai-integration[bot]
184969df37
fix(models): add fireworks deprecation date for glm 5.2 fast serverless rows (#42866)
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-23 20:28:04 -07:00
devin-ai-integration[bot]
a3d791f348
test: add tag rpm limit and tag budget reset integration coverage (#42859)
Adds two integration tests to tests/integration/spend/test_tag_budget_enforcement.py

test_key_tag_rpm_limit_rejects_the_second_request_carrying_that_tag proves
a key metadata tag_rpm_limit of 1 rejects the second request carrying that
tag with 429 while a request carrying a different tag still passes

test_tag_budget_duration_resets_spend_and_unblocks_the_tag boots an owned
proxy with a 2 to 3 second budget rescheduler, creates a tag with
max_budget 0.0001 and budget_duration 5s, observes the spend block, then
observes the tag serving again once ResetBudgetJob zeroes the tag spend

Mutation evidence

get_key_tag_rpm_limit forced to return None: the second tagged request
returned 200 instead of 429, test red

_queue_budget_linked_resets for uow.tags disabled in
_commit_budget_cascade_once: the tag stayed blocked at 422 for the full
70 second recovery window, test red

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-24 02:49:01 +00:00