Commit graph

52864 commits

Author SHA1 Message Date
shrey kharbanda
4347e20149 fix(router): expose input_cost_per_reference_pixel on ModelGroupInfo
Some checks failed
LiteLLM Rust / rust-lint (push) Has been cancelled
LiteLLM Rust / rust-test (push) Has been cancelled
LiteLLM Rust / rust-wheel (push) Has been cancelled
Terraform Modules / fmt, validate, test (aws) (push) Has been cancelled
Terraform Modules / fmt, validate, test (gcp) (push) Has been cancelled
Terraform Provider / gofmt, vet, build, test (push) Has been cancelled
Terraform Provider / Provider endpoints vs proxy OpenAPI schema (push) Has been cancelled
2026-09-23 23:54:38 +00:00
shrey kharbanda
6a1bf0e704 fix(images): keep the marker prefix at a JPEG fill chunk boundary 2026-09-23 16:24:54 +00:00
shrey kharbanda
314e3e8592 fix(images): skip arbitrarily long JPEG fill runs in one hop 2026-09-23 16:03:17 +00:00
shrey kharbanda
24155efd34 fix(images): keep JPEG marker alignment after a fill byte before a large segment 2026-09-23 15:47:11 +00:00
shrey kharbanda
b929e4d22d fix(terraform): forward explicit zero input_cost_per_reference_pixel 2026-09-23 14:41:58 +00:00
shrey kharbanda
1eaddc47e9 feat(terraform): add input_cost_per_reference_pixel to model resource 2026-09-23 07:27:22 +00:00
shrey kharbanda
0bb9c54d11 fix(images): harden reference header measurement and cover pricing edge cases 2026-09-23 07:27:22 +00:00
shrey kharbanda
93ebd1116e chore(ui): regen schema.d.ts for input_cost_per_reference_pixel 2026-09-23 06:57:38 +00:00
shrey kharbanda
6db3435299 fix(azure_ai): price reference pixels from the resolved deployment model info 2026-09-23 06:38:32 +00:00
shrey kharbanda
4dc2f3be1a fix(images): read BITMAPCOREHEADER bitmaps and name every measured format 2026-09-23 02:56:41 +00:00
shrey kharbanda
d3b1205381 test(schema): allow input_cost_per_reference_pixel in cost map schema 2026-09-23 02:51:46 +00:00
shrey kharbanda
80c1eb6572 fix(images): suppress non-exhaustive match in header dispatch 2026-09-23 02:35:12 +00:00
shrey kharbanda
848484d68c fix(images): measure GIF and BMP reference image headers 2026-09-23 02:27:27 +00:00
shrey kharbanda
770343bba6 fix(model-catalog): round-trip input_cost_per_reference_pixel in ModelInfo 2026-09-23 02:19:37 +00:00
shrey kharbanda
b9b3078618 style(azure_ai): justify model_cost cast for type discipline gate 2026-09-23 02:14:54 +00:00
shrey kharbanda
f6e792058b test(images): cover malformed JPEG, WebP and PNG headers 2026-09-23 02:10:10 +00:00
shrey kharbanda
661d6992ca refactor(images): make header parser returns explicit 2026-09-23 02:10:10 +00:00
shrey kharbanda
05439eecc1 fix(azure_ai): price FLUX.2-flex megapixels as 1024x1024 pixels 2026-09-23 02:10:10 +00:00
shrey kharbanda
eea8528880 refactor(images): walk JPEG segments with a bounded loop 2026-09-23 02:03:47 +00:00
shrey kharbanda
5d04e97f8c fix(azure_ai): bill FLUX.2 edit reference images by measured pixels
Azure meters FLUX.2 Flex image edits per generated megapixel and per reference megapixel, but the proxy priced only the generated image.

The shared image edit handler now reads the PNG, JPEG or WebP header of each reference before the request transform consumes the stream, sums the pixels and stores one integer under hidden_params["reference_pixels"] on the ImageResponse. The Azure AI image cost calculator multiplies that value by the new input_cost_per_reference_pixel rate when no provider usage block is present, so token reporting models keep their existing cost. The rate on azure_ai/FLUX.2-flex comes from the Azure retail prices API Flex Ref Megapixel row (0.05 USD per megapixel)
2026-09-23 02:03:47 +00:00
devin-ai-integration[bot]
b0ac23d385
feat(logger): dispatch Python logging through the Rust diagnostics processor (#42616)
* feat(logger): add shared Rust diagnostics and Python logging bridge

* feat(logger): dispatch diagnostic processing through Rust

* chore: regenerate Cargo.lock after rebase

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: allowlist bounded logging tree walkers in recursive detector

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* perf(logger): skip decoding plain access arguments

* test(logger): skip embedded-python logger test when litellm deps are absent

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style: cargo fmt

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: expect NativeDiagnosticProcessor in the native public surface

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(stub): export NativeDiagnosticProcessor via __new__ in _native.pyi

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(tracing): rename logger crate and document host sink contract

* test(logger): cover exc, stack, and nested extras in the diagnostic filter

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(logger): keep rendered redacted line when template scan flags a key pattern

The blanket REDACTED for a changed msg/color template discarded lines
whose rendered form was already redacted by the same pipeline, e.g.
'password=%s' became 'REDACTED' instead of 'password=REDACTED'. Only
fall back to REDACTED when the rendered form did not change either,
which is where interpolation can mangle the key pattern the scrub
would otherwise see.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci(rust): install python deps so the logger bridge test runs

The end-to-end bridge test skipped silently when litellm's Python deps
were absent. uv sync --no-install-project installs them without a
maturin build, and PYTHONPATH makes them visible to the embedded
interpreter

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-22 18:44:15 -07:00
devin-ai-integration[bot]
2d2b7e8fa0
test(cost): repoint the Azure image cost test at gpt-image-2 (#42631)
#42435 removed the OpenAI dall-e-3 cost-map row that
test_dalle_3_azure_cost_tracking pinned, and Azure retired DALL-E 3
inference on 2026-03-04, so the test now mirrors a real Azure
gpt-image-2 response and derives the expected cost from the cost map.

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-22 18:37:41 -07:00
devin-ai-integration[bot]
1953603043
feat(ci): close open pull requests superseded by a merged fix on their linked issue (#42609)
* feat(ci): close open pull requests superseded by a merged fix on their linked issue

* fix(ci): replace the mixed-anchor release-line regex with a plain predicate

* fix(ci): recover a half-done close, page linked pull requests, trust only the workflow's marker

* fix(ci): close superseded pull requests whatever closed the issue and fail the job when the script throws

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-22 18:24:35 -07:00
devin-ai-integration[bot]
327515a3ba
fix(mcp): preserve credential authority in DCR bridge authentication (#42563)
* fix(mcp): admit dcr_bridge envelope alongside an explicit litellm credential and mint under jwt principals

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(mcp): suppress LIT002 on concrete dict header payloads

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(mcp): mint bridge envelope for jwt mapped to a key without a user_id

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(mcp): mint and admit bridge envelopes under the master key

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(mcp): bind mapped JWT envelopes to stored key tokens

* fix(mcp): preserve master envelope scope enforcement

* fix(mcp): reject bridge minting that loses JWT restrictions

---------

Co-authored-by: joshua <joshua@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
2026-09-22 18:09:52 -07:00
tin-berri
5b287f66d7
feat(ui): add native LiteAdmin assistant (#42443) 2026-09-22 18:06:40 -07:00
moe-berri
2157351004
feat(ui): simplify auto-router setup and clarify feature limits (#42625)
* feat(ui): simplify auto-router setup and clarify feature limits

* fix(ui): validate auto-router drafts before saving

* fix: keep auto-router allowances consistent after deletes and refreshes
2026-09-22 18:03:32 -07:00
yujonglee
630c4624f6
test(e2e): add secret manager lanes for HashiCorp Vault and CyberArk Conjur (#42503)
* test(e2e): add a HashiCorp Vault secret manager lane

key_management_system had no end-to-end coverage: the Rust crates and the
Python unit tests all run against mocked managers. This adds a secret_manager
suite that drives a proxy configured with hashicorp_vault against a real Vault.

The tests seed a fresh secret name per test with the runner's OPENAI_API_KEY and
register a deployment pointing at os.environ/<name>. The proxy's env never holds
that name, so get_secret's os.environ fallback cannot mask a broken manager, and
a bogus value in Vault must come back as the provider's 401. Virtual keys are
checked written to and removed from Vault under prefix_for_stored_virtual_keys.

The setting is global to the proxy, so the lane has its own config and the
secret_manager_vault opt-in marker, and stays out of the per-PR selector.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(e2e): make the secret manager suite backend-agnostic

One marker and opt-in (secret_manager / E2E_SECRET_MANAGER=<system>) pick the
backend from secret_backends.BACKENDS. The tests reach the manager through a
SecretStore protocol, and each backend contributes a secret_store_<system>.py
module, a registry entry, and gateway/secret_manager_<system>_ci_config.yml.
requires_capability deselects tests a backend cannot support (CyberArk does
not delete), and test_secret_backends.py checks every lane config against its
backend without a live stack.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(e2e): add a CyberArk Conjur secret manager lane

Adds cyberark as the second secret_manager backend: a Conjur store over its
REST API (policy-declared variables, raw-text values, policy-patch teardown),
its lane config, and a registry entry without deletes_stored_keys, since the
proxy's CyberArk delete answers not_supported and Conjur keeps the key.

secret_manager/backend.sh up|down <system> boots any backend in Docker and
writes proxy.env and tests.env, so every lane runs the same way; the registry
test checks the script boots exactly the registered backends. e2e_http gains
send_text_external for APIs that speak raw text rather than JSON.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(e2e): give the secret manager suite a client with .proxy and address review

The shared resources fixture reads client.proxy, so a bare ProxyClient errored every
live test at setup. backend.sh now writes its env under a per-user directory with
umask 077, the markerless unit tests are gone per tests/e2e/AGENTS.md, and routine
comments are trimmed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-09-22 18:02:32 -07:00
devin-ai-integration[bot]
944f44d82b
fix(utils): isolate callback errors in async_post_call_success_deployment_hook (#42535)
* fix(utils): isolate callback errors in async_post_call_success_deployment_hook

A callback that raises inside async_post_call_success_deployment_hook no longer
fails the completed request. The exception is logged with the callback class and
call_type, the response stays as it was, and later callbacks still run. Guardrail
callbacks are exempt because raising is how a post-call guardrail blocks

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(utils): drop unrelated ruff autofixes from test_utils

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(utils): drop fastapi import from guardrail propagation regression

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(utils): cover every success deployment hook call type with a raising hook

Parametrize the unit regression over video, embedding, responses, image, rerank,
transcription, chat and anthropic messages responses and assert the failure log
names the callback and call type. Run the integration test through a real proxy
for /v1/chat/completions, /v1/embeddings, /v1/responses and /v1/videos

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(integration): move raising success hook cases into the existing callback delivery file

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-22 17:59:50 -07:00
devin-ai-integration[bot]
a80379baf8
fix(proxy): keep the in-flight daily spend batch when shutdown cancels the flush (#42593)
* fix(proxy): keep the in-flight daily spend batch when shutdown cancels the flush

A daily spend batch drained from the in-memory queue was dropped for good when
the scheduler tick was cancelled by shutdown, because asyncio.CancelledError
bypasses the except Exception requeue. The flush now requeues the drained rows
on cancellation and re-raises, and each daily batch upsert runs in an
interactive transaction so a statement that already reached Postgres is rolled
back with the cancel instead of committing behind the requeue

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): requeue the cancelled daily spend batch before its rollback returns

Behind a lock the rollback of the cancelled interactive transaction only
returns once the blocked statement does, which is after the shutdown flush
has already run. The commit now runs as a shielded task so the cancelled
tick requeues the batch at once and lets the rollback finish in the
background. The final flush then finds the rows and writes them exactly once

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): give the recording db a transaction seam for the bulk upsert tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): route the mocked daily tag spend upsert through the transaction seam

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): restore the drained Redis tag batch when shutdown cancels its commit

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-22 17:58:08 -07:00
devin-ai-integration[bot]
21a2d828df
ci(test-unit): drop dead misc shard paths and skip missing paths with a warning (#42603)
Eight directories the misc shard named moved to tests/unit on 2026-09-20, and one
missing path makes pytest-xdist collect [0 items] for the whole shard, which the
exit-5 tolerance turned into a green required check running nothing. The shared
Run tests step now drops a path that does not exist with a :⚠️: and runs
pytest over the rest, keeping option tokens verbatim.

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-23 00:56:32 +00:00
devin-ai-integration[bot]
8ee6bab529
fix(bedrock): treat blank AWS_S3_* env vars as unset for batch jobs (#42528)
* test(e2e): pin bedrock batch create with blank S3 env vars

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(bedrock): treat blank S3 env vars as unset for batch jobs

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(e2e): trim blank S3 env gateway config

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(e2e): register blank_s3_env capability and clean gateway tempdir

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(e2e): move blank S3 env batch test to its own module

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-22 17:55:22 -07:00
devin-ai-integration[bot]
38f0eb876b
test(realtime): drop legacy InvalidStatusCode tests and pin websockets imports (#42624)
The two redaction tests raised the deprecated InvalidStatusCode, which the
websockets 15 asyncio client never raises, and asserted the raw 403 close
code that the handshake refusal path replaced with 1008. The refusal path
builds its close reason from the status code alone, so there is no secret
to redact there, and the handshake refusal tests already cover the error
event and the 1008 close.

Those refusal tests only passed when run after a sibling test had imported
websockets.asyncio.client, since websockets lazy-loads its exceptions
submodule. Importing InvalidStatus, Response, and Headers from their own
submodules makes them pass in any order.

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-22 17:42:12 -07:00
mubashir1osmani
b4ccb5b747
fix(s3): replace colons in generated log filenames (#40452)
Some checks are pending
Unit Tests: Documentation Validation / documentation (push) Waiting to run
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests / caching-local (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests / All Other Providers (push) Waiting to run
Unit Tests / Vertex AI (push) Waiting to run
Unit Tests / mcp-integration (push) Waiting to run
Unit Tests / misc (push) Waiting to run
Unit Tests / proxy-auth (push) Waiting to run
Unit Tests / proxy-endpoints (push) Waiting to run
Unit Tests / proxy-extras (push) Waiting to run
Unit Tests / proxy-infra (push) Waiting to run
Unit Tests / proxy-server (push) Waiting to run
Unit Tests / responses-caching-types (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
Unit Tests / core-utils (push) Waiting to run
Unit Tests / enterprise-package (push) Waiting to run
Unit Tests / enterprise-routing (push) Waiting to run
Unit Tests / integrations (push) Waiting to run
* fix(s3): replace colons in generated log filenames

Bedrock and Vertex AI batch file uploads use s3:// and gs:// URIs as
response ids. The shared filename sanitizer replaced slashes but kept
the scheme colon, producing log object keys that Hadoop-style consumers
reject as a relative path in an absolute URI.

Fixes #40234

* test(s3): drop docstrings flagged by review

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-22 17:33:44 -07:00
devin-ai-integration[bot]
7688f56256
fix(pricing): drop the duplicate cache_read_input_token_cost_batches key from 23 entries (#42623)
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-22 17:28:48 -07:00
yuneng-jiang
4f93e2c3da
test: point CircleCI-only suites at models still in the cost map (#42617)
* test: point CircleCI-only suites at models still in the cost map

#42435 removed cost map entries past their deprecation date and #42437 added
litellm_uisettings to the config-synced tables, but both only updated
tests/test_litellm. The CircleCI-only suites (local_testing, llm_translation,
logging_callback_tests, litellm_utils_tests, unit) kept using the removed
models or the old table list and went red on main.

Each test keeps its assertions and swaps the removed model for a current one
with the same provider and capabilities. The fireworks tests pick a vision
model from the cost map because #34941 set supports_vision false on
minimax-m3, and the vertex image provider test injects the image model set
because #42435 removed every vertex_ai-image-models entry.

* test(vertex_ai): register the image model through add_known_models in the provider test
2026-09-22 17:28:34 -07:00
devin-ai-integration[bot]
ca95fc2bd4
fix: answer get_api_base for github_copilot and chatgpt without running the login flow (#42602)
* fix: answer get_api_base for github_copilot and chatgpt without running the login flow

* refactor(get_api_base): dispatch the provider helpers with if-chains

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-22 17:23:44 -07:00
Pawan Shahane
9bad2c35e5
fix(ollama): send PNG and JPEG images without requiring Pillow. (#41979)
* fix(ollama): send PNG and JPEG images without requiring Pillow

The ollama/ completion transport imported Pillow before it looked at the image, so every image request failed with a 500 on installs without Pillow. That includes the Docker image, where Pillow is only a CI dependency

Detect PNG and JPEG from their leading bytes and pass them through untouched. Pillow is now imported only when another format has to be re-encoded as JPEG, and that case still raises the same install hint

* fix(ollama): address Greptile findings on image conversion

Catch all exceptions on Pillow import, not just ImportError, so the helpful
install hint always appears. Break a line that exceeded 120 characters
2026-09-22 17:09:39 -07:00
devin-ai-integration[bot]
cf08cb89e8
test(utils): accept the per-size image cost keys in the price-map schema check (#42612)
The cost map's fal_ai/fal-ai/trellis-2 entry prices its output by resolution
with output_cost_per_image_512, output_cost_per_image_1024, and
output_cost_per_image_1536, which litellm/types/utils.py types and the fal_ai
cost calculator reads, but INTENDED_SCHEMA in
test_aaamodel_prices_and_context_window_json_is_valid never allowed them, so
the test fails on main with "Additional properties are not allowed". Add the
three keys next to output_cost_per_image in the schema and in the cost-under-1
field list so a per-size image price is validated like the per-size video ones

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-22 17:04:13 -07:00
devin-ai-integration[bot]
13691426c4
test(cost_calculator): point image-generation deployment price test at a live gemini row (#42615)
The test priced gemini/gemini-3.1-flash-image-preview, which #42435 removed
from the cost map as deprecated, so the calculator had no per-token rates to
keep and the hardcoded expected value no longer matched. Price the live
gemini/gemini-3.1-flash-image row instead and derive the expected cost from
that row in litellm.model_cost, so a rate change on it cannot break the test
while a calculator that drops the map's token rates still fails it.

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-22 16:52:05 -07:00
devin-ai-integration[bot]
da82ea8e94
fix(ui): let the Create Key user picker find users by user_id, not just email (#41687)
* feat(ui): search users by id or email when assigning a key owner

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): label users without an email by user id in key owner picker

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): freeze merged user-filter where, format create key test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(proxy): suppress module-global patch findings in ui_view_users search test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): keep merged user-filter where as a plain dict for prisma serialization

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): forward search param from userFilterUICall to /user/filter/ui

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): mention user ID in the Create Key user picker helper text

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: jesus <jesus@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-22 16:30:29 -07:00
devin-ai-integration[bot]
dd327156c8
feat(rust): add immutable model catalog crate (#42605)
* feat(rust): add immutable model catalog crate

* feat(rust): add typed model info mirror, error module, and schema feature

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(rust): split catalog module, rstest tests, and repo data parity tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-22 23:07:02 +00:00
devin-ai-integration[bot]
3c3803f37a
test(bedrock): point unit tests at model ids still in the cost map (#42606)
#42521 retired the cohere.command-r ids from the cost map, and a bare Bedrock id resolves its provider through that map, so test_model_group_info and the cohere cases in test_bedrock_dynamic_auth_params_unit_tests failed with LLM Provider NOT provided. The completion tests keep the cohere invoke request and URL assertions through the bedrock/ prefix, the bare parametrize entry moves to amazon.nova-2-lite-v1:0 with the mock response shape picked by the real Bedrock route, and the router test builds its group info from bedrock/amazon.nova-2-lite-v1:0.

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-22 16:04:44 -07:00
devin-ai-integration[bot]
392e807172
feat(logging): add normalized_error cluster key to error_information (#41715)
* feat(logging): add normalized_error cluster key to error_information

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(logging): stop classifying parameter length errors as context window errors

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(e2e): assert failure spend rows share normalized_error across provider wording

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(logging): map agent model access denials and ignore non-string proxy error types

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(logging): cover budget exceeded errors with custom wording

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(logging): cluster router no-healthy and provider-budget wording correctly

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(logging): let the exception class win over router fallback wording

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(logging): cluster peer closed connection errors as provider connection errors

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(logging): cluster tag routing denials as 403_MODEL_ACCESS_DENIED

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: shivam <shivam@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng <yucheng@berri.ai>
2026-09-22 15:56:50 -07:00
devin-ai-integration[bot]
7177d3b6d1
feat(rust): add standalone cost calculator (#42604)
* feat: add standalone Rust text pricing crate

* feat(rust): harden standalone cost calculator

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
2026-09-22 22:50:28 +00:00
devin-ai-integration[bot]
58a05a9eae
fix(anthropic): return 400 instead of 500 when a content list holds a bare string (#42420)
* fix(anthropic): skip non-dict content items in beta-header and file-id helpers so malformed content lists return 400 instead of 500

Fixes #42094
Supersedes #42101

Co-authored-by: Pawan-Shahane <shahanepawan511@gmail.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(e2e): spawn the DB-less regression proxy with -P so the cwd cannot shadow the pinned checkout

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(e2e): launch the DB-less proxy via -I -c with an explicit sys.path so python 3.10 works, drop DIRECT_URL, remove restating docstrings

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(e2e): gate the self-booted DB-less proxy behind the owned_gateway opt-in the Buildkite container cannot satisfy

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(anthropic): move the bare string content item repro to tests/integration

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(tests): wrap the anthropic bare string wire test to the 120 column limit

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(tests): wrap anthropic common_utils test literals to the 120 column limit

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style: fix ruff findings in touched test files

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Pawan-Shahane <shahanepawan511@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-22 15:44:30 -07:00
devin-ai-integration[bot]
3db94b932e
fix(spend): return 400 from /spend/calculate for a model with no pricing row (#42497)
* fix(spend): return 400 from /spend/calculate for a model with no pricing row

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(spend): assert error type and param for unpriced /spend/calculate

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(spend): move the repro to tests/integration

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: alias ModelNotMappedError re-export to satisfy F401

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(utils): raise ModelNotMappedError only when the pricing row is missing

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-22 15:44:00 -07:00
devin-ai-integration[bot]
238f434153
fix(otel): record the GenAI exception event through the Logs API on both OpenTelemetry lines (#42431)
* fix(otel): record the GenAI exception event without the removed Events API

OpenTelemetry removed opentelemetry._events in 1.44.0, so the three imports of
it broke 7 modules under litellm.integrations.otel, including the entry point.
Two things then failed quietly: with LITELLM_OTEL_V2 set the otel callback
resolved to None and nothing was exported, and with it unset the newrelic
callback was dropped as well, because that branch imports the v2 logger
ungated

Build and emit the event through the Logs API, which both lines carry. The
event name keeps riding the event.name attribute: the event_name log record
field that replaces it only exists from 1.44.0, and this package pins 1.28.0,
so the attribute is the only form both can write. It is also what the Events
API wrote, so exported events keep their shape

Emitting a plain record drops the default the Events SDK applied, so the
timestamp now falls back to time_ns() here

* style(otel): trim the event name key and regression test prose

Keep only the constraint a reader cannot infer from the code, that the
event_name record field does not exist on the pinned OpenTelemetry line

* fix(otel): export the GenAI exception event on both OpenTelemetry 1.28 and 1.44 lines

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(otel): drop the record selection comment

Co-authored-by: Pawan-Shahane <shahanepawan511@gmail.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(otel): collapse the record selection conditional for ruff format

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(otel): build the record fields with a dict literal

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(otel): set the native event_name on the 1.44 log record

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(otel): spell out the record kwargs so the type gate sees each call

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(otel): suppress the version-gated kwargs for the type gate

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(otel): drop the version-window prose and correct the event_name suppression reason

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(otel): wrap the compat test docstring to the line limit

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Pawan-Shahane <shahanepawan511@gmail.com>
Co-authored-by: yucheng <yucheng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-22 15:16:46 -07:00
berriai-litellm-provider-info-sync[bot]
95d4613bd3
chore(prices): sync xAI prices: 3 models, 3 new [3 with gaps] (#42591)
* chore(prices): sync xAI prices: 3 models, 3 new [3 with gaps]

xai/grok-code-fast: supports_vision, supports_prompt_caching, input_cost_per_token, output_cost_per_token, input_cost_per_image_token, cache_read_input_token_cost, input_cost_per_token_above_200k_tokens, output_cost_per_token_above_200k_tokens, cache_read_input_token_cost_above_200k_tokens, supports_function_calling, supports_tool_choice, supports_response_schema
xai/grok-code-fast-1: supports_vision, supports_prompt_caching, input_cost_per_token, output_cost_per_token, input_cost_per_image_token, cache_read_input_token_cost, input_cost_per_token_above_200k_tokens, output_cost_per_token_above_200k_tokens, cache_read_input_token_cost_above_200k_tokens, supports_function_calling, supports_tool_choice, supports_response_schema
xai/grok-code-fast-1-0825: supports_vision, supports_prompt_caching, input_cost_per_token, output_cost_per_token, input_cost_per_image_token, cache_read_input_token_cost, input_cost_per_token_above_200k_tokens, output_cost_per_token_above_200k_tokens, cache_read_input_token_cost_above_200k_tokens, supports_function_calling, supports_tool_choice, supports_response_schema

* fix(prices): add context limits and reasoning flag to xai grok-code-fast aliases

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(prices): keep the xai sync diff limited to the alias fields

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: berriai-litellm-provider-info-sync[bot] <328147090+berriai-litellm-provider-info-sync[bot]@users.noreply.github.com>
Co-authored-by: kerry <kerry@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-22 15:14:48 -07:00
devin-ai-integration[bot]
a173657dfb
fix(caching): keep embedding cache hits aligned with request inputs (#42571)
* fix(caching): keep embedding cache hits aligned with request inputs

Partial hits now send only the uncached inputs to the provider and merge
fresh vectors back into their original positions. Responses whose item
count differs from the input count (one input scoring many documents)
are no longer written to the per-input cache, since a later hit would
return a single item.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(caching): drop mutable collection builds flagged by the type discipline gate

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(caching): bypass embedding cache entries written before the per input cardinality check

Embedding cache entries now carry format_version and readers treat entries without it as
misses, so entries that only hold the first row of a multi row response are refetched instead
of served until their TTL expires. The provider call also receives a copy of the request kwargs
with the uncached inputs rather than mutating the caller's mapping

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(caching): assert a partial embedding cache hit becomes a full hit on repeat

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(caching): await pending embedding cache writes before asserting on cache hits

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(caching): validate cached embeddings without mutating responses or request kwargs

Validate cache rows through a frozen pydantic model so import does not depend on
TypeAdapter support for ReadOnly TypedDicts, accept string embeddings, build the
merged partial hit response instead of mutating the cached one, and hand the
provider request mapping to post call hooks

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(caching): keep cache_hit and response_ms on merged partial embedding hits

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-22 17:14:44 -05:00
devin-ai-integration[bot]
b9fcfb26d0
test(e2e): run the memory cell alone on the shared stack (#42518)
* test(e2e): run the memory cell alone on the shared stack

* test(e2e): hold the stack lock for every collected test, marker or not

* test(e2e): prove the stack lock's reader sharing and writer preference across processes

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-22 15:13:37 -07:00