* feat(xai): use billed image cost and resolution prices
* test(xai): mark image generation tests as a package
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): forward Codex agent messages that carry encrypted content intact
Codex multi-agent v2 sends worker messages as agent_message items whose
encrypted_content block only the model server can decrypt. Bedrock Mantle and
Bedrock Runtime used to 400 on agent_message, so LiteLLM rewrote every one into
assistant text, pasting the ciphertext into the prompt and dropping the sender
and recipient. Bedrock now decrypts these items itself, so the rewrite left the
recipient model reading an opaque token instead of its task.
Agent messages with an encrypted_content block now go to Bedrock untouched on
both endpoints. The Bedrock configs declare supports_encrypted_agent_messages,
and the shared normalizer keys off that flag so the two cannot drift. Mantle's
/v1/responses route for open-weight models still rejects agent_message in any
form, so it keeps the old rewrite. Plaintext-only agent messages keep the
rewrite on every route.
* fix(bedrock): only keep worker messages locked for OpenAI GPT models
Gemma and Grok on Mantle's OpenAI route never lock Codex worker messages, so
Codex puts plain text in the locked slot and Bedrock rejected it with
invalid_encrypted_content once the route alone decided to forward it intact.
Decide from the model instead, and restore the full-value rewrite assertion.
* feat(videos): add first-class OpenRouter video request fields
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(videos): remove recursive image parameter fallback
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(videos): preserve EdenAI native parameter passthrough
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(videos): deduplicate frame roles in linear time
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(videos): preserve image selection and dropped parameter semantics
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(videos): preserve provider scalar parameters and raw extra body
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(videos): retain accepted scalar inputs for Vertex and DashScope
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(bedrock): add Kimi K3.1 rows inherited from Kimi K3
* fix(bedrock): sync published Kimi K3.1 Highspeed prices
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(ui): format dashboard values through i18next formatters
Register named formats on i18next's formatter (usd, compact, duration,
relativetime, number and five date presets) with a cache keyed on the
language and options, add format.* keys to en.json, and move the usage
stat strip onto them.
* fix(ui): keep number formats in USD and the format cache bounded
Force USD on the number and compact formats, clean underscore language
codes like i18next does, fall back when options cannot be serialized,
keep formatted placeholders alone in format keys, and pin the remaining
stat strip values in the German test.
* refactor(ui): format durations with Intl.DurationFormat and load its polyfill on demand
Browsers without Intl.DurationFormat (Chrome and Edge before 129, Firefox before 136) load the pinned FormatJS polyfill through a dynamic import and re-render translations once it is installed. The milliseconds-to-units split and the 0s fallback stay, so the output matches the previous hand-built formatter.
* test(ui): cover duration edge cases on the polyfill path
* refactor(ui): drop the unused duration format and its polyfill
Nothing outside tests called format.duration, and its 1m 15s style doesn't match the decimal-second latencies the dashboard shows today. A duration format will come with the pages that need it, in the style they already use.
#50009 moved Aria out of the dashboard header into the sidebar with its own
page, and #50195 renamed the Skills nav link and AI Hub tab to Skills Hub and
relabeled the skills search box. The UI e2e specs still looked for the old
header button, dialog and labels, so the scheduled litellm-e2e run has been
red since build 815.
* fix(otel): seed request metadata baggage for team-only OTel v2 destinations
A team newrelic (or other preset) destination exported spans without the
operator-allowlisted litellm.metadata.* keys unless a proxy-wide otel callback
was listed, because only the OTel logger's async_pre_call_hook seeded
requester_metadata into Baggage and that hook runs only for loggers in
litellm.callbacks. ProxyLogging.pre_call_hook now re-seeds identity with the
request data once requester_metadata exists.
Resolves LIT-9783
* fix(otel): seed only request metadata from the proxy pre-call hook
The pre-call re-seed also passed the request model, which by then can be an
alias target, mcp-tool-call or an internal embedding model, so it rewrote
gen_ai.request.model on the server span for V2 deployments without otel listed.
Seed only the request metadata and keep the model auth seeded. Adds a unit test
for the pre-call seed and the guardrails_only skip, drops the explanatory
comments, and requires one server span plus the model span in the integration
test.
* test(otel): annotate new test variables Final and build test config without mutation
Addresses review: the new unit and integration tests now follow AGENTS.md
(LIT010 Final on every variable, no in-place mutation, read-only parameter
types, one for-clause per comprehension).
* test(otel): cover key-level callbacks and assert tenant spans carry no credentials
Parametrizes the tenant metadata test over team and key callbacks, and adds a
test that sends credential headers and a planted New Relic key with an
allowlist naming credential headers, then asserts no exported span carries the
master key, the virtual key, the team's New Relic key or any credential header
value, and that every export used only the team's own New Relic key.
#40154 made the deployment post_call hook really run apply_guardrail guardrails, so the proxy
checked the same response a second time (LIT-10481). The deployment hook now records guardrail +
response hash on the request and on the response, and the proxy skips a guardrail only on a
matching record. The hash covers every field except ids, usage, model and other metadata set
between layers, so a response rewritten between the layers (OCR included) is still checked.
Guardrails with their own post_call hook keep their pre-#40154 behavior
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix(usage): tolerate entity metadata database failures
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(usage): keep spend when metadata lookups hit database failures
Treat transport errors and the query engine crash on a closed connection as database
failures in the key, user and guardrail lookups of the aggregated, keys, users,
model_top_keys and cache_leakage_keys routes, and keep prisma out of module-level imports
so a proxy install without prisma still imports the usage endpoints.
* fix(usage): fall back only on database outages, on every daily activity route
Data-layer Prisma errors such as a rejected query or a missing table return 500 again, a key whose details failed to load reports key_exists null instead of false, and the non-aggregated daily activity routes keep spend when the key or entity metadata lookup hits an outage
* fix(proxy): skip the key metadata lookup when the daily rows carry no keys
* fix(usage): keep the guardrail breakdown fallback for any Prisma error
A missing guardrail usage table or column made the Usage call return 500 after the outage-only narrowing. The guardrail lookup now falls back on outages and on any Prisma error, as it did before this PR
* fix(usage): keep spend when the team, organization, customer, or agent name lookup drops
---------
Co-authored-by: mateo <277851410+mateo-berri@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(azure_ai): jais-30b-chat price was per 1K tokens, store it per token
* fix(watsonx): jais-13b-chat price was per 1K tokens, store it per token
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(proxy): give Codex accurate model entries for every model in /v1/models
* fix(router): read a deployment's upstream model without a not-required key access
* fix(proxy): keep Codex tiers for unpriced deployments and match stock entries by declared provider
A deployment with no token price no longer votes a Codex service tier off,
an undescribed model keeps Codex's default reasoning summary parameter,
a Bedrock deployment naming its provider in custom_llm_provider gets Codex's
stock entry, and Bedrock provider namespaces follow the AWS model id format.
* fix(proxy): import cost tracking and provider helpers lazily in the Codex catalog
lite codex imports the catalog before the SDK is initialized, so importing
cost tracking, the Bedrock helpers or provider resolution at module level hit
a circular import and broke every lite command.
* fix(proxy): drop Codex's fast speed tier wherever its priority tier is dropped
Codex shows /fast for a leftover fast speed tier even when service_tiers
no longer offers priority, so an alias, an empty configured tier list or
a configured list without priority showed a /fast toggle that sends nothing.
One helper now sets service tiers, speed tiers and the default together.
* fix(proxy): give an older Codex a translated entry for models past its version
Codex's own entries name the oldest release that can read them in minimal_client_version. A release older than that can fail to decode the entry, such as a reasoning effort it has no name for, which fails the whole catalog, so it now gets the translated entry, whose levels are trimmed to that release
* feat(proxy): serve Codex its own release's model catalog, fetched from GitHub
When Codex asks /v1/models for its catalog, the proxy now reads models.json and prompt.md at that
release's tag on GitHub and serves those entries, falling back to the bundled 0.166.0 copy when the
fetch fails, times out after 2 seconds, does not validate, the version has no tag, or
LITELLM_OFFLINE_MODE is on. A failed fetch is remembered per version for an hour and only the eight
most recent versions are kept. The integration tests ask as the bundled release so their expectations
hold whether or not the proxy can reach GitHub
* fix(proxy): bound the GitHub fetches a caller can trigger through Codex catalog versions
Failed versions now live in their own cache so they never push out a fetched catalog, requests for a
version already being fetched share that fetch, each worker starts at most six fetches a minute and
serves the bundled catalog past that, and the version check only accepts ASCII digits
* fix(proxy): keep a failed Codex version's cooldown past the fetched catalog cap
Failures are tiny, so they get their own cap of 256 instead of sharing the eight-catalog cap, which let nine
failing versions retry each other well before their hour was up. The TypeVar takes Final like the rest of the
codebase
* refactor(typing): clear basedpyright Any errors (wave 1)
* refactor(typing): clear basedpyright Any errors (wave 2)
* refactor(typing): clear basedpyright Any errors (wave 3)
* refactor(typing): clear basedpyright Any errors (wave 4)
* refactor(typing): clear basedpyright Any errors (wave 5)
* refactor(typing): clear basedpyright Any errors (wave 6)
* fix(typing): keep the cached-plan fallback's direct query_first calls so span naming resolves
* fix(logging): keep callbacks running when litellm_params is a params model
should_run_callback and the sync success and failure handlers read litellm_params through any object with a get method again, as they did before this branch, so a logging hook that returns a GenericLiteLLMParams no longer stops sync logging for that request
Adds behavior tests for the changed lines codecov listed as not run
* fix: read the regional uplift for every data residency again and reuse one dict TypeAdapter
The typed lookup only read the eu and us multipliers, so a deployment that set regional_processing_uplift_multiplier_fedramp stopped applying it. /model/update and the model write paths now share one module-level TypeAdapter instead of building one per call.
* fix: bill regional uplift multipliers float() accepts, bytes included
* fix: raise the in operator's own error from one shared json_contains, and name the bad uplift type
* fix(bedrock): run realtime guardrails on Nova Sonic sessions
* fix(bedrock): quote the spoken violation message and skip it after a blocked Nova Sonic tool output
* fix(bedrock): screen every message role, end Nova Sonic sessions on blocked transcripts, drop unparseable frames
* fix(realtime): apply guardrail masks, keep block reasons out of spoken notices, fail closed on guardrail crashes
* fix(realtime): screen joined text parts, keep the documented notice, honor key opt-outs
Text parts of one message are joined before screening again, so a phrase split
across parts is still blocked, and a mask collapses them into one part. The
spoken notice is realtime_violation_message or the block reason, quoted only on
Nova Sonic. Client-written output_text and text parts are screened. Key and team
guardrail opt-outs reach regular and translation realtime sessions. A non-string
upstream transcript skips screening as before.
* fix(realtime): honor guardrail role scope, clear stale response.create drops, screen split keywords, report Nova guardrail outages
* fix(realtime): keep every per-part mask when a joined text masks differently
* fix(realtime): screen one joined text per item and pass the session's tenant context to guardrails
Multi-part items reach each guardrail once as the space-joined text, as on main, and only an item a
guardrail rewrites gets its parts screened again to keep per-part masks. Guardrails now receive the
session request data with its metadata, so key and team policies apply, and a ModifyResponseException
block is reported as a block. Nova scopes every message item as the user turn it sends to Bedrock.
* fix(realtime): catch phrases split mid-word across text parts without re-screening single-part items
* fix(realtime): screen each joined text in its own guardrail call and keep a Nova opening message for the retry while it is screened
* fix(realtime): screen text parts in a loop so large messages keep their masks
* fix(realtime): fail closed on a non-string transcript under a guardrail and give each turn its own guardrail metadata
* fix(realtime): fail a non-string transcript only when a guardrail would screen the user's turn
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(router): pin /v1/responses requests for prompt caching affinity
* fix(router): pin previous_response_id turns and keep native tools in the Responses pin key
A Responses turn that continues a stored response now goes to the deployment
that answered it, unless that deployment already failed this request. Native
Responses tools are hashed into the pin key as sent instead of converted, so
two requests with different native tools no longer share a pin and a native
request no longer logs a false dropped-tool warning. The pin keeps its
cacheable deployment ids in a sorted tuple so loggers can serialize request
metadata.
* fix(router): leave a Responses request with a malformed tool to the provider
* fix(router): keep the Responses prompt cache pin out of logged metadata
The pre-call check now keeps the pin in a router-owned in-flight store and
puts only a random token in litellm_metadata, so loggers that ship metadata
never see the routing key or deployment ids. The store is shared across check
instances, so turning prompt_caching off and on at runtime keeps pinning, and
a retry drops its previous attempt's token. The skipped-input debug log now
names only the error type.
* fix(router): apply the prompt caching pin after tag, region and pre-call filters, and leave it on retries
The prompt caching check no longer narrows the candidates inside the callback
filters. It names the deployment it prefers, and the router core applies that
preference at the pinned deployment step, after tag, region, pre-call and plugin
filters, only when the deployment is still a candidate. A pin can no longer
send a request to a deployment its tags do not allow.
A retry skips the pin when the pinned deployment was already tried in this
request, read from the retry breadcrumbs in both metadata buckets and the
excluded and retry-skipped ids. When the retry count outruns the breadcrumbs
the check stands down, so a pinned deployment that keeps failing is not
retried on every attempt.
* test(router): list the prompt caching check as an in-memory cache caller
The check's in-flight Responses pins live in a process-local InMemoryCache, so it opens no Redis span and needs no key family, like the other in-memory callers
* fix(router): pin a lone healthy deployment, free every in-flight Responses pin, and clear a reused metadata dict's retry records
* test(router): lift test cooldowns through the cooldown store's evict_local
* fix(router): key Responses tools like their chat twin and drop the pin token after success
A Responses function tool now hashes only the function fields it sent, so a chat request and a Responses request with the same tool share one pin. The success event also removes the pin token from the caller's metadata dict.
* test(router): cover untyped tool parameters and custom tool grammars in the Responses pin key
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(ui): stop parsing formatted dates and spend back into values
Regenerating a key stored its new created_at as locale text and parsed it
back for display, which shows "Invalid Date" on browsers whose date format
JavaScript cannot parse. It now stores an ISO timestamp.
The model health table stored check times as locale text and sorted by
parsing that text, which misorders or drops rows outside US English. It now
keeps the raw timestamp and formats it in the table cell, so the shown text
is unchanged.
getSpendString decided whether spend rounds to zero by stripping commas from
its own formatted output. It now compares the number against half of the
smallest shown unit, which gives the same output for every value and no
longer depends on how the digits are rendered.
* fix(ui): keep getSpendString exact at high precision
0.5 / 10 ** decimals drifts by one ulp once 10 ** decimals is no longer exact
(past 22 decimals), so a value right at the threshold could flip from "< $..."
to a row of zeros. Parsing 5e-(decimals + 1) gives the correctly rounded
threshold for every precision.
