A credential_provider object reaches the resolver as a ProviderHandle, so it
resolves to Auth::Provider and two RedisCaches share a client only when they
share the provider. The process registry of native clients now holds each
client weakly, so a discarded Router drops its client and its provider
instead of keeping them for the life of the process.
NativeRedisClient resolves a RedisCache's redis_kwargs through the RedisCache
kwargs surface and hands every native store in the process one shared client,
breaker and ServiceLogging event sink. A setting Rust ignores is logged with
its consequence, and a config it cannot use leaves the stores on memory
NativeRequestRedis carries one request's pre-call and post-call batches from
RequestRedisBatches into the stores. The keyspaces that use these land in the
PRs above
RedisSpec carries RedisCache.redis_kwargs into the resolver as plain values
plus the names of keys whose value is a Python object, instead of replacing
each object with its type name. ConfiguredRedis::with_host_objects takes those
names, so the resolver reports them in Resolution::host_objects, warns for the
ones Rust cannot honor and lets each hide its REDIS_* value. A
credential_provider object without a ProviderHandle fails with
ProviderWithoutHandle, so the stores stay on memory instead of connecting
without auth
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every cache setter now takes a pydantic model or plain data (str, number, bool,
None, sequences, str-keyed mappings), which is what Redis can hold and what the
Rust auth cache accepts. No production writer passes anything else, and pyright
rejects a new caller that does
Adds publish(channel, payload) through the guarded command path and
subscribe(channel), a Stream of raw payloads on its own RESP3 connection that
resubscribes after a lost connection with the Python subscriber's 5s to 60s
backoff, on standalone, cluster and Sentinel. Channel names skip the
namespace, matching the Python publisher, so the auth cache invalidation
channel can move onto the Rust client
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The REDIS_CIRCUIT_BREAKER_* settings are read once in resolve into
Resolution::breaker, like get_env_int and get_env_float read them. A value
that does not parse is warned about and keeps its default. Before, a typo
failed connect with InvalidBreakerSetting and put every native store on
per-process memory while RedisCache kept its default breaker.
Python-shaped input becomes typed Rust only in crates/config, which reads
config.yaml with litellm-python-compat, and crates/python-bridge, which reads
live Python objects through PyO3. A crate that stands in for a Python library,
like redis-resolve for redis-py, keeps that library's semantics because they are
its compatibility contract. The skill names the code that still coerces past
the boundary, so it moves there when touched instead of growing
redis_py_kwargs.py reads each redis-py version's signatures in a throwaway uv
environment and the REDIS_* names from litellm._redis under the locked
version, then writes the catalog. Run on this branch it reproduces every arg row
of the hand-made file, so only the meta text changes.
test_redis.py now loads the script's arg_signatures instead of keeping its own
copy of the introspection, so the code that writes the file is the code that
checks it in every redis-compat CI job
Both ask for X.509 checks such as CRL revocation that rustls cannot perform,
and redis-py 5.3.1's async client already refuses them with a TypeError, so a
set value now fails with Error::Unenforceable like the OCSP settings instead of
connecting without the check. An empty list asks for nothing.
A Reject row now says which values ask for its check: true for the OCSP flags,
any present value for the verify flag lists. tests/catalog.rs fails if the
locked redis-py's async, cluster or connection signature accepts a Reject row,
so a redis-py bump forces the decision again
RedisError becomes Error, the one top-level error the rust-error skill asks of
each crate, and gateway nests it as litellm_redis_resolve::Error.
NotPlainData moves to gateway as Error::RedisNotPlainData, since only the
gateway merge turns a YAML value into JSON. Unenforceable now carries the check
a Reject row asks for, so its message is not hard-coded to OCSP
Each row now carries its handling with the warning's consequence, whether the
value is plain data or a Python object, whether a url query carries it, its
REDIS_* name and its get_secret fallbacks. HOST_OBJECTS, consequence(),
is_met_by_rustls and the url setting lists in tls.rs fold into those columns.
The 29 kwargs that had no row get one, so setting them now warns instead of
passing through silently. redis-py-kwargs.json keeps only the Python facts:
which redis-py versions name each kwarg and whether Python reads its REDIS_*
name. tests/catalog.rs fails until SETTINGS has exactly one row per kwarg in
it with the same REDIS_* flag, and test_redis.py checks the file against the
redis-py each redis-compat CI job installs
config now only deserializes the Redis YAML. The new litellm-redis-resolve crate
turns an already merged redis-py kwargs dict into a Resolution without reading the
environment or secrets, so both hosts can share it. The REDIS_* layering, the
get_secret fallbacks and the Azure REDIS_USERNAME read move to gateway, the only
host that needs a Rust copy of the merge.
litellm/_redis.py gets _merge_redis_kwargs, the merge stage of
_get_redis_client_logic, and the contract cases gain a merged column that pins
the Python merge, the Rust merge and the Rust interpret step to the same dict.
ssl_validate_ocsp and ssl_validate_ocsp_stapled were dropped with a warning, so
a config that asked for revocation checks connected without them. rustls has no
client-side OCSP, so a true value from the kwargs, REDIS_* or a url query now
fails resolution with RedisError::Unenforceable. redis-py 5.3.1's async client
refuses both with a TypeError, so this matches the proxy's path. A false value
asks for nothing and is dropped silently
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
rustls only reads unencrypted keys, so ssl_password was dropped with a warning
and the TLS handshake failed. The resolver now reads it into TlsOptions and
redis-client decrypts a PKCS#8 ENCRYPTED PRIVATE KEY before handing it to
redis-rs, like the sync redis-py client's load_cert_chain(password=...). A
missing or wrong password, or a legacy Proc-Type encrypted PEM key, fails at
client build with a TlsKey error naming the file
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A Redis setting the Rust client cannot honor is now warned and dropped with
what that means, so a config that works on Python never fails to build on
Rust. health_check_interval, unix_socket_path, ssl_ca_data and ssl_ca_path
are read and applied like redis-py.
health_check_interval resolves to a three-state HealthCheck, so 0 stays off
on a cluster instead of turning on Python's 25 second cluster default, which
only an unset interval takes. unix_socket_path resolves to Target::Unix. A
url's ?db=, ?username= and ?password= are read like parse_url reads them
instead of being dropped, and ssl_min_version no longer warns at TLS 1.2 or
lower, which rustls already meets.
resolve now returns litellm_redis_types::Resolution, the value both client
projections build from. Target replaces Endpoint, a url carries the TLS parsed
from it only when it is rediss, Sentinel carries its own monitor_auth, and Auth
is a tagged union whose Provider variant holds a ProviderHandle the host passes
with ConfiguredRedis::with_credential_provider. A credential_provider key with
no handle is an error instead of a connect-time failure.
Settings the resolver does not model are no longer dropped. Plain ones land in
passthrough and object-valued ones in host_objects, so a redis-py projection can
reattach them. TlsOptions no longer derives Default, which turned certificate
verification off, and TlsOptions::verified() is the one place defaults come from
Each config surface now turns its Redis section into a ConfiguredRedis
(coordination, cache_params, or a RedisCache.redis_kwargs dict), and one
resolve returns the ResolvedRedis plus the ignored settings by source, so
hosts log gaps instead of the resolver. The resolve/resolve_cache flag and
the coordination ssl normalization are gone
Resolution reads the layered kwargs into a typed Options struct once, and
endpoint and auth selection work from that struct alone. An empty value is
unset everywhere, and a flag Python surfaces read differently is an error
ResolvedRedis and RedisAuth move to redis-types, so redis-connect no longer
depends on litellm-config
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every command is now the one RedisCache sends. Pipelined scripts go out as
EVALSHA, and a NOSCRIPT op alone runs again after the pipeline like
RedisBatch's _settle_alone. Only a ScriptCall whose reply was NOSCRIPT can be
sent again, so nothing can resend a command that already ran. Increments are
plain INCRBY and INCRBYFLOAT with TTL and EXPIRE in seconds, as
increment_cache, async_increment and RedisBatch send them, instead of Lua.
SET uses EX, and numbers are written the way redis-py encodes them.
* fix(auth): align cloud credential loading and expiry with the Python SDKs
* feat(auth): resolve consumer-scoped default identities for AWS and Azure
AwsAuthService::default_chain_credentials and AzureAuthService::identity_token give a consumer the SDK's own default identity without LiteLLM's Bedrock or Azure OpenAI settings. Moved here from #49510 so every shared auth crate change lives in one PR
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(auth): include identity hints in secret snapshots
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* feat(billing): support Cognito bearer auth for metrics
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): refetch billing token on 401 and require https
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): import Unpack from typing_extensions
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): make billing metrics bearer only and drop mtls
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style(proxy): format billing metrics module
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The bridge moves leading system messages into the Responses instructions
field. With response_format json_object, Azure requires the word json in
input and ignores instructions, so a request that follows the documented
rule 400s after translation. Send the system text as a system input item
when json_object is requested, the same shape already used for
system-only requests
Resolves LIT-10426
* chore(cost-map): set openai gpt-5.6-cyber and Daybreak Red max input to 272K and source the sora-2-pro-high-res deprecation
* Absorb GovCloud Bedrock, Azure GPT-6 Flex and GMI registry rows
Roll the verified registry data and Azure Flex cost coverage into the rolling price sync
Co-authored-by: kerry-berri <326648561+kerry-berri@users.noreply.github.com>
Co-authored-by: dguthruf <102602010+danielguthruf@users.noreply.github.com>
Co-authored-by: hisamotodev <hisamotodev@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: hisamotodev <hisamotodev@users.noreply.github.com>
* fix(azure): answer 401 when Entra rejects the deployment's credential
A ClientAuthenticationError from the Entra token provider (wrong, expired or rotated client secret) used to surface as a retryable 500 carrying the app's client id and Entra trace ids. Map it to a 401 AuthenticationError that keeps the AADSTS code and first sentence with GUIDs redacted, on both the direct token fetch and the provider handed to the OpenAI SDK client. Network failures stay 500.
* fix(azure): keep network failures to Entra a retryable 500
azure-identity wraps a transport failure in ClientAuthenticationError, so the
401 mapping now skips errors whose cause chain holds ServiceRequestError or
ServiceResponseError
* fix(azure): cover async token providers and Entra 5xx, and scope caller-sent Entra credentials
* fix(azure): map only AADSTS-coded Entra rejections to 401 and keep caller credential failures out of cooldown
An Entra ClientAuthenticationError now answers 401 only when its message carries an AADSTS code, its HTTP
status is not 429 or 5xx, and the code is not the AADSTS50196 throttle, so a chained credential's outage or
throttle stays a retryable error. Caller-sent Entra fields no longer change the deployment id; a 401 from a
value the caller sent skips the shared deployment's cooldown instead. The Azure AI Foundry agent token fetch
maps the same rejection, and the A2A endpoint answers it with HTTP 401.
* fix(router): count router default_litellm_params as configured Entra credentials when deciding cooldown
* fix(azure): keep transient Entra codes retryable and scope 401 handling to Entra rejections
ClientSecretCredential drops Entra's HTTP status, so throttling and transient
codes (AADSTS50033, 50196, 90006, 90024, 90033, 90055) now stay retryable
instead of mapping to 401.
The A2A endpoint answers 401 only for an AuthenticationError caused by an
Entra token rejection, and the router skips cooldown for a caller-sent Entra
field only on Azure-family deployments. The azure/ Decisions route and the
Bing grounding search tool now map an environment-credential rejection to the
same 401.
* fix(azure): map only known Entra credential rejections to 401 and cover the OIDC token exchange
Unlisted AADSTS codes keep the retryable 500, the router skips the 401 cooldown only when an Entra rejection caused it, and a federated token Entra rejects answers the same 401 on Azure chat, Responses and A2A agents
* fix(azure): answer 401 for Entra rejections on the /azure passthrough and token counting
The fallback hop skips cooldown when Entra rejected a credential the caller sent,
Prometheus counts a deployment's own Entra rejection as a deployment failure again,
and an OIDC exchange error keeps Entra's headers so Retry-After reaches the router.
* fix(azure): keep a no-op caller Entra field from skipping cooldown and redact the passthrough 401 message
A caller-sent Entra field now skips the deployment's cooldown only when it
differs from the deployment, its named credential, the router defaults, the
AZURE_* environment variables and the default scope. An empty value counts as
unset, so a deployment whose own secret is wrong still cools down when the
caller's field changes nothing about the token request.
The /azure passthrough's Entra 401 message goes through the same internal
hostname redaction as other client-facing errors.
* fix(router): compare a caller's Entra field with the value the deployment uses
A caller-sent Entra field skipped the cooldown only when it matched no
configured source, so a value equal to an environment default the deployment
overrides (a stale AZURE_CLIENT_SECRET, say) counted as the deployment's own
and let a caller's rejected credential cool down a working deployment.
The check now resolves the value the deployment would use without the caller,
in order from the router defaults, the deployment, its named credential, the
AZURE_* environment variables and the default scope, and skips the cooldown
only when the caller's value differs from it.
* fix(azure): follow the settings merge for empty Entra fields and keep non-rejection OIDC exchange errors unchanged
* fix(azure): count only the failed attempt's Entra rejection and ignore a non-callable token provider
* fix(azure): keep the streaming Entra rejection inside the failed attempt
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(guardrails): check Anthropic-format request tools in tool_permission pre_call
The pre_call hook only collected tools shaped as {"type": "function", "function": {...}},
so flat tools from /v1/messages ({"name", "input_schema"}) and /v1/responses
({"type": "function", "name"}) were never checked and deny rules silently passed.
Reuse the shared anthropic_tool_names helper so every request tool name is checked,
and handle Anthropic tool_choice {"type": "tool"} in both block and rewrite modes.
Fixes#40583
* fix(guardrails): detect Responses API named tool_choice in tool_permission
Responses API named choices put the name at the top level ({"type": "function", "name": ...}),
so rewrite mode removed the denied tool but left a tool_choice pointing at it.
* test(guardrails): run tool_permission pre_call tests through real default_on gating
Build the guardrail as an always-on pre_call guardrail instead of patching should_run_guardrail.
* test(guardrails): type the new tool_permission pre_call test parameters
* fix(guardrails): keep the Responses API custom tool type distinct
Anthropic's user-defined tools carry type "custom", so pre_call mapped
that to "function" for tool_type rules. The Responses API has its own
custom tool type, and mapping it let a custom tool slip past a
tool_type deny rule when the default action is allow.
Normalize only for the formats where custom means a plain function, and
read tool names directly in the rewrite path, which never needed a type.
* test(guardrails): cover flat tools through Messages and Responses
Add blocked and permitted flat-tool cases through the running proxy, with
endpoint coverage and metadata for the current e2e harness
Require a guardrail policy verdict when checking denied tools so an
unrelated provider error cannot satisfy the regression test
Co-authored-by: Christopher Krah <christopher.krah@plexito.de>
* fix(guardrails): check namespace and nameless tools and relax tool_choice in tool permission rewrite
* fix(guardrails): check forced, grouped, loaded, and hosted tools by their real type
* fix(guardrails): drop references to removed tools and count loaded tools in tool permission rewrite
* fix(guardrails): check tools sent outside top-level tools in tool permission
* fix(guardrails): judge namespace members by their bare name as a fallback and keep rewrites valid for Gemini, Bedrock and hosted tools
A namespace member is checked by namespace__name first, and by its bare name when no rule names the joined one. Rewrite mode now prunes Gemini allowedFunctionNames and turns off forced calling when no function is left, drops a removed hosted tool from allowed_tools, reads Bedrock toolSpec and systemTool blocks by name, skips cachePoint markers, and leaves a tools field that is not a list for the provider to reject instead of failing with a 500
* fix(guardrails): let only name rules decide a namespace member's bare-name fallback
* fix(guardrails): judge malformed tool types, empty Gemini name lists and pass-through Anthropic tools like the endpoints do
A tool or tool_choice whose type is not a string now reads as untyped instead of crashing the guardrail with a 500, and a null type key is never read as a Gemini tool part. An empty allowedFunctionNames list no longer turns off the functions a rewrite kept. An Anthropic custom tool reads as a function by its input_schema on every route, so a pass-through route judges it the same as /v1/messages. A Responses WebSocket frame now forwards the extra_body tools a rewrite stripped
* fix(guardrails): judge a named hosted tool by its type and keep a nested chat custom tool custom
* fix(guardrails): keep judging Anthropic client tools and type-named server tools by their name
* fix(guardrails): judge a named tool by its name alone only when the name is its type's own
* fix(guardrails): judge the function a tool names beside a declaration list or a top-level name
* fix(guardrails): judge Gemini hosted keys whatever the tool type and a namespace's function
* fix(guardrails): judge legacy functions entries like tools and send Gemini a namespace's members
* fix(guardrails): judge a namespace member by every name a provider reads and send Gemini its schema converted
* fix(guardrails): judge a legacy functions entry as a function and relax a tool_choice in extra_body
* fix(gemini): keep a namespace whole when inlining its members would repeat a function name
* fix(guardrails): judge a namespace member by every bare name that shares its qualified name
* fix(vertex_ai): declare a repeated namespace member name once instead of sending namespace names to Gemini
* fix(guardrails): judge a tool_choice sent in extra_body like a top-level one
A forced name only extra_body.tool_choice carried was never checked, so block mode let a denied name through to the provider
* fix(guardrails): judge a Gemini namespace member returned by its bare name and force a named namespace's members
* test: keep formatter churn out of the tool_permission and Gemini test files
* fix(guardrails): judge extra_body.input tools, argument-aware alias rules, and exact forced Gemini functions
Tools loaded in extra_body.input are now checked and stripped like top-level input. A returned
namespace member is judged by the alias whose first name rule its arguments satisfy. A forced
Gemini name that matches a top-level function keeps that function. Alias grouping and Gemini
declaration dedupe now run in one pass instead of quadratic scans.
* fix(guardrails): judge and rewrite every tool field sent inside extra_body
extra_body is merged into the upstream body, so functions, function_call,
mcp_servers and toolConfig nested there reached the model unjudged. Treat
extra_body as a second request layer that is collected and rewritten the
same way as the top-level request.
* fix(guardrails): judge Gemini hosted keys by key and count only the tools a provider receives after extra_body
* fix(guardrails): relax a tool_choice that names a dropped namespace member by its bare name
Gemini and Anthropic chat send namespace members under their bare names, so rewrite mode now treats a dropped member's bare name, and an emptied namespace's own name, as removed when no kept tool still carries it
* fix(gemini,guardrails): keep a top-level function over a same-named namespace member and leave malformed tool values to the provider
Gemini and Vertex chat now declare a top-level function over a namespace member that shares its name, so a forced call gets the top-level schema. post_call judges a returned name a top-level tool carries as that tool, and pre_call no longer raises on an unhashable or non-string tool name, input item type, or function_call name.
* fix(guardrails): judge a returned namespace member named like a hosted MCP server as that member
* fix(gemini,guardrails): send a namespace as main did when another tool has a member's name
Gemini chat inlines a namespace's members only when no other tool, namespace
member, or legacy functions entry declares the same name, and otherwise sends the
namespace by its own name as main did. A tool_choice naming such a namespace
forces the namespace itself. The tool permission post_call reads top-level names
the way Gemini declares them and judges a call the Responses bridge tags with a
namespace as that member.
* fix(gemini): send colliding namespace members as namespace__name functions and judge them in post_call
When a namespace member shares a name with another tool, Gemini chat now gets that namespace's members as namespace__name functions instead of one function named after the namespace, so every function it can call is a name the rules judged. Declared names are counted once, and a forced member maps to each holder's qualified name.
post_call judges a returned namespace__name as that member, by its qualified or bare name the way pre_call does, so a bare-name allowlist or denylist applies to it.
* fix(gemini): keep generated member names unique and let a top-level function own its exact name
A namespace that listed a member twice, or a generated namespace__name that
repeated a top-level function, sent Gemini two declarations with one name, and
Gemini answered 400 where main answered 200. A name a namespace lists twice is
now one member, and a top-level function keeps its exact name.
post_call now judges a returned name a top-level function declares as that
function, never as the namespace member the name could also mean.
* fix(gemini): qualify namespace members by the namespace's top-level name
The guardrail judges a member as `name__member` from the namespace's top-level
name. Gemini built the name from `function.name` when the tool also carried one,
so a request could pass pre-call and reach Gemini under a name no rule judged.
* fix(vertex_ai): force only the namespace members Gemini is sent
A forced namespace or member on Gemini chat could still allow a generated name that a top-level function or an earlier namespace owns, so tool_choice forced an unrelated function. The forced names now come from the same collision checks the declarations use
* fix(vertex_ai): reject a forced namespace Gemini gets no member of, and resolve forced names in one pass
* fix(guardrails): judge a namespace member by a rule on its namespace's name
---------
Co-authored-by: Christopher Krah <christopher.krah@plexito.de>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* fix(proxy): charge a video's provider-reported cost once, on the first poll that sees it completed
A custom video provider can report a video's cost on the status poll that
sees it completed. Every poll of a video shared the video id as its spend
log request_id, so the paid poll's row was dropped as a duplicate and every
later poll of the finished video charged the cost again.
Video status rows now carry their own `_video_cost` request_id, and a poll
that reports a cost goes through the same settle-once claim batches use, so
the first poll that sees the video completed charges it and writes the row,
and every later poll charges nothing.
* fix(proxy): charge a video's provider-reported cost once on model, router and session budgets too
A video status poll gets the _video_cost row suffix only when it reports a cost, so a $0 poll keeps the bare video id and adds no row of its own, and the spend log writer claims a row by that suffix.
The batch charge-once marker now covers videos, keyed on the video id under {spend_key}:video:{id}, with the batch marker key unchanged. Model and router budgets use it, and per-session budgets now charge a repolled batch or video once and ignore a cost that is not a finite number
* fix(proxy): charge a finished video's own cost once, keyed by the id the client polled
Guardrail cost on a running poll no longer takes the charge-once claim, only a completed poll charges, the session marker TTL refreshes like the others, and two deployments whose provider ids match each keep their own charge.
* fix(proxy): charge repeat poll guardrails, key video charges by canonical id and survive redaction
* fix(proxy): keep video status polls in-process for the collector and give guardrail-only job polls an unclaimable row id
* fix(proxy): keep repeat batch polls free and group guardrail-only batch rows under their batch's trace
* fix(proxy): settle a finished job whose own cost is below its poll's guardrail cost under the job's charge row
* test(proxy): count charged_poll_cost as covered through the finished job guardrail poll tests
* fix(proxy): settle a finished batch that cost nothing itself on its first priced retrieve
* fix(proxy): charge no budget for a batch retrieve before its cost is final, and read the retrieved batch from before redaction
* fix(proxy): charge sessions a poll's guardrail-inclusive cost, count a job cost row's guardrail run, and keep batch_cost_is_final importable from litellm.batches
* fix(proxy): claim a job and charge the session budget in one Redis call
The session limiter claimed the job marker and then raised spend in a second Redis call. A failure between the two left the job claimed while its charge sat only in local memory, so later polls skipped it. One script now does both, and a refused script claims and charges locally
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* feat: update EmpirioLabs models in model_prices_and_context_window.json
* chore(endpoints): regenerate provider endpoint support for the EmpirioLabs cost map rows
* fix(empiriolabs): drop the retired max_tokens field, route embeddings and remove the unrouted rerank row
The cost map test rejects any row carrying max_tokens, so every EmpirioLabs row keeps max_input_tokens and max_output_tokens only. The provider JSON now lists /v1/embeddings so embedding(model="empiriolabs/text-embedding-v4") reaches the OpenAI-compatible embeddings route instead of failing provider lookup. The qwen3-rerank row goes away because no rerank route exists for the provider. Endpoint support regenerated from the cost map
* fix(empiriolabs): mark audio input and image embedding input on the rows the provider lists as multimodal
The three vision embedding rows get supports_embedding_image_input and the eleven chat rows whose input_modalities on GET https://api.empiriolabs.ai/v1/models include audio get supports_audio_input, so capability checks stop treating them as text only
* fix(empiriolabs): carry the long-context price tiers on the 24 rows the provider bills in bands
EmpirioLabs bills 24 of these models at a higher rate once the prompt passes 32K, 128K, 256K, 272K or 512K tokens, and the rows carried only the first band, so a 300K-token qwen3-7-plus request was priced at a third of the real cost. Each of those rows now carries input_cost_per_token_above_<N>k_tokens, output_cost_per_token_above_<N>k_tokens and, where the row already has cache_read_input_token_cost, cache_read_input_token_cost_above_<N>k_tokens for every band, taken from pricing[].min_context on GET https://api.empiriolabs.ai/v1/models for the base rows and from the Variants rate card on docs.empiriolabs.ai/models/<id> for the six variant rows
* feat(empiriolabs): bill the per-call web search fee reported in usage.tool_usage
EmpirioLabs charges a flat fee per invoked web_search, web_search_image or image_search call and reports the counts in usage.tool_usage on the chat response. The JSON provider config now carries that count as usage.server_tool_use.web_search_requests when providers.json sets special_handling.web_search_requests_from_tool_usage, the per-call web search cost path bills the usage-reported count instead of flooring to one call, and the 39 EmpirioLabs rows the provider lists a per-call search price for carry it as search_context_cost_per_query, from unit_pricing on GET /v1/models for base rows and the Variants rate card on docs.empiriolabs.ai for variant rows
* fix(openai_like): mark the dict signature of transform_parsed_response_dict as matching BaseConfig for LIT001
* fix(openai_like): mark both dict annotations of transform_parsed_response_dict for LIT001
---------
Co-authored-by: Adam Dalloul <47503782+Adam-Dalloul@users.noreply.github.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Moves the page-authored English on /playground, the /chat app and /moyai
into t() and Trans with keys in en.json, as a pure text move. The rendered
English is unchanged
Route the page-authored English on the Logs, Usage and Telemetry pages,
the log detail drawer and the shared usage widgets through t() and Trans,
with 1,250 messages added to en.json under 10 new top-level keys. The
rendered English is unchanged.
Moves the page-authored English on the virtual keys, projects, cost tracking,
cost optimization and budgets pages, plus the create key form and the shared
key and team pickers, into t() and Trans with 1,318 messages under 32 new
top-level keys in en.json. The English on screen is unchanged
key_edit_view.tsx grows past 800 lines, so it gets a max-lines entry in
eslint-suppressions.json instead of a split
* fix(router): stamp the failed deployment on every endpoint's attempt
Weighted failover (#27980) re-picks within a model group by excluding the
deployment named in the error's failed_deployment_id. #27980 stamped that id
in only three attempt functions: _completion, _acompletion and
_ageneric_api_call_with_fallbacks_helper. Embedding, image generation,
transcription, speech, text completion, rerank, adapter completion, the sync
generic call and vector store create never stamped it, so with
enable_weighted_failover on, a failure on one of their deployments skipped the
in-group re-pick and went straight to the cross-group fallbacks, or raised.
Each of those attempts now stamps the deployment it picked, read from the
model_info that _update_kwargs_with_deployment put on the request (so a
client-side-credential call names its own dynamic deployment, as on
completions). A failed pick stamps nothing. File and batch creation and batch
cancellation are left alone: they are bound to a resource on the deployment
they picked, so re-picking another one cannot help.
* test(router): narrow the failed-pick raise to the no-healthy-deployments error
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013FDwtoMjnHa4xxazGJJNSm
* fix(router): exclude the configured deployment on failover for client-side credentials
Weighted failover excluded the per-request id that client-supplied credentials
generate, so the failed configured deployment stayed eligible for the re-pick.
Stamp the configured id alongside it and exclude that. Also drop the vector
store create stamp, which no fallback path reads, and redundant test comments.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013FDwtoMjnHa4xxazGJJNSm
---------
Co-authored-by: Claude <noreply@anthropic.com>
* feat(decisions): add EmpirioLabs AI as a decisions provider
empiriolabs/aplomb-1 now works with litellm.decisions and litellm.systemone, the proxy /v1/decisions and /v1/systemone routes, the decision model guardrail and the Admin UI. Requests go to POST /v1/decisions on the EmpirioLabs AI API with the System One body, keyed by EMPIRIOLABS_API_KEY with an optional EMPIRIOLABS_API_BASE
Chat models keep routing through the existing OpenAI-compatible JSON provider, and a test pins that
* ci: run the EmpirioLabs AI decisions tests in the llms unit shard
* fix: align EmpirioLabs registration with CI gates
---------
Co-authored-by: Adam Dalloul <47503782+Adam-Dalloul@users.noreply.github.com>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(accounting): replace event scaffolding with SpendLogs projection
* test: preserve spend timestamps without removed capture callback
* fix: guard accounting event removal across schema update paths
* refactor(accounting): retain schema and prepare durable payloads
* fix(accounting): preserve dates in serialized payloads
* fix(accounting): retain schema push compatibility
* fix(accounting): retain span registration for the legacy schema
* test(accounting): remove obsolete capture acknowledgement test
* fix(accounting): date every daily projection row the way the model rollup does
A space-separated startTime kept the whole timestamp as the daily key, while the model usage rollup took the calendar date. Every daily row now uses the same date helper
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* chore(ui): regenerate dashboard API types after rebasing on main
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* refactor(accounting): project only from the stored accounting payload
project() now reads entities only from the payload's accounting_entities, so
the raw SpendLogs fallback branches are gone. build_accounting_payload() builds
the metadata once instead of dumping it to JSON and parsing it back. Guardrail
usage dates use a tolerant UTC parse, so a malformed startTime drops only the
guardrail rows instead of failing the whole projection.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Joshua Valluru <326636767+joshua-berri@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* fix: prevent reasoning field from being copied into provider_specific_fields
* fix: ensure reasoning field in provider_specific_fields is redacted
* fix: redact reasoning_details in provider_specific_fields
* fix: format redact_messages.py
* ci: retrigger
* ci: retrigger
* ci: retrigger
* fix(logging): keep provider_specific_fields.reasoning in chat responses
Redaction already blanks provider_specific_fields.reasoning and reasoning_details in logs, so chat responses can keep the field as they did before.
* fix(logging): redact reasoning_text, citations, and web search results in provider_specific_fields
* fix(logging): redact vector store, RAG, Groq tool, Databricks agent and tool search results in provider_specific_fields
* fix(logging): redact Bedrock streamed reasoningContent and file annotations in provider_specific_fields
---------
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
* test(e2e): retire the quiet_stack lock
quiet_stack held an exclusive host-wide lock for six tests, draining every
other xdist worker while they ran. On the full e2e lane that left 35-38% of
worker time idle. Each of those tests now lives where it can run without
pausing the suite
- the two memory tests move to perf/ behind the perf marker, so the perf lane
runs them on its own fresh stack. Their refusing deployments use a literal
key because the perf stack carries no provider keys
- the stored-request size check from the memory test becomes an integration
test on an owned proxy that stores prompts
- the discount and margin spend-row tests are dropped from e2e. Their
breakdown assertions join the existing integration pricing test, which
already owns the discount and margin repricing flow
- the Bedrock guardrail cost test stays in e2e without the pricing reset
fixture, since nothing else on the stack writes pricing config now
- the MCP semantic tool filter test turns on a proxy-wide setting, so it gets
a new proxy_wide_config opt-in marker and runs after the main run
* test(e2e): register the claude_code deployments as one batch
The claude_code session fixture registered its 24 deployments one at a time on every xdist worker, and each one waited out the 15 s propagation window on its own, about 6 minutes per worker. ProxyClient.register_models posts the whole batch, polls each model until every replica serves it, then waits out propagation once from the last write. A model that fails to post or never becomes servable comes back as ModelNotRegistered, and an unservable one is deleted, so the fixture keeps reporting per-deployment failures the way it did
* feat(openai): bill gov.api.openai.com FedRAMP endpoint with 1.1x uplift
* chore(openai): trim FedRAMP uplift diff
* refactor(openai): bill the FedRAMP endpoint through its own fedramp_uplift_multiplier key
* fix(ui): regenerate fedramp_uplift_multiplier doc comment in schema.d.ts
* style: ruff format utils.py and reorder fedramp_uplift_multiplier in schema.d.ts
* fix(openai): infer data_residency from OPENAI_BASE_URL and litellm.api_base when api_base is unset
* test(cost): cover get_fedramp_uplift_multiplier fallback paths
* fix(cost): apply fedramp_uplift_multiplier in batch_cost_calculator
* fix(openai): derive data_residency from a supplied client's base_url
* test: type the supplied-client data_residency test params
---------
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
