The TPM pre-call reservation read `max_tokens or max_completion_tokens`, so a
request declaring both was charged for whichever field came first. A caller
sending `max_tokens=1` with `max_completion_tokens=10000` reserved 2 tokens and
was then free to consume ten thousand, since the provider honours the modern
field and litellm's own param mapping drops the legacy one for the gpt-5 and
o-series families.
Reserve against the larger of the declared budgets instead. Over-reserving is
the safe direction for a limiter: post-call reconciliation refunds the
difference between the reservation and actual usage, while under-reserving lets
the window be exceeded before anything notices.
redis-py >= 7.4 decorates AbstractConnection.__init__ with @deprecated_args,
whose wrapper is declared (self, *args, **kwargs). _init_arg_names introspects
the wrapper directly, so from redis-py 7.4 the MRO walk loses every real
connection parameter and the from_url allowlist silently drops socket_timeout
and socket_connect_timeout again - the exact regression the allowlist rework
fixed, reintroduced one dependency version later. A url-configured Redis that
blackholes packets then blocks callers indefinitely instead of timing out.
Follow the __wrapped__ chain with inspect.unwrap before introspecting; a no-op
for undecorated __init__s.
Measured across redis-py lines (socket_timeout present in the allowlist):
6.4.0 before/after: yes/yes. 7.1.0: yes/yes. 7.4.1: NO/yes. 8.1.0: NO/yes.
tests/test_litellm/test_redis.py at redis-py 8.1.0: 10 failures before, 3
after (the residual trio is sentinel/cluster password handling, failing
identically without this change).
Two tests: a decorated-fake proving the unwrap mechanism, and a live-invariant
assertion that the installed redis-py's allowlist carries the socket timeouts -
the first thing to go red if a future redis-py changes signature declaration
again.
Co-authored-by: yuneng-jiang <yuneng@berri.ai>
* fix(model_map): flag native structured outputs on Anthropic-direct claude-sonnet-5 and claude-haiku-4-5
The Bedrock twins of both models already carry
supports_native_structured_output, but the Anthropic-direct entries do not,
so response_format requests to anthropic/claude-sonnet-5 and
anthropic/claude-haiku-4-5 fall back to the json_tool_call emulation and
inherit its nested-envelope failure modes (#8898) despite the API supporting
output_format natively.
Verified live against the Anthropic API on 2026-08-05: both models accept
output_format (structured outputs beta header) and return exact schema
instances, including a large nested production schema validated with
pydantic. Same two lines applied to the bundled backup map.
* fix(model_map): cover the versioned claude-haiku-4-5-20251001 alias
Exact-match capability lookup of anthropic/claude-haiku-4-5-20251001
resolved the versioned entry, which lacked the flag, so response_format
for that identifier still took the tool-emulation path. Flag it in both
the root and bundled maps, matching its unversioned alias.
* fix(anthropic): bound $defs inlining in output_format with the shared schema-bomb budget
map_response_format_to_anthropic_output_format called unpack_defs with
no max_inlined_bytes, so an authenticated caller could send a compact
schema whose repeated $refs expand without bound before reaching the
provider. Reuse the existing 10MB inlining budget (renamed from
_LEGACY_DEFS_MAX_INLINED_BYTES to DEFS_MAX_INLINED_BYTES now that two
call sites share it); overflow raises ValueError instead of
materialising the expansion.
Regression tests: a compact schema bomb is rejected, a normal $defs
schema still resolves; the bomb test fails when the bound is removed.
* chore: retrigger CI (benchmarks job flaked on a PyPI download timeout)
---------
Co-authored-by: Anmol Jaiswal <anmolg1997@users.noreply.github.com>
* fix(panw_prisma_airs): surface scan_id and scan metadata on allowed requests
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* style: ruff format panw guardrail
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(panw_prisma_airs): expose scan id header only
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(panw_prisma_airs): inject http client instead of patching private api
Adds an http_client seam so the scan-id tests drive the real AIRS request/parse path through a mock transport, plus direct coverage for the scan-id header helper.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): expose guardrail scan id header to browser clients
Keeps the panw optional_fields block untouched to avoid a needless conflict with a sibling PR that deletes it.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Registering the five managed-batch fields in all_litellm_params stops them leaking into
extra_body, but two of them never reached the transformation that reads them.
CredentialLiteLLMParams is a whitelist, so get_deployment_credentials_with_provider
round-tripped the deployment and silently dropped s3_output_bucket_name and bedrock_tags
before the files/batch/passthrough callers saw them. s3_bucket_name, s3_region_name and
aws_batch_role_arn were added to that model for #25104; these two are the remainder of
the same deployment config
bedrock_tags is typed as a plain list rather than a stricter shape so a malformed value
still reaches _validate_bedrock_tags and gets its own error message instead of a Pydantic
one
The preservation assertion previously round-tripped through GenericLiteLLMParams, which is
extra="allow" and would hold even for a field nothing declares. It now also reproduces the
CredentialLiteLLMParams normalization the proxy actually performs, and fails naming
exactly the dropped fields without this change
A Bedrock managed-batch deployment carries aws_batch_role_arn, s3_bucket_name,
s3_region_name, s3_output_bucket_name and bedrock_tags in its litellm_params,
and the batch and files transformations read all five from there. None was
registered in all_litellm_params, so the param builder swept them into
extra_body on every other route that deployment serves: Bedrock answers
"aws_batch_role_arn: Extra inputs are not permitted" on Anthropic models and
"extraneous key [aws_batch_role_arn] is not permitted" on Nova, Llama and
Titan, so configuring batch turns every chat and embedding request to that
model into a 400.
Register them alongside the agentic-loop and callback-credential fields, which
are listed for exactly this reason. The batch path is unaffected because
GenericLiteLLMParams is extra="allow" and preserves them into litellm_params
for the transformations that consume them.
Before this, batch could only be configured on a deployment dedicated to
batch; the same model group could not serve both.
The Global Control Plane (formerly documented as the HA Control Plane) is
documented as an Enterprise feature, but `worker_registry` carried no premium
check, so any OSS install could run one. Gate it at config load, matching the
`enforced_params` precedent, and fail startup rather than ignoring the registry
silently: a silently dropped registry degrades a control plane into an ordinary
proxy with no signal to the operator.
Also declare `worker_registry` and `general_settings.control_plane_url`, both
load bearing today and neither previously declared, so they appear in the
generated config schema.
Dashboard tests located controls through antd's own class names
(.ant-form-item, .ant-select-selector, .ant-select-item-option). Those
break when a page moves to shadcn without any behaviour changing, and
they miss regressions a user would notice.
Replace them with role, label, title and accessible icon-name queries
where antd exposes one, and add local/no-antd-class-selectors to keep
them out. The rule is enabled as an error at zero violations, so there is
no budget file and no suppressions baseline. It found eight more sites a
'.ant-' search missed, written as bare class names.
Eleven couplings remain and carry an inline suppression naming why:
antd puts role="option" only on a hidden mirror list, so the visible
options have no role, no aria-disabled and a tooltip in title; Skeleton
and the modal mask expose nothing at all; and one assertion's whole
purpose is that no antd modal renders.
7231 tests passed before, 7233 pass after: one conflated ModelSelector
case became three focused ones.
antd Form.Item injects an id into its child so the rendered label can point
at the control. BudgetDurationDropdown destructured its props without
forwarding that id, so the label resolved to nothing and assistive tech
announced the field unnamed. Forward it to SelectTrigger.
Google's Gemini deprecations page lists no shutdown date for the 2.5 GA models and May 14, 2028 for gemini-embedding-001; keep only the DeepSeek V4 max output correction.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Revert gemini-embedding-001 to its published 2028-05-14 shutdown, move
gpt-4-turbo-preview to the 2026-03-26 shutdown of the gpt-4-0125-preview
snapshot it aliases, and drop the unannounced Gemini 2.5 shutdown dates
Create Key offered two options labelled "Never resets" in the Reset Budget
dropdown. BudgetDurationDropdown renders its unset item using the caller's
placeholder, and create_key_button passed placeholder="Never resets" alongside
showNeverResets, so the omit option and the explicit-null option looked
identical while behaving differently. An omitted budget_duration picks up
default_key_generate_params and the linked budget tier's schedule, whereas the
"none" sentinel is converted to an explicit null and truly never resets. The
unset item now reads "Not set", matching getBudgetDurationLabel, and the
create-key test mock passes the placeholder through so a future collision fails
the suite
The rest is migration cleanup found during manual QA. The models and endpoints
tab strip hides its scrollbar and fades at the right edge using a vendored copy
of the shadcn scroll-fade utility, keeping the CLI package out of the build.
globals.css neutralises the @tailwindcss/forms resting-state rules for
combobox-chip-input, which lets twelve call sites drop the same copy-pasted
className workaround. Guardrails moves to the line tab variant and stops
clipping its textarea focus ring, the log drawer JSON tree takes the app
background, the audit log empty state centres, the caching page selects no
longer stretch to the row height, the usage page team filter shares its row
with the Export button through a new filterSlot prop, and the cost optimization
and vector store tab strips drop their leftover full-width divider
* fix(proxy): requeue spend logs when the DB write fails with a transport error
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): hardcode the spend log queue cap and drop the stale re-export
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(proxy): keep the spend log requeue within the type discipline budget
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): apply the spend log queue cap to producer appends too
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): lower the spend log queue cap to 1k and make it env configurable
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): bound the spend log queue by bytes instead of row count
A row cap cannot bound memory: a row carries the whole prompt under store_prompts_in_spend_logs, so a cap that rides out an outage of counter-only rows is an OOM once prompts are stored. Every enqueue and dequeue now goes through one pair that tracks what the queue costs and drops the oldest rows past a 64 MB budget.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): make the spend log queue byte budget env configurable
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): use a string default for the spend log queue byte budget env read
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(proxy): make the spend log queue byte total a public attribute
The queue it accounts for is already public, and a private name only bought reportPrivateUsage errors at every call site.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: shivam <shivam@berri.ai>
