Replace ChatUI cards, inputs, dialogs, popovers, MCP selects, uploads,
tooltips, and icons with shadcn/Base UI and Lucide. Update ChatUI tests
to drive searchable combobox controls instead of Ant Design selectors
Continue the Playground Chat Ant Design/Tremor migration: shared MultiSelect,
upload validation with semantic file inputs, collapsible message widgets, and
AdditionalModelSettings on Base UI controls
Bare fireworks_ai/<slug> only resolved to accounts/fireworks/models/<slug>,
so Fireworks routers (served at accounts/fireworks/routers/<id>, e.g.
glm-latest and firerouter) could not be reached without passing the full
resource id. Add a shared resolve_fireworks_resource_name helper that maps an
explicit routers/<id> or models/<id> segment to the right resource path, keeps
the existing -fast router heuristic, and defaults bare slugs to models/ for
backward compatibility. Wire it into both the chat and text-completion
transforms, which had drifted (completion lacked router handling entirely)
PR #35956 added the max_input_chars passthrough to the AutoRouter
constructor but left this mock assertion in tests/router_unit_tests
unchanged, so test_init_auto_router_deployment_success has been failing
on litellm_internal_staging ever since.
The passthrough itself is intentional and its behaviour is already
covered by TestAutoRouterMaxInputCharsWiring in tests/test_litellm, so
only the stale expected kwargs need updating. Assert the shared constant
rather than the literal 2000 so tuning the default does not break this
test again.
Add test_abort_upstream_logs_warning_when_aclose_raises: verifies that _abort_upstream swallows and logs any exception raised by the upstream's aclose() method instead of propagating it.
Add test_enqueue_for_client_returns_false_when_already_detached: verifies that _enqueue_for_client returns False immediately without touching the queue when client_detached is already set before the call.
Add test_enqueue_for_
Strip net-new inline # blocks from streaming_iterator.py, the unit test file,
and the live-proxy regression test to comply with the no-new-comments rule.
Add test_async_sse_wrapper_aborts_upstream_when_detached_drain_cap_reached:
verifies that when the detached-drain cap is already full, the pump calls
aclose() on the upstream so the provider stops generating and billing
instead of continuing to stream while we record only the partial prefix.
Also fixes LIT001 (bare dict in AsyncIterator union) by replacing dict
with Mapping[str, object] across all three stream-type annotations, and
adds the required LIT003 reason strings to the three noqa: BLE001 directives.
Add ANTHROPIC_MESSAGES_MAX_DETACHED_STREAM_DRAINS and ANTHROPIC_MESSAGES_STREAM_RELAY_QUEUE_MAXSIZE to the excluded set. These are advanced internal infrastructure parameters for streaming/queue management with sensible defaults that most users should not modify.
The relay queue was unbounded, so a client reading a long stream more slowly
than Bedrock produced it let the pump accumulate every pending SSE chunk in
memory, and detached post-disconnect drains had no concurrency bound, so an
authenticated client could open many large streams and read slowly to pin
unbounded worker state.
Bound the relay queue and make the pump apply backpressure while the client is
connected (it blocks on a full queue, racing the disconnect signal), so a slow
reader throttles the upstream read exactly as the old direct yield did. Cap how
many detached drains run at once; over the cap a disconnected pump bills what it
collected instead of draining further. Detached-drain lifetime is otherwise
bounded by the upstream stream/read timeout. Both limits are tunable via env.
The detached pump previously caught every upstream exception (Bedrock read,
decode, provider-response, or chunk-conversion error) and terminated the
client stream normally, masking the original provider exception and its
status so downstream failure handling never ran.
Now, when the upstream fails while the client is still connected, forward the
original exception through the queue so the client-facing generator re-raises
it and the proxy's failure handling (status code, post_call_failure_hook)
runs unchanged. Only when the client has already disconnected, where there is
no one to propagate to and no failure hook will fire, fall back to salvaging
partial spend from the collected chunks.
On the /v1/messages -> bedrock/ invoke streaming path a client disconnect
raises CancelledError inside the httpx socket read, which unwinds the whole
upstream generator chain before any finally can drain it. Bedrock keeps
generating and billing the full response, so spend tracking logged only the
truncated partial the client drained (output tokens ~1-15 vs the real count)
and undercounted against AWS invocation logs.
Move the upstream read into a detached background task that fully drains the
provider stream to its terminal message_delta/message_stop and bills there.
The client-facing generator only relays chunks off a queue, so a disconnect
tears down the relay but not the pump. A client_detached event stops
enqueueing after disconnect so the queue can't grow unbounded.
The provider's release gate in project-releaser publishes only when the
topmost released heading in terraform/provider/CHANGELOG.md moves past the
tag the mirror already carries. That heading has been 0.2.2 since
2026-05-13, so every stable release since has correctly decided there was
nothing to publish and the registry has gone stale.
Two things were blocking a release:
1. The mirror shipped 0.3.0 out-of-band on 2026-07-13 (pricing_base_model,
BerriAI/terraform-provider-litellm#47) after the source move, so that
code exists only in the mirror. The publish rsyncs monorepo -> mirror
with --delete, so publishing without this port would have deleted a
released feature from the registry.
2. Nothing here declared a new version.
Port #47 verbatim (resource_model.go and resource_model_crud.go are now
byte-identical to the mirror's released files), backfill the 0.3.0
changelog entry it shipped under, and cut 0.4.0 covering the changes made
here since the source move. 0.3.0 is not reusable as the next version --
the mirror holds that tag and the publish workflow's tag guard rejects it.
When store_model_in_db is true, general_settings are persisted to the
LiteLLM_Config DB table. On subsequent startups and periodic reloads,
_add_general_settings_from_db_config() unconditionally overwrites the
in-memory general_settings with DB-cached values, including
store_prompts_in_spend_logs.
This means a YAML config change (e.g. store_prompts_in_spend_logs: false)
deployed via CI/CD has no effect because the stale DB value (true) always
wins. The admin must manually update via /config/update API after every
deploy, defeating config-as-code.
Fix: track which general_settings keys were explicitly set in YAML at
startup (_yaml_general_settings_keys). During DB config merge, prefer the
YAML value for tracked keys. The DB value is only used as fallback when
YAML does not set the key, preserving the admin UI's ability to change
settings at runtime.
Steps to reproduce:
1. Start proxy with store_model_in_db: true, store_prompts_in_spend_logs: true
2. Change YAML to store_prompts_in_spend_logs: false, restart
3. Send a request, query LiteLLM_SpendLogs - prompts still stored
4. Check LiteLLM_Config table - DB still has true, overriding YAML
Slack thread: https://dataset-jsonhackathon.slack.com/archives/C0ACUS7LM29/p1785835131860139
Map the remaining gateway-documented effort keys: thinking as an alias
for enable_thinking (enable_thinking wins when both are present),
reasoning_budget to an integer reasoning_effort (skipped when thinking
is explicitly off), and low_effort=true to reasoning_effort=low (budget
wins when both are set). guided_json and guided_choice response_format
wrappers now include the name field (response and choice) to match the
gateway wire shape.
GET /v1/files filters data down to the caller's own managed files but left first_id and last_id as the upstream page's, so a non-owner got back file ids belonging to other users even with an empty data array
The native files and batches routes declare /{provider}/v1/... and their routers are mounted before the passthrough router, so /openai_passthrough/v1/files and /openai_passthrough/v1/batches matched them with provider="openai_passthrough" and 500'd on the LlmProviders lookup instead of reaching openai_proxy_route.
Move the dedicated /openai_passthrough prefix onto its own router mounted ahead of the batches and files routers. /openai/... and every other provider prefix keep their current behavior.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The base branch ratcheted the same limits in 28a277e9, so the conflicting
files were reset to base and the ratchet re-run against the new merge-base
rather than resolved by hand. Each limit is now the base value minus this
branch's own delta, so both ratchets survive: basedpyright -653 across 48
rules, strict ruff -80, LIT -85.
Seven wide-reaching but semantically neutral commits landed since the
last entry, together rewriting roughly 162k lines across ~4,700 file
touches. Blame on any line they reflowed points at the sweep instead of
the commit that wrote the logic.
They cover the safe ruff autofix pass, the collections.abc import move,
the f-string !s cleanup, lazy log message construction, the LIT010 and
LIT011 Final and frozen-parameter rollout, ruff coverage for
litellm/types, and the inert type: ignore strip.
Smaller ratchet commits are left out on purpose: each touches a few
hundred lines at most, so listing them would grow the file faster than
it buys back blame accuracy
Revert the LIT001 build-clause inserts, phrase the LIT002 menu as
'or (if it really must be dynamic) a MappingProxyType wrapping a dict
literal or comprehension', and fold the two freezing-wrapper exemption
sentences into one that names MappingProxyType beside tuple/frozenset.
LIT001/LIT002 and the typing.Dict ban all steered dict-shaped values to
frozen dataclasses or suppression even though the checker already accepts
MappingProxyType as a freezing wrapper; the messages now name it so the
dict-shaped freeze path is actually discoverable at fix time.
Storing a baseline used to prune every other cache entry, so gate runs in
concurrent worktrees kept evicting each other's baselines and forcing full
recomputes: this bit six times across two nights of benchmarking. The store
now writes alongside existing entries and evicts only the oldest beyond
eight, keyed as before by merge-base and environment fingerprint, so
parallel worktrees' baselines simply coexist
Prompt Security drops tool and function rows unless check_tool_results
is on, so it now reports scan-only support from that setting and the
registry refuses the pairing at boot. Pairing scan_only_tool_results
with skip_tool_message_in_guardrail excludes every message, so guardrail
initialization now rejects that combination too.
prisma resolves its prisma-client-py generator through a plain /bin/sh PATH
lookup, never through the interpreter that ran prisma generate, so the gate's
generate step landed the client in whatever venv the caller had on PATH: the
owned env never received one, every gate run regenerated, the caller's venv
was mutated instead, and any invocation without a venv on PATH (the rewritten
publisher workflow) failed outright
The generate now runs with the target interpreter's bin directory pinned to
the front of the child PATH. The prisma schema joins the environment
fingerprint so clientless counts recorded before this commit can never be
compared against clientful ones, a cold provision announces itself on stderr
instead of sitting silent for two minutes, and the CI gate step reuses the
job's prisma binary cache
Under scan_only_tool_results, legacy OpenAI function-role messages now count as tool results, and duplicate names among guardrail-returned tools keep only the first occurrence. CustomGuardrail.structured_messages_cover_full_request lets CrowdStrike AIDR declare that its writeback already rebuilds the whole conversation, so handlers install it as-is instead of merging it into the full message list a second time and duplicating out-of-scope rows. Lint budget ceilings ratchet down to match the tree
