supports_mid_conversation_system joins the other supports_* helpers in
litellm.utils, so the chat transformation stops importing the private
_supports_factory.
Codecov flagged the 5m ttl branch and the empty-system path of the wire
builder; both now have a test. Greptile asked for full typing on the new
test helpers.
Same priming and assertions as the /v1/messages cases, through
/v1/chat/completions with OpenAI-format messages, for first-party Anthropic
and Bedrock Invoke on a flagged (Opus 4.8) and an unflagged (Haiku 4.5) model.
The reminder sits between the assistant turn and the next user turn, the shape
OpenAI-style agent frameworks send, which is the placement the chat path has
to translate.
translate_system_message hoisted every role=system message, at any index, into
the top-level system block. On a conversation carrying a mid-session reminder
that rewrites the cached prefix, so the provider re-bills the whole history at
cache-write pricing on every turn (#36559). #36968 fixed this on /v1/messages;
the chat completions path, shared by first-party Anthropic, Vertex, Azure AI
and Bedrock Invoke, still hoisted.
Only the leading system run becomes the system prompt now. Later system
messages go through the placement policy, and anthropic_messages_pt emits a
system message instead of rejecting the role. The caller's message list is no
longer mutated. Tests pin the two-turn prefix invariant across all four chat
configs and both flag states.
Pure functions over the OpenAI-format message list: split off the leading
system run, keep later system messages as role=system at a placement Anthropic
accepts on models flagged supports_mid_conversation_system (after a user turn,
before an assistant turn or the end, never adjacent), and convert them to user
turns in place elsewhere, keeping tool_result first in a merged user turn.
POST /prompts silently stored an empty template when litellm_params.prompt_id
was combined with prompt_data keyed by template name, because the loader
wrapped the already-keyed dict under prompt_id a second time. The loader now
wraps only a flat template (a dict carrying a content key), and create,
update, and patch reject the ambiguous keyed+prompt_id combination with a 400
that names both valid shapes. The API also returned version null on every
create and lost version, environment, and created_by on registry reload; both
now carry through. Versioned ids like my-prompt.v1, which the create API
itself returns, now resolve to their base template on the SDK prompt hooks,
and a flat DB prompt with no litellm_params.prompt_id registers under its base
API id instead of garbage.
A model_group_alias whose target is an auto-router shows up in /v1/models and
/model_group/info but 400s on call with "Unmapped LLM provider for this
endpoint. You passed model=complexity_router, custom_llm_provider=auto_router".
async_pre_routing_hook picks the strategy using the name the caller passed,
while the alias is only resolved further down in
_common_checks_available_deployment, so the four strategy registries, all keyed
by the marker deployment's model_name, never match. The hook then declines, and
the auto_router/ marker deployment goes out as if it were a real model
Resolve the alias once at the top of the hook, for lookups only, so the
registries, the tag-filtering escape hatch and the marker's forwardable params
all see the name they are keyed under. The caller-facing name is untouched:
spend metadata is stamped before routing and the response still carries the
alias the client sent
Second half, so the same symptom cannot reach a provider through the entry
points this does not fix (the sync selection path that never runs the hook, a
team-scoped router keyed on its internal name), a group that resolves only to
strategy markers is no longer callable: it raises a BadRequestError naming the
marker instead of handing the auto_router/ pseudo-model to the provider
Drop the notifications_manager.tsx keyword tweak: staging replaced the
substring classifier with lib/toast.ts, which already titles
validation_error responses as Validation Error
Cache-hit success callbacks in short-lived SDK scripts enqueue
Logging.async_success_handler onto the global logging worker; the worker
loop dequeues the task and asyncio.run cancels the processing task before
it ever starts, so the coroutine leaves the queue unawaited and the atexit
flush finds an empty queue and rescues nothing. Track dequeued-but-unfinished
tasks with strong refs and have the atexit flush run any whose coroutine
never started
The realtime health check always built the Azure websocket URL with the
default beta protocol, so GA-only transcription models such as
azure/gpt-realtime-whisper got probed at /openai/realtime and were
rejected with HTTP 400 on every /health run, while real calls through
the proxy resolved the GA path via intent=transcription and worked.
The probe now resolves the protocol the way the real call path does:
an explicit realtime_protocol (argument, deployment litellm_params, or
LITELLM_AZURE_REALTIME_PROTOCOL) wins, transcription-only models fall
back to GA with intent=transcription, and everything else keeps beta.
Transcription-only detection reads both mode and supported_endpoints
from get_model_info because a live proxy overwrites the catalog mode
with the operator's deployment model_info (mode: realtime) during
router registration, while supported_endpoints survives it.
get_model_info now propagates supported_endpoints from the cost map;
it declared the field but never populated it.
