The model Azure Model Router served was recovered by checking whether the text
"model_router" or "model-router" appeared in a model string. Spend logs applied that
check to the litellm model path, where the route prefix guarantees a match, but the
proxy applied it to the client's model group alias, which carries no prefix. A model
group named anything else therefore lost the selected model in both the response and
the spend row.
AzureModelRouterConfig now stamps the served model onto _hidden_params, and the spend
log payload and the proxy's response restamping read that stamp. The name heuristic
survives as a fallback for callers with no response in hand, routed through
get_azure_ai_route so it lives in one place.
Mantle 400s ("Invalid 'input': value did not match any expected variant")
on the Codex history item types agent_message, context_compaction, and
local_shell_call, killing every Codex multi-agent session on the first
sub-agent turn. Rewrite agent_message into an assistant output_text message
(preserving encrypted_content slot payloads, which carry the plaintext task
through Mantle), context_compaction into Mantle's supported compaction
spelling, and local_shell_call into the function_call its recorded
function_call_output already pairs with.
Read strict from the caller's output_format/output_config.format instead
of hardcoding true, defaulting to false to match OpenAI's API default.
Explicit true/false values are preserved and output_format still takes
precedence over output_config.format.
* fix(http_handler): dispose aiohttp session when finalized without a running loop
AsyncHTTPHandler.__del__ can only schedule an async close when a running
event loop exists at finalization time; in any other context (worker
threads whose loop has closed, sync contexts, interpreter shutdown) the
RuntimeError from get_running_loop() is swallowed and the underlying
aiohttp ClientSession is abandoned to GC, emitting 'Unclosed client
session' / 'Unclosed connector' warnings.
This is the disposal gap left after the recycle-time fix: clients created
for short-lived event loops (the loop-id-keyed LLM client cache mints one
handler per loop) are never recycled - they live and die with their loop,
and their finalization is precisely the loop-less case.
Fix:
- no running loop: fall back to the connector's synchronous teardown via
LiteLLMAiohttpTransport._mark_connector_closed - the same finalizer-safe
path used for dead-loop recycles - honoring _owns_session so a shared
session is never closed.
- running loop: keep the async close, but hold a strong reference to the
scheduled task until it completes (a bare create_task() result may be
collected before running), mirroring _background_close_tasks.
Tests: loop-less finalization closes a dead-loop session; running-loop
finalization registers and drains the close task; the sync fallback
respects session ownership. All three fail without the fix.
* lint: conform new finalizer code to the type-discipline budget
Final on the five never-rebound locals (LIT010); the class-level task
registry keeps its mutable set with the sanctioned mutable-ok reason,
mirroring the aiohttp transport's registry (LIT001).
* lint: reasoned pyright ignore on the cross-class teardown call
The handler deliberately reuses the transport's finalizer-safe connector
teardown; no public seam exists and an async close can never run at
loop-less finalization. Clears the net-new reportPrivateUsage the
basedpyright budget gate flagged once the LIT stage passed.
* fix(http_handler): retrieve exceptions from finalizer close tasks
A bare discard done-callback dropped the task without consuming its
exception, so a failing aclose() emitted "Task exception was never
retrieved" at GC, the same noise class this path exists to remove.
Mirror the transport's _on_close_task_done: discard, early-return on
cancellation, retrieve and debug-log the exception.
* fix(http_handler): dispose foreign-loop sessions instead of scheduling aclose on the live loop
GC on a live loop (e.g. the app's) of a handler whose session belongs to
another, possibly dead, loop scheduled aclose() on the current loop, the
cross-loop path the transport refuses. Route both that case and the
loop-less case through the transport's lifecycle-aware
_close_recycled_session, which picks async close on the session's own
loop, threadsafe handoff, or the synchronous connector teardown.
Regression test: a dead-loop session collected while another loop runs
is disposed without scheduling anything on that loop.
* chore: retrigger CI (test_mcp_logging payload-order flake, also failed on litellm_spendlogs_fallback_metadata minutes earlier)
* test(mcp): select the MCP tool-call payload instead of the last-delivered one
TestMCPLogger kept a single last-writer slot; an async success event from
another call (a mocked acompletion whose log task lands late) races the
MCP event for it, so the cost assertions intermittently read the wrong
payload. This PR's finalizer change shifts task interleaving on the loop
and tips that latent race over (also seen on an unrelated PR minutes
earlier). Collect call_type=call_mcp_tool payloads in their own list and
assert on those.
* test(mcp): MCPLoggerHook inherits the order-independent payload capture
It duplicated TestMCPLogger's init and success handler verbatim; the
hook test reads the same MCP payload selection, so subclass instead.
The video edit endpoint parsed the multipart body but dropped the uploaded
source video, only normalizing it to an id. When a raw file is uploaded it now
flows through videos.main -> the http handler -> the provider transform, which
emits multipart/form-data with the source video as a file part, matching the
official OpenAI SDK's videos.edit wire format. Edit-by-id still egresses JSON.
OpenAI videos never fetches input URLs or invents download size caps. Drop MAX_VIDEO_MEDIA_* and the fetch-and-inline path. Keep MAX_IMAGE_URL_DOWNLOAD_SIZE_MB for chat image handling. Validate Omni reference URLs, then send them as form fields the way OpenAI videos sends input_reference as a file
Suppress the three reportUnknownArgumentType diagnostics with reasons at
the untyped provider-params boundary, collapse the early return, and
assign extra_body via a TypedDict-annotated literal so the file's
basedpyright profile matches the merge base exactly. The user-supplied
extra_body merge is covered end-to-end through get_optional_params.
Route hosted_vllm video generation through /v1/videos as multipart form data so Omni extra fields such as width and extra_params reach the server instead of a JSON body Omni rejects
A reasoning model whose map entry names no effort flag now resolves to None, so
the API omits the field and the dashboard keeps its six-level fallback, and a
deployment counts as catalog-known only when the map supplied its mode, so an
operator writing model_info on an off-map deployment no longer empties the
levels its mapped siblings agree on.
Also drops the ultra level nothing asked for, forwards every level the public
literal names across the chat to Responses bridge, and removes the unreachable
supported_reasoning_efforts validator.
The chat-completions gate only ever owned xhigh. Widening it to max and ultra
made gpt-5.6 answer 400 on requests litellm itself converts to /v1/responses,
where max is valid, because the gate runs before the bridge decision. No map
entry asserts either flag, so the widened gate could only ever reject.
An empty per-group intersection now falls back to the capability-blind level
list in the dashboard, matching what the picker showed before the field
existed, and ModelGroupInfo tolerates whatever shape an operator writes under
supported_reasoning_efforts instead of failing the whole /model_group/info
response.
/v1/chat/completions rejects reasoning_effort=max on every gpt-5.6 snapshot with
"Unsupported value: 'reasoning_effort' does not support 'max' with this model.
Supported values are: 'none', 'low', 'medium', 'high', and 'xhigh'", so the 16
gpt-5.6 entries that asserted supports_max_reasoning_effort had model groups
advertising a level routing would always get a 400 for.
The flag reads as chat-surface capability everywhere else in the map: before
this branch no openai or azure entry carried it at all, only anthropic-family
ones whose chat endpoint really does take max. Dropping it lines the gpt-5.6
advertisement up with what the endpoint accepts and lets the chat gate refuse
the level with our own error instead of forwarding it into a provider 400.
/v1/responses does accept max for gpt-5.6 and is unaffected: the responses
config gates none alone, so the cursor thinking-max variant keeps resolving.
Extract a shared _valid_max_results predicate that rejects bools (an int
subclass) and non-positive values, and reuse it from both the connection-mode
request count and the response-side cap so both paths honor the same contract.
Address Greptile review comments and the strict lint budgets:
- read supports_adaptive_thinking from the model cost map instead of
substring-matching the model name, so aliases and newly onboarded
adaptive-only models need no code change
- add tencent/minimax-m3 to the pricing JSON (and backup), which also
fixes cost tracking for the model
- type the thinking/extra_body payloads with ReadOnly TypedDicts
- build the merged extra_body without rebinding or in-place mutation
- send a caller api_key via the Azure api-key header instead of Authorization: Bearer
- cap web_search results to the requested max_results (the tool has no count knob)
- surface a Foundry failed/incomplete response status as a 502 error
- zero the per-query cost in web_search mode; keep the map price for connection mode
- trim the example config to terse env-var pointers
AzureVideoConfig subclasses OpenAIVideoConfig and so inherits the new
use_multipart_form_data() -> True. Azure's /openai/v1/videos surface is
OpenAI-SDK-compatible, so the JSON->multipart flip is intentional; assert it
through the real handler so the inherited behavior can't silently regress.
* fix(anthropic): reconcile enum with declared type in output_format schema
Anthropic cross-validates `enum` against `type` in structured outputs: every
enum value must match a single declared type. A union `type` array, or an enum
value whose JSON type differs from a scalar `type`, is rejected with
"Invalid schema: Enum value 'low' does not match declared type '['string','null']'"
filter_anthropic_output_schema had no enum/type reconciliation, so both keys
reached Anthropic untouched. Drop the conflicting `type` -- `enum` is the
tighter constraint, and an enum with no `type` is accepted
The drop is conditional: `type` is only removed when it is a union array, or
when some enum value does not match the scalar type. A matching enum plus
scalar `type` is left exactly as-is, so existing behaviour is unchanged
Pydantic emits the failing shape for Optional[SomeEnum], so this affects any
caller with a nullable enum field on the native output_format path. vertex_ai
is unaffected because it is forced onto the permissive tool-use path
Fixes#37881
* refactor(anthropic): make enum/type reconciliation immutable and precisely typed
Address review: the predicate registry was a mutable `dict[str, Any]`, and the
reconciliation removed `type` by mutating the built result with `pop`
- registry is now `Final[Mapping[str, Callable[[Any], bool]]]` wrapped in
`MappingProxyType`, so predicate signatures are statically checked and the
table cannot be mutated
- the conflict decision moves into a pure helper evaluated once against the
input schema, and the conflicting `type` key is skipped at build time in the
existing loop instead of being popped afterwards, so nothing is mutated
Behaviour is unchanged; all 27 tests in the schema-filter suite still pass
Bring the Entra ID / OAuth auth work for Azure AI Foundry routes up to date
with staging and fix the lint-budget regressions the merge surfaced:
- widen get_azure_ai_auth_headers return type to Mapping[str, str] (LIT001)
- build the azure_ai image_generation request headers into a new Final local
instead of rebinding the Final headers dict (reportGeneralTypeIssues)
- order HuggingFace rerank validate_environment params to match BaseRerankConfig
so litellm_params lines up positionally (reportIncompatibleMethodOverride)
- add a match= to the credential-error test and document the handler-boundary
patches the auth wiring tests rely on
Six defects in the RunwayML video provider:
- transform_video_create_request hardcoded /image_to_video, so text-to-video 400'd and video-to-video was unreachable; the endpoint is now selected from the inputs present (promptVideo/videoUri, promptImage, or text only)
- get_error_class raised instead of returning, turning a provider 4xx into a proxy 500 APIConnectionError; it now returns a RunwayMLError
- VideoObject.progress was typed int while Runway sends a 0..1 float, 500'ing status polls while RUNNING; it is now scaled to a 0..100 percent
- custom per-deployment pricing stored under litellm_metadata was ignored for video; the deployment model_info lookup now checks both metadata keys
- stale cost-map entries (gen3a_turbo, gen4_aleph) were removed and current models added, with output_cost_per_second_480p/_4k tier keys plumbed through the model-info and router types
- video cost now falls back to Runway's estimatedCost from the create response when no custom pricing is configured, and custom pricing always wins over it
Fixes#36483
