`AnthropicCacheControlHook` spends the configured injection points on the first
message list it is shown and drops the message points that matched nothing. That is
right when the messages it sees are the ones going upstream. It is wrong for
/v1/responses: the system prompt lives in `instructions`, which only becomes a system
message once the chat-completion bridge builds one, so a role-targeted point matched
nothing and was thrown away before the message it wanted existed. Injection silently
did nothing across the whole surface.
Hand those points back instead, stamped as judged, when the caller says its message
list is provisional. The stamp is what makes carrying them safe: without it the next
pass re-judges the points against messages this pass has already marked and stands the
whole configuration down. Callers holding the final messages -- /chat/completions and
/v1/messages -- do not raise the signal and keep dropping unmatched points as before.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
An auto_router deployment is a marker, not something a probe can contact, so
`_run_model_health_check` returns `{}` for it and it lands healthy whatever is
behind it. This derives its verdict from the models it actually resolves.
Rules and owners:
- `strategy_router_dependencies` is the single answer to "what does this router
call": tier, default, classifier and embedding names per router kind, aligned
with what init and the request path actually use.
- `_health_check_eligible` is the single probe-eligibility gate, applied to the
requested set and to the pool a router's dependencies are drawn from alike, so
an opted-out deployment cannot re-enter through a router that depends on it.
- `_resolved_deployment_ids` resolves names through `get_model_list`, the same
composition of alias, routing-group and wildcard channels a request uses.
- A dependency reds its router only when *every* deployment behind the name is
known unhealthy. A replica this run never judged, hidden from the caller or
opted out of health checks, can still serve what the dead one drops, so
partial evidence leaves the verdict green. Absent information never reds.
- Verdicts settle over rounds, because a marker never fails a probe of its own
and a parent whose tier is a red router must inherit that fault. Both sweeps
are bounded loops, so a router cycle terminates green.
- Dependency probes are added only on the targeted `/health?model_id=` path the
dashboard uses per deployment, and are dropped from the response.
Resolves LIT-6073
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
get_fireworks_session_id fell back to litellm_trace_id when no session id was
given. That id is generated per request (uuid4 when absent), so x-session-affinity
carried a different value every time and Fireworks prompt caching never hit;
cached_tokens stayed 0 across identical prompts.
The None path the original change described was effectively unreachable because
of it. Drop the fallback so affinity comes only from an id the caller actually
supplied: litellm_session_id, session_id, or metadata.session_id.
Callers who were relying on a trace id for affinity can pass litellm_session_id
instead, which is stable across the requests they want grouped.
Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
The image_edit health probe sent a 512x512 solid-gray PNG with the generic
chat prompt "test from litellm", an ambiguous pair OpenAI's gpt-image-1
output moderation sometimes rejects as moderation_blocked, which reported a
working deployment as unhealthy. The probe now sends a blue circle on a
white background with a descriptive edit prompt, and a provider moderation
verdict (ContentPolicyViolationError or a moderation_blocked error body) is
treated as proof the endpoint works rather than as an unhealthy deployment.
Live probes against every gemini_native_audio model on both providers show
setup accepts a valid prebuilt voice and 1007s only unknown voice names, so
the strip predicate rested on a false premise and silently discarded the
client's voice on AI Studio native-audio sessions
The dashboard resolved the complexity-router tier set three different ways: a
private TIER_KEYS in build_complexity_router_config.ts, TIER_ORDER in
complexity_router_tiers.ts, and TIER_KEYS in ComplexityRouterConfig.tsx. The
edit modal went further and re-implemented the whole create payload builder,
kept in sync only by a comment reading "Mirrors buildComplexityRouterConfig".
tier_rows.ts now owns the tier set. Every consumer reads activeTierRows(value)
and a row carries its own id, so the plan-mode floor and per-model params point
at a row rather than at a position, and the leaves that already wanted entries
(buildAutoRouterTestTargets, getRequiredModels, model_info_view) take them.
buildUpdatedComplexityRouterConfig becomes preserve-unmanaged-keys around the
shared builder instead of a second copy of it.
Also drops the literal ", ]" that renders as visible text in two DialogFooter
blocks on the auto-router routing-test and connection-test dialogs, left over
from a JSX array-to-fragment conversion.
No behaviour change: all 566 tests over the touched modules pass with fixture
shape changes only, no assertion edited.
The three member-budget tests patched litellm internals and asserted only on
the mock, which tripped the TQ002 and TQ008 test-quality ratchet. Fake the
prisma budget table on the shared client and assert on the row that reaches
the database plus the returned team payload.
POST /prompts silently stored an empty template when litellm_params.prompt_id
was combined with prompt_data keyed by template name, because the loader
wrapped the already-keyed dict under prompt_id a second time. The loader now
wraps only a flat template (a dict carrying a content key), and create,
update, and patch reject the ambiguous keyed+prompt_id combination with a 400
that names both valid shapes. The API also returned version null on every
create and lost version, environment, and created_by on registry reload; both
now carry through. Versioned ids like my-prompt.v1, which the create API
itself returns, now resolve to their base template on the SDK prompt hooks,
and a flat DB prompt with no litellm_params.prompt_id registers under its base
API id instead of garbage.
The member duration dropdown reused its placeholder as "Never resets", so a
team with no member budget yet showed "Never resets" while sending nothing and
inheriting the team's own reset period. Use the dropdown's never-resets
sentinel for an explicit null and label the untouched state as inheriting.
The Default Budget Duration field in Team Member Settings only offered daily, weekly and monthly, so a team member budget could never be set to never reset. It now uses the shared BudgetDurationDropdown, and /team/update writes an explicitly null duration through to the member budget row along with its reset time.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
A model_group_alias whose target is an auto-router shows up in /v1/models and
/model_group/info but 400s on call with "Unmapped LLM provider for this
endpoint. You passed model=complexity_router, custom_llm_provider=auto_router".
async_pre_routing_hook picks the strategy using the name the caller passed,
while the alias is only resolved further down in
_common_checks_available_deployment, so the four strategy registries, all keyed
by the marker deployment's model_name, never match. The hook then declines, and
the auto_router/ marker deployment goes out as if it were a real model
Resolve the alias once at the top of the hook, for lookups only, so the
registries, the tag-filtering escape hatch and the marker's forwardable params
all see the name they are keyed under. The caller-facing name is untouched:
spend metadata is stamped before routing and the response still carries the
alias the client sent
Second half, so the same symptom cannot reach a provider through the entry
points this does not fix (the sync selection path that never runs the hook, a
team-scoped router keyed on its internal name), a group that resolves only to
strategy markers is no longer callable: it raises a BadRequestError naming the
marker instead of handing the auto_router/ pseudo-model to the provider
Drop the notifications_manager.tsx keyword tweak: staging replaced the
substring classifier with lib/toast.ts, which already titles
validation_error responses as Validation Error
