* feat(langfuse): support langfuse_environment as a per-key dynamic callback param
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor(langfuse): type the langfuse_environment constructor param
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(langfuse): only pass environment when the SDK client supports it
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(langfuse): drop the request-body metadata test for langfuse_environment
The proxy bans request-body callback params by default (derived from
_supported_callback_params in auth_utils), so the metadata channel this
test asserted is rejected with a 401 on the proxy. The supported channel
is admin-set key/team callback_vars, with LANGFUSE_TRACING_ENVIRONMENT
as the deployment-wide fallback.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(langfuse): validate langfuse_environment, avoid redundant clients, honor it in langfuse_otel
Closes the review gaps on the langfuse_environment param:
- Validate values against Langfuse's environment pattern at save time
(/key/generate, /key/update, /team callback all 400 on e.g. 'Production'
instead of 200-then-silently-dropping every trace server-side) and at
logger init; non-string values are str()-coerced instead of crashing
the SDK's regex check per event.
- Treat empty/whitespace values and values equal to the deployment-wide
LANGFUSE_TRACING_ENVIRONMENT as non-dynamic so an environment-only
override that changes nothing no longer mints a duplicate SDK client
against MAX_LANGFUSE_INITIALIZED_CLIENTS.
- langfuse_otel now reads the per-key/team langfuse_environment from
standard_callback_dynamic_params instead of only the env var.
- Advertise the param on the discovery surfaces: callback_configs.json
(langfuse + langfuse_otel), the dashboard callback registry, and the
/team/{team_id}/callback docstring (schema.d.ts regenerated).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* style: ruff format langfuse files
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(lint): remove duplicate test import, LIT002 dict literal, and mock-echo otel test
- drop redundant in-function import of callback_config_error (F811)
- avoid the `or {}` mutable literal in _set_langfuse_specific_attributes (LIT002)
- rewrite the dynamic-env otel test to observe span.set_attribute output
instead of patching litellm internals (TQ002/TQ008)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: milan <milan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: yucheng-berri <yucheng@berri.ai>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
`AnthropicCacheControlHook` spends the configured injection points on the first
message list it is shown and drops the message points that matched nothing. That is
right when the messages it sees are the ones going upstream. It is wrong for
/v1/responses: the system prompt lives in `instructions`, which only becomes a system
message once the chat-completion bridge builds one, so a role-targeted point matched
nothing and was thrown away before the message it wanted existed. Injection silently
did nothing across the whole surface.
Hand those points back instead, stamped as judged, when the caller says its message
list is provisional. The stamp is what makes carrying them safe: without it the next
pass re-judges the points against messages this pass has already marked and stands the
whole configuration down. Callers holding the final messages -- /chat/completions and
/v1/messages -- do not raise the signal and keep dropping unmatched points as before.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
An auto_router deployment is a marker, not something a probe can contact, so
`_run_model_health_check` returns `{}` for it and it lands healthy whatever is
behind it. This derives its verdict from the models it actually resolves.
Rules and owners:
- `strategy_router_dependencies` is the single answer to "what does this router
call": tier, default, classifier and embedding names per router kind, aligned
with what init and the request path actually use.
- `_health_check_eligible` is the single probe-eligibility gate, applied to the
requested set and to the pool a router's dependencies are drawn from alike, so
an opted-out deployment cannot re-enter through a router that depends on it.
- `_resolved_deployment_ids` resolves names through `get_model_list`, the same
composition of alias, routing-group and wildcard channels a request uses.
- A dependency reds its router only when *every* deployment behind the name is
known unhealthy. A replica this run never judged, hidden from the caller or
opted out of health checks, can still serve what the dead one drops, so
partial evidence leaves the verdict green. Absent information never reds.
- Verdicts settle over rounds, because a marker never fails a probe of its own
and a parent whose tier is a red router must inherit that fault. Both sweeps
are bounded loops, so a router cycle terminates green.
- Dependency probes are added only on the targeted `/health?model_id=` path the
dashboard uses per deployment, and are dropped from the response.
Resolves LIT-6073
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Vertex Gemini 3.x models route through cost_per_character (the cost_router
token-path gate only matches gemini-2), and its token fallbacks dropped
service_tier, so ON_DEMAND_FLEX responses were still billed at the standard
rate. Pass the tier through the call site and all four fallbacks.
get_fireworks_session_id fell back to litellm_trace_id when no session id was
given. That id is generated per request (uuid4 when absent), so x-session-affinity
carried a different value every time and Fireworks prompt caching never hit;
cached_tokens stayed 0 across identical prompts.
The None path the original change described was effectively unreachable because
of it. Drop the fallback so affinity comes only from an id the caller actually
supplied: litellm_session_id, session_id, or metadata.session_id.
Callers who were relying on a trace id for affinity can pass litellm_session_id
instead, which is stable across the requests they want grouped.
Co-authored-by: mubashir1osmani <mubashir.osmani777@gmail.com>
Gemini API Maps-grounded prompts were billed as web search and Vertex AI Maps-grounded prompts were not billed at all. Classify grounding metadata per candidate into web search vs Maps requests, carry a distinct google_maps_grounding_requests usage counter through non-streaming and streaming paths, and price it via the new google_maps_grounding_cost_per_query cost map key with per-query and per-prompt defaults keyed off web_search_billing_unit. Fixes#35906
The image_edit health probe sent a 512x512 solid-gray PNG with the generic
chat prompt "test from litellm", an ambiguous pair OpenAI's gpt-image-1
output moderation sometimes rejects as moderation_blocked, which reported a
working deployment as unhealthy. The probe now sends a blue circle on a
white background with a descriptive edit prompt, and a provider moderation
verdict (ContentPolicyViolationError or a moderation_blocked error body) is
treated as proof the endpoint works rather than as an unhealthy deployment.
Live probes against every gemini_native_audio model on both providers show
setup accepts a valid prebuilt voice and 1007s only unknown voice names, so
the strip predicate rested on a false premise and silently discarded the
client's voice on AI Studio native-audio sessions
The dashboard resolved the complexity-router tier set three different ways: a
private TIER_KEYS in build_complexity_router_config.ts, TIER_ORDER in
complexity_router_tiers.ts, and TIER_KEYS in ComplexityRouterConfig.tsx. The
edit modal went further and re-implemented the whole create payload builder,
kept in sync only by a comment reading "Mirrors buildComplexityRouterConfig".
tier_rows.ts now owns the tier set. Every consumer reads activeTierRows(value)
and a row carries its own id, so the plan-mode floor and per-model params point
at a row rather than at a position, and the leaves that already wanted entries
(buildAutoRouterTestTargets, getRequiredModels, model_info_view) take them.
buildUpdatedComplexityRouterConfig becomes preserve-unmanaged-keys around the
shared builder instead of a second copy of it.
Also drops the literal ", ]" that renders as visible text in two DialogFooter
blocks on the auto-router routing-test and connection-test dialogs, left over
from a JSX array-to-fragment conversion.
No behaviour change: all 566 tests over the touched modules pass with fixture
shape changes only, no assertion edited.
The three member-budget tests patched litellm internals and asserted only on
the mock, which tripped the TQ002 and TQ008 test-quality ratchet. Fake the
prisma budget table on the shared client and assert on the row that reaches
the database plus the returned team payload.
