* fix(vertex_ai): graft default vertex path when api_base has a version-only path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(vertex_ai): keep query and fragment placement when grafting vertex path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(vertex_ai): merge alt=sse into existing query when streaming
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(spend_tracking): persist router metadata in spend logs for internal router models
* test(spend_tracking): expect router_metadata key in exact-payload tests, type the routed-kwargs helper
* test(newrelic): cover static default_team_settings per-team routing
The dynamic POST /team/{team_id}/callback path for New Relic is tested, but
the static default_team_settings twin had no regression coverage. Add a test
that drives default_team_settings -> add_team_based_callbacks_from_config and
asserts the resolved trusted vars dispatch to BOTH the per-team metrics logger
(cost/usage) and the trace logger (LLM/agent spans), so a config-file customer
gets the same per-team routing as the API customer.
Also correct the /team/callback docstring: callback_name is a str validated
against the credential-capable callbacks, not a fixed langfuse/langsmith/gcs
Literal, and document the newrelic_api_key / newrelic_region vars.
* chore(ui): sync schema.d.ts with the /team/callback docstring
Regenerate the dashboard OpenAPI types for the add_team_callbacks description
change: callback_name is a validated str (not a langfuse/langsmith/gcs
Literal) and the newrelic_api_key / newrelic_region vars are documented.
* docs(newrelic): note LITELLM_OTEL_V2 prerequisite, trim test comments
Address review: team-scoped New Relic config is rejected with a 400 unless the
proxy runs with LITELLM_OTEL_V2=true, so document that in the /team/callback
endpoint and sync schema.d.ts. Drop the narrative setup comments in the new
test per the repo comment convention; the test name and docstring already say why.
* fix(proxy): deliver budget alerts on webhook-only alerting and accept ALERTING_WEBHOOK_URL
ProxyLogging.budget_alerts forwarded to the alerting pipeline only when
'slack' was in general_settings.alerting, so alerting: ['webhook'] plus
WEBHOOK_URL silently never delivered a budget alert (the config
/health/services?service=webhook exists to test). Forward when 'webhook'
is present too; SlackAlerting.send_alert already fans out per channel.
Also accept a provider-neutral ALERTING_WEBHOOK_URL env fallback for the
Slack-format channel (any Slack-compatible receiver works), mark it as a
sensitive var, and de-brand the admin UI alerting copy.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): format settings.tsx with prettier
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): regenerate schema.d.ts for updated alerting description
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: retrigger checks after ALERTING_WEBHOOK_URL docs merged
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Default remains true so lite claude turns tool search back on through
a proxy. An explicit false or auto in the env or settings is left alone
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Claude Code turns tool search off when ANTHROPIC_BASE_URL is a proxy.
lite claude, lite up, login --config-claude, and autoroute now force
ENABLE_TOOL_SEARCH=true so MCP tools stay deferred through the proxy
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* fix(otel/v2): detach credential-routed tenant spans into their own trace
Multi-tenant OTel v2 routes a team or key's LLM-call span to that tenant's
own vendor account (New Relic, Arize, Langfuse, Weave) via dynamic OTLP
credential headers, while the request-root, auth, and db spans stay on the
operator's default backend. The span was still parented into the request
trace, so the tenant account received a child whose parent it never got,
and New Relic rendered it as a fragmented trace with a missing parent.
Detach a credential-routed span the same way a project-routed (Phoenix)
span already detaches: root a fresh trace in the tenant account and link
back to the request trace for correlation. Service-name routing keeps
parenting, since it only relabels service.name on the same operator
backend where the parent is present.
Guard the detach on the callback actually owning an OTLP exporter the
credentials can reach: a callback owning only a console or in_memory
exporter has nowhere to stamp them, so the span would export to the
default backend unchanged and detaching would orphan it on the very
backend that holds its parent. In that case warn once and keep the
default tracer.
* fix(otel/v2): derive tenant-route routability from resolved exporter transport
A denylist classified an owned exporter as routable whenever its kind was
not console/in_memory, so a typo'd or unavailable kind (e.g. "otlp",
"grcp") passed the check while _exporter_from_spec falls it back to a
header-ignoring console exporter. Detaching such a span would root a fresh
trace that only ever reaches the operator console, never the tenant
backend, orphaning it on both sides.
Route on a shared exporter_transport() predicate that resolves the kind the
same way _exporter_from_spec builds it (registered factories + otlp_http
aliases -> http, otlp_grpc aliases -> grpc, else headerless), so an
unresolvable kind is headerless and stays parented. Fixes the same latent
gap in project routability.
The componentized chart's Ingress renders a fixed path set: the UI paths, a
hardcoded gateway prefix list mirroring gateway/routes/allowlist.py, and a `/`
Prefix catch-all to the backend. values.yaml exposes no path knob, so an
operator who needs a route the chart does not know about has to fork the
template.
That prefix list is a snapshot of the data plane at release time and drifts
from it. `/watsonx` is in GATEWAY_PATH_PREFIXES with no ingress rule and no
backend prefix, so `/watsonx/*` falls into the catch-all, reaches the backend,
and 404s. A provider passthrough prefix added upstream after the chart version
an operator runs has the same shape, and a custom
general_settings.pass_through_endpoints route has a path only the operator
knows, so the chart can never ship a rule for it at all.
ingress.extraPaths takes a list of {path, service, pathType} entries rendered
in addition to the built-in paths, never in place of them. They render after
every built-in path and before the backend catch-all, and an entry repeating a
path the chart already routes is rejected: duplicates within one rule are
resolved by position or by controller-specific tie breaking, so an entry at
`/` Prefix would swallow the whole backend management API rather than add to
it. `service` picks the component Service — gateway by default, or backend or
ui — so an operator does not have to reconstruct release-templated Service
names, and `pathType` defaults to Prefix. A non-mapping entry, an entry with
no path, an unknown service, an unknown pathType, and a duplicate each abort
the render naming the offending index rather than emitting an Ingress that
misroutes traffic.
The duplicate check needs the built-in paths as data, so the UI paths move
from six literal YAML blocks into a $uiPaths list rendered by one range, and
their explanatory comments move from the rendered manifest into template
comments. The Ingress object this produces is unchanged: the same 92 paths in
the same order with the same backends, verified by parsing both renders.
The target component still answers only the paths its own route allowlist
keeps, so this makes a route routable, not servable.
The Add Model page's connection test dialog rendered a literal ", ]" beside the Close button, left over from converting an array of footer buttons into JSX children.
The driver retries rate-limit-shaped failures twice with a 65s backoff, but
RATE_LIMIT_SHAPED_RE only matches 429 / rate limit / too many requests /
throttled / CLI timeout. A saturated provider does not always say any of
those: litellm-e2e-pr build 182 turned a green cell red on
status 503: litellm.ServiceUnavailableError: BedrockException -
{"message":"Bedrock is unable to process your request."}
which matched nothing and so was never retried.
Add TRANSIENT_UPSTREAM_SHAPED_RE for 503, 529, "service unavailable",
"overloaded" and Bedrock's "unable to process your request", and have the
retry loop ask is_retryable_shaped (either shape) instead of
is_rate_limit_shaped.
Kept as a second pattern rather than widened into the first on purpose. The
conftest feeds RATE_LIMIT_SHAPED_RE into the rate-limit summary, which the
binary-search helper reads to decide whether to lower a provider's request
rate. A 503 says the provider is out of capacity, not that we are asking too
often, so folding it in there would keep ratcheting our rate down against a
condition our rate never caused.
Retry-loop logging now names which shape fired, so a saturated upstream is
distinguishable from a throttled one in the run output.
Verified against the literal failure text from build 182: is_retryable_shaped
is True while is_rate_limit_shaped stays False.
Note this covers CLI-driven rows only. HTTP-probe rows (tool_search,
count_tokens) have no retry layer at all, so the specific cell that failed in
build 182 is still unprotected -- that is a separate change to http_probe.
#34940 widened the mask-in-place safety guard so a Responses-API
`instructions` field (and a combined messages+input body) skips the
PII masking branch. With `on_flagged: "monitor"` that fell straight
through to "allow", so PII that used to be masked now reaches the
model unredacted.
Monitor means "don't block", not "don't redact". Recover the one shape
whose payload is still fully writable: mask it and write the redacted
instructions back into `data["instructions"]` directly, since
apply_redacted_messages_back has no path for that field and would
otherwise fold the instructions text into `data["input"]`.
The combined messages+input and multimodal shapes stay unmasked - both
are unsafe to write back, not merely unwritable - and now log an error
naming the reason instead of passing silently.
No block/allow decision changes: block and inject_system_message keep
the exact outcomes #34940 shipped.
