* fix(otel): emit cache token counts on OTel v2 LLM spans
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): trim comment in LLMUsage adapter
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): drop casts in LLMUsage cache token adapter
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(deps): bump restrictedpython to 8.3 for GHSA-ffg3-p8fm-mjx2
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
`ChatCompletionFileObject` is in the union `_count_content_list` accepts, but
`file` was missing from its match, so every local count of a Responses
`input_file` raised `Invalid content item type: file`. On
/v1/responses/input_tokens that surfaced as an opaque 500 whenever the model's
provider counting API refused the block and the local tokenizer took over.
Count it the way the module already counts the same thing in Anthropic's
dialect: the filename like a document title, the inline bytes through the
image pricer.
The Responses-to-chat transform dropped the filename OpenAI requires next to
file_data, so a request carrying an inline PDF counted 13 tokens instead of 36
and a real completion through the chat bridge got a 400.
Assistant list content was forwarded to /v1/responses/input_tokens as chat
`text` blocks, which the Responses API rejects (it accepts only output_text
and refusal inside an assistant turn). The 400 sent the whole request to the
local tokenizer, so any conversation with an assistant turn silently lost
provider-exact counting, including the image counting added in 73ab647b1c.
Assistant content now collapses to the plain string the Responses API counts
identically, and image parts are kept to user turns where they are legal.
* fix(vertex_ai): graft default vertex path when api_base has a version-only path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(vertex_ai): keep query and fragment placement when grafting vertex path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(vertex_ai): merge alt=sse into existing query when streaming
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(spend_tracking): persist router metadata in spend logs for internal router models
* test(spend_tracking): expect router_metadata key in exact-payload tests, type the routed-kwargs helper
The chat-to-Responses reverse transform kept only text blocks, so an image
input was dropped before the count went to OpenAI. A 256x256 image request
counted 13 tokens instead of 268.
/v1/responses/input_tokens returned 200 with a count for an empty
"input" ("" or []), while OpenAI returns a 400 missing_required_parameter.
The route also went through optimistic budget reservation, which is only
released by LLM success/failure callbacks that a token count never
reaches, so every call leaked a reservation until TTL expiry and could
429 real traffic. Both routes plus the /openai alias now join
/utils/token_counter in the reservation exemption set.
* fix(proxy): deliver budget alerts on webhook-only alerting and accept ALERTING_WEBHOOK_URL
ProxyLogging.budget_alerts forwarded to the alerting pipeline only when
'slack' was in general_settings.alerting, so alerting: ['webhook'] plus
WEBHOOK_URL silently never delivered a budget alert (the config
/health/services?service=webhook exists to test). Forward when 'webhook'
is present too; SlackAlerting.send_alert already fans out per channel.
Also accept a provider-neutral ALERTING_WEBHOOK_URL env fallback for the
Slack-format channel (any Slack-compatible receiver works), mark it as a
sensitive var, and de-brand the admin UI alerting copy.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(ui): format settings.tsx with prettier
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(ui): regenerate schema.d.ts for updated alerting description
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* ci: retrigger checks after ALERTING_WEBHOOK_URL docs merged
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Default remains true so lite claude turns tool search back on through
a proxy. An explicit false or auto in the env or settings is left alone
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
Claude Code turns tool search off when ANTHROPIC_BASE_URL is a proxy.
lite claude, lite up, login --config-claude, and autoroute now force
ENABLE_TOOL_SEARCH=true so MCP tools stay deferred through the proxy
Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
* fix(otel/v2): detach credential-routed tenant spans into their own trace
Multi-tenant OTel v2 routes a team or key's LLM-call span to that tenant's
own vendor account (New Relic, Arize, Langfuse, Weave) via dynamic OTLP
credential headers, while the request-root, auth, and db spans stay on the
operator's default backend. The span was still parented into the request
trace, so the tenant account received a child whose parent it never got,
and New Relic rendered it as a fragmented trace with a missing parent.
Detach a credential-routed span the same way a project-routed (Phoenix)
span already detaches: root a fresh trace in the tenant account and link
back to the request trace for correlation. Service-name routing keeps
parenting, since it only relabels service.name on the same operator
backend where the parent is present.
Guard the detach on the callback actually owning an OTLP exporter the
credentials can reach: a callback owning only a console or in_memory
exporter has nowhere to stamp them, so the span would export to the
default backend unchanged and detaching would orphan it on the very
backend that holds its parent. In that case warn once and keep the
default tracer.
* fix(otel/v2): derive tenant-route routability from resolved exporter transport
A denylist classified an owned exporter as routable whenever its kind was
not console/in_memory, so a typo'd or unavailable kind (e.g. "otlp",
"grcp") passed the check while _exporter_from_spec falls it back to a
header-ignoring console exporter. Detaching such a span would root a fresh
trace that only ever reaches the operator console, never the tenant
backend, orphaning it on both sides.
Route on a shared exporter_transport() predicate that resolves the kind the
same way _exporter_from_spec builds it (registered factories + otlp_http
aliases -> http, otlp_grpc aliases -> grpc, else headerless), so an
unresolvable kind is headerless and stays parented. Fixes the same latent
gap in project routability.
#34940 widened the mask-in-place safety guard so a Responses-API
`instructions` field (and a combined messages+input body) skips the
PII masking branch. With `on_flagged: "monitor"` that fell straight
through to "allow", so PII that used to be masked now reaches the
model unredacted.
Monitor means "don't block", not "don't redact". Recover the one shape
whose payload is still fully writable: mask it and write the redacted
instructions back into `data["instructions"]` directly, since
apply_redacted_messages_back has no path for that field and would
otherwise fold the instructions text into `data["input"]`.
The combined messages+input and multimodal shapes stay unmasked - both
are unsafe to write back, not merely unwritable - and now log an error
naming the reason instead of passing silently.
No block/allow decision changes: block and inject_system_message keep
the exact outcomes #34940 shipped.
Direct access on /model/info was read from the user record alone, so an
unrestricted user calling with a key limited to a few models saw every non-team
deployment, including ones the key gets a 403 on. Resolve the key's grant the
same way and intersect the two.
Resolving a grant now also expands access groups, which the key path needs and
the user path was missing.
Claude-Session: https://claude.ai/code/session_01XL7LBFEew4wi8gphVCDq6n
Model access group budgets shipped API-only, so the only way to give a group a
budget was a curl. Adds an Access Group Budgets tab under Models & Endpoints
that lists every group with the spend drawn against its shared pool, and a
modal to set, edit or clear the budget.
/access_group/list now carries each group's budget and spend inline, so the
table renders from one read instead of one follow-up request per row.
