xAI states the amount it charged in usage.cost_in_usd_ticks, at 10^10 ticks to
the dollar, and that figure covers tokens and every server-side tool invocation
together. The xAI chat and responses transformations restate it in USD on
usage.cost, the field litellm already carries a provider-stated cost in, and the
xAI cost calculator bills from it the way the perplexity calculator does
Routing it through usage.cost rather than a private field means the streaming
chunk assembler carries it too, and no provider-neutral file has to learn about
an xAI wire field
Only a finite, non-negative amount is trusted, so an endpoint a caller can
point litellm at cannot report a negative amount to subtract from its own
recorded spend, and cannot report a NaN, which Usage stores unvalidated and
which compares false against every budget threshold, disabling enforcement for
the key rather than mispricing one request. Absent a usable figure nothing
changes: the existing token math and the
$5 per 1,000 web search calls fallback both run as before
The web search surcharge is suppressed once the reported total applies, since
that total already covers the search calls
Adds Playwright coverage for the RC checklist items an audit marked
automatable today: Playground to Logs hand-off, public Agent/MCP hub
tabs, team models in the Playground dropdown via a team key, Add Model
with a stored credential, internal user team key creation, a second
admin account, team model deletion, and Presidio guardrail CRUD without
a live sidecar. Seeds e2e-team-keygen with the /key/generate member
permission so the internal user key flow avoids the team-list cache lag
The base added router_metadata to SpendLogsMetadata in #39001 without
updating this fixture, and its CI run never executed logging_testing,
so the job now fails on every branch merged with current staging.
The consolidated popup test only asserted the options never cover the
trigger, so opening above the trigger with room below it, the regression
PR #38554 fixed, would have passed. Split it back into a below-trigger
case and a cramped-viewport case. The header test accepted a single pixel
of vertical intersection; require the refresh control's centre to sit
within the tab row instead.
The migration smoke waited on `getByRole("button", { expanded: false })`
after clicking it. Playwright re-resolves that locator on every retry, so
once the clicked group flipped to expanded it matched the next collapsed
group instead, and the assertion could never pass. Count the remaining
collapsed groups and wait for that count to drop by one.
precached_prompt_tokens is a subset of prompt_tokens (OpenAI cached_tokens
semantics), so map it to prompt_tokens_details.cached_tokens instead of
adding it on top of prompt/total. Emit stream usage from any final chunk
carrying it rather than only finish_reason stop, which dropped tokens for
function_call and length streams. Merge auth metadata into a new dict in
the gigachat router handler instead of mutating the shared parsed-body
cache in place.
The UI e2e suite had a class of assertions that pin how the dashboard is
built rather than what it does, so an ordinary refactor turns them red
without any user-visible change.
Geometry. The auto-router template select had two tests made of pixel
arithmetic plus a data-side="bottom" check, which is Base UI's own
positioner signal. The regression they guard (#38554) is a popup opening
on top of the control that spawned it, so both cases collapse to one
invariant: the options never cover the trigger. It now runs at both
viewport heights and reads the popup as role=listbox. The models header
test compared the tabs and refresh centers within 2px, which a padding
change flips; it now asserts the two share a row.
Structure. The logs drawer test walked xpath=../../.. from a text node
and read collapsed state off chevron icon classes. SectionHeader now
renders a real disclosure button with aria-expanded, and its two copy
buttons carry distinct names instead of both being "Copy". Sidebar group
toggles expose aria-expanded too, so the migration spec can ask for a
collapsed group by state rather than by nesting depth.
Positional lookups. keyRow.locator("button").first(), row.locator("td")
.first() and getByTestId(grid).locator("div").first() all named a
position where they meant an action; they now name the control. Table
scoping moves from "table tbody" to role=row.
Timing. Nine waitForTimeout calls are gone. Every assertion that followed
them already retried to its own timeout, so the sleeps only slowed the
run down.
Both files under tests/users/ were wrapped in test.skip("...", () => {}),
which registers one skipped test and never runs the body, so the four
tests inside had never executed and were written against a UI that has
since changed (the search placeholder is "Search by email…", the ID
filters moved into a drawer, pagination is labelled "Go to previous
page"). Rewritten against the current surface: the suite goes from 104
collected tests to 107.
Left in place deliberately: the chip and dialog-footer data-slot
selectors, because the accessible names they work around live in
components/ui/, which is shadcn CLI-managed and not hand-edited.
* fix(otel): emit cache token counts on OTel v2 LLM spans
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): trim comment in LLMUsage adapter
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(otel): drop casts in LLMUsage cache token adapter
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(deps): bump restrictedpython to 8.3 for GHSA-ffg3-p8fm-mjx2
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
The advisory was modified 2026-08-31 and flags mlflow 3.13.0 through
3.15.2 with no fixed release published, so every osv-scan run fails
with nothing to bump. Same treatment as the existing diskcache entry.
`ChatCompletionFileObject` is in the union `_count_content_list` accepts, but
`file` was missing from its match, so every local count of a Responses
`input_file` raised `Invalid content item type: file`. On
/v1/responses/input_tokens that surfaced as an opaque 500 whenever the model's
provider counting API refused the block and the local tokenizer took over.
Count it the way the module already counts the same thing in Anthropic's
dialect: the filename like a document title, the inline bytes through the
image pricer.
Since the requires-python cap moved to <3.15, uv resolved the project
python to 3.14, downloaded a managed interpreter under
/root/.local/share/uv that the runtime stage never receives, and every
layer-cache-miss image build broke: first at uvloop's cp314 sdist
configure step, then, with file/make added, at the runtime stage where
the copied venv's python symlink dangles and prisma imports fall through
to the system python. UV_PYTHON_DOWNLOADS=0 (already the convention in
migrations/backend/gateway) roots the venv on the apk python3.
The wolfi-base digest bump is required alongside it: the pinned 08-22
base ships glibc-2.43 while the current apk python-3.13 needs
GLIBC_2.44, and wolfi version-names glibc packages so apk upgrade
cannot cross that boundary.
With the venv on system 3.13 every dependency installs from wheels
again, so the file and make packages added for the sdist build are
reverted.
The Responses-to-chat transform dropped the filename OpenAI requires next to
file_data, so a request carrying an inline PDF counted 13 tokens instead of 36
and a real completion through the chat bridge got a 400.
- sync llm_passthrough_route: read and close an error-status streaming
response before mapping it, so upstream 4xx/5xx surface as the provider
error instead of httpx.ResponseNotRead
- AsyncPassthroughStreamingResponse: expose aiter_bytes() and carry
_hidden_params so the router attaches headers in place instead of
wrapping the stream in HiddenParamsAsyncIteratorWrapper, which 500'd
every streaming azure router-model passthrough request
- logging: swap the passthrough httpx result for the transformed
ModelResponse/EmbeddingResponse when firing success callbacks
- get_llm_provider: resolve gigachat from its api base and drop the dead
gigachat_models elif branch
- constants: register the gigachat api base in openai_compatible_endpoints
Adapts streaming_model_restamp.py to the LIT001/LIT010 gates that landed
on staging since this branch was cut (Final annotations, Mapping in
annotations instead of dict).
Assistant list content was forwarded to /v1/responses/input_tokens as chat
`text` blocks, which the Responses API rejects (it accepts only output_text
and refusal inside an assistant turn). The 400 sent the whole request to the
local tokenizer, so any conversation with an assistant turn silently lost
provider-exact counting, including the image counting added in 73ab647b1c.
Assistant content now collapses to the plain string the Responses API counts
identically, and image parts are kept to user turns where they are legal.
* fix(vertex_ai): graft default vertex path when api_base has a version-only path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(vertex_ai): keep query and fragment placement when grafting vertex path
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix(vertex_ai): merge alt=sse into existing query when streaming
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* feat(spend_tracking): persist router metadata in spend logs for internal router models
* test(spend_tracking): expect router_metadata key in exact-payload tests, type the routed-kwargs helper
When the upstream errors while the client is still connected, the pump
forwards the exception through the relay queue so the proxy's failure
handling re-raises it. If the client disconnects before consuming that
queued exception, neither the failure hook nor billing ran and the spend
row was lost. The pump now waits for client detach and, if the exception
was never consumed, salvages partial spend like the post-disconnect
error path.
Also rewrites the bedrock disconnect logging test to the detached-pump
contract: billing fires after the upstream drain completes, not
synchronously at aclose().
The chat-to-Responses reverse transform kept only text blocks, so an image
input was dropped before the count went to OpenAI. A 256x256 image request
counted 13 tokens instead of 268.
/v1/responses/input_tokens returned 200 with a count for an empty
"input" ("" or []), while OpenAI returns a 400 missing_required_parameter.
The route also went through optimistic budget reservation, which is only
released by LLM success/failure callbacks that a token count never
reaches, so every call leaked a reservation until TTL expiry and could
429 real traffic. Both routes plus the /openai alias now join
/utils/token_counter in the reservation exemption set.
