Gemini returns a thoughtSignature on the first function call of a parallel
batch and leaves the siblings bare. When replaying that assistant turn,
litellm gave every unsigned call the skip_thought_signature_validator
placeholder, so a three-call turn went back with three signatures where
Gemini had produced one. Keep the placeholder for the first call only and
forward the siblings with whatever signature they actually carry, which is
usually none.
An unknown reasoning split now falls back to reasoning_tokens=0 in the
chat-to-responses usage translation, since the OpenAI SDK requires
output_tokens_details with an int reasoning_tokens, and the streaming
chunk builder caps the tokenized reasoning estimate at completion_tokens
and fills text_tokens with the remainder
Greptile flagged that lazy per-slug pool initialization could double-build
under concurrent replay calls, splitting consumption across a discarded
pool. Pools are now built once at ReplaySource construction and per-key
consumption is a single atomic deque pop, with a barrier-synchronized
regression test that fails 10/10 under the lazy-init mutant
A trusted hostname over plain http would expose the bearer token or a
replayable SigV4 signature to network observers. Credentials now only ride
https, with localhost exempt so local MCP stubs keep working.
Web-search connector 1.1.0 and later return the machine-readable results in result.structuredContent and may leave the text block as prose, which the parser dropped.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Replay previously matched interactions by transport verb and path in
recorded order, so a request whose body drifted from the recording
silently replayed the stale response, and reordering two independent
calls broke replay even though both were recorded. Match keys are now
canonical: fixture_canonical.py strips volatile headers and credential
fields, replaces unique markers, generated ids, uuids, and timestamps
with fixed placeholders, sorts object keys, and hashes what remains, so
a key is stable across runs and machines while any real content drift
is a hard ReplayMiss naming the computed key, the closest recorded key
with its file, and a content diff, with no fallthrough to a live call.
Matching is order-independent across distinct keys and FIFO within one
key. Recording now also redacts credential body and form fields (not
just auth headers) so provider keys never land in bundles.
Resolves LIT-5741
An OpenAPI-backed MCP tool whose upstream answered 401 came back as a
successful tool result carrying the upstream's rejection as its content, so a
caller saw {"error":"invalid_token"} presented as data and the gateway recorded
the request in its own spend log as call_mcp_tool | success.
Three layers each erased the outcome. The request function returned
response.text whatever the status, _handle_local_mcp_tool caught every exception
and returned it as ordinary TextContent, and both dispatch sites then stamped
isError=False unconditionally. Fixing only the first, which is the obvious fix,
changes nothing, because the two above it still map failure onto the
success-shaped value.
The status is now classified where the response is held: a 401 becomes
MCPUpstreamAuthError so the caller is told to re-authenticate, and every other
non-2xx becomes MCPOpenApiUpstreamError, which carries the status and drops the
upstream body rather than serving it as tool content. _handle_local_mcp_tool no
longer swallows, and the call_tool arm keeps the auth error's type. Nothing new
renders these: call_mcp_tool and call_tool_rest_api already turn them into an
isError result naming the status and into a real 401 with WWW-Authenticate, and
the OpenAPI path simply never reached them.
The result is now byte-identical to the regular MCP path for the same failure.
Two or more reasoning keyword matches promoted a request straight to the
REASONING tier no matter what the weighted score said, so "hi, step by step,
pros and cons" scored 0.100 and still bought the most expensive tier.
Require the score to clear the simple_medium boundary before the override
applies. Promotion from MEDIUM or COMPLEX is unchanged; only prompts the
scorer already placed in the cheapest band stay there.
* chore(ui): upgrade the dashboard to React 19
Bumps react and react-dom from 18.3.1 to 19.2.8 with matching @types. Next 16 already required a React 19 peer, so this aligns the dashboard with what the framework expects and unblocks Base UI and shadcn work that assumes the React 19 ref model.
React 19 passes ref through as a regular prop, so the setup file's forwardRef tripwire and the ref-forwarding test's forwardRef case no longer describe real behavior; both now assert the React 19 contract instead. useRef<T>(null) now yields RefObject<T | null>, which is the one prop type MessageList had to widen.
* test(ui): wait for a Base UI select popup to open before clicking an option
The option lands in the DOM one render before the popup finishes entering, while its positioner still carries pointer-events: none, so clicking it throws. Waiting on the option's text alone was a race that React 19's flush timing loses, which is why four ToolPolicies cases went red on the bump.
chooseSelectOption in test-utils opens the trigger, finds the option by role, waits for it to stop being pointer-blocked, then clicks. It also replaces the last-match-by-text hack, which only worked because the popup happens to portal after the table.
Moves all 33 antd Alert usages across 20 dashboard files onto
src/components/shared/Alert, following the composition the rest of the
dashboard already uses: message becomes AlertTitle, description becomes
AlertDescription, showIcon becomes a lucide icon child, and closable
becomes an AlertAction ghost button.
antd type="success" has no counterpart on the shared Alert, so the two
success sites land on the default variant with a CircleCheck icon, which
is what cloudzero_export_modal and CloudZeroIntegrationSettings already
do for the same case.
LoginPage's dismissible SSO notice moves into its own SsoEnabledNotice
component in the same file: antd's closable carried its own dismiss
state, and inlining it pushed LoginPageContent past the complexity
budget.
Four files lose their last antd symbol, so their no-restricted-imports
suppressions are pruned by hand. antd import sites drop from 115 to 111
across 107 to 103 files, and the no-restricted-imports ratchet drops
from 119 to 115 over 110 to 106 files.
One test asserted antd's own ant-alert-info class; it is repointed to
the shared Alert's text-info variant class, which keeps the same
"info, not warning" check. Every other colocated test passes untouched.
* feat(auto-router): scope shadow eval jobs to multiple keys
A shadow eval job now covers a set of keys instead of exactly one, and each
key carries its own max_turns budget, so one key exhausting its budget leaves
its siblings sampling. The existing job row already is the per-key unit
(api_key_id, max_turns, stopped_at, and the one-active-per-key-and-direction
partial unique index all live on it), so multi-key is grouping rather than
schema surgery: a new group_id column ties N sibling rows written atomically
by one create_many, the API's job id becomes the group id, and pre-existing
jobs backfill group_id = id so their ids keep resolving. The sampler hot path
is untouched; its test file has a zero-line diff
Results come back pooled plus a per-key breakdown and responses list every key
with its own budget, stop state and read-time labels. The dashboard is adapted
minimally to the new shapes (the picker stays single-key and submits a one-key
list); the multi-select picker and per-key table land in the stacked UI PR
* fix(shadow_eval): derive completed from spent budgets and record operator stops
* fix(shadow_eval): stamp stops atomically and freeze counts at the stamp
The stop endpoint wrote stopped_by and stopped_at as two separate updates, so
a failure between them left a job reading stopped while its unstamped legs
kept sampling, and the retry got 400 already stopped. One UPDATE now stamps
stopped_by and every missing stopped_at together, preserving the stopped_at a
leg earned from its own budget via COALESCE
Attempt counts now exclude attempts that land after a leg's stopped_at, so an
in-flight attempt finishing just after an operator stop can never push a
legacy pre-stopped_by job over its budget and flip it from stopped to
completed at read time
* fix(shadow_eval): backfill stopped_by so legacy stops never read as completions
* chore(ui): regenerate api types for the shadow eval stop fields
* fix(shadow_eval): let the stop statement pick one winner under racing stops
Two operators can both pass the derived-status guard in the race window. The
stop UPDATE now claims only legs with stopped_by still null and the endpoint
judges by its row count, so exactly one caller ever gets the 200 and the loser
gets the same already-stopped 400 a late caller gets
* refactor(shadow_eval): make the stop statement the whole state machine
The status guard ran before the UPDATE, so a stop racing the last budgeted
attempt still claimed the job and it read stopped forever instead of
completed. The statement now claims the job only while a leg still samples
inside the window with no stop recorded, and the endpoint reads once after
writing: a racing operator, a same-instant budget spend, and a repeat stop all
get the 400 naming the status the job actually holds. The pre-write guard and
the hand-built response go away
* chore(ui): regenerate api types for the stop route description
The flat-cost rollup reads deployments only from LiteLLM_ProxyModelTable, so a PTU
deployment declared in config.yaml never accrues flat cost. Those deployments live in
llm_router.model_list as plain dicts whose id sits in model_info rather than on the entry,
so they do not satisfy the shape _parse_ptu_model reads.
Adds a frozen record in that shape and a factory that maps a router entry onto it, leaving
_parse_ptu_model byte-identical so the existing cases stand as evidence of no behaviour
change. Nothing calls the factory yet; the caller lands with the loader union.
_decode_model_info also stops handing back valid JSON that is not an object. It decoded
a list or a scalar and returned it as a mapping, so the caller read fields off it and
raised, losing the whole run rather than the one bad deployment.
* test(ui): split the vitest suite into unit, component and integration tiers
Every test file booted jsdom, including the ~1800 that assert pure functions
and never render. They now run as a separate vitest project in the node
environment, where the whole tier finishes in under four seconds.
The tiers are vitest projects rather than a naming convention, so CI can run
them as independent jobs. A .test.ts that renders React, a hook test being the
usual case, is listed explicitly and stays in the jsdom tier.
* test(ui): report per-test duration against a per-tier budget
A timeout only catches a hung test, and it has to stay generous enough to
survive a loaded runner, so it never reports the multi-second render tests that
make CI fail the moment the box is busy. Budgets are separate and far tighter:
50ms unit, 1s component, 3s integration.
The counts are laptop measurements, so the CI job is report-only for now.
Flipping it to blocking is one line once CI has published its own numbers.
* test(ui): run the tiers as separate CI jobs and stop clicking popups by text
The old job ran every file in one process, so the single slowest file set the
wall clock and a bigger box bought nothing. The tiers now run as separate jobs
with the component tier sharded four ways.
getByText and findByText match hidden nodes, so they resolve against a closed
Base UI popup whose positioner still carries pointer-events: none, and the
click lands or not depending on how far the open transition got. Two files
failed this way, one three runs in five and one every run. Querying the option
by role waits for it to be visible, and both are now stable. A lint rule keeps
the pattern from coming back.
* test(ui): give React Testing Library's async queries a CI-sized window
findBy* and waitFor run on asyncUtilTimeout, which defaults to 1000ms and is
independent of vitest's testTimeout. Raising the vitest timeout therefore did
nothing for them: a query still gave up after one second while the test had 59
seconds of budget left, which is why a loaded runner produced 'Unable to find
role=...' rather than a timeout.
UserSearchModal is the worked example. The role query it makes resolves in
249ms on a laptop and blew past 1000ms on CI, failing the run at 1494ms. Five
seconds keeps the same assertions and only widens the window a failing query
waits before reporting; a passing query still resolves the moment the element
appears.
* test(ui): calibrate the tier budgets from real CI numbers and report by default
The first CI run showed the laptop counts were badly off: component 176 local
against 326 on CI, integration 87 against 128. The maxima now come from that
run with headroom.
continue-on-error still painted the check red, which is the opposite of the
point, so the report-only decision moves into test-budgets.json as an explicit
enforce flag. The job passes and prints the counts; flipping enforce to true
makes it a gate.
* docs(ui): drop the CLAUDE.md edits from the tier split
Keeping this PR to the vitest, CI and test changes.
* ci(ui): run every tier in one job instead of eight check rows
Sharding bought nothing. Measured on the first run of this branch, the
component tier unsharded finishes in 198s while the integration tier is floored
at 384s by a single file, so integration was always the critical path and the
four component shards only added rows. One job running every project comes in
around 384s against the 426s the split jobs took.
Eight rows named things like 'component (2)' also told a reviewer nothing, on a
PR page that already carries forty checks.
The job keeps the id ui-unit-tests because guard-internal-staging requires that
exact context; renaming the jobs had silently stopped it reporting, which would
have blocked every merge on a check that no longer existed. The workflow's
display name becomes UI Tests since it runs more than unit tests.
The tier split itself is untouched: it lives in the vitest projects config, so
the unit tier still runs in node with no jsdom, and each tier keeps its own
timeout and budget.
* fix(ui): stop the type check from running the whole suite a second time
test:types was 'vitest --run --typecheck.only'. Under test.projects that flag
is ignored and the root-level typecheck block is not inherited, so the step
collected each project's normal include and ran all 8464 runtime tests instead
of type-checking. It took 542s on CI against 33s on the flat config it
replaced, and the job then ran the same suite again in the next step.
Typecheck now belongs to a project of its own, with an empty include so it
contributes no runtime tests, and the CI job runs one vitest invocation for all
four. The type tier adds about 3s to a full run and reports 'Type Errors: no
errors' rather than a suite of tests.
Verified it still catches things: breaking SortingState in DataTable.test-d.tsx
fails with 'Type number is not assignable to type string' and exit 1, and
restoring it passes.
* test(ui): scope the split down to the vitest tier projects
Removes everything from this branch that was not the tier split.
The three lint rules brought 1381 lines of grandfathered suppressions in
eslint-suppressions.json, which is 81% of the branch's added lines and
debt nobody is going to pay down. The per-test duration budget does not
scale as a CI step. Both are gone, along with the two query rewrites the
no-click-by-text rule forced: those files pass 10/10 at this base, quiet
and under load, so there was no failure behind them.
The workflow is byte-identical to the base again. It already runs
npm run test:types and then vitest related on pull requests, so PR cost
is unchanged; the split only repoints test:types at the new project.
That project is required, not optional: vitest silently ignores
--typecheck.only under test.projects, so without it the type script
collects the whole suite instead of the one typed file.
Restores the base 60s testTimeout on the unit tier. The 5s cap was not
part of the split and failed ChatShell.serverRootPath.test.ts, a 960ms
test, under load.
Moves all 58 antd Button JSX sites across 24 files onto the shadcn
Button, leaving zero antd Button importers.
Prop mapping follows what already merged rather than a new convention:
type="primary" to the default variant, a bare button to outline (the
house default), type="text" to ghost, type="link" to link,
type="dashed" to outline plus border-dashed, danger to destructive,
size="small" to sm, htmlType to type, block to w-full, the icon prop to
a child, and loading to disabled plus aria-busy. Icon-only buttons take
the matching icon-* size. Inline style props that had a direct utility
equivalent moved to className, and the opacity toggle on the create key
submit is dropped since the base cva already carries disabled:opacity-50.
Base UI's Button defaults type to "button" for native buttons, the same
default antd used, so bare buttons inside a form do not start
submitting.
guardrail_info.tsx and tag_info.tsx lose their last antd import, so
their no-restricted-imports suppressions are pruned. The other 22 files
keep other antd symbols and keep their entries.
One behavior change: the MCP transports docs link now opens in a new
tab, matching every other external docs link in the dashboard, instead
of navigating the dashboard away.
The ModelSettingsModal loading assertion moves off antd's spinner
element onto aria-busy, which is what the rest of the suite already
asserts; that markup cannot survive removing antd.
