Address review feedback on #36762:
- Only use the parsed 5m/1h split when it fully accounts for
cacheWriteInputTokens; an unrecognized ttl or missing entry now falls
back to the aggregate (previous behavior) instead of silently
understating cost.
- Mark TypedDict fields ReadOnly (AWS response data, never constructed
by us) to satisfy the repo's type-discipline lint gate.
- Trim comments and add Final to locals per repo style.
Co-Authored-By: pi (Claude/GPT via @earendil-works/pi-coding-agent) <noreply@earendil.works>
Types 28 files with Protocols, TypedDicts, and Pydantic validation in place
of Any, cutting basedpyright reportAny by 974 and reportExplicitAny by 261
(1411 errors total across 48 rules), and ratchets the basedpyright, ruff
strict, and type discipline budgets down to match
AmazonConverseConfig._transform_usage only read the aggregate
cacheWriteInputTokens field, so cache_creation_token_details was always
unset for Bedrock Converse responses. calculate_cache_writing_cost bills
the whole cache-write count at the 5m rate whenever that field is None,
so 1-hour TTL cache writes on the standard Bedrock chat path were always
undercounted, even though Bedrock returns the 5m/1h split in
usage.cacheDetails.
Parse cacheDetails (when present) into CacheCreationTokenDetails so the
correct rate applies to each portion. No cacheDetails in the response
(older models/regions) keeps the previous behavior.
Fixes#36760
Co-Authored-By: pi (Claude/GPT via @earendil-works/pi-coding-agent) <noreply@earendil.works>
langfuse_* request headers land in metadata as strings, but the trace path reads
mask_input/mask_output with a bare truthiness check and iterates update_trace_keys
directly. A header saying mask_input: false redacted the payload it was asked to
keep, and update_trace_keys was walked one character at a time so every requested
key silently failed to match
The template rendered affinity and tolerations but never nodeSelector, so a
values file that pinned the chart to a node pool got the gateway and every
subchart placed correctly while the migration Job silently fell through to
whatever the cluster's default pool was.
That is worse than an outright failure. On EKS Auto Mode the default pool hands
out 3 GiB nodes and the migration container needs roughly 3.6 GB, so the Job
was OOM-killed on a pool it was never meant to run on, while the values file
that would have placed it on a large enough node looked correct.
The new test fails against the old template with "unknown path
spec.template.spec.nodeSelector".
* fix(guardrails): scan and re-emit raw Anthropic SSE streams in the bedrock post-call hook
* fix(guardrails): keep upstream id and model on a blocked Anthropic stream
* fix(guardrails): deliver a blocked Anthropic stream as an error frame
* fix(guardrails): deliver an unscannable Anthropic stream as an error frame
* fix(guardrails): emit the guardrail block detail as JSON in the stream error frame
* fix(guardrails): deliver an Anthropic block through the shared block-SSE builder
* fix(guardrails): keep the shared SSE assembler behavior-identical for existing callers
* fix(guardrails): keep the stream error message a string and drop an unreachable branch
* chore(guardrails): drop a comment that repeated its own docstring
* fix(guardrails): let bedrock service failures keep their status instead of framing them as blocks
* fix(guardrails): key the streamed block decision on status, not detail shape
InvokeGuardrailChecks details a Mapping on its 500 for an unparseable response,
so a detail-shape test read that outage as a policy block and framed it as a 200
guardrail_error. Both block sites raise 400, so gate on the status too.
* refactor(guardrails): narrow the SSE error-frame helper to the input it actually takes
Both callers pass a string, so the Mapping overload and its json.dumps branch
were unreachable. Folds the block branch's narrative comment into the rebind
suppressions that already carry a reason.
* test(ui): pin behaviour of guardrails-monitor, projects and logs components before migration
Adds role- and text-based characterisation tests for EvaluationSettingsModal,
GuardrailDetail and AuditLogDrawer, which had none, and moves the remaining
antd-specific assertions (.ant-spin, the icon role of an antd Spin indicator)
onto library-neutral ARIA queries. Also covers the enterprise banner on the
deleted keys and deleted teams pages, which no test reached.
All of these pass against the current antd and Tremor components.
* refactor(ui): migrate guardrails-monitor, projects and logs to shadcn
Replaces antd and Tremor with installed shadcn primitives across the files
these three routes exclusively own. Markup only, except where noted below.
Deletes AntDLoadingSpinner, an antd-only primitive living in the shadcn ui/
folder, and moves its single call site onto ui/ui-loading-spinner.
Two behaviour notes. The logs tab handler previously mapped every tab past
the first to "audit logs", so the audit panel kept polling while Deleted
Keys or Deleted Teams was on screen; each tab now reports its own value and
panels stay mounted via keepMounted. The evaluation settings dialog is
bounded to the viewport and scrolls internally, which the antd Modal got
from being top-anchored on a scrolling page.
The tests added in the previous commit pass unedited against these
components.
* test(interactions): follow Google spec drift replacing Turn with typed steps
* test(interactions): send step and content-list input to the live Gemini API
The migration turned each collapse row into a real button, but the existing
tests only click, so a regression in Enter or Space activation would still
pass. Add one test per component that tabs to the row, expands with Enter and
collapses with Space, asserting visibility rather than markup.
Both fail against the antd version and pass against the migrated one.
TruncatedValue swaps antd Tooltip and Typography for the shadcn Tooltip
and the shared CopyButton, so the full value now lives in the page and is
clipped with CSS rather than shortened in the text node.
OutputCard drops antd Typography for token-styled markup and folds its two
duplicated branches into one. Its border and the drawer's mono font stay on
the module's existing constants so the card still matches InputCard, which
is out of scope here.
Both files retire their no-restricted-imports suppression.
Replace antd Typography and inline hex styling in the log drawer's message
and tool-call blocks with plain elements and design tokens. Both components'
existing tests pass unedited before and after.
Retires their no-restricted-imports entries from the eslint suppressions
baseline.
The assertion hard-coded en-US separators while toLocaleString follows the
host locale, so it failed under de_DE. Building the expected string the same
way keeps it deterministic everywhere and still catches a dropped
toLocaleString wherever the locale groups at all.
Drops the antd Typography import from both. TokenFlow renders inside an antd
Descriptions.Item that already sets the colour, font size, line height and
wrapping the Text wrapper restated, so a bare span is pixel-identical there.
JsonViewer's placeholder moves onto the muted-foreground token.
The characterisation tests from the previous commit are unchanged and stay
green, which is what shows the markup swap did not move behaviour.
Both log-drawer collapse rows hand-rolled the same pattern: a click
handler on a plain div, hover tracked in React state, and a max-height
tween on an always-mounted panel. Move both onto the ui/collapsible
primitive with lucide chevrons, so the row is a real button that keyboard
users can reach and the open state lives in the primitive.
CollapsibleContent keeps keepMounted, which preserves the existing
contract that panel content stays in the DOM while collapsed.
Neither test file is touched: both were already role and text based, and
they pass unedited against the new markup.
Both components are shared by the logs, guardrails-monitor and tool-policies
routes and had no test. These assert on rendered text and roles only, so they
hold against antd Typography and against its replacement.
* fix(langfuse): emit otel trace version and release on the keys langfuse v4 reads
The langfuse_otel exporter wrote version to langfuse.generation.version and
langfuse.trace.version, and release to langfuse.trace.release. Langfuse v4
recognizes neither, so both landed in the generic span attribute bag and every
trace reported version and release as null. v4 has a single langfuse.version
key, lifted to the trace when it sits on the root span, plus langfuse.release.
Also routes the otel v2 preset's per-request headers through the shared builder
so key-scoped and team-scoped exports carry x-langfuse-ingestion-version like
the other three exporter paths already do.
* fix(langfuse): give trace_version precedence over version on the shared v4 key
Matches the documented contract in docs/observability/langfuse_integration.md
and the legacy langfuse SDK callback, which both treat trace_version as the
authoritative trace version with version as its fallback.
Replace Any-typed seams with real types in the files carrying the highest
remaining reportAny/reportExplicitAny density: Literal-keyed structural
Protocols for deployment dicts in tag-based routing, typed Prisma table
wrappers and row protocols in the key and internal-user management
endpoints, TypedDict views for websearch interception kwargs, typed
streaming state in the responses iterator and background polling, and
concrete request/response types in the google_genai, vertex_ai files,
runwayml, rubrik, anthropic context-management, and guardrail translation
modules. Mutable annotations introduced along the way were rewritten as
read-only views (Mapping/Sequence/tuple) built functionally.
No casts, no type: ignore, no noqa, no suppression comments, no new Any
annotations, no behavior changes. Whole-tree basedpyright: reportAny
15,496 -> 14,523, reportExplicitAny 5,356 -> 5,102, all rules
145,547 -> 143,989, with no rule increased repo-wide or per-file.
Budgets ratcheted: basedpyright -1,545, ruff-strict -73,
type-discipline -237.
cost_per_token reads litellm.model_cost, which CI loads from the
model_prices_and_context_window.json on main, so the new entries were
missing until merge. Use the same local_model_cost_map fixture the
sonnet 5 metadata test uses to force the branch's own backup map.
Pre-routing now reads the request's tags on every request with a registered
strategy, including the single-strategy case that used to short-circuit before
looking at tags. Metadata is request-controlled, so a caller that sends
`litellm_metadata` (or `tags`) as a string or any other non-dict shape crashed
tag lookup with an AttributeError instead of routing untagged.
The virtual-key edit view never rendered the router settings stored on a
key, so fallbacks configured at creation could not be verified or changed
afterwards. The key info panel now summarises them and the edit view
embeds the router settings accordion.
The accordion is a fixed-field editor, so its value is merged over the
stored object instead of replacing it. Routing fields it cannot render,
such as tag_routing_prefix or model_group_retry_policy, survive an
unrelated edit, while a field it does own that the admin emptied still
goes out as null so the clear reaches the server. Emptying every field
sends {}, which the proxy reads as no key-level override, so the key
falls back to its team and global routing rather than being pinned to a
blob of nulls.
