Commit graph

53679 commits

Author SHA1 Message Date
Ishaan Jaff
2207a79922
fix(lens): start review history over when a run is reclaimed 2026-10-03 18:33:54 -07:00
Ishaan Jaff
98a810a71c
refactor(lens): keep update retry settings next to the repository 2026-10-03 18:31:59 -07:00
Ishaan Jaff
e9f3d8a24d
fix(lens): keep worker settings inside the standalone worker package 2026-10-03 18:31:58 -07:00
Ishaan Jaff
68ca72041e
style(lens): format live run files 2026-10-03 18:19:40 -07:00
Ishaan Jaff
8b9e799a19
refactor(lens): name inline objects in the live run 2026-10-03 18:19:40 -07:00
Ishaan Jaff
775a808003
refactor(lens): name now reading conditions 2026-10-03 18:19:40 -07:00
Ishaan Jaff
2c05090b1a
refactor(lens): name the run now handler in investigations view 2026-10-03 18:19:40 -07:00
Ishaan Jaff
a38bbe3465
style(lens): format endpoint tests 2026-10-03 18:15:46 -07:00
Ishaan Jaff
ca9db83e17
Merge remote-tracking branch 'upstream/main' into litellm_lens_live_review
# Conflicts:
#	litellm/proxy/lens/worker.py
#	tests/unit/proxy/lens/test_worker.py
#	ui/litellm-dashboard/src/components/lens/investigations/InvestigationsView.tsx
#	ui/litellm-dashboard/src/components/lens/investigations/detail/InvestigationDetail.tsx
#	ui/litellm-dashboard/src/lib/http/schema.d.ts
2026-10-03 18:15:36 -07:00
Ishaan Jaff
4db3c3df22
fix(lens): give demo jobs an empty in-flight list 2026-10-03 18:09:50 -07:00
Ishaan Jaff
3c8309272f
fix(lens): resolve the analysis provider logo from the model catalog 2026-10-03 18:09:50 -07:00
Ishaan Jaff
6fb7e8124f
feat(lens): put the now reading stage above the trace list in View run 2026-10-03 18:09:50 -07:00
Ishaan Jaff
3577f3bebf
feat(lens): show a now reading stage that types each trace's reasoning 2026-10-03 18:09:47 -07:00
Ishaan Jaff
69b5265a17
feat(lens): model live reading lanes from in-flight runs and reviews 2026-10-03 18:09:47 -07:00
devin-ai-integration[bot]
62fb808d4b
feat: improve Lens dev seeding and live UI (#44468)
* feat: seed Lens dev with configurable load profiles

* feat: run Lens UI live through the dev launcher

* fix: verify Lens UI startup before seeding

* fix(ui): render run timestamps on one compact line

The agent runs table printed the long locale form with timezone, which
wrapped to two lines per row. Use a fixed-width 24h form with
milliseconds that matches the timeline axis, keep the long form in the
hover title, and show the timezone once in the column header

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(ui): use the root query client for the Lens demo

Drop the demo's nested QueryClient. Cache keys are already partitioned by
scope, and the root client now skips retries on 4xx ApiErrors, which covers
the demo's not-in-demo and read-only rejections and live 4xx alike.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore: drop unused synthetic_spend reference from query help

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(traces): drop the deeplite fixtures and map every SDK to a logo

The deeplite captures predate the example repository and carried
synthetic spend rows, which leaked a fixture-only column into the
query help SQL and pinned tests to its shape. Replace them with the
SDK captures in the ClickHouse round trip and query API tests, and
derive the seed tenant lookup from the capture metadata

The runs table only knew the two Anthropic framework slugs. Register
the slugs the normalizer emits for LangChain, LangGraph, Deep Agents,
CrewAI, Google ADK, LlamaIndex, OpenAI Agents, Pydantic AI, Strands,
Vercel AI SDK, Codex, Cursor and Copilot, and fall back to the generic
agent glyph when a run has no known framework

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(ui): inject the Lens sample through data sources instead of demo checks

Components no longer ask whether they are in the demo. The traces source
carries live and handoff, navigation state comes from a URL or memory
route, and the preview action comes from context instead of onDemo props

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(ui): infinite scroll for the agent runs list

Replace the Load more button with a sentinel that fetches the next
cursor page as the list nears its end. Placeholder rows hold the tail
while more runs exist, and a failed page stops auto-loading until Retry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(ui): keep the whole Lens view in the URL

Lens navigation now lives entirely in query params through nuqs: the
sample session (demo=true, with a Demo data switch in the header), the
open run (trace, trace_ref), the selected step, view and detail section
(span, view, span_tab) and the list filters and range (q, agent, status,
hours). Any Lens view is a shareable link and the back button walks runs

RunView takes its selection injected: the drawer feeds it URL state and
the investigations evidence sheet keeps a local one, so a finding's
original run never writes step ids into the URL. Leaving the sample
session clears every Lens key except the tab so sample ids never point
at live data

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* style(traces): cargo fmt captures tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-04 01:03:11 +00:00
Ishaan Jaff
96e30ad045
chore(ui): regenerate api types for lens in-flight runs 2026-10-03 18:02:02 -07:00
Ishaan Jaff
66329e7154
test(lens): cover in-flight reporting under original run ids 2026-10-03 18:01:39 -07:00
Ishaan Jaff
46f16ea03e
test(lens): cover in-flight runs across progress, old workers and terminal states 2026-10-03 18:01:39 -07:00
Ishaan Jaff
e4387d8ab2
feat(lens): send in-flight runs with worker progress 2026-10-03 18:01:39 -07:00
Ishaan Jaff
1d18be780e
feat(lens): report each run as in flight when its review starts 2026-10-03 18:01:39 -07:00
Ishaan Jaff
45909929ff
refactor(lens): route progress, cancel and results through shared job transitions 2026-10-03 18:01:38 -07:00
Ishaan Jaff
687d94879a
feat(lens): store in-flight runs from progress and clear them when a job ends 2026-10-03 18:01:38 -07:00
Ishaan Jaff
2a05b50c55
feat(lens): add in-flight runs to jobs and worker progress 2026-10-03 18:01:38 -07:00
Ishaan Jaff
28f70ea41d
feat(lens): slide one model rectangle over the traces being read
A single rounded rectangle carrying the provider logo and model wraps the real in-flight rows from job.reading. It translates and resizes over 250ms as traces finish in place. Before job.reading arrives it sits on a top slot showing the honest progress line, and when the run completes it fades out over 400ms. Rows have a fixed height and stable execution_id keys, so polls don't cause jumps or flicker.
2026-10-03 17:58:18 -07:00
Ishaan Jaff
5724190869
feat(lens): sum up a finished live run with time taken
doneLine reads like "Reviewed 30 traces in 31s with", measured from when reading started.
2026-10-03 17:58:16 -07:00
Ishaan Jaff
b174232308
feat(lens): show what the worker is reading and make View run obvious
Each trace in flight gets a highlighted row with the model and a live timer, and becomes its completed row in place. Completed rows show the real review time. View run is an outline button next to the progress line, and clicking anywhere on the strip opens it too.
2026-10-03 17:55:12 -07:00
Ishaan Jaff
0376a8d7c6
refactor(lens): drop client-side replay in favour of real in-flight rows
Removes the playback reducer and its pacing. liveRows lists the traces the worker is reading, from job.reading, followed by completed reviews newest first, keyed by execution_id so a trace keeps its row when it finishes.
2026-10-03 17:55:10 -07:00
moe-berri
8520626e7a
fix(lens): preserve approved worker digests and harden its image (#44467)
* fix(lens): pin worker dependencies and support approved image digests

* fix(lens): include locked dependencies and release identity in build context

* fix(lens): select the dev worker package for SHA-tagged charts
2026-10-03 17:54:17 -07:00
Ishaan Jaff
b00cb8d733
feat(lens): simplify View run to traces and conclusions
The left pane is the trace list. A soft highlighter carrying the provider and model slides to the trace being reviewed, and clicking a row shows just Lens's reasoning and verdicts. The right pane keeps one conclusion card per check.
2026-10-03 17:47:02 -07:00
Ishaan Jaff
55d9425a82
fix(lens): group live conclusions by check with short labels
There is now one group per check_id: issue traces are the main count and pattern traces a secondary note, so there are no duplicate red and grey cards. A long instruction falls back to the humanized check id. Adds briefReasoning and traceRows for the simplified trace list, and drops helpers nothing uses.
2026-10-03 17:46:52 -07:00
Ishaan Jaff
10260da169
fix(lens): split live conclusions into issues and patterns
A check could show up twice with the same label, once as an issue and once as a pattern.
2026-10-03 17:41:20 -07:00
Ishaan Jaff
7ef76eabff
fix(lens): feed the live run from the reviews endpoint and keep View run open
LiveRun now gets its reviews from useJobReviews instead of the list, which no longer carries them. View run stays clickable while a run is queued or running, and before the first trace the opened view says what the worker is doing.
2026-10-03 17:40:35 -07:00
Ishaan Jaff
2f5fb3ef44
feat(ui): add a lens reviews query that polls the index cursor while live 2026-10-03 17:40:32 -07:00
Ishaan Jaff
db0e4df4f5
feat(lens): page job reviews by index cursor
Adds api.reviews for GET /lens/{id}/runs/{job}/reviews?after=N, with a demo implementation. appendPage adds pages in arrival order and keeps the latest 200. liveJob now keys off reviewed, since the list no longer carries reviews.
2026-10-03 17:40:30 -07:00
Ishaan Jaff
73ad24f7de
chore(ui): regenerate api types for the lens reviews endpoint 2026-10-03 17:39:38 -07:00
Ishaan Jaff
c3007bb667
test(lens): cover index cursors across out-of-order and rolled-over reviews 2026-10-03 17:37:48 -07:00
Ishaan Jaff
ccc7a1b2ac
feat(lens): take an index cursor on the reviews endpoint 2026-10-03 17:37:48 -07:00
Ishaan Jaff
8ffe1222e0
fix(lens): page live reviews by index so out-of-order reviews are never skipped 2026-10-03 17:37:48 -07:00
Ishaan Jaff
89087a0202
feat(lens): add a review page model carrying the total reviewed count 2026-10-03 17:37:48 -07:00
Ishaan Jaff
a75dab2f41
feat(lens): show the queue reason and what the worker is doing in the live strip
The progress header and the strip replace "Queued for your worker" with the concrete reason. While waiting, the strip lists the busy worker's investigations; click one to open it.
2026-10-03 17:36:52 -07:00
Ishaan Jaff
12738b0d79
feat(lens): explain why a queued investigation is waiting
Works out whether no worker is connected, the worker is busy (with its running investigations and an estimated start time), or it is just being picked up.
2026-10-03 17:36:49 -07:00
Ishaan Jaff
15e570e4c7
test(lens): cover list summaries, review polling and review access 2026-10-03 17:34:13 -07:00
Ishaan Jaff
8737002ce6
perf(lens): strip reviews and run attributes from the lens list and serve reviews separately 2026-10-03 17:34:13 -07:00
Ishaan Jaff
a8a5d41644
feat(lens): add list summaries and an incremental review filter 2026-10-03 17:34:13 -07:00
berriai-litellm-provider-info-sync[bot]
a66adb4ff8
chore(cost-map): sync openrouter prices from the models API (#44466)
Price-Sync: litellm-providers

Co-authored-by: berriai-litellm-provider-info-sync[bot] <328147090+berriai-litellm-provider-info-sync[bot]@users.noreply.github.com>
2026-10-03 17:34:05 -07:00
devin-ai-integration[bot]
9d16412341
fix(caching): count tool_call cache_control marks in the injection census (#43556)
* fix(caching): count tool_call cache_control marks in the injection census

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(caching): remove cache census casts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(caching): only skip injection on message or content marks

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(caching): skip injection on messages whose tool calls carry marks

Reverts 9f08d8aef8. A default 5m mark injected on assistant text lands
before the client's 1h tool_use mark, which Anthropic rejects with a 400
because a 1h breakpoint must not follow a 5m one. Keeping the full census
in the skip check leaves the client's tool_call breakpoint as the only
one on that message.

* fix(caching): count every client tool_call cache mark in the breakpoint census

The census gated tool_call marks on type function and dict shape, so a client mark on a call without a type or with a string cache_control slipped past the count and injection overflowed the 4 breakpoint cap. Count any non-None tool_call mark, keep the server tool exclusion, and add integration cells for the capped surfaces, the yaml stand-down, Bedrock and Gemini, and router affinity

* test(integration): hold the upstream so the worker kill lands mid-burst

* refactor(caching): reuse the transform's server tool lookup in the breakpoint census

The census now calls the same helper the Anthropic transform uses to decide
whether a marked tool call becomes a server tool block, so the two cannot
drift apart. The owned-proxy burst test waits up to 90 seconds for the burst
to reach the wire before it kills a worker

* refactor(anthropic): move the server tool rebuild check under llms/anthropic

* test(integration): audit the tool call mark census across chat, messages, responses, and chaos

Adds the /audit cells for the breakpoint census on assistant tool_calls marks: the
Responses stream bridge, the OpenAI and Anthropic SDK clients, in-process Pydantic
messages, response cache twins, request-level points, a provider 401 on a capped request,
malformed provider_specific_fields and tool_call ids, null or empty points, a points
update mid-burst, a proxy restart mid-burst, and a worker SIGKILL that picks the worker
holding the burst's upstream connections

* test(integration): close the SDK clients the cache census cells open

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-10-03 17:33:48 -07:00
Ishaan Jaff
183d07c73e
feat(lens): add worker slot and poll interval settings 2026-10-03 17:33:08 -07:00
Ishaan Jaff
fab6291929
fix(lens): run several investigations per worker and poll every two seconds 2026-10-03 17:33:00 -07:00
Ishaan Jaff
c4fcd4b457
feat(lens): show the live run as a two-pane trace and conclusions view
Left pane: the trace being reviewed as a readable timeline, followed by Lens's reasoning and the verdict. Right pane: ranked conclusion groups with share bars, plus a trace list you can filter.
2026-10-03 17:31:34 -07:00
Ishaan Jaff
1dfcdf223e
feat(lens): keep the live run ambient until View run is clicked
The drawer no longer opens on Run or when entering a running investigation. LiveRun takes reviews as a prop so it can move to a dedicated reviews endpoint.
2026-10-03 17:31:32 -07:00