Commit graph

19 commits

Author SHA1 Message Date
moe-berri
22ad3c5fff
refactor(lens): connect LiteLLM to the independent Lens service (#45529)
* feat(lens): consume the independent Lens service and shared UI

* fix(lens): keep embedded setup stable and vendor UI before builds

* chore(lens): pin embedded UI provenance to published Lens source

* fix(lens): complete embedded UI extraction across CI builds

* docs(lens): record latest main and removal-boundary validation

* fix(lens): wire external chart credentials and ingestion modes

* test(lens): cover adapter failures and fix extraction CI

* docs(lens): refresh bundled chart setup instructions

* test(lens): restore gateway adapter test package marker

* build(lens): pin clean-cut UI and supported storage chart

* test(ui): await guardrail scope menu before checking options

* perf(lens): omit unused trace-team lookup from product requests

* docs(lens): record final-image adapter permission checks

* test(lens): assert delegated investigation authorization

* refactor(lens): remove copied trace runtime and preserve spend logging

* test(lens): verify independent gateway image upgrades and outages

* test(lens): pin the chart qualification database image

* fix(lens): update embedded UI metadata filters

* test: wait for Lens chart pod identity convergence

* fix(e2e): select Lens chart pods by deployment ownership

* fix(e2e): select Helm migration ownership for Lens upgrades

* test(lens): retain migration Jobs through rollout assertions

* docs(helm): require existing database for migration hooks

* test(lens): qualify release boundaries with embedded navigation update

* docs(lens): record browser qualification for embedded tabs

* fix(lens): remove unrelated CI and guardrail test changes

* test(lens): give qualification tenants unique key aliases

* feat(lens): embed the latest canonical Lens interface

* fix(lens): qualify split inference through the gateway

* fix(lens): preserve scoped auth and isolate delegated credentials

* chore(lens): remove unrelated documentation and lint changes

* fix(lens): bound concurrent forwarding buffers through response delivery

* fix(lens): preserve OAuth2 dispatch without inference policies

* chore(lens): adopt current shared onboarding UI

* fix(lens): refresh shared setup UI and review context

* fix(lens): bound uploads before gateway authentication

* docs(lens): remove extraction evidence from the gateway repo

* docs(lens): refresh shared setup prompt for paired releases
2026-10-09 16:17:05 -07:00
moe-berri
e9cfba2c17
feat(lens): isolate ingestion and investigations in a Rust service (#45148)
* feat(lens): isolate trace storage and investigation in a Rust service

* fix(lens): include Rust sources in the image build context

* feat(lens): wire service setup, scoped delivery receipts and lease attempts

* fix(lens): complete service routing and reject stale investigation results

* fix(lens): retry key propagation and validate isolated Compose setup

* fix(lens): seed through isolated ingestion and preserve upstream queue fixes

* chore: sync schema.prisma copies from root

* fix(lens): bind nullable due timestamps as text for Prisma

* chore(ui): remove stale lint suppressions

* fix(lens): address CI failures and review findings

* refactor(lens): remove retired Python worker and run evaluations in Rust

* fix(lens): reuse control connections and satisfy review checks

* test(lens): install and upgrade both Helm charts on Kubernetes

* test(lens): run connection reuse coverage as an integration test

* fix(ui): upgrade Next.js to 16.3.8 security release

* fix(lens): fence stale attempts and preserve reviewed evidence

* Revert "fix(ui): upgrade Next.js to 16.3.8 security release"

This reverts commit 2f79a51b25.

* fix(lens): stop failed investigations and stream history excerpts

* test(lens): cover model tool and result contracts

* test(lens): fix retired routes and reuse installation build artifacts

* test(lens): use portable grep in Helm installation smoke

* test(lens): wait for migrations before forwarding Helm services

* fix(lens): keep failed evidence reads retryable

* fix(lens): preserve sandbox output during process exit

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-10-07 17:18:58 -07:00
ishaan-berri
bf9bd35469
feat(lens): scope traces to one agent with a header picker (#45202)
* feat(lens): add trace_agents rollup query

* feat(lens): register the trace_agents read query

* feat(lens): add trace_agents params and row types

* feat(lens): dispatch the trace_agents query

* feat(lens): export trace_agents wire schemas

* test(lens): pin the trace_agents query name

* test(lens): cover trace_agents scope, window and failure counts

* feat(lens): cap the agent list size

* feat(lens): declare the trace_agents bridge query

* feat(lens): read trace agents from clickhouse storage

* feat(lens): add trace agent response models

* feat(lens): list agents within the reader's trace scope

* feat(lens): add GET /v1/traces/agents

* chore(lens): regenerate trace models with trace_agents

* chore(lens): regenerate trace types with trace_agents

* chore(lens): regenerate read query name schema

* chore(lens): add trace agent row schema

* chore(lens): add trace agents params schema

* test(lens): cover the trace agents route

* test(lens): cover agent listing scope and timestamps

* chore(ui): regenerate api types with trace agents route

* feat(lens): derive trace agent types from the schema

* feat(lens): fetch the agent list from the traces api

* feat(lens): roll up demo runs into agents

* feat(lens): serve the agent list in demo data

* feat(lens): load agents seen in the last two weeks

* feat(lens): remember the selected agent per browser

* feat(lens): add the agent picker

* feat(lens): wire agent selection into the lens header

* feat(lens): show the agent picker next to the lens title

* refactor(lens): drop the toolbar agent filter in favor of the header picker

* test(lens): remove tests for the toolbar agent filter

* test(lens): cover agent resolution and rollup

* test(lens): cover scoping, switching and remembering the agent

* test(lens): read only the create request in guided setup

* test(lens): assert the toolbar agent filter is gone

* test(lens): read stubbed requests without their abort signal
2026-10-07 23:48:04 +00:00
moe-berri
a3e74a5223
fix(tracing): preserve optional provider evidence in fixture replay (#44930)
* fix(tracing): preserve optional provider evidence in fixture replay

* fix(tracing): keep copied provider identities consistent
2026-10-06 14:00:40 -07:00
moe-berri
2190003bb8
fix(lens): reconstruct native coding agent conversations (#44711)
* fix(lens): reconstruct native coding agent conversations

* fix(lens): display timestamps used for conversation ordering

* fix(lens): preserve coding trace identity and message provenance

* fix(lens): complete capture checks after trace pagination

* fix(lens): show loaded replies during trace pagination
2026-10-05 17:51:05 -07:00
devin-ai-integration[bot]
45e7be1abc
feat(tracing)!: return only data from SQL queries (#44609)
* feat(tracing): add generated SQL response contract

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat(proxy): return data-only trace SQL responses

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(rust): format trace response schema

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-05 12:11:59 -07:00
devin-ai-integration[bot]
9b6a6a0b71
refactor(tracing): generate existing HTTP request models from Rust schemas (#44591)
* test(tracing): pin HTTP request compatibility

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(tracing): generate existing HTTP request models from Rust schemas

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(tracing): bind trace query params to generated request models

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* ci(rust): raise the native wheel size gate to 48 MB

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* perf(tracing): read the trace list clock without a thread-pool dispatch

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(traces): share the trace page-size bounds between schema and reader

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(ui): type trace request queries against the generated OpenAPI schema

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* docs: encode the trace contract boundary in AGENTS.md

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(ui): format trace request aliases

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-05 17:37:52 +00:00
devin-ai-integration[bot]
97fc72e5bb
fix(ci): send the Lens preview body as selection in the tracing endpoint tests (#44525)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-04 13:22:16 -07:00
devin-ai-integration[bot]
461a58c40a
refactor(lens): storage-independent trace reads, shared keyset pager, typed read failures (#44422)
* feat(lens): own trace reads behind a cached TraceStore port

Move storage-independent trace reads into litellm-traces-cache behind a
TraceStore port that ClickHouse implements. One keyset pager drives the
span, list span and spend reads, and a run list batch reads spend once.

Trace opens, pages and list summaries share one resolved read per trace
in an in-process cache with single-flight loading. Live traces and reads
with unknown spend expire after 5s, quiet traces after 10 minutes, failed
reads are never cached, and the accepted list page size is remembered per
scope.

Trace read failures map to their own status and code (400, 409, 413, 503
with Retry-After), and the trace drawer retries temporary failures while
offering only a refresh for changed or oversized traces.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* perf(lens): seed large profiles with server-side copies and long sessions

Replay one copy through the proxy, then copy it inside ClickHouse and
PostgreSQL with INSERT ... SELECT, rewriting trace, span and call IDs so
every copy keeps its own spend. Add three long single-trace sessions for
drawer paging and the oversized read path

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chores

* style(lens): float the investigation setup badge on the tab edge

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(lens): restructure the trace drawer and polish its layout

Split the 547-line TraceDrawer into run/, tree/, span/, content/ and
conversation/ modules. Step rows now sit on one line with colored span
family tiles, and the per-row timing bar moved into an optional Waterfall
layout with a time axis. The steps and details panes are separated by the
shadcn Resizable handle, with the split remembered per orientation.

Span payloads go through one pure classifier (payloadView) that picks
messages, a tool result, a nested field tree or text. JSON-encoded field
values unfold into a tree, prose renders as markdown, repr and tracebacks
stay monospace, and every section offers a Raw view. LangChain's
serialized messages now render as conversation cards.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* typesafety

* wip

* fmt

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 12:39:47 -07:00
moe-berri
ad8babae33
fix(lens): paginate trace reads within ClickHouse limits (#44384) 2026-10-03 10:38:40 -07:00
devin-ai-integration[bot]
e340e546e2
feat(traces): tracing development seed (#44363)
* feat(dev): seed linked tracing and spend fixtures

* chore(dev): use OpenAI model in tracing config

* chore(dev): align tracing credentials with UI E2E

* fix(dev): update fixture seeder query scope

* feat(dev): seed linked tracing and spend fixtures

* chore(dev): use OpenAI model in tracing config

* chore(dev): align tracing credentials with UI E2E

* fix(dev): update fixture seeder query scope

* wip

* wip

* wip

* chore(trace): checkpoint ongoing Rust migration

* refactor(trace): group Python bridge under trace package

* refactor(traces): read span conventions through a Convention trait

Each span format (Claude Code, LangSmith, OpenInference, gen_ai) now lives under
normalize/convention/ as a unit struct implementing Convention, owning both its
detection and its extraction. Precedence is one ordered registry instead of an
if-chain in mod.rs that reached into each module differently.

The modules now share one way to read attributes: present() for the first
non-empty key and Payload for a text that also reports the key it consumed,
replacing three different idioms and the &mut Vec threaded through payload
readers. Instrumentation::adjust returns a new Extraction instead of mutating
one, with each SDK rule as its own function, and the LangChain middleware
suffix list exists once.

* feat(trace): export Rust-owned wire schemas and enforce contract bounds

* fix(trace): bound quoted counts in ClickHouse wire schemas

* feat(trace): generate Python wire contracts with datamodel-code-generator

* test(trace): validate migrated callers and generated contracts at the native boundary

* refactor(traces): rename normalization convention to format

* fix(traces): reconcile spend evidence and preserve unknown costs

* feat(traces): normalize additional telemetry formats

* test(traces): cover captured normalization fixtures

* refactor(traces): isolate SDK normalization rules

* feat(tracing): seed all trace exports for local dashboard

* fix(clickhouse): preserve custom LiteLLM request metadata

* docs(traces): define normalization module boundaries

* docs(traces): define resolution and OTLP boundaries

* fix(ui): normalize nullable trace message names

* refactor(traces): split resolver modules and cover resolution behavior

* test(traces): replace normalization snapshots with behavior assertions

* fix(ui): align dashboard API contracts with generated types

* refactor(traces): type normalization and storage boundaries

* fix(traces): seed captured SDK spend and preserve provider identities

* wip

* test(traces): verify guide discovery and content ordering

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-03 09:59:51 +00:00
yujonglee
677205b3f5
fix(proxy): share ownership permissions for spend logs and traces (#44239)
* refactor(proxy): extract shared spend log read policy

* test(proxy): use named bindings for spend scope regression

* test(proxy): reuse existing spend log query harness

* test(proxy): cover spend log permission lookup adoption

* chore(proxy): relocate existing spend query baseline

* refactor(proxy): make scope query returns explicit

* refactor(proxy): inject deferred log permission lookup

* test(proxy): cover teamless management compatibility lookup

* refactor(proxy): compose user and team log grants

* refactor(proxy): share generic authorization composition

* refactor(proxy): compose trace read permissions

* refactor(proxy): centralize spend and trace authorization

* refactor(proxy): strengthen spend and trace scope types

* refactor(proxy): flatten log read scope into owned logs

Replace the AnyOf grant tree with a flat OwnedLogs(user_id, team_ids) scope,
and OwnedTraces(logs, api_key_hash) for traces, since every consumer flattened
the tree back into that shape.

A caller with no user id now gets an empty scope instead of matching ownerless
rows through Prisma's IS NULL. The dead request_id guard in ui_view_spend_logs
is removed, and the management facets inject the log team lookup and reuse
read_scope_sql instead of the list shim.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(proxy): run spend scope tests through one SQLite emulator

Replace the string-matching payload emulator and the hand-rolled Prisma where
interpreter with one SQLite helper that runs the real scope SQL. Session scope
tests now go through the endpoint, including the no-user caller that must not
match ownerless rows. Drop duplicated lookup-failure and trace mapping cases.

load_permitted_log_team_ids returns no teams without a database instead of
relying on the resolver's broad except.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(proxy): unify log and trace ownership permissions

* test(tracing): align fixtures with ownership read scopes

* refactor(tracing): align query scopes with row ownership

* refactor(spend): make ownership SQL predicates explicit

* test(spend): validate ownership SQL against PostgreSQL

* docs(traces): drop key-row visibility from query help guide

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(spend): reach the empty-memberships branch in team lookup test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(ui): regenerate dashboard API types

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-03 01:38:54 +00:00
devin-ai-integration[bot]
dd86ca5175
refactor(traces): type the ClickHouse query help response (#44285)
* refactor(traces): type the ClickHouse query help response

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(traces): include agent names and frameworks in named contract round trips

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(traces): cover native query help validation in the storage adapter

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-02 18:01:27 -07:00
yujonglee
688d791fa0
feat(traces): type queries and align read access with log visibility (#44228)
* wip

* wip

* test(traces): separate root status from diagnostic error counts

* test(traces): cover normalization precedence and fallbacks

* chore(cache): remove stray comments from trace PR

* test(traces): name lens test for shared query path

* fix(traces): place query implementation before test module

* test(traces): use unified read scope in migration tests

* ci(rust): allow feature checks to finish

* ci(mcp): allow dependency resolution to finish

* fix(traces): preserve key visibility and safe spend attribution
2026-10-02 21:55:42 +00:00
yujonglee
8d28e8d776
feat(tracing): add scoped SQL queries and schema-aware help (#44085)
* feat(tracing): add SQL queries and schema-aware query help

* test(tracing): verify help requests and sync API types

* refactor(tracing): render query help with Askama

* refactor(tracing): use jinja extension for query guide

* fix(tracing): preserve query help when discovery fails

* feat(tracing): enforce team SQL scope with managed ClickHouse readers

* test(tracing): verify reads with one ClickHouse URL

* fix(tracing): revoke rotated trace reader credentials

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(tracing): streamline query help catalog assembly

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(tracing): run query help discovery sequentially

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(tracing): update reader setup request expectations

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-01 22:06:14 -07:00
yujonglee
ae6b762190
refactor(tracing): normalize agent spans in Rust (#44071)
* refactor(tracing): normalize agent spans in Rust

* refactor(tracing): generate dashboard trace types from API

* test(tracing): use complete trace response fixtures

* fix(ui): expose generated span error response type

* fix(tracing): retain full tool call payloads

* fix(tracing): preserve decoded attribute tuple shape

* fix(tracing): type consumed attributes as tuple
2026-10-01 18:01:20 -07:00
yujonglee
ec605826d4
feat: improve trace ingestion and trace details (#43975)
* refactor: separate OTLP HTTP decoding from trace codec

* feat: complete trace ingestion and read paths

* fix: encode OTLP protobuf errors in Rust

* fix: raise OTLP body limit to 16 MiB

* test: cover OTLP auth body parsing boundary

* refactor: parse OTLP media type into enum

* fix: enforce OTLP body size at HTTP boundary

* perf: preserve shared OTLP metadata across ingestion

* bench: compare owned and shared trace resource fanout

* refactor: extract shared storage and Python conversion caches

* refactor: keep shared storage owned by traces

* test: keep trace loopback coverage in Rust

* test(proxy): adapt trace coverage to injected access context

* fix(tracing): satisfy stacked branch lint checks

* refactor(tracing): use immutable ingestion payloads

* fix(tracing): declare native error encoder export

* test(proxy): resolve trace access through dependency

* fix(tracing): align merged normalizer types and bridge tests

* fix(tracing): address ingestion and diagnostic review findings

* fix(proxy): preserve body parsing for partial request scopes

* test(proxy): use valid HTTP scopes in request fixtures

* test(proxy): complete auth request flow scopes
2026-10-01 13:45:33 -07:00
yujonglee
be67fce26a
refactor(proxy): inject tracing receiver and access context (#44035)
* refactor(proxy): inject tracing receiver and access context

* refactor(proxy): own tracing resources through FastAPI lifespan

* test(proxy): pass tracing dependency in Lens lifecycle

* refactor(proxy): stop tracing logger cooperatively

* refactor(proxy): derive tracing permissions in one place

* refactor(proxy): compose application lifespan state

* refactor(proxy): give Lens tracing storage directly

* refactor(tracing): name shared ClickHouse storage explicitly

* refactor(tracing): extract shared ClickHouse storage crate

* test(proxy): isolate db push timeout from Lens safety check

* fix(tracing): drain spend retries during shutdown
2026-10-01 13:45:32 -07:00
devin-ai-integration[bot]
a76b59db9f
test(proxy): move middleware, spend_tracking, pass_through, common_utils and root proxy tests into tests/unit/proxy (#44015)
Co-authored-by: yuneng <yuneng@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-01 18:23:31 +00:00
Renamed from tests/test_litellm/proxy/test_tracing_endpoints.py (Browse further)