Commit graph

14 commits

Author SHA1 Message Date
yujonglee
7b60fd061a
refactor(rust): adopt one error discipline across litellm-rust with source chains and Report (#48042)
* build(rust): upgrade reqwest to 0.13 and move MCP out of litellm-http

The workspace moves to reqwest 0.13, so litellm-http no longer carries a
second reqwest just for rmcp. The mcp feature, its pool and its proxy
conversion go away, and the gateway hands rmcp a pooled NoRedirect client
that already carries the TLS, proxy and timeout settings

The Client test helpers collapse into for_test, a no-redirect client
without TLS or proxy settings, and wrap_for_test for the few tests that
need a hand-built reqwest client

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(http): honor transport settings for Rust WebSockets

The Responses WebSocket connected through tokio-tungstenite with its own
native-roots TLS, so it ignored the configured CA bundle, verify flag,
client certificate, proxies and timeouts. The handshake now goes through
a pooled reqwest client (ClientVariant::WebSocket, HTTP/1.1, no
redirects) and upgrades the response, so it shares the settings every
other outbound request uses

The upgrade response is checked for the Sec-WebSocket-Accept key, the
Upgrade and Connection headers and a subprotocol that was offered

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(http): build outbound requests as reqwest::Request

OutboundRequest was a hand-rolled copy of reqwest::Request. Routes now build
a reqwest::Request directly and RequestSigner reads the body bytes back out
of it, so the signer still covers exactly what is sent. SigV4 signing moves
onto HeaderMap and takes the method, which drops the BTreeMap plumbing and
the header casing table

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(http): make litellm_http::Error the one opaque error

transport::Error and media::Error fold into litellm_http::Error, an
Arc<Repr> newtype whose private variants keep the typed reqwest, io,
rustls and tungstenite causes and own every message. Callers can no
longer build an error with a message of their own. They ask through
dispatched, is_request, status, upstream_response and tls_source

One From<reqwest::Error> now classifies connect, network and timeout
and strips the URL. The WebSocket handshake reports each failed check
as its own variant and owns its connect timeout. OCR drops the media
variants it mirrored, and inference::Error maps a header the signer
computes to Unsupported for every route, not only chat

The error rules move out of AGENTS.md into the rust-error skill, which
also records the decisions behind this design

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(rust): split rust-error skill into thin SKILL.md with references

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(rust-error): render causes once and ask predicates at the boundaries

The four WebSocket handshake variants collapse into Handshake(HandshakeFault), and media download failures no longer answer dispatched() since they happen before the provider call

ErrorDetail in llms and auth-types, the inference error and the OCR error now store their cause in a field named cause with no #[source], so to_string() carries it exactly once and source() is None. Tests that walked the chain now destructure the variant and downcast the cause

The gateway asks is_unsupported() and is_missing_credential() on the inference and OCR errors instead of matching through nested enums, and litellm_auth::Error answers is_missing_credential() for both

The rust-error skill gains axum and pyo3 references, boundaries.md describes the real predicate order and the known divergences, the testing example asserts dispatched() correctly, and shape, classification, conversions, messages-and-causes and recipes agree with the code again

* docs(rust): focus the rust-error gateway page on the error flow and add a rust-gateway skill

The former axum page toured gateway-inference: its status match, the envelope
table, an extractor, the unsupported route and the Sse closure. Those details
belong to the gateway crates and go stale with them. The page is now gateway.md
and follows an error from the provider failure through each From into
IntoResponse, where it is classified once and rendered by surface and framing.

The gateway-structure material moves to a new rust-gateway skill: envelopes per
API surface, extractors and path registration, SSE relay, and the router test
harness.

* test(inference): pin provider URL redaction on every route's transport error

Chat, Messages, Responses and transcription each get a route-level test that
fails if a reqwest error reaches the caller with its URL. The Messages stream
decoder also takes body errors through litellm_http::Error like every other
reqwest error

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(rust-error): adopt std source chains and add litellm-error Report

Messages describe their own layer and the cause is the source, boundaries
render the chain once with litellm_error::Report. Corrects skill claims
the audit found stale against the code.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(rust-error): keep causes as sources across http, auth, secrets, cache and utility crates

Messages describe their own layer and every cause is a typed source,
predicates match exhaustively, and boundaries render with
litellm_error::Report. Media downloads no longer count as dispatched,
Error::upstream bounds the provider body itself, and a forwarded
header SigV4 cannot sign is the caller's fault.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(rust-error): keep causes as sources in llms and the gateway crates

ErrorDetail loses its string variant, OCR provider variants become data,
host faults stop counting as the caller's, a tunnelled transport error
classifies the same on every stream path, and each gateway crate has one
Error with one status() and extractors that reject in its envelope.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(rust-error): nest llms errors in inference and ask predicates in route tests

litellm_inference::Error nests litellm_llms::Error instead of mirroring
its variants, host driver faults get their own variant, every predicate
matches exhaustively, and route tests ask predicates instead of matching
transport variants.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(rust-error): map bridge and gateway-inference errors once and render them with Report

RustUpstreamError has one constructor that always sets headers, argument
checks raise typed errors mapped once, OCR asks upstream_response(), the
Responses stream error event comes from the error type, gateway-inference
loses its string variants, and auth faults classify through one
litellm_auth predicate on every route.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(rust-error): carry OCR provider headers as a HeaderMap and keep outages off the caller

The OCR provider error and get_error_class take reqwest's HeaderMap, the
bridge turns it into Python's (name, value) pairs once. Credential
acquisition and provider SDK failures are no longer the caller's fault,
the UI hides auth and backend causes, and chat keeps a provider answer
as is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(rust-error): correct skill claims against the aligned code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(rust-error): fix the remaining sweep findings in gateway, llms, auth and cache crates

Auth rejections hide their backend cause on every inference renderer,
path parameters reject in the surface envelope, Textract's unsupported
document and Azure DI's unparseable poll URL get typed variants, the
credential chain exposes its last failure as the source, gateway-mcp
asks predicates over rmcp errors, and the S3 and Azure blob transports
classify through litellm_http::Error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(rust-error): fix the remaining sweep findings on the Python side

host-python keeps a native error's chain and returns blocking-pool
cancellation instead of panicking, Vault transport failures convert
through litellm_http::Error so no URL reaches Python, the bridge asks
predicates instead of matching three levels into Vault errors, and the
last string-built PyErrs become typed errors mapped once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* build(deps): refresh compatible dependencies flagged by OSV

* refactor(rust): remove redundant client construction comment

* test(rust): cover configured MCP proxy and CA bundle

* fix(http): preserve native WebSocket trust and optional protocols

* fix(http): allow plain WebSockets without native roots

* fix(http): normalize provider URLs before building outbound requests

* fix(rust): preserve OCR download status and reduce error size

* fix(rust): preserve upstream bodies and bound diagnostics only

* test(rust): preserve bridge serialization boundary in body regression

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-28 09:22:11 -07:00
yujonglee
6c0585b8f1
build(rust): upgrade reqwest to 0.13 and move MCP out of litellm-http (#48039)
* build(rust): upgrade reqwest to 0.13 and move MCP out of litellm-http

The workspace moves to reqwest 0.13, so litellm-http no longer carries a
second reqwest just for rmcp. The mcp feature, its pool and its proxy
conversion go away, and the gateway hands rmcp a pooled NoRedirect client
that already carries the TLS, proxy and timeout settings

The Client test helpers collapse into for_test, a no-redirect client
without TLS or proxy settings, and wrap_for_test for the few tests that
need a hand-built reqwest client

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* build(deps): refresh compatible dependencies flagged by OSV

* refactor(rust): remove redundant client construction comment

* test(rust): cover configured MCP proxy and CA bundle

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-28 07:32:40 -07:00
yujonglee
6edbd27f74
feat(rust): add redis-client crate for standalone Redis (#48269)
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-26 18:41:46 -07:00
Mateo Wang
3069a467d0
docs(agents): ban semicolons and colons in human-facing text (#45684)
* docs(agents): ban semicolons and colons in human-facing text

Tighten the writing rule so ";" and ":" are only allowed where the sentence would be nonsensical without them, rewrite the existing semicolons in AGENTS.md to follow it, and note that docs (litellm-docs) keep their trailing "."

* docs(agents): rewrite semicolons in nested AGENTS.md files

* docs(agents): hand-maintain the dashboard Next.js warning and sync e2e CONTRIBUTING lines

* docs(agents): point at the PR template path that exists

* docs(agents): restore the generated Next.js block in the dashboard AGENTS.md
2026-10-09 16:36:32 -07:00
moe-berri
22ad3c5fff
refactor(lens): connect LiteLLM to the independent Lens service (#45529)
* feat(lens): consume the independent Lens service and shared UI

* fix(lens): keep embedded setup stable and vendor UI before builds

* chore(lens): pin embedded UI provenance to published Lens source

* fix(lens): complete embedded UI extraction across CI builds

* docs(lens): record latest main and removal-boundary validation

* fix(lens): wire external chart credentials and ingestion modes

* test(lens): cover adapter failures and fix extraction CI

* docs(lens): refresh bundled chart setup instructions

* test(lens): restore gateway adapter test package marker

* build(lens): pin clean-cut UI and supported storage chart

* test(ui): await guardrail scope menu before checking options

* perf(lens): omit unused trace-team lookup from product requests

* docs(lens): record final-image adapter permission checks

* test(lens): assert delegated investigation authorization

* refactor(lens): remove copied trace runtime and preserve spend logging

* test(lens): verify independent gateway image upgrades and outages

* test(lens): pin the chart qualification database image

* fix(lens): update embedded UI metadata filters

* test: wait for Lens chart pod identity convergence

* fix(e2e): select Lens chart pods by deployment ownership

* fix(e2e): select Helm migration ownership for Lens upgrades

* test(lens): retain migration Jobs through rollout assertions

* docs(helm): require existing database for migration hooks

* test(lens): qualify release boundaries with embedded navigation update

* docs(lens): record browser qualification for embedded tabs

* fix(lens): remove unrelated CI and guardrail test changes

* test(lens): give qualification tenants unique key aliases

* feat(lens): embed the latest canonical Lens interface

* fix(lens): qualify split inference through the gateway

* fix(lens): preserve scoped auth and isolate delegated credentials

* chore(lens): remove unrelated documentation and lint changes

* fix(lens): bound concurrent forwarding buffers through response delivery

* fix(lens): preserve OAuth2 dispatch without inference policies

* chore(lens): adopt current shared onboarding UI

* fix(lens): refresh shared setup UI and review context

* fix(lens): bound uploads before gateway authentication

* docs(lens): remove extraction evidence from the gateway repo

* docs(lens): refresh shared setup prompt for paired releases
2026-10-09 16:17:05 -07:00
devin-ai-integration[bot]
8f85de740f
perf(lens): prune ClickHouse partitions when sampling and sample in one pass (#45087)
* perf(lens): prune ClickHouse partitions when sampling and sample in one pass

Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>

* test(lens): allow unused query fixture field in load tests

Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>

* fix(lens): shrink the sample page when a response exceeds the read limit

Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>

* fix(lens): qualify request sample window columns

Co-Authored-By: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Ishaan Jaffer <155045088+ishaan-berri@users.noreply.github.com>
2026-10-07 12:24:25 -07:00
devin-ai-integration[bot]
fdc0e7dc2b
refactor(rust): track ClickHouse migrations in a checksummed ledger (#44580)
* refactor(rust): track ClickHouse migrations in a checksummed ledger

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(clickhouse): harden migration execution and reuse retention SQL

* refactor(rust): track ClickHouse migrations in a checksummed ledger

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* docs(rust): keep ClickHouse retention TTLs in the current policy list

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-05 16:04:44 +00:00
devin-ai-integration[bot]
62fb808d4b
feat: improve Lens dev seeding and live UI (#44468)
* feat: seed Lens dev with configurable load profiles

* feat: run Lens UI live through the dev launcher

* fix: verify Lens UI startup before seeding

* fix(ui): render run timestamps on one compact line

The agent runs table printed the long locale form with timezone, which
wrapped to two lines per row. Use a fixed-width 24h form with
milliseconds that matches the timeline axis, keep the long form in the
hover title, and show the timezone once in the column header

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(ui): use the root query client for the Lens demo

Drop the demo's nested QueryClient. Cache keys are already partitioned by
scope, and the root client now skips retries on 4xx ApiErrors, which covers
the demo's not-in-demo and read-only rejections and live 4xx alike.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore: drop unused synthetic_spend reference from query help

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* chore(traces): drop the deeplite fixtures and map every SDK to a logo

The deeplite captures predate the example repository and carried
synthetic spend rows, which leaked a fixture-only column into the
query help SQL and pinned tests to its shape. Replace them with the
SDK captures in the ClickHouse round trip and query API tests, and
derive the seed tenant lookup from the capture metadata

The runs table only knew the two Anthropic framework slugs. Register
the slugs the normalizer emits for LangChain, LangGraph, Deep Agents,
CrewAI, Google ADK, LlamaIndex, OpenAI Agents, Pydantic AI, Strands,
Vercel AI SDK, Codex, Cursor and Copilot, and fall back to the generic
agent glyph when a run has no known framework

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* refactor(ui): inject the Lens sample through data sources instead of demo checks

Components no longer ask whether they are in the demo. The traces source
carries live and handoff, navigation state comes from a URL or memory
route, and the preview action comes from context instead of onDemo props

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(ui): infinite scroll for the agent runs list

Replace the Load more button with a sentinel that fetches the next
cursor page as the list nears its end. Placeholder rows hold the tail
while more runs exist, and a failed page stops auto-loading until Retry.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(ui): keep the whole Lens view in the URL

Lens navigation now lives entirely in query params through nuqs: the
sample session (demo=true, with a Demo data switch in the header), the
open run (trace, trace_ref), the selected step, view and detail section
(span, view, span_tab) and the list filters and range (q, agent, status,
hours). Any Lens view is a shareable link and the back button walks runs

RunView takes its selection injected: the drawer feeds it URL state and
the investigations evidence sheet keeps a local one, so a finding's
original run never writes step ids into the URL. Leaving the sample
session clears every Lens key except the tab so sample ids never point
at live data

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* style(traces): cargo fmt captures tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-04 01:03:11 +00:00
moe-berri
ad8babae33
fix(lens): paginate trace reads within ClickHouse limits (#44384) 2026-10-03 10:38:40 -07:00
devin-ai-integration[bot]
dd86ca5175
refactor(traces): type the ClickHouse query help response (#44285)
* refactor(traces): type the ClickHouse query help response

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(traces): include agent names and frameworks in named contract round trips

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(traces): cover native query help validation in the storage adapter

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Yujong Lee <yujong@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-02 18:01:27 -07:00
yujonglee
688d791fa0
feat(traces): type queries and align read access with log visibility (#44228)
* wip

* wip

* test(traces): separate root status from diagnostic error counts

* test(traces): cover normalization precedence and fallbacks

* chore(cache): remove stray comments from trace PR

* test(traces): name lens test for shared query path

* fix(traces): place query implementation before test module

* test(traces): use unified read scope in migration tests

* ci(rust): allow feature checks to finish

* ci(mcp): allow dependency resolution to finish

* fix(traces): preserve key visibility and safe spend attribution
2026-10-02 21:55:42 +00:00
yujonglee
276fc9c63a
fix(tracing): unify ClickHouse storage configuration (#43941)
* fix(tracing): use ClickHouse URL for reads by default

* fix(tracing): unify ClickHouse storage configuration

* fix(tracing): update dashboard setup copy for one URL

* test(tracing): make tests/unit/tracing a package

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(config): drop legacy string tracing store variant

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(tracing): own ClickHouse defaults in constants and reject unset env references

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(tracing): use raw regex patterns in config tests

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(tracing): read ClickHouse env defaults when tracing config resolves

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): split audit log query guard to fit condition-chain budget

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-10-02 16:31:26 +00:00
yujonglee
ec605826d4
feat: improve trace ingestion and trace details (#43975)
* refactor: separate OTLP HTTP decoding from trace codec

* feat: complete trace ingestion and read paths

* fix: encode OTLP protobuf errors in Rust

* fix: raise OTLP body limit to 16 MiB

* test: cover OTLP auth body parsing boundary

* refactor: parse OTLP media type into enum

* fix: enforce OTLP body size at HTTP boundary

* perf: preserve shared OTLP metadata across ingestion

* bench: compare owned and shared trace resource fanout

* refactor: extract shared storage and Python conversion caches

* refactor: keep shared storage owned by traces

* test: keep trace loopback coverage in Rust

* test(proxy): adapt trace coverage to injected access context

* fix(tracing): satisfy stacked branch lint checks

* refactor(tracing): use immutable ingestion payloads

* fix(tracing): declare native error encoder export

* test(proxy): resolve trace access through dependency

* fix(tracing): align merged normalizer types and bridge tests

* fix(tracing): address ingestion and diagnostic review findings

* fix(proxy): preserve body parsing for partial request scopes

* test(proxy): use valid HTTP scopes in request fixtures

* test(proxy): complete auth request flow scopes
2026-10-01 13:45:33 -07:00
yujonglee
be67fce26a
refactor(proxy): inject tracing receiver and access context (#44035)
* refactor(proxy): inject tracing receiver and access context

* refactor(proxy): own tracing resources through FastAPI lifespan

* test(proxy): pass tracing dependency in Lens lifecycle

* refactor(proxy): stop tracing logger cooperatively

* refactor(proxy): derive tracing permissions in one place

* refactor(proxy): compose application lifespan state

* refactor(proxy): give Lens tracing storage directly

* refactor(tracing): name shared ClickHouse storage explicitly

* refactor(tracing): extract shared ClickHouse storage crate

* test(proxy): isolate db push timeout from Lens safety check

* fix(tracing): drain spend retries during shutdown
2026-10-01 13:45:32 -07:00