Commit graph

53360 commits

Author SHA1 Message Date
Ishaan Jaff
69ed633ac4
feat(ui): add agent trace API calls 2026-09-30 06:48:19 -07:00
Ishaan Jaff
5481cdd6c2
test(proxy): protobuf bodies are not parsed as JSON 2026-09-30 06:48:19 -07:00
Ishaan Jaff
7f63248e81
test(clickhouse): cover clickhouse spend-log callback 2026-09-30 06:48:19 -07:00
Ishaan Jaff
7a34f89c46
test(proxy): cover /v1/traces endpoints and scoping 2026-09-30 06:48:19 -07:00
Ishaan Jaff
295ea7326a
test(tracing): cover TraceReceiver ingest, tenant stamping and backpressure 2026-09-30 06:48:19 -07:00
Ishaan Jaff
f5a2da83de
test(tracing): cover trace store row mapping and agent aggregation 2026-09-30 06:48:19 -07:00
Ishaan Jaff
a252acb42c
test(tracing): cover OTLP decoding and span normalization 2026-09-30 06:48:19 -07:00
Ishaan Jaff
a1c8e1e884
test(tracing): add real LangSmith deep agent OTLP export fixture 2026-09-30 06:48:19 -07:00
Ishaan Jaff
b3d9db4113
feat(tracing): add tracing and clickhouse constants 2026-09-30 06:48:19 -07:00
Ishaan Jaff
e1207f3f51
feat(logging): add clickhouse to custom logger callbacks 2026-09-30 06:48:19 -07:00
Ishaan Jaff
23e332a607
feat(logging): register the clickhouse callback 2026-09-30 06:48:19 -07:00
Ishaan Jaff
e73ab9eaf9
fix(proxy): don't parse protobuf request bodies as JSON 2026-09-30 06:48:19 -07:00
Ishaan Jaff
c5e24539a6
feat(proxy): allow virtual keys on /v1/traces routes 2026-09-30 06:48:19 -07:00
Ishaan Jaff
29b6723b01
feat(proxy): enable agent tracing from general_settings.tracing 2026-09-30 06:48:19 -07:00
Ishaan Jaff
bdc634a2d2
feat(proxy): add /v1/traces OTLP ingest and trace read endpoints 2026-09-30 06:48:19 -07:00
Ishaan Jaff
4d350c3787
feat(clickhouse): add clickhouse spend-log callback 2026-09-30 06:48:19 -07:00
Ishaan Jaff
f512c161f0
feat(clickhouse): add agent names to the per-trace rollup 2026-09-30 06:48:19 -07:00
Ishaan Jaff
eaa97090f4
refactor(tracing): move tracing types out of litellm/types 2026-09-30 06:48:19 -07:00
Ishaan Jaff
b5636a8519
docs(tracing): document the trace contract, payloads and correlation 2026-09-30 06:48:19 -07:00
Ishaan Jaff
fa00072a58
feat(tracing): export TraceReceiver from litellm.tracing 2026-09-30 06:48:19 -07:00
Ishaan Jaff
fd1dd9454e
feat(tracing): add TraceReceiver, the single entry point for ingest and reads 2026-09-30 06:48:19 -07:00
Ishaan Jaff
aff5c28a33
feat(tracing): add ClickHouse trace store with batched writes and team-scoped reads 2026-09-30 06:48:19 -07:00
Ishaan Jaff
bab0811ba1
feat(tracing): decode OTLP traces and normalize LangSmith, GenAI and OpenInference spans 2026-09-30 06:48:19 -07:00
Ishaan Jaff
3a612cc682
feat(tracing): add agent tracing types (Trace, Span, AgentNode, LiteLLMRequest) 2026-09-30 06:48:19 -07:00
Ishaan Jaff
fdaa4cf7a4
test(agent-tracing): cover ClickHouseBatchLogger batching, backpressure, retry 2026-09-30 06:48:19 -07:00
Ishaan Jaff
a82445aa53
feat(agent-tracing): add ClickHouseBatchLogger built on CustomBatchLogger 2026-09-30 06:48:19 -07:00
Ishaan Jaff
c533ebaeca
style(agent-tracing): apply ruff format to schema tests 2026-09-30 06:48:19 -07:00
Ishaan Jaff
db66e23393
style(agent-tracing): use builtin generics in agent tracing types 2026-09-30 06:48:19 -07:00
Ishaan Jaff
edc30c7de6
style(agent-tracing): apply ruff lint + format to clickhouse schema 2026-09-30 06:48:19 -07:00
Ishaan Jaff
b4743ced3d
style(agent-tracing): apply ruff lint to clickhouse writer 2026-09-30 06:48:19 -07:00
Ishaan Jaff
a8f947c5a5
style(agent-tracing): apply ruff lint + format to clickhouse client 2026-09-30 06:48:19 -07:00
Ishaan Jaff
475b25a85b
test(agent-tracing): cover clickhouse schema ddl 2026-09-30 06:48:19 -07:00
Ishaan Jaff
a1951584c6
test(agent-tracing): cover clickhouse writer batching and backpressure 2026-09-30 06:48:19 -07:00
Ishaan Jaff
2f35ca9881
feat(agent-tracing): add clickhouse schema for traces and spend logs 2026-09-30 06:48:19 -07:00
Ishaan Jaff
1b60554037
feat(agent-tracing): add bounded batched clickhouse writer 2026-09-30 06:48:19 -07:00
Ishaan Jaff
bcdb558ed2
feat(agent-tracing): add async clickhouse http client 2026-09-30 06:48:18 -07:00
Ishaan Jaff
039d757954
feat(agent-tracing): add span, spend log and trace response types 2026-09-30 06:48:18 -07:00
Ishaan Jaff
5241821b84
feat(agent-tracing): add clickhouse + otlp tracing constants 2026-09-30 06:48:18 -07:00
devin-ai-integration[bot]
314ff111e5
fix(router): carry per-request routing reads on context variables instead of public method kwargs (#43814)
Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-30 07:34:06 +00:00
devin-ai-integration[bot]
d79600987e
perf(router): honour the cooldown read interval in the routing prefetch (#43815)
Resolves LIT-9043

Co-authored-by: yassin <yassin@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-30 00:29:20 -07:00
shrey-berri
6cf51383bf
fix(params): filter internal traceback flag from provider requests (#43783) 2026-09-30 00:02:41 -07:00
yuneng-jiang
d02ff435bf
test(bedrock): accept regional aliases that inherit Converse routing (#43785)
* test(bedrock): accept regional aliases that inherit Converse routing

* test(bedrock): cover regional alias metadata independently of catalog
2026-09-29 23:30:06 -07:00
yuneng-jiang
ba6d6d1a95
test(ci): repair MCP Responses and budget fixtures (#43788)
* test(ci): repair MCP Responses and budget fixtures

* test(auth): verify delegated budget changes persist
2026-09-29 23:29:52 -07:00
devin-ai-integration[bot]
b71f02dbcf
fix(ui): keep MCP permissions visible after key, team and MCP server saves (#43810)
* fix(ui): keep MCP permissions visible after key, team and MCP server saves

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(proxy): type the object_permission include as a prisma TypedDict

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): do not block key save confirmation on cache refetch

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: ryan <ryan@berri.ai>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-09-29 22:49:20 -07:00
yuneng-jiang
7d9cc28dce
test(ci): refresh retired OpenAI tool-call models (#43676) 2026-09-29 21:53:59 -07:00
berriai-litellm-provider-info-sync[bot]
cd0ac30881
fix(cost-map): add deprecation_date to two together_ai nvidia rows (#43809)
Price-Sync: litellm-providers

Co-authored-by: berriai-litellm-provider-info-sync[bot] <328147090+berriai-litellm-provider-info-sync[bot]@users.noreply.github.com>
2026-09-29 21:24:04 -07:00
devin-ai-integration[bot]
61a73c59b0
fix(proxy): look up hashed key names with two spend log rows per key (#43656)
* fix(proxy): look up hashed key names with two spend log rows per key

The spend-log fallback for keys missing from the key table read every row per key to check that all named rows agreed, which passed the 5s statement timeout on busy keys even with the (api_key, startTime) index. Probe only the oldest and newest named row per key, so the lookup stays two index reads per key however much the key logged.

* fix(proxy): cap each spend log name probe at 100 rows per key

* fix(proxy): bound the newest-row probe at where the oldest probe stopped

The newest-row probe now starts at the row where the oldest-row probe gave up, so a key with under 200 rows in the window is read once instead of twice, and the lookup transaction turns bitmap scans off so the planner walks the (api_key, startTime) index instead of every row of a busy key when statistics or the visibility map are stale.

* test(integration): add spend log alias probe cells for the daily activity routes

---------

Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-29 20:21:55 -07:00
devin-ai-integration[bot]
8afabe81f1
fix(ui): surface x-litellm-call-id in Logs search, table and drawer (#42436)
* fix(ui): surface x-litellm-call-id in Logs search, table and drawer

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore(ui): regenerate api types for spend logs search description

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(e2e): drop redundant comments from the call id logs helpers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* style(e2e): format logs call id helper and spec

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(ui): keep one id per Logs row, move x-litellm-call-id to hover and drawer

The Request ID cell shows only request_id again. When the row's litellm_call_id
differs, the cell tooltip lists it as x-litellm-call-id with its own copy button,
and the drawer header labels the second line x-litellm-call-id: instead of the
call id caption. Stacking two ids in every row made the column noisy for the
common case where the viewer only needs the row they searched for.

* test(e2e): cover the Request ID tooltip hover and copy path

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: poll the clipboard after the tooltip copy and drop a jsdom aside

The e2e read navigator.clipboard right after the click, so a slow async write
could fail the check even though copy works. The unit test's fireEvent choice
(jsdom has no layout, so a real pointer move off the trigger closes the tooltip
before the click lands) is documented here instead of inline.

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
2026-09-30 02:16:56 +00:00
yuneng-jiang
82eb7405f5
chore(deps): bump pyjwt, moment and brace-expansion to clear osv-scan (#43792)
pyjwt 2.13.0 -> 2.14.0 (uv.lock only, pyproject floor unchanged), moment
2.30.1 -> 2.31.0 and the brace-expansion override 5.0.9 -> 5.0.12 in the
dashboard. oauthlib's only fixed release (4.0.0, 2026-09-28) is still inside
the 3-day uv exclude-newer cooldown, so its two findings are ignored until
2026-10-02
2026-09-29 19:11:39 -07:00
yuneng-jiang
d098b02ed9
fix(auth): give UI/CLI session tokens their own AES-GCM context and header-safe shape (#43790)
Some checks are pending
Unit Tests / misc (push) Waiting to run
Unit Tests: Documentation Validation / documentation (push) Waiting to run
Unit Tests: Proxy DB Operations / assert-shard-coverage (push) Waiting to run
Unit Tests: Proxy DB Operations / auth-checks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / budgets (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / custom-logging (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / db-and-spend (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / endpoints-and-responses (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / guardrails-hooks (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / jwt-and-keys (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / key-generation (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / logging-misc (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-runtime (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-server-core (push) Blocked by required conditions
Unit Tests: Proxy DB Operations / proxy-utils (push) Blocked by required conditions
Unit Tests / caching-local (push) Waiting to run
Unit Tests / core-utils (push) Waiting to run
Unit Tests / enterprise-package (push) Waiting to run
Unit Tests / enterprise-routing (push) Waiting to run
Unit Tests / integrations (push) Waiting to run
Unit Tests / All Other Providers (push) Waiting to run
Unit Tests / Vertex AI (push) Waiting to run
Unit Tests / mcp-integration (push) Waiting to run
Unit Tests / proxy-auth (push) Waiting to run
Unit Tests / proxy-endpoints (push) Waiting to run
Unit Tests / proxy-extras (push) Waiting to run
Unit Tests / proxy-server (push) Waiting to run
Unit Tests / proxy-infra (push) Waiting to run
Unit Tests / responses-caching-types (push) Waiting to run
GitHub Actions Security Analysis / zizmor (push) Waiting to run
* refactor(auth): bind UI/CLI session tokens to their own AES-GCM context

UI and CLI session tokens are now always encrypted with AES-256-GCM and a
fixed session associated-data value, and the session-token check only accepts
AES-GCM values carrying that same value. Stored secrets keep their current
encryption and decrypt unchanged, so nothing needs migrating.

encrypt_value_helper and decrypt_value_helper take an optional aad. XSalsa20
cannot bind associated data, so an AAD-bound value is always written as
AES-256-GCM, and an AAD-bound decrypt refuses the legacy format.

Session tokens issued before the upgrade stop validating, so UI and CLI users
sign in once more after upgrading.

* test(e2e): cover real SSO login through the dashboard and the lite CLI

Adds two specs under tests/e2e/ui/oidc, run by playwright.oidc.config.ts
against a live Keycloak stack. The dashboard spec checks that the SSO
session authorizes the Virtual Keys and Models data requests. The CLI
spec runs a real lite login in an isolated HOME with the keyring
disabled, then lists models and sends one chat completion with the
stored session. The main Playwright config now ignores oidc/.

* fix(auth): encode UI/CLI session tokens as unpadded base64url

Session tokens carried the v2:gcm: storage prefix and base64 padding. Basic-auth parsers split on the first colon and browsers reject ':' and '=' in WebSocket subprotocols, so Langfuse pass-through and the realtime playground could not use them

Tokens are now plain unpadded base64url, the same header-safe shape as any bearer token

* fix(auth): prefix UI/CLI session tokens with litellm_login_

A prefix-less token starts with sk- about once in 262,144 logins and is then routed as a virtual key, so that login gets a 401. The prefix also makes session tokens easy to spot in logs

The prefix doubles as the token's AES-GCM associated data, so the visible kind and the encrypted kind cannot disagree

---------

Co-authored-by: ryan-crabbe-berri <ryan@berri.ai>
2026-09-29 18:42:24 -07:00