litellm/litellm-rust/crates
ishaan-berri 03ba79d261
feat(lens): show end-user feedback on traces, stored in ClickHouse (#45171)
* feat(lens): add lens_feedback ClickHouse table for human trace scores

ReplacingMergeTree(UpdatedAt, IsDeleted) keyed like agent_traces_by_key so
feedback joins traces in sort order, with a bloom filter on TraceId, a score
CHECK, and the same retention as traces

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(lens): add scoped ClickHouse reads for trace feedback

feedback_target resolves a visible trace's team, key and trace_ref before a
write; feedback lists the latest live entry per author; feedback_summary
returns count, average and lowest score for a batch of traces

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(lens): pin the Claude session to trace id hash shared with feedback

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(lens): expose feedback queries to the Python trace bridge

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(lens): add typed feedback models and ClickHouse feedback store

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(lens): add /lens/feedback API for 0-10 human scores on traces

PUT saves or replaces the caller's score and comment, GET lists every
entry, DELETE removes the caller's, POST /summary feeds the trace list.
Callers can target a trace_id or a Claude Code session_id

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(lens-ui): add typed feedback calls to the traces API

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(lens-ui): flag rated traces and filter the list by feedback

A Feedback column shows each run's average score and rating count, marked
red when any score is 4 or below, and a filter narrows to rated or
low-score runs through the URL

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(lens-ui): show and edit human feedback inside a trace

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(lens): let app keys post end-user feedback on their own traces

Any key can now PUT/DELETE feedback on traces its team or key sent, naming
the end user in an optional user field. Reading feedback stays with Lens
admins

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* feat(lens-ui): show end-user feedback first in a run and flag low-scored rows

Replace the edit popover and feedback filter with a read-only view: the
list shows each run's score in a fixed-width badge and tints the row red
when a user scored it 4 or below, and opening a run shows what users said
with their score right under the header

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 20:14:30 -07:00
..
auth refactor(rust): prepare inference and auth foundations for the gateway (#43287) 2026-09-25 23:12:48 -07:00
auth-aws refactor(rust): prepare inference and auth foundations for the gateway (#43287) 2026-09-25 23:12:48 -07:00
auth-azure refactor(rust): share anthropic types, request helpers, and streaming contracts across crates (#43426) 2026-09-27 14:53:12 -07:00
auth-gcp refactor(rust): share anthropic types, request helpers, and streaming contracts across crates (#43426) 2026-09-27 14:53:12 -07:00
auth-types refactor(rust): centralize host execution and compose callbacks (#43515) 2026-09-28 19:20:27 +00:00
cache fix(caching): skip the cache past max_messages and keep tool_result text in semantic prompts (#43878) 2026-10-07 17:32:39 +00:00
cache-azure-blob feat: improve trace ingestion and trace details (#43975) 2026-10-01 13:45:33 -07:00
cache-disk feat(rust): add litellm-db and litellm-db-testing workspace scaffolding (#43504) 2026-09-27 18:48:45 -07:00
cache-gcs feat: improve trace ingestion and trace details (#43975) 2026-10-01 13:45:33 -07:00
cache-memory refactor(rust): align the cache crates with Python and wire every native backend (#42530) 2026-09-22 13:13:02 -07:00
cache-qdrant-semantic refactor(http): hand out an owned Client and route all providers through the pool (#43245) 2026-09-25 18:31:15 -07:00
cache-redis refactor(rust): align the cache crates with Python and wire every native backend (#42530) 2026-09-22 13:13:02 -07:00
cache-redis-semantic refactor(rust): align the cache crates with Python and wire every native backend (#42530) 2026-09-22 13:13:02 -07:00
cache-response feat: improve trace ingestion and trace details (#43975) 2026-10-01 13:45:33 -07:00
cache-s3 feat: improve trace ingestion and trace details (#43975) 2026-10-01 13:45:33 -07:00
cache-testing refactor(rust): align the cache crates with Python and wire every native backend (#42530) 2026-09-22 13:13:02 -07:00
cache-valkey-semantic refactor(rust): align the cache crates with Python and wire every native backend (#42530) 2026-09-22 13:13:02 -07:00
callbacks-legacy-python fix(lens): price agent traces by joining gen_ai.response.id to spend logs (#44738) 2026-10-06 19:55:26 +00:00
config fix(tracing): unify ClickHouse storage configuration (#43941) 2026-10-02 16:31:26 +00:00
core-utils refactor(rust): add shared llms wire type derives (#43730) 2026-09-29 09:19:33 -07:00
coroutine refactor(rust): extract the host coroutine into its own crate (#43129) 2026-09-25 04:20:06 +00:00
cost fix(cost_calculator): bill ultrafast prompts above 272k at the ultrafast long-context rates (#43764) 2026-09-30 07:32:16 -07:00
db feat(rust): add litellm-db and litellm-db-testing workspace scaffolding (#43504) 2026-09-27 18:48:45 -07:00
db-testing feat(rust): add litellm-db and litellm-db-testing workspace scaffolding (#43504) 2026-09-27 18:48:45 -07:00
framer refactor(rust): rename litellm-framing crate to litellm-framer (#44683) 2026-10-05 22:32:00 +00:00
gateway chore(rust): prune inference deps and rewrite layering docs (#44836) 2026-10-06 07:17:27 -07:00
gateway-auth fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
gateway-inference chore(rust): prune inference deps and rewrite layering docs (#44836) 2026-10-06 07:17:27 -07:00
gateway-management feat(rust): add virtual key storage contracts (#43468) 2026-09-27 19:18:59 -07:00
gateway-mcp feat(rust): add the MCP gateway (#43470) 2026-09-27 19:37:41 -07:00
gateway-ui feat(ui): adopt the new LiteLLM logo and monogram (#43913) 2026-09-30 14:51:06 -07:00
host feat(cache): select Rust caching through explicit cache objects (#43601) 2026-09-29 00:01:44 +00:00
host-http refactor(rust): centralize host execution and compose callbacks (#43515) 2026-09-28 19:20:27 +00:00
host-native feat(cache): select Rust caching through explicit cache objects (#43601) 2026-09-29 00:01:44 +00:00
host-python refactor(rust-bridge): share field and response marshaling (#45187) 2026-10-07 17:22:23 -07:00
http feat(rust): add the MCP gateway (#43470) 2026-09-27 19:37:41 -07:00
inference refactor(rust): extract inference-testing crate (#44873) 2026-10-06 16:33:06 +00:00
inference-chat refactor(rust): extract inference-testing crate (#44873) 2026-10-06 16:33:06 +00:00
inference-messages refactor(rust-bridge): unify native call inputs and Messages settings (#45126) 2026-10-07 15:05:43 -07:00
inference-ocr refactor(rust): extract inference-testing crate (#44873) 2026-10-06 16:33:06 +00:00
inference-responses refactor(rust): extract inference-testing crate (#44873) 2026-10-06 16:33:06 +00:00
inference-testing refactor(rust): extract inference-testing crate (#44873) 2026-10-06 16:33:06 +00:00
inference-transcription refactor(rust): extract inference-testing crate (#44873) 2026-10-06 16:33:06 +00:00
lens feat(lens): show end-user feedback on traces, stored in ClickHouse (#45171) 2026-10-07 20:14:30 -07:00
litellm refactor(rust): expand logging and test coverage across gateway and Anthropic messages (#43295) 2026-09-26 08:04:19 +00:00
llms chore(rust): prune inference deps and rewrite layering docs (#44836) 2026-10-06 07:17:27 -07:00
llms-types chore(rust): prune inference deps and rewrite layering docs (#44836) 2026-10-06 07:17:27 -07:00
model-catalog fix(model_prices): consolidate claude-haiku-5-5 over-100k pricing and capability flags (#45151) 2026-10-07 14:14:47 -07:00
python-bridge refactor(rust-bridge): share field and response marshaling (#45187) 2026-10-07 17:22:23 -07:00
python-compat feat(rust): add python-compat crate for Python data formats (#42510) 2026-09-22 12:16:56 -07:00
router refactor(rust-bridge): unify native call inputs and Messages settings (#45126) 2026-10-07 15:05:43 -07:00
secrets feat: improve trace ingestion and trace details (#43975) 2026-10-01 13:45:33 -07:00
secrets-aws feat: improve trace ingestion and trace details (#43975) 2026-10-01 13:45:33 -07:00
secrets-azure feat: improve trace ingestion and trace details (#43975) 2026-10-01 13:45:33 -07:00
secrets-cyberark feat: improve trace ingestion and trace details (#43975) 2026-10-01 13:45:33 -07:00
secrets-google feat: improve trace ingestion and trace details (#43975) 2026-10-01 13:45:33 -07:00
secrets-hashicorp feat: improve trace ingestion and trace details (#43975) 2026-10-01 13:45:33 -07:00
secrets-types refactor(rust): prepare inference and auth foundations for the gateway (#43287) 2026-09-25 23:12:48 -07:00
storage-clickhouse perf(lens): prune ClickHouse partitions when sampling and sample in one pass (#45087) 2026-10-07 12:24:25 -07:00
testkit refactor(http): hand out an owned Client and route all providers through the pool (#43245) 2026-09-25 18:31:15 -07:00
token-counter refactor(rust): add shared llms wire type derives (#43730) 2026-09-29 09:19:33 -07:00
token-counter-fast refactor(rust): share anthropic types, request helpers, and streaming contracts across crates (#43426) 2026-09-27 14:53:12 -07:00
token-counter-huggingface refactor(rust): share anthropic types, request helpers, and streaming contracts across crates (#43426) 2026-09-27 14:53:12 -07:00
token-counter-tiktoken refactor(rust): share anthropic types, request helpers, and streaming contracts across crates (#43426) 2026-09-27 14:53:12 -07:00
traces feat(lens): show end-user feedback on traces, stored in ClickHouse (#45171) 2026-10-07 20:14:30 -07:00
traces-cache feat(lens): show agent, user and slack thread first in the run header (#45261) 2026-10-07 20:01:55 -07:00
traces-clickhouse feat(lens): show end-user feedback on traces, stored in ClickHouse (#45171) 2026-10-07 20:14:30 -07:00
tracing feat(rust): add structured route lifecycle tracing (#43466) 2026-09-27 19:18:58 -07:00