litellm/.github
ishaan-berri 5fb23eabd8
feat(lens): add datasets built from real traces (#44765)
* feat(lens): add dataset case and size limits

* feat(lens): add dataset, case and build models

* feat(lens): build dataset cases from traces, findings and text

* feat(lens): store dataset revisions insert-only

* feat(lens): add dataset routes for build, save, export and eval cases

* feat(lens): mount the dataset router before lens routes

* feat(lens): add LiteLLM_LensDataset table

* feat(lens): add LiteLLM_LensDataset table to proxy schema

* feat(lens): add LiteLLM_LensDataset table to extras schema

* feat(lens): add migration that creates the dataset table

* test(lens): cover dataset case building, dedupe and limits

* test(lens): cover dataset revisions, conflicts and eval cases

* chore(ui): regenerate API types for lens datasets

* feat(lens): add dataset UI types

* feat(lens): add datasets API client

* feat(lens): add dataset query and mutation hooks

* feat(lens): add case selection and expected edit logic

* test(lens): cover case selection and expected edits

* feat(lens): add the add to dataset dialog

* test(lens): cover saving picked cases from the dialog

* feat(lens): add datasets list

* feat(lens): add dataset detail with revisions and export

* test(lens): cover editing, revisions and export in datasets tab

* feat(lens): expose datasets on the lens API

* feat(lens): add in-memory datasets for demo mode

* feat(lens): wire demo datasets into the demo lens API

* feat(lens): add optional lens API hook

* feat(lens): add optional onboarding hook

* feat(lens): add datasets tab and dataset routing

* feat(lens): show the datasets tab

* feat(lens): add to dataset from the trace header

* feat(lens): add a single turn to a dataset from a step

* feat(lens): add finding evidence to a dataset

* docs(lens): add datasets screenshots for the PR

* test(lens): cover dataset revision storage against Postgres

* test(lens): cover dataset trace paging, findings and route errors

* test(lens): cover dataset build fallbacks and no_content skips

* fix(lens): register the dataset table for postgres span names

* feat(lens): add invalid skip reason for unparseable case lines

* fix(lens): keep valid JSONL cases, provenance and size limits; stop reading past the case cap

* test(lens): cover malformed JSONL, re-import provenance, size fields and early cap

* chore(ui): regenerate API types for the invalid skip reason

* feat(lens): show text for the invalid skip reason

* feat(lens): render datasets in the same inspector table as investigations

* test(lens): open a dataset by clicking its table row

* docs(lens): update the datasets list screenshot

* style(lens): format the datasets table

* fix(lens): normalize JSON text span input and output into messages when building cases

* test(lens): cover JSON text span normalization for dataset cases

* Revert "fix(lens): normalize JSON text span input and output into messages when building cases"

This reverts commit 29c3e34962fa12427dbd516c08f2599db997a905.

* fix(traces): normalize agent assistant summaries into UI messages

* feat(lens): add dataset case view helpers built on the trace parsers

* test(lens): cover dataset case view helpers

* feat(lens): show dataset cases in an inspector table

* feat(lens): open a dataset case in a side panel with trace message cards

* feat(lens): rebuild the dataset page header and layout

* feat(lens): keep the open dataset case in the URL

* test(lens): drive dataset edits through the case table and panel

* docs(lens): update dataset view screenshots

* Revert "test(lens): cover JSON text span normalization for dataset cases"

This reverts commit 30229ae8ae90265c09d755c43c1f1eeaa66050c7.

* fix(lens): import StateMessage from shared in dataset detail

* fix(lens): import StateMessage from shared in datasets list

* fix(lens): give the datasets migration a unique timestamp after review checkpoints
2026-10-06 23:04:43 +00:00
..
actions ci: split slow unit shards and build the Rust bridge once per run (#44622) 2026-10-05 23:43:01 +00:00
assets feat(lens): add datasets built from real traces (#44765) 2026-10-06 23:04:43 +00:00
codeql ci(codeql): retain Python log-injection coverage 2026-09-14 16:09:27 -07:00
e2e-stack ci: move Postgres, MCP and Redis suites to CircleCI integration (#44453) 2026-10-05 09:33:14 -07:00
ISSUE_TEMPLATE ci: classify new issues into domain, provider, kind, priority and lift labels 2026-09-18 11:37:55 -07:00
observatory Add observatory test workflow for RC/stable releases 2026-03-01 15:30:09 -03:00
prompts ci: classify new issues into domain, provider, kind, priority and lift labels 2026-09-18 11:37:55 -07:00
PULL_REQUEST_TEMPLATE fix(security): remove the publicly known master key from the repo (#44718) 2026-10-06 10:55:24 -07:00
screenshots fix(team_endpoints): auto-add SSO team members to org on move (proxy admin only) (#26377) 2026-04-24 08:36:25 -07:00
scripts ci: run unit selections from GHA test-path and drop the CircleCI unit jobs (#44461) 2026-10-05 10:49:16 -07:00
workflows fix(lens): use async-timeout on python 3.10 for budget reservation timeouts (#44911) 2026-10-06 14:19:37 -07:00
ci-coverage-allowlist.yml ci: move Postgres, MCP and Redis suites to CircleCI integration (#44453) 2026-10-05 09:33:14 -07:00
CODEOWNERS chore(codeowners): drop UI, migration, and CODEOWNERS self owners (#43653) 2026-09-29 01:48:22 +00:00
dependabot.yaml chore: fixes 2026-04-05 01:30:57 -07:00
deploy-on-aws.png feat: add LiteLLM Rust workspace with Mistral OCR bridge (#31033) 2026-06-23 13:16:47 -07:00
deploy-on-gcp.png feat: add LiteLLM Rust workspace with Mistral OCR bridge (#31033) 2026-06-23 13:16:47 -07:00
deploy-to-aws.png Add files via upload 2023-10-25 16:33:53 -07:00
FUNDING.yml Update FUNDING.yml 2023-09-22 09:51:35 -07:00
issue-labels.json ci(issue-classifier): name every workflow, script and job after the issue it works on 2026-09-18 11:37:55 -07:00
merge-smoke-tests.json test(proxy): move auth, hooks, policy_engine and client tests into tests/unit/proxy (#43998) 2026-10-01 10:11:45 -07:00
mutmut-coverage.rc fix(ci): let the mutation workflow find covered lines so it generates mutants 2026-08-25 23:16:40 -07:00
template.yaml fix(proxy): drop legacy telemetry key from persisted WORKER_CONFIG before initialize 2026-09-20 03:14:11 +00:00