litellm/litellm-proxy-extras/litellm_proxy_extras
ishaan-berri 118ce3cc91
feat: add model leaderboard page (#43649)
* feat(proxy): add model leaderboard analytics

* feat: add model insights task and range constants

* feat: record task type from task tags in model usage rollup

* feat: serve 365 days of model insights by UTC date

* test: cover task tag resolution in model usage rollup

* test: update model insights range limit test to 365 days

* chore: regenerate dashboard api types for model insights

* feat: add model insights aggregation helpers

* test: cover model insights aggregation helpers

* feat: redesign model leaderboard with stacked bars, treemap and ranking

* test: update model leaderboard view test

* feat: mark model leaderboard as beta in sidebar

* chore: sync schema.prisma copies from root

* fix: only treat task: prefixed tags as model insight tasks

* feat: add metric type for model insights ranking

* fix: rank model insights by selected metric and scope detail queries to ranked deployments

* test: plain tags are not model insight tasks

* test: cover metric ranking, deployment scoping and rollup round trip

* fix: build model insights weeks and halves from the requested date range

* test: cover empty weeks and range-based change comparison

* fix: refetch by metric, show load errors and ignore stale responses

* test: cover metric refetch and error state

* feat: define model insight tasks in a JSON file

* feat: return task labels and categories from model insights

* feat: load model insight tasks from JSON

* refactor: validate rollup task tags against the JSON task list

* feat: serve the task list with model insights

* refactor: drop hardcoded task list from constants

* build: ship model insight tasks JSON in the wheel

* test: cover model insight task JSON

* refactor: take task labels and categories from the API

* test: pass task info to task tile builder

* refactor: color treemap by API-provided category

* test: include tasks in model leaderboard fixture

* fix: make daily model usage migration idempotent

* feat: bound the model insights task query size

* fix: compute task breakdown independent of the chart metric

* test: task breakdown is stable across chart metrics

* chore: regenerate lazy openapi snapshot for model insights

* chore: regenerate dashboard api types for model insights

* fix: keep previous ranking dimmed while a new metric loads

* test: cover stale metric state in model leaderboard

* refactor: drop task row cap constant

* fix: return the full task breakdown instead of a truncated one

* test: task query is not truncated

* feat: add task summary types for model insights

* feat: summarise tasks server-side on a separate model insights endpoint

* test: cover the model insights tasks endpoint

* chore: regenerate lazy openapi snapshot for model insights tasks

* chore: regenerate dashboard api types for model insights tasks

* refactor: drop client-side task aggregation

* test: remove client-side task aggregation tests

* feat: load task breakdown separately from the chart metric

* test: task breakdown is not refetched on chart metric change

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-09-28 19:40:04 -07:00
..
migrations feat: add model leaderboard page (#43649) 2026-09-28 19:40:04 -07:00
__init__.py install prisma migration files - connects litellm proxy to litellm's prisma migration files (#9637) 2025-03-29 15:27:09 -07:00
_logging.py fix(proxy): honor LITELLM_LOG for uvicorn and proxy extras loggers 2026-09-15 21:55:37 +00:00
migration_lock.py fix(proxy): coordinate v2 migration startup and add container regression CI 2026-09-12 18:24:59 -07:00
migration_recovery.py fix(proxy): coordinate v2 migration startup and add container regression CI 2026-09-12 18:24:59 -07:00
prisma_toolchain.py fix(proxy): coordinate v2 migration startup and add container regression CI 2026-09-12 18:24:59 -07:00
py.typed fix(lint): bring basedpyright rule counts back under their budget limits 2026-08-05 10:23:02 -07:00
replica_identity.py fix(proxy-extras): kill the whole Prisma process group when a command times out 2026-09-02 18:29:55 -07:00
schema.prisma feat: add model leaderboard page (#43649) 2026-09-28 19:40:04 -07:00
utils.py test: isolate migration CI selection and exercise resolver boundaries 2026-09-12 18:49:58 -07:00