scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-06 02:48:13 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 103
litellm/ui/litellm-dashboard
History
Zachary Lyon 19c13ac971
feat(proxy): add TinyFish Agent API passthrough with per-step billing (#41099) 
* feat(proxy): add TinyFish Agent API passthrough with per-step billing

* chore(ui): regenerate dashboard API types for /tinyfish passthrough

* fix(proxy): satisfy strict lint budget for tinyfish passthrough

* style: ruff format tinyfish passthrough handler

* test(proxy): exercise tinyfish route through the app with a faked upstream

* refactor(proxy): make cross-module tinyfish billing hooks public

* fix(proxy): tolerate transient tinyfish poll failures instead of dropping the charge

* fix(proxy): defer billing for disconnected tinyfish SSE runs to the background poller

* Revert "fix(proxy): defer billing for disconnected tinyfish SSE runs to the background poller"

This reverts commit ef0bcfb4a0.

* fix(proxy): bill tinyfish SSE runs via detached poller and only COMPLETED runs

Disconnected run-sse clients previously left completed runs unbilled: the stream-end
handler saw a still-RUNNING run and logged $0. The poller now spawns from the streaming
path on the first run_id frame, outlives the disconnect, and writes the one spend row when
the run turns terminal; the stream-end path only logs the $0 fallback for run_id-less
streams. Costs now apply only to COMPLETED runs ($0 for FAILED/CANCELLED, matching the
upstream invoice), spend rows carry the request's litellm_call_id (previously NULL
request_ids collided and were silently dropped), and the GET /v1/runs listing is blocked
so callers behind the shared key cannot discover each other's runs.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pxz11TQKYvCXvKawNUa2g4

* fix(proxy): drop GET /v1/runs from the tinyfish allowlist error message

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pxz11TQKYvCXvKawNUa2g4

* chore(proxy): sync openapi artifacts for tinyfish docstring, suppress LIT011 on flag write

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pxz11TQKYvCXvKawNUa2g4

* style(proxy): ruff-format the sse poller flag write

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pxz11TQKYvCXvKawNUa2g4

* style(proxy): keep the rebind-ok suppression on the flag write's own line

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pxz11TQKYvCXvKawNUa2g4

* fix(proxy): harden tinyfish billing paths from review findings

Skip failure dispatch when the SSE poller owns billing (a failure row collided with the
poller's billed row on request_id and dropped the charge), late-spawn the poller for
run_ids that arrive in unterminated frames instead of mispricing RUNNING runs at $0,
thread litellm_params into poller-billed standard logging objects so SLO consumers see
attribution, untype the run error field so upstream error-shape drift cannot void a
billable run, normalize a schemeless TINYFISH_AGENT_API_BASE, extend the poll budget to
cover queue wait (3600s) with ~60s outage tolerance, and log poller cancellation.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pxz11TQKYvCXvKawNUa2g4

* chore(proxy): satisfy ratcheted BLE001/LIT002 budgets from main in tinyfish handler

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pxz11TQKYvCXvKawNUa2g4

* style(proxy): drop stray blank line from merge resolution

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pxz11TQKYvCXvKawNUa2g4

* fix(proxy): reject passthrough envelope controls on tinyfish route, raise blocking-run timeout

The generic passthrough unwraps a caller-supplied custom_body as the forwarded request and
honors a caller stream flag, so custom_body.use_vault bypassed the credentialed-run 403 and
stream: true flipped a blocking run into the streaming pipeline. The route now 400s the
envelope fields (custom_body, stream, query_params); streaming comes from the endpoint.
Blocking runs also get a 1500s default timeout covering the upstream 1200s run cap, unless
the operator configured pass_through_request_timeout.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pxz11TQKYvCXvKawNUa2g4

* style(proxy): resolve operator timeout without a dict-literal default

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pxz11TQKYvCXvKawNUa2g4

* test(passthrough): list the TinyFish route among protocol-constrained pass-through routes

* chore(proxy): regenerate the lazy OpenAPI snapshot after merging main

* chore(proxy): keep the lazy OpenAPI snapshot as CI's Python 3.12 renders it

* fix(proxy): reject TinyFish POST bodies that are not a JSON object

A form-encoded or text body carried stream and use_vault past both field
gates, because the gates only saw fields the body parsed to as JSON. The
route now checks the content type before reading the body and answers 400
for anything that is not a JSON object.

* fix(tinyfish): reject submit paths with extra slashes so run-async always bills

The allowlist dropped empty path segments, so POST /v1/automation/run-async/
was forwarded upstream while the billing dispatch only recognises the exact
path and would have logged the submit at $0 without starting the poller.
Any path with a trailing or doubled slash now returns 403 before forwarding.

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: mateo-berri <277851410+mateo-berri@users.noreply.github.com>
2026-09-21 21:21:43 -07:00
..
public feat(edenai): add Eden AI as a provider 2026-09-15 06:50:43 +01:00
scripts fix(ui): lint z-index utilities behind arbitrary Tailwind variants 2026-08-25 17:34:45 -07:00
src feat(proxy): add TinyFish Agent API passthrough with per-step billing (#41099) 2026-09-21 21:21:43 -07:00
tests fix(ui): fall back to user id in Top Virtual Keys user column 2026-09-19 11:04:46 -07:00
.env.development
.env.production
.npmrc chore(build): move the Admin UI toolchain to Node 24 (#35801) 2026-08-04 12:36:07 -07:00
.nvmrc chore(build): move the Admin UI toolchain to Node 24 (#35801) 2026-08-04 12:36:07 -07:00
.prettierignore
.prettierrc
AGENTS.md chore: consolidate CLAUDE.md into AGENTS.md 2026-09-19 02:30:35 +00:00
build_release_ui.sh
build_ui.sh chore(build): move the Admin UI toolchain to Node 24 (#35801) 2026-08-04 12:36:07 -07:00
build_ui_custom_path.sh chore(build): move the Admin UI toolchain to Node 24 (#35801) 2026-08-04 12:36:07 -07:00
components.json refactor(ui): switch shadcn primitives from Radix to Base UI (#32124) 2026-07-07 09:55:41 -07:00
eslint-budgets.json refactor(ui): render the Virtual Keys page without the legacy user dashboard 2026-09-05 17:49:29 -07:00
eslint-suppressions.json Merge pull request #39308 from BerriAI/litellm_ui_per_second_video_pricing 2026-09-18 10:05:26 -07:00
eslint.config.mjs test(ui): budget DOM-structure assertions in dashboard tests 2026-09-01 00:13:05 -07:00
knip.json fix(ui): register type-test files as knip entry points 2026-08-10 15:42:09 -07:00
next.config.mjs fix(ui): retain compatible build and test commands 2026-09-08 15:54:13 -07:00
package-lock.json chore(ui): bump smol-toml to fix GHSA-7w5x-hrqm-74c2 osv-scan failure 2026-09-10 00:46:54 +00:00
package.json fix(ui): align SDK mocks and remaining dependency patches 2026-09-08 16:03:34 -07:00
postcss.config.js feat(ui): shadcn migration foundation: Tailwind v4, shadcn init, antd cascade fix (#31995) 2026-07-02 19:02:27 -07:00
README.md
tsconfig.json test(e2e): move Admin UI Playwright suite to tests/e2e/ui (#34196) 2026-07-22 19:43:10 +00:00
tsconfig.tsbuildinfo fix(auto_router): clear the non-reasoning tier when the classifier changes 2026-09-08 12:30:42 -07:00
vitest.config.ts test(ui): split the vitest suite into unit, component, integration and type projects (#37488) 2026-08-19 20:40:14 +00:00

README.md

This is a Next.js project bootstrapped with create-next-app.

Getting Started

First, run the development server:

npm run dev
# or
yarn dev
# or
pnpm dev
# or
bun dev

Open http://localhost:3000 with your browser to see the result.

You can start editing the page by modifying app/page.tsx. The page auto-updates as you edit the file.

This project uses next/font to automatically optimize and load Inter, a custom Google Font.

Learn More

To learn more about Next.js, take a look at the following resources:

You can check out the Next.js GitHub repository - your feedback and contributions are welcome!

Deploy on Vercel

The easiest way to deploy your Next.js app is to use the Vercel Platform from the creators of Next.js.

Check out our Next.js deployment documentation for more details.