mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-13 23:11:40 +00:00
2e2b2624ed
Fold each auto-routed turn into a per-(api_key, session, auto-router) row when it happens, and serve the benchmarks dashboard by summing those rows. Nothing in the feature reads LiteLLM_SpendLogs. The row carries what each tier the session used left in the prompt cache, so a turn's bucket is a question about one model's own record and the upsert answers it against the row it is already writing. Absent from that record means a first visit; present means warm or expired, on one comparison of the idle gap against the TTL the entry was written with. Nothing is read before the write, so there is no state to load or validate, and the statement is atomic. |
||
|---|---|---|
| .. | ||
| routes | ||
| Dockerfile | ||
| main.py | ||