mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-11 08:38:24 +00:00
9c4096d0e3
The session view grouped every row in the date window to show 50 sessions and count them, so each page read the whole window. Read the newest rows that match the filters instead, group those, and keep only sessions that are complete in what was read: a session whose last activity is newer than the oldest row read has every row in the read. The first read is cap + offset + page size + 2 rows, and at most 2 x cap + 2 for a page past the cap, so a huge page number cannot overflow the read's LIMIT. It grows 8x, then 64x, then reads everything, only while the page or the count is not settled; a read that does not settle is discarded, so results are the same as before and the count still stops at 10,000. Ascending order still reads every row in one query. On a busy key, where few long sessions fill the newest rows, a read that the last read's rows per session say cannot hold enough sessions is skipped. An offset page's read that holds no session of the page hands its sessions to the next read, which reads only the rows at or below that read's oldest row, up to the rest of its budget. So a deep page reads each row about once instead of re-reading the newest rows at every step: the last page of a key with 2,000 sessions of 50 rows read about 307-313k rows before and about 195-205k now, against 205k for grouping every row. A descending cursor page reads from the cursor's startTime down and drops sessions that also have a newer row. When the sessions after the cursor do not pass the cap, the window's count is read separately from the newest rows, starting at the first read size that the last read's rows per session say can pass the cap, so an unindexed filter such as key_alias or status does not pay a full scan per step. The status filter keeps or drops whole sessions, so the newer-row probe skips it for sessions already on the page. The representative fetch checks it again on just those sessions, since a row written after the page read can change a session's outcome; as before, such a session is dropped from the page. Its subquery gets the date window as literals: Prisma reuses prepared statements, and from the 6th run Postgres plans a $N window as tiny, which turned the subquery into a ~2 min nested loop. The literal is built only from a UTC datetime; anything else raises. |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| test_background_interaction_settlement.py | ||
| test_baseline_accounting.py | ||
| test_budget_reservation.py | ||
| test_budget_reservation_redis_failure.py | ||
| test_carried_budget_state.py | ||
| test_cloudzero_endpoints.py | ||
| test_compression_savings.py | ||
| test_input_tokens.py | ||
| test_key_metadata_recovery.py | ||
| test_ptu_feature_flag.py | ||
| test_ptu_flat_cost_rollup.py | ||
| test_savings.py | ||
| test_search_api_logging.py | ||
| test_spend_capture_rate.py | ||
| test_spend_counter_batch.py | ||
| test_spend_event.py | ||
| test_spend_event_producer.py | ||
| test_spend_log_error_logger.py | ||
| test_spend_management_endpoints.py | ||
| test_spend_query_optimization.py | ||
| test_spend_tracking_utils.py | ||