partition_spend_logs.sql and unpartition_spend_logs.sql hardcode every
index Prisma defines on LiteLLM_SpendLogs, because `LIKE ... INCLUDING
DEFAULTS INCLUDING GENERATED` copies columns but not indexes. They also
rename the old table's indexes aside first, since index names are unique
per schema and a surviving name makes `CREATE INDEX IF NOT EXISTS` a
silent no-op.
The new (api_key, startTime) index was in neither list, so an operator who
partitions (or unpartitions) after this migration lands gets a replacement
table without it, and Prisma will not recreate it: the migration is already
recorded as applied, and `migrate deploy` skips its drift sanity check when
there is nothing pending.
Verified on Postgres 18 against the shipped migration statements:
* unpatched partition script -> parent table has no api_key index, and
re-running the migration's own `CREATE INDEX IF NOT EXISTS` reports
success while being skipped, because the legacy table still owns the
name. The recovery an operator would reach for silently does nothing.
* patched -> index survives partitioning, propagates to every partition
(LiteLLM_SpendLogs_p*_api_key_startTime_idx) and to DEFAULT, is chosen
by the planner for the key+date-range query shape the endpoints use,
and survives the unpartition round-trip with all rows intact.
High-volume deployments see LiteLLM_SpendLogs grow unbounded because
retention via DELETE leaves dead tuples that autovacuum cannot reclaim
fast enough. With a range-partitioned table, retention drops whole
partitions instead: an instant metadata operation that returns disk to
the OS immediately.
The feature is gated behind general_settings.use_spend_logs_partitioning
(default false). With the flag off, the cleanup job never queries the
catalog and behaves exactly as today. With it on, the job verifies the
table is partitioned, pre-creates upcoming partitions, and drops expired
ones; expired rows the drops cannot reach (DEFAULT partition, partitions
spanning the cutoff) are still deleted row-wise so retention is never
bypassed. If the table is not partitioned it falls back to batched
DELETE only.
Converting an existing table is a manual, documented operation in
db_scripts/partition_spend_logs.sql; db_scripts/unpartition_spend_logs.sql
rolls it back. Both scripts rename the old table's indexes aside before
recreating them, since a table rename keeps the schema-unique index names
and would otherwise silently skip the CREATE INDEX IF NOT EXISTS block.
Granularity and pre-create lookahead are tunable via
SPEND_LOG_PARTITION_INTERVAL (day/week/month, invalid values fall back to
day) and SPEND_LOG_PARTITION_PRECREATE_AHEAD.
