mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-12 23:01:41 +00:00
7a9a9f0c79
* fix: batch-limit stale managed object cleanup to prevent 300K row UPDATE (#25257) * Add STALE_OBJECT_CLEANUP_BATCH_SIZE constant Configurable batch limit (default 1000) for stale managed object cleanup, preventing unbounded UPDATE queries from hitting 300K+ rows at once. * Batch-limit stale managed object cleanup with single bounded SQL query Two fixes to _cleanup_stale_managed_objects: 1. Replace unbounded update_many with a single execute_raw using a subquery LIMIT, capping each poll cycle to STALE_OBJECT_CLEANUP_BATCH_SIZE rows. Zero rows loaded into Python memory — everything stays in Postgres. Uses the same PostgreSQL raw-SQL pattern as spend_log_cleanup.py (the proxy requires PostgreSQL per schema.prisma). 2. Extract _expire_stale_rows as a separate method for testability. Keeps the file_purpose='response' filter to avoid incorrectly expiring long-running batch or fine-tune jobs that legitimately exceed the staleness cutoff. * docs: add STALE_OBJECT_CLEANUP_BATCH_SIZE to env vars reference * test: remove deprecated embed-english-v2.0 cohere embedding tests |
||
|---|---|---|
| .. | ||
| auth | ||
| common_utils | ||
| hooks | ||
| management_endpoints | ||
| vector_stores | ||
| __init__.py | ||
| audit_logging_endpoints.py | ||
| enterprise_routes.py | ||
| proxy_server.py | ||
| readme.md | ||
| utils.py | ||
LiteLLM Proxy Enterprise Features - Readme
Overview
This directory contains enterprise features used on the LiteLLM proxy.
Format
Create a file for every group of endpoints (e.g. key_management_endpoints.py, user_management_endpoints.py, etc.)
If there is a broader semantic group of endpoints, create a folder for that group (e.g. management_endpoints, auth_endpoints, etc.)