scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-09 03:18:44 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 109
litellm/docs
History
Exact
AlexsanderHamir 5910750ed6 fix: prevent memory blowout in LoggingWorker 
Previously, tasks were executed sequentially, awaiting each task before
starting the next. With large queues (10k+ tasks), this caused objects to
accumulate in memory, holding references to heavy resources and leading
to memory blowouts.

This update:

1. Introduces a semaphore to allow a configurable number of concurrent tasks,
   improving throughput and preventing queue buildup.
2. Implements a configurable cleaning mechanism for when the queue reaches its
   limit, ensuring tasks are not dropped.
2025-11-21 11:31:57 -08:00
..
my-website fix: prevent memory blowout in LoggingWorker 2025-11-21 11:31:57 -08:00