Two-pronged fix for the reported memory leak where RSS grows from ~1.9 GiB
to ~3.5 GiB under load (1000-1500 concurrent users) and never returns to
baseline:
1. Logging._cleanup_after_logging(): Clears large payload fields (input,
original_response, raw_request_typed_dict) and streaming chunk lists from
the Logging object after all success/failure callbacks have consumed them.
This prevents per-request data (which can be kilobytes per request) from
being held in memory by async task references.
2. _periodic_memory_cleanup(): A scheduled background job (every 60s by
default, configurable via LITELLM_MEMORY_CLEANUP_INTERVAL) that runs
gc.collect() + malloc_trim(0) on Linux. Python's pymalloc allocator does
not return freed memory arenas to the OS by default; malloc_trim forces
glibc to release unused heap pages, allowing RSS to shrink after traffic
spikes subside.
Together these ensure that:
- Per-request memory is released promptly after logging completes
- The OS reclaims freed heap memory periodically, so RSS drops during idle
periods instead of staying at the peak level indefinitely
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
* feat(ui): add user filtering to usage page
Adds "User Usage" as a new view option in the usage page dropdown,
allowing admins to view and filter usage data by individual users
via the existing /user/daily/activity backend endpoint.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(ui/): working usage filtering
* fix(ui): use single-select for user filter and add tests
The user entity type's backend endpoint only accepts a single user_id,
so the filter now uses single-select mode instead of multi-select.
Added tests for the new user entity type in EntityUsage and
UsageViewSelect. Updated CLAUDE.md and AGENTS.md with guidance on
UI/backend contract consistency and test coverage for new entity types.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* revert: remove unintended package-lock.json changes
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* revert: restore package-lock.json to merge base state
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* chore: remove development setup files from repository
Removes VERTEX_ENV_SETUP.md and setup_vertex_env.sh from the
repository as they are not referenced in documentation or tests.
These files were added in PR #15824 alongside the VertexAI Search
feature. The setup information is already well-documented in the
official docs at:
docs/my-website/docs/pass_through/vertex_ai_search_datastores.md
* docs: add poetry run python usage to CLAUDE.md and AGENTS.md
