mirror of
https://github.com/BerriAI/litellm.git
synced 2026-11-06 07:48:06 +00:00
f94bee8d4f
Retention cleanup could delete up to 500,000 rows per table per run with no wall-clock bound and no per-statement timeout, so a large backlog turned into a single long, lock-holding run. A run now carries a wall-clock budget shared across every table it prunes, each delete batch runs under a Postgres statement_timeout and lock_timeout, and batch size, batch cap, budget and timeout are all operator-configurable from general_settings and the admin dashboard. Prometheus metrics report rows deleted, batch duration, remaining eligible rows, batch failures, and why each run ended. The budget is a hard bound rather than an advisory one. Every statement is issued with its timeout clamped to the budget that is left, so a batch started just under the deadline is cancelled at the deadline instead of running a further batch timeout past it, and no statement is issued once the budget is spent. Measured against a real Postgres with a deliberately slow delete, a 5s budget ran 20.8s before and 5.0s after. Partition maintenance is DDL that cannot be cut short, so it is only started while budget remains Those four bounds also reach the running job. The scheduler holds one long-lived SpendLogCleanup, so they are re-read at the start of every run rather than captured at construction, and the runtime config reload copies them into general_settings, which it previously did only for the retention period itself. Clearing one from the dashboard restores its shipped default, while a bound set in config.yaml is left alone Partition maintenance is bounded too. Every statement it issues, the two catalog lookups as well as the CREATE and DROP, carries a timeout, and the DDL also carries a lock_timeout so a DROP that cannot take its ACCESS EXCLUSIVE lock gives up rather than queueing behind a long-running reader for as long as that reader lives. The bound is injected as a callable rather than a number, so it is re-read before every statement and a loop stops once the budget is gone instead of issuing one statement per partition against a bound taken before the first of them Clearing a dashboard override of a bound that config.yaml also declares now restores the YAML value rather than leaving the deleted override in memory Saving the form no longer asks the proxy to clear a field that was never stored. That request answers 400 when no general_settings row exists, which is every deployment that has not saved these settings, and it aborted the whole save Resolves LIT-5439 |
||
|---|---|---|
| .. | ||
| agent_tests | ||
| audio_tests | ||
| base_sdk_tests | ||
| basic_proxy_startup_tests | ||
| batches_tests | ||
| benchmarks | ||
| code_coverage_tests | ||
| documentation_tests | ||
| e2e | ||
| enterprise | ||
| guardrails_tests | ||
| image_gen_tests | ||
| integration | ||
| litellm | ||
| litellm-proxy-extras | ||
| litellm_core_utils | ||
| litellm_utils_tests | ||
| llm_responses_api_testing | ||
| llm_translation | ||
| load_tests | ||
| local_testing | ||
| logging_callback_tests | ||
| mcp_tests | ||
| multi_instance_e2e_tests | ||
| ocr_tests | ||
| old_proxy_tests/tests | ||
| openai_endpoints_tests | ||
| otel_tests | ||
| pass_through_tests | ||
| pass_through_unit_tests | ||
| proxy_admin_ui_tests | ||
| proxy_behavior | ||
| proxy_e2e_anthropic_messages_tests | ||
| proxy_migration_tests | ||
| proxy_security_tests | ||
| proxy_unit_tests | ||
| router_unit_tests | ||
| scim_tests | ||
| search_tests | ||
| spend_tracking_tests | ||
| store_model_in_db_tests | ||
| test_litellm | ||
| unified_google_tests | ||
| vector_store_tests | ||
| windows_tests | ||
| __init__.py | ||
| _fake_openai_endpoint_server.py | ||
| _flush_vcr_cache.py | ||
| _live_test_helpers.py | ||
| _openai_record_replay_proxy.py | ||
| _vcr_conftest_common.py | ||
| _vcr_redis_persister.py | ||
| _ws_vcr.py | ||
| eval_swe_bench.py | ||
| fake_openai_endpoint.py | ||
| gettysburg.wav | ||
| large_text.py | ||
| openai_batch_completions.jsonl | ||
| pyrightconfig.json | ||
| README.MD | ||
| test_anthropic_compaction_usage.py | ||
| test_budget_management.py | ||
| test_callbacks_on_proxy.py | ||
| test_debug_warning.py | ||
| test_default_encoding_non_root.py | ||
| test_end_users.py | ||
| test_fallbacks.py | ||
| test_gpt5_azure_temperature_support.py | ||
| test_health.py | ||
| test_keys.py | ||
| test_litellm_proxy_responses_config.py | ||
| test_logging.conf | ||
| test_models.py | ||
| test_new_vector_store_endpoints.py | ||
| test_openai_endpoints.py | ||
| test_organizations.py | ||
| test_otel_thread_leak.py | ||
| test_presidio_latency.py | ||
| test_proxy_server_non_root.py | ||
| test_ratelimit.py | ||
| test_resource_cleanup.py | ||
| test_service_logger_otel.py | ||
| test_spend_logs.py | ||
| test_team.py | ||
| test_team_logging.py | ||
| test_team_members.py | ||
| test_users.py | ||
In total litellm runs 1000+ tests
[02/20/2025] Update:
To make it easier to contribute and map what behavior is tested,
we've started mapping the litellm directory in tests/test_litellm
This folder can only run mock tests.