scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-13 23:11:40 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 101
litellm/tests/test_litellm/proxy
History
Exact
ryan-crabbe-berri be71a8fdbf
fix(alerting): dedupe scheduled Slack spend reports across pods (#36489) 
* fix(alerting): dedupe scheduled Slack spend reports across pods

Every pod ran its own weekly/monthly spend report jobs, prometheus
fallback stats cron, and daily report loop, so deployments with
multiple replicas or uvicorn workers received one copy per pod.

Gate each scheduled send behind the shared PodLockManager redis lock.
The lock is never released: its TTL (the full reporting window for the
weekly interval job, whose per-pod anchors drift by boot time and
jitter) doubles as a sent-this-window marker. acquire_lock returning
None (no redis wired) proceeds, preserving single-pod behavior.

Also generalize the pod lock could-not-acquire log line, which claimed
to be about spend tracking for every consumer.

Fixes #14809

* fix(alerting): harden spend report locks after adversarial review

Weekly lock TTL gets an hour haircut: with ttl equal to the interval,
the winner re-fires just before its own key expires, reacquires without
a TTL refresh, and the key then lapses in time for a trailing pod to
re-send. Job/lock ids move to litellm/constants.py per convention, and
spend_report_frequency now rejects non-positive day counts, which
previously coerced to an every-second schedule and would now compute a
negative lock TTL that silently never sends.

Adds the missing test coverage the review flagged: startup_event's
pod_lock_manager wiring (identity-asserted), the prometheus closure's
positive path, and the ungated immediate prometheus send pinned to
exactly one await.

* test(alerting): consolidate spend_report_frequency validator coverage

Drops a duplicate non-positive-days test and parametrizes the survivor
over the suffix half of the validator too

* fix(alerting): route the startup prometheus fallback send through the pod lock

Greptile caught that the boot-time send still ran once per pod when
PROMETHEUS_URL is set, the same duplication class this PR removes

* fix(alerting): make report lock acquisition non-reentrant

Greptile caught that a pod booting within an hour of the fallback stats
cron sent twice: the startup send takes the lock, then the cron fire
hits acquire_lock's reacquire branch, which returns True for the
holder. Window-marker gates now pass allow_reentrant=False so a live
lock blocks everyone including its holder; leader-election consumers
keep the reentrant default

* test(proxy): give spec'd ProxyLogging mocks a db_spend_update_writer

_initialize_slack_alerting_jobs now reads it for the pod lock manager,
and spec=ProxyLogging blocks instance-only attributes
2026-08-11 12:41:11 -07:00
..
_experimental/mcp_server perf: build log messages lazily so filtered-out log records cost nothing (#35703) 2026-08-04 04:34:52 +00:00
a2a fix(a2a): reject malformed protocolVersion suffixes while keeping semver prereleases 2026-07-21 13:24:58 -07:00
agent_endpoints fix(proxy): deny agent access when key and team grants resolve to nothing (#36221) 2026-08-07 20:44:11 +00:00
analytics_endpoints feat(ui): split failed requests into their own series on the cache dashboard (#34862) 2026-07-29 09:48:17 -07:00
anthropic_endpoints fix(types): correct annotations that were false about their runtime values 2026-08-06 04:37:48 +00:00
auth fix(proxy): ban caller-supplied aws identity selectors in request bodies 2026-08-10 22:05:49 -07:00
batches_endpoints fix(proxy): authenticate managed ids before routing 2026-08-08 02:42:34 -07:00
client feat(cli): read base_url from persistent config file (#35015) 2026-07-29 18:25:05 -07:00
common_utils fix(reset_budget_job): atomic budget cascade with chunked reset scans (#36287) 2026-08-10 14:42:36 -07:00
config_resolvers feat(proxy): add SAML 2.0 SSO for the admin UI (#31429) 2026-07-24 12:51:28 -07:00
credential_endpoints fix(proxy): return the real status code when a credential update is rejected (#36166) 2026-08-07 08:02:52 +00:00
db fix(alerting): dedupe scheduled Slack spend reports across pods (#36489) 2026-08-11 12:41:11 -07:00
discovery_endpoints chore: litellm oss staging160626 (#30527) 2026-06-16 18:23:13 -07:00
enterprise_billing feat(proxy): push-based OTLP billable-request metering for enterprise deployments (#31592) 2026-07-15 12:12:52 -07:00
experimental/mcp_server
fine_tuning_endpoints fix(proxy): authenticate managed ids before routing 2026-08-08 02:42:34 -07:00
google_endpoints chore: litellm oss staging160626 (#30527) 2026-06-16 18:23:13 -07:00
guardrails fix(guardrails): chunk oversized Bedrock ApplyGuardrail requests instead of failing (#36119) 2026-08-07 19:44:24 -07:00
health_endpoints fix(health): drop a stored-credential reference along with the credentials it names 2026-08-05 16:07:04 -07:00
hooks fix(email): stop duplicate legacy invitation email and fix its onboarding link (#36455) 2026-08-10 23:26:27 -07:00
image_endpoints fix(proxy)!: share one destination check between body and path-supplied model 2026-08-05 15:17:43 -07:00
logging_endpoints feat(proxy): add POST /v1/callbacks/logs to replay logging payloads through callbacks (#31134) 2026-06-24 15:25:10 -07:00
management_endpoints feat(proxy): per-key prompt caching toggle via enable_prompt_caching (#36466) 2026-08-11 11:53:11 -07:00
management_helpers feat(team): custom metadata validation hook for team create and update (#33353) 2026-08-03 18:37:45 -07:00
memory fix(proxy): give proxy_admin_viewer read parity with proxy_admin (#35851) 2026-08-05 18:33:55 +00:00
middleware feat(sgr): make the gateway middleware the source of truth for successful requests (#35717) 2026-08-05 12:40:47 -07:00
openai_files_endpoint Merge pull request #36273 from BerriAI/litellm_dbless_hook_registration 2026-08-08 17:25:23 -07:00
pass_through_endpoints fix(proxy): preserve crlf line endings when injecting streamed usage cost 2026-08-11 00:45:32 -07:00
policy_engine fix(policy_engine): warn that the config-defined policy reactivates when all DB versions are deleted 2026-07-30 19:48:35 -07:00
prompts fix(proxy): give proxy_admin_viewer read parity with proxy_admin (#35851) 2026-08-05 18:33:55 +00:00
proxy_server fix(alerting): dedupe scheduled Slack spend reports across pods (#36489) 2026-08-11 12:41:11 -07:00
public_endpoints fix(proxy): derive config agent ids from agent_name so grants survive secret rotation (#36020) 2026-08-07 19:19:49 +00:00
rag_endpoints fix(rag): track LLM completion usage and spend for /v1/rag/query (#32438) 2026-07-17 17:45:27 +00:00
realtime_endpoints chore: litellm oss 170626 (#30637) 2026-06-17 21:11:12 -07:00
response_api_endpoints fix(proxy): enforce per-model budgets against resolved cursor model variants 2026-08-04 14:36:04 -07:00
shutdown feat(proxy): native /health/drain preStop hook for graceful shutdown (#29439) 2026-06-02 16:30:44 -07:00
spend_tracking fix: net prompt-caching savings against the cache-write premium (#36452) 2026-08-10 18:52:03 -07:00
test_configs
types_utils fix(proxy): resolve router_settings.plugins dotted paths and load plugins from installed packages (#33644) 2026-07-17 11:23:18 -07:00
ui_crud_endpoints feat(ptu): gate PTU flat-cost attribution behind an opt-in env var (#36138) 2026-08-10 12:23:20 -07:00
utils fix(alerting): dedupe scheduled Slack spend reports across pods (#36489) 2026-08-11 12:41:11 -07:00
vector_store_endpoints feat(proxy): add GET /v1/indexes to list vector store indexes (#36289) 2026-08-10 15:09:59 -07:00
vector_store_files_endpoints fix(proxy): authenticate managed ids before routing 2026-08-08 02:42:34 -07:00
video_endpoints test(videos): add 1:1 test file scaffold for videos component paths (#30631) 2026-06-29 09:12:51 +05:30
__init__.py
conftest.py fix(proxy): avoid DB outage during planned RDS IAM rotation (#34749) 2026-07-28 13:27:50 -07:00
test_aiohttp_cleanup_closed.py
test_aiohttp_session_recovery.py
test_api_key_masking_in_errors.py
test_audio_speech_prometheus_hooks.py fix(proxy): extend response headers hook to streaming, TTS, image gen, and pass-through (#24232) 2026-06-09 22:10:23 +02:00
test_batch_expiry.py fix(proxy): strip LiteLLM policy tracking from OpenAI batch metadata (#28425) 2026-05-26 11:35:42 -07:00
test_batch_metadata_none_fix.py
test_batch_retrieve_bedrock.py
test_batch_x_litellm_model_encoding.py test(batches): move orphan tests into tests/test_litellm for CI coverage (#30510) 2026-06-16 10:20:59 -07:00
test_blocked_response_usage.py feat: litellm oss staging (#31935) 2026-07-03 09:27:31 +05:30
test_budget_reservation.py feat(proxy): add apply_user_budget_to_team_keys opt-in (#36102) 2026-08-07 15:40:13 +00:00
test_caching_routes.py Litellm oss staging 040626 (#29671) 2026-06-04 11:07:20 -07:00
test_chat_completion_metadata.py
test_common_request_processing.py fix(proxy): inject streaming usage cost on openai passthrough streams 2026-08-10 19:56:34 -07:00
test_component_allowlists.py fix(gateway): keep the Prometheus /metrics Mount in the gateway route trim (#32317) 2026-07-07 18:36:38 +03:00
test_cors_config.py
test_custom_proxy.py
test_dynamic_mcp_route.py Litellm oss staging 040626 (#29671) 2026-06-04 11:07:20 -07:00
test_empty_model_list.py
test_enforce_user_param.py
test_fallback_management_endpoints.py
test_fastapi_offline_routes.py
test_filter_models_by_team_access_group.py
test_health_check_functions.py feat(proxy): skip disable_background_health_check models on GET /health when flag set (#27716) 2026-05-13 09:49:05 -07:00
test_health_check_max_tokens.py fix: skip health check for semantic auto_router deployments (#31668) 2026-06-29 19:47:20 -07:00
test_langfuse_passthrough_security.py
test_lazy_openapi_snapshot.py feat(proxy): add GET /v1/indexes to list vector store indexes (#36289) 2026-08-10 15:09:59 -07:00
test_litellm_pre_call_utils.py feat(proxy): per-key prompt caching toggle via enable_prompt_caching (#36466) 2026-08-11 11:53:11 -07:00
test_max_budget_env_var.py
test_mcp_asgi_response.py fix(mcp): surface upstream 401 for token-forwarding MCP servers (#27847) 2026-05-13 12:03:36 -07:00
test_model_based_routing_files_batches.py test(batches): move orphan tests into tests/test_litellm for CI coverage (#30510) 2026-06-16 10:20:59 -07:00
test_model_dump_with_preserved_fields.py
test_model_id_header_propagation.py feat(proxy): add x-litellm-model-name response header with deployment model string (#33698) 2026-07-17 20:29:42 -07:00
test_model_info_default_limits.py fix(proxy): align /v1/model/info with router deployments (#30025) 2026-06-10 19:38:21 -07:00
test_model_level_guardrails.py fix(proxy): apply key/team router_settings.model_group_alias (#35486) 2026-08-03 22:09:47 +00:00
test_model_list_healthy_only.py feat: litellm oss 110626 (#30202) 2026-06-11 22:30:26 -07:00
test_modify_response_streaming_passthrough.py fix(proxy): use e.request_data for logging_obj in ModifyResponseException streaming passthrough (#30800) 2026-06-18 23:29:08 -07:00
test_openapi_schema_validation.py
test_plugin_routes.py feat: litellm plugin architecture v2 (#30688) 2026-06-20 20:37:22 -07:00
test_pricing_field_strip.py chore: litellm oss staging160626 (#30527) 2026-06-16 18:23:13 -07:00
test_prometheus_cleanup.py
test_provider_url_destination_guard.py fix(proxy)!: share one destination check between body and path-supplied model 2026-08-05 15:17:43 -07:00
test_proxy_cli.py feat(proxy): bound DB statement and lock time via general_settings (#35496) 2026-08-01 13:54:17 -07:00
test_proxy_logging_hook_detection.py fix(proxy): recognize inherited apply_guardrail overrides and keep masking guardrails on their own stream hook 2026-07-30 15:33:49 -07:00
test_proxy_server.py fix(alerting): dedupe scheduled Slack spend reports across pods (#36489) 2026-08-11 12:41:11 -07:00
test_proxy_types.py fix(proxy): allow /key/update to identify the key by key_alias (#34851) 2026-07-29 09:48:08 -07:00
test_proxy_utils.py fix(proxy): add config_updated_at audit timestamp for virtual keys (#36488) 2026-08-11 11:02:57 -07:00
test_pyroscope.py
test_read_model_list.py feat: add minimal rust router + axum ai-gateway calling router.realtime (2/2) (#31135) 2026-06-23 19:16:34 -07:00
test_redis_auth_cache_flag.py fix(proxy): share CLI SSO login sessions across workers without enable_redis_auth_cache (#33261) 2026-07-22 10:29:34 -07:00
test_response_model_sanitization.py perf: eliminate per-request callback scanning on proxy hot path (#27858) 2026-05-14 09:28:31 -07:00
test_route_a2a_models.py
test_route_llm_request.py fix(ci): let the E2E proxy accept the mock testing params its suite sends 2026-08-01 14:57:42 -07:00
test_sensitive_route_auth.py
test_shared_health_check.py feat(proxy): skip disable_background_health_check models on GET /health when flag set (#27716) 2026-05-13 09:49:05 -07:00
test_spend_log_cleanup.py feat(spend): rebuild the auto-router benchmarks backend as a per-session rollup (#35910) 2026-08-05 20:06:32 +00:00
test_swagger_chat_completions.py
test_team_member_update.py feat(ui): add budget duration to edit team member form (#29717) 2026-06-06 17:24:55 -07:00
test_team_org_move.py
test_tools_allowlist_enforcement.py fix(responses-bridge): custom tool round-trip and allowlist preservation for Codex CLI (#32258) 2026-07-06 17:34:27 -07:00
test_update_llm_router_resilience.py fix(proxy): stop model writes 500ing on another pod's delete (#35400) 2026-07-31 18:10:48 -07:00