mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-05 08:07:05 +00:00
0bb26c3f1b
* feat(proxy): add Prisma DB pool and engine health metrics to Prometheus Add a PrismaMetricsCollector that periodically queries pg_stat_activity and the Prisma engine process to expose connection pool and engine health as Prometheus gauges/counters. Auto-enabled when prometheus_system is in service_callback. New metrics: - litellm_db_pool_active_connections (Gauge) - litellm_db_pool_idle_connections (Gauge) - litellm_db_pool_total_connections (Gauge) - litellm_db_pool_waiting_connections (Gauge) - litellm_db_engine_up (Gauge) - litellm_db_engine_restarts_total (Counter) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address Greptile review feedback - Only increment engine_restarts counter on heavy reconnects (engine actually dead), not lightweight network-blip reconnects - Fix potential KeyError in _get_or_create_gauge/counter fallback path when REGISTRY._names_to_collectors is absent - Rename litellm_db_pool_waiting_connections to litellm_db_pool_lock_waiting_connections to clarify it measures lock contention, not pool slot queuing Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: warn when prometheus_system enabled but watchdog disabled Log a warning when users have prometheus_system in service_callback but PRISMA_HEALTH_WATCHDOG_ENABLED=false, since DB pool and engine metrics won't be collected in that configuration. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * ci: retrigger CI checks Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * refactor: use labeled gauge for DB pool connection metrics Replace 3 separate pool gauges (active, idle, total) with a single `litellm_db_pool_connections` gauge using a `state` label. This is more Prometheus-idiomatic and exposes all pg_stat_activity states (active, idle, idle in transaction, etc.) without ambiguity about what "total" includes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: address Greptile review — stale labels and fallback re-registration - Zero out known pg_stat_activity states that are absent from the current query result, preventing stale gauge values from persisting. - Simplify _get_or_create_gauge/counter by removing the fallback loop that could re-register an already-registered metric (ValueError). - Add test for stale label clearing across collection cycles. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: include "unknown" in _PG_STATES for stale label clearing Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: collect immediately on start and consolidate into single query - Move sleep to end of loop so metrics appear on /metrics immediately after startup instead of after a 30s delay. - Combine pool state and lock waiting queries into a single SQL query using conditional aggregation, halving per-cycle DB overhead. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: prevent tight spin loop on collection error Move asyncio.sleep outside the try/except so it always executes even when _collect_engine_health() or _collect_pool_metrics() raises. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add multiprocess_mode to _get_or_create_gauge initialization - Include `multiprocess_mode` parameter to properly support multiprocessing in Gauge creation. - Ensure consistent behavior for labeled and unlabeled Gauges. * fix: handle invalid env var and document watchdog prerequisite - Add try/except ValueError for PRISMA_METRICS_COLLECTION_INTERVAL_SECONDS to prevent proxy startup crash on non-numeric values (e.g. "30s") - Document that DB metrics require both prometheus_system callback and PRISMA_HEALTH_WATCHDOG_ENABLED=true Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: use defensive null coalescing for query_raw row values Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * test: add invalid env var fallback test and fix mock signature - Add test for non-numeric PRISMA_METRICS_COLLECTION_INTERVAL_SECONDS - Add **kwargs to mock _patched_get_or_create_gauge for forward compat Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| adding_provider | ||
| anthropic_unified | ||
| caching | ||
| completion | ||
| contribute_integration | ||
| contributing | ||
| debugging | ||
| embedding | ||
| extras | ||
| guides | ||
| integrations | ||
| langchain | ||
| observability | ||
| pass_through | ||
| projects | ||
| provider_registration | ||
| providers | ||
| proxy | ||
| search | ||
| secret_managers | ||
| troubleshoot | ||
| tutorials | ||
| vector_stores | ||
| a2a.md | ||
| a2a_agent_headers.md | ||
| a2a_agent_permissions.md | ||
| a2a_cost_tracking.md | ||
| a2a_invoking_agents.md | ||
| a2a_iteration_budgets.md | ||
| aiohttp_benchmarks.md | ||
| anthropic_count_tokens.md | ||
| apply_guardrail.md | ||
| assistants.md | ||
| audio_transcription.md | ||
| batches.md | ||
| bedrock_converse.md | ||
| bedrock_invoke.md | ||
| benchmarks.md | ||
| budget_manager.md | ||
| contact.md | ||
| container_files.md | ||
| containers.md | ||
| contributing.md | ||
| data_retention.md | ||
| data_security.md | ||
| default_code_snippet.md | ||
| enterprise.md | ||
| evals_api.md | ||
| exception_mapping.md | ||
| files_endpoints.md | ||
| fine_tuning.md | ||
| generateContent.md | ||
| image_edits.md | ||
| image_generation.md | ||
| image_variations.md | ||
| index.md | ||
| interactions.md | ||
| load_test.md | ||
| load_test_advanced.md | ||
| load_test_rpm.md | ||
| load_test_sdk.md | ||
| mcp.md | ||
| mcp_control.md | ||
| mcp_cost.md | ||
| mcp_guardrail.md | ||
| mcp_oauth.md | ||
| mcp_openapi.md | ||
| mcp_public_internet.md | ||
| mcp_semantic_filter.md | ||
| mcp_troubleshoot.md | ||
| mcp_usage.md | ||
| migration.md | ||
| migration_policy.md | ||
| moderation.md | ||
| ocr.md | ||
| oidc.md | ||
| old_guardrails.md | ||
| projects.md | ||
| proxy_api.md | ||
| proxy_auth.md | ||
| proxy_server.md | ||
| rag_ingest.md | ||
| rag_query.md | ||
| realtime.md | ||
| reasoning_content.md | ||
| rerank.md | ||
| response_api.md | ||
| response_api_compact.md | ||
| router_architecture.md | ||
| routing.md | ||
| rules.md | ||
| scheduler.md | ||
| sdk_custom_pricing.md | ||
| secret.md | ||
| set_keys.md | ||
| skills.md | ||
| text_completion.md | ||
| text_to_speech.md | ||
| traffic_mirroring.md | ||
| troubleshoot.md | ||
| vector_store_files.md | ||
| vertex_batch_passthrough.md | ||
| videos.md | ||
| wildcard_routing.md | ||