Unit Tests: Proxy DB Operations / proxy-db (auth-checks, tests/proxy_unit_tests/test_auth_checks.py tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
Unit Tests: Proxy DB Operations / proxy-db (remaining, tests/proxy_unit_tests --ignore=tests/proxy_unit_tests/test_key_generate_prisma.py --ignore=tests/proxy_unit_tests/test_auth_checks.py --ignore=tests/proxy_unit_tests/test_user_api_key_auth.py, 20, 8) (push) Has been cancelled
- Add _truncate_large_payload_for_logging() to truncate embedding responses,
long strings, large lists, and nested structures before serialization
- Addresses ~3-4s dumps/post_call cost from embedding_profile_percentages.md
- Add 15 tests covering huge payloads, edge cases, and max depth
- Add pyroscope-io to requirements.txt for proxy deployments
- Remove Windows platform marker from pyroscope-io in pyproject.toml
so it is installed on all platforms when using proxy extra
- Add optional PYROSCOPE_SAMPLE_RATE env (integer, no default)
- Pass sample_rate to pyroscope.configure() as int for pyroscope-io
- Replace print with verbose_proxy_logger (info/warning)
- Document PYROSCOPE_SAMPLE_RATE in config_settings.md
- No defaults for PYROSCOPE_APP_NAME or PYROSCOPE_SERVER_ADDRESS; fail at startup if unset when Pyroscope is enabled
- Set LANG/LC_ALL to C.UTF-8 when unset to reduce malformed_profile (invalid UTF-8) rejections
- Startup message suggests PYTHONUTF8=1 if server rejects profiles
- Simplify LITELLM_ENABLE_PYROSCOPE in config_settings; document Pyroscope env vars as required with no default
- Add pyroscope_profiling to sidebar (Alerting & Monitoring)
- pyproject.toml: pyroscope-io as required dep on non-Windows (marker), in proxy extra
Make asyncio.gather resilient to partial failures. If key, team, or
user lookup raises, log at debug level and allow the others to
complete. Prevents one failing lookup from blocking the rest.
Replace sequential await of key, team, and user budget metric lookups
with asyncio.gather() so they run in parallel. Reduces latency from
T_key + T_team + T_user to max(T_key, T_team, T_user) when lookups
hit DB or cache.
Implemented TracerProvider caching in OpenTelemetry class to prevent linear thread growth when using dynamic headers (e.g., per-team Langfuse keys). Added formal unit test in tests/test_otel_thread_leak.py.
