scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-23 00:41:40 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 99
litellm/docs/my-website/docs/proxy/pyroscope_profiling.md
Alexsander Hamir 30b28da2b7
Add pyroscope for observability (#21167) 
* Pyroscope: require PYROSCOPE_APP_NAME and PYROSCOPE_SERVER_ADDRESS, add UTF-8 locale hint

- No defaults for PYROSCOPE_APP_NAME or PYROSCOPE_SERVER_ADDRESS; fail at startup if unset when Pyroscope is enabled
- Set LANG/LC_ALL to C.UTF-8 when unset to reduce malformed_profile (invalid UTF-8) rejections
- Startup message suggests PYTHONUTF8=1 if server rejects profiles
- Simplify LITELLM_ENABLE_PYROSCOPE in config_settings; document Pyroscope env vars as required with no default
- Add pyroscope_profiling to sidebar (Alerting & Monitoring)
- pyproject.toml: pyroscope-io as required dep on non-Windows (marker), in proxy extra

* proxy: add PYROSCOPE_SAMPLE_RATE env, use verbose logging, fix int type

- Add optional PYROSCOPE_SAMPLE_RATE env (integer, no default)
- Pass sample_rate to pyroscope.configure() as int for pyroscope-io
- Replace print with verbose_proxy_logger (info/warning)
- Document PYROSCOPE_SAMPLE_RATE in config_settings.md

* Address Greptile PR feedback: Pyroscope optional, docs, tests, docstring

- pyproject.toml: mark pyroscope-io as optional=true (proxy extra only)
- Add docs/my-website/docs/proxy/pyroscope_profiling.md (fix broken sidebar link)
- Add tests/test_litellm/proxy/test_pyroscope.py for _init_pyroscope()
- proxy_server: fix _init_pyroscope docstring (required server/app name, sample rate as int)

* Update litellm/proxy/proxy_server.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

---------

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
2026-02-13 17:32:29 -08:00

1.8 KiB
Raw Blame History

Grafana Pyroscope CPU profiling

LiteLLM proxy can send continuous CPU profiles to Grafana Pyroscope when enabled via environment variables. This is optional and off by default.

Quick start

  1. Install the optional dependency (required only when enabling Pyroscope):

    pip install pyroscope-io

    Or install the proxy extra:

    pip install "litellm[proxy]"

  2. Set environment variables before starting the proxy:

    Variable Required Description
    LITELLM_ENABLE_PYROSCOPE Yes (to enable) Set to true to enable Pyroscope profiling.
    PYROSCOPE_APP_NAME Yes (when enabled) Application name shown in the Pyroscope UI.
    PYROSCOPE_SERVER_ADDRESS Yes (when enabled) Pyroscope server URL (e.g. http://localhost:4040).
    PYROSCOPE_SAMPLE_RATE No Sample rate (integer). If unset, the pyroscope-io library default is used.

  3. Start the proxy; profiling will begin automatically when the proxy starts.

    export LITELLM_ENABLE_PYROSCOPE=true
export PYROSCOPE_APP_NAME=litellm-proxy
export PYROSCOPE_SERVER_ADDRESS=http://localhost:4040
litellm --config config.yaml

  4. View profiles in the Pyroscope (or Grafana) UI and select your PYROSCOPE_APP_NAME.

Notes

  • Optional dependency: pyroscope-io is an optional dependency. If it is not installed and LITELLM_ENABLE_PYROSCOPE=true, the proxy will log a warning and continue without profiling.
  • Platform support: The pyroscope-io package uses a native extension and is not available on all platforms (e.g. Windows is excluded by the package).
  • Other settings: See Configuration settings for all proxy environment variables.