lite login used to write the minted cli-session key in cleartext to
~/.litellm/token.json. The secret material (key plus any JWT) now goes
to the OS keychain through the optional keyring package, with the 0600
file kept for non-secret metadata and as the fallback on headless boxes.
Legacy plaintext files keep authenticating and are migrated into the
keychain, then scrubbed, on first read. A secret still on disk always
outranks the keychain entry, so a failed keychain write can never
resurrect a stale key. LITELLM_PROXY_API_KEY and --api-key precedence
is unchanged, lite logout clears both stores and warns when the
keychain will not release the entry, and ~/.litellm is created 0700
(tightened from 0755 where an older CLI left it broader).
LITELLM_CLI_DISABLE_KEYRING=1 forces the file fallback.
The existing dashboards in this folder chart the litellm_* Prometheus metrics.
Nothing charted the gen_ai.* metrics the OpenTelemetry v2 integration emits, and
Grafana's own prebuilt GenAI dashboards cannot: twenty of their twenty-two panels
filter on telemetry_sdk_name="openlit", a label LiteLLM does not carry and has no
setting to add.
Ten panels over the six gen_ai instruments: spend, tokens, request count and p95
duration as stats, then request rate, spend per hour, tokens per minute split by
input and output, and p95 duration, time to first token, and provider generation
time by model. Template variables for data source, service, and model.
Verified against a live Grafana Cloud stack with real traffic across three
models. The readme documents the attribute filter the panels depend on, since the
default attribute set gives nearly every request its own series and makes every
rate-based panel read zero.
* Prompt Management API - new API to interact with Prompt Management integrations (no PR required) (#17800)
* feat: initial commit adding prompt management api
* feat: initial commit adding prompt management api
* fix: refactoring to make sure get prompt is async
* fix: additional fixes
* fix: partially working generic api prompt management
* fix: cli auth with SSO okta
* fix: add LITTELM_CLI_SERVICE_ACCOUNT_NAME
* fix: get_litellm_cli_user_api_key_auth
* use existing_key CLI
* fix: use existing key
* test auth commands
* test_cli_sso_callback_regenerate_vs_create_flow
* feat: add CLI Token Utilities
* fix: get_stored_api_key
* move file
* fix: get_valid_models
* fix config.yaml
* TestCLITokenUtils
* TestGetValidModelsWithCLI
* fix: tie user id to keys created through CLI
* fix: add teams interface to CLI
* add /keys/update to the list client commands
* fix /sso/cli/poll to return the user_id
* fix: working TeamsManagementClient
* fix CLI Login command
* fixes for auth
* Potential fix for code scanning alert no. 3400: Clear-text logging of sensitive information
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* ruff fix
---------
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
This commit updates the Grafana dashboard configuration to include a datasource template variable. This allows users to dynamically select the datasource directly within the Grafana dashboard, improving flexibility and user experience.
