diff --git a/docs/my-website/docs/proxy/security_encryption_faq.md b/docs/my-website/docs/proxy/security_encryption_faq.md new file mode 100644 index 00000000000..690f67d79a3 --- /dev/null +++ b/docs/my-website/docs/proxy/security_encryption_faq.md @@ -0,0 +1,354 @@ +# LiteLLM Self-Hosted Security & Encryption FAQ + +## Data in Transit Encryption + +### Does the product encrypt data in transit? + +**Yes**, LiteLLM encrypts data in transit using TLS/SSL. + +### Available in both OSS and Enterprise? + +**Yes**, TLS encryption is available in both Open Source and Enterprise versions. + +### In transit between the calling client and the product? + +**Yes**, HTTPS/TLS is supported through SSL certificate configuration. + +**Configuration:** +```bash +# CLI +litellm --ssl_keyfile_path /path/to/key.pem --ssl_certfile_path /path/to/cert.pem + +# Environment Variables +export SSL_KEYFILE_PATH="/path/to/key.pem" +export SSL_CERTFILE_PATH="/path/to/cert.pem" +``` + +**Documentation Reference:** `docs/my-website/docs/guides/security_settings.md` + +### In transit between the product and the LLM providers? + +**Yes**, all connections to LLM providers use TLS encryption by default. + +**Implementation Details:** +- Uses Python's `ssl.create_default_context()` +- Leverages HTTPX and aiohttp libraries with SSL/TLS enabled +- Uses certifi CA bundle by default for SSL verification + +**Code Reference:** `litellm/llms/custom_httpx/http_handler.py` (lines 43-105) + +### Are TCP sessions to the LLM providers shared? + +**Yes**, TCP connections are pooled and reused. + +**Details:** +- Connection pooling is enabled by default +- Default: 1000 max concurrent connections with keepalive +- Sessions are maintained across requests to the same provider +- Reduces overhead of TLS handshakes + +**Code Reference:** `litellm/llms/custom_httpx/http_handler.py` (lines 704-712) + +### Or does the product negotiate a new TLS session with the same LLM provider for every sequential call? + +**No**, TLS sessions are reused through connection pooling. New TLS handshakes are not performed for every request. + +### How is it encrypted? + +**TLS 1.2 and TLS 1.3** + +Uses Python's default SSL context which supports both TLS 1.2 and TLS 1.3. The specific version negotiated depends on: +- Python version +- System SSL library (typically OpenSSL) +- Server capabilities + +**Implementation:** `ssl.create_default_context()` in Python + +### How are these added to the product's configuration? + +#### x.509 Certificate + +**Method 1: CLI Arguments** +```bash +litellm --ssl_certfile_path /path/to/certificate.pem +``` + +**Method 2: Environment Variable** +```bash +export SSL_CERTFILE_PATH="/path/to/certificate.pem" +``` + +#### Private Key + +**Method 1: CLI Arguments** +```bash +litellm --ssl_keyfile_path /path/to/private_key.pem +``` + +**Method 2: Environment Variable** +```bash +export SSL_KEYFILE_PATH="/path/to/private_key.pem" +``` + +#### Certificate Bundle/Chain + +**For client-to-proxy connections:** +Use standard SSL certificate setup with intermediate certificates bundled in the certfile. + +**For proxy-to-LLM provider connections:** + +**Method 1: Config YAML** +```yaml +litellm_settings: + ssl_verify: "/path/to/ca_bundle.pem" +``` + +**Method 2: Environment Variable** +```bash +export SSL_CERT_FILE="/path/to/ca_bundle.pem" +``` + +**Method 3: Client Certificate Authentication** +```yaml +litellm_settings: + ssl_certificate: "/path/to/client_certificate.pem" +``` + +or + +```bash +export SSL_CERTIFICATE="/path/to/client_certificate.pem" +``` + +### Documentation Coverage + +**Primary Documentation:** +- `docs/my-website/docs/guides/security_settings.md` - SSL/TLS configuration guide + +**Additional References:** +- `litellm/proxy/proxy_cli.py` (lines 455-467) - CLI options +- `docs/my-website/docs/completion/http_handler_config.md` - Custom HTTP handler configuration + +--- + +## Data at Rest Encryption + +### Does the product encrypt data at rest? + +**Partially**. Only specific sensitive data is encrypted at rest. + +### What data is stored in encrypted form? + +#### Encrypted Data: +1. **LLM API Keys** - Model credentials in `LiteLLM_ProxyModelTable.litellm_params` +2. **Provider Credentials** - Stored in `LiteLLM_CredentialsTable.credential_values` +3. **Configuration Secrets** - Sensitive config values in `LiteLLM_Config` table +4. **Virtual Keys** - When using secret managers (optional feature) + +#### NOT Encrypted: +1. **Spend Logs** - Request/response data in `LiteLLM_SpendLogs` +2. **Audit Logs** - Change history in `LiteLLM_AuditLog` +3. **User/Team/Organization Data** - Metadata and configuration +4. **Cached Prompts and Completions** - Cache data is stored in plaintext + +### Cached prompts and completions? + +**No**, cached prompts and completions are **NOT encrypted**. + +Cache backends (Redis, S3, local disk) store data as plaintext JSON. + +**Code References:** +- `litellm/caching/redis_cache.py` +- `litellm/caching/s3_cache.py` +- `litellm/caching/caching.py` + +### Configuration data? + +**Partially encrypted**. + +#### What IS Encrypted: +- LLM API keys and credentials in model configurations +- Sensitive values in `LiteLLM_Config` table +- Credential values in `LiteLLM_CredentialsTable` + +#### What is NOT Encrypted: +- Model names and aliases +- Rate limits and budget settings +- User/team/organization metadata +- Non-sensitive configuration parameters + +**Code Reference:** `litellm/proxy/management_endpoints/model_management_endpoints.py` (lines 275-308) + +### Log data? + +**No**, log data is **NOT encrypted**. + +Log data stored in database tables is in plaintext: +- `LiteLLM_SpendLogs` - Contains request/response data, tokens, spend +- `LiteLLM_ErrorLogs` - Error information +- `LiteLLM_AuditLog` - Audit trail of changes + +**Note:** You can disable logging to avoid storing sensitive data: + +```yaml +general_settings: + disable_spend_logs: True # Disable writing spend logs to DB + disable_error_logs: True # Disable writing error logs to DB +``` + +**Documentation:** `docs/my-website/docs/proxy/db_info.md` (lines 52-60) + +### Where is it stored? + +#### In the DB? + +**Yes**, encrypted data is stored in PostgreSQL database. + +**Key Tables with Encrypted Data:** +- `LiteLLM_ProxyModelTable` - Model configurations with encrypted API keys +- `LiteLLM_CredentialsTable` - Credential values +- `LiteLLM_Config` - Configuration secrets + +**Schema Reference:** `schema.prisma` + +#### In the filesystem? + +**No**, encrypted data is not stored in the filesystem by default. + +**Note:** If using disk cache (`disk_cache_dir`), cached data is stored unencrypted. + +#### Somewhere else? + +**Optional:** When using secret managers (AWS Secrets Manager, Azure Key Vault, HashiCorp Vault), encrypted data can be stored externally. + +**Configuration:** +```yaml +general_settings: + key_management_system: "aws_secret_manager" # or "azure_key_vault", "hashicorp_vault" +``` + +**Documentation:** `docs/my-website/docs/secret.md` + +### How is it encrypted? + +**Algorithm:** NaCl SecretBox (XSalsa20-Poly1305 AEAD) + +**NOT AES-256** - LiteLLM uses NaCl (Networking and Cryptography Library) which provides: +- XSalsa20 stream cipher +- Poly1305 MAC for authentication +- Equivalent security to AES-256 + +**Key Derivation:** +1. Takes `LITELLM_SALT_KEY` (or `LITELLM_MASTER_KEY` if salt key not set) +2. Hashes with SHA-256 to derive 256-bit encryption key +3. Uses NaCl SecretBox for authenticated encryption + +**Code Reference:** `litellm/proxy/common_utils/encrypt_decrypt_utils.py` (lines 69-112) + +**Implementation:** +```python +import hashlib +import nacl.secret + +# Derive 256-bit key from salt +hash_object = hashlib.sha256(signing_key.encode()) +hash_bytes = hash_object.digest() + +# Create SecretBox and encrypt +box = nacl.secret.SecretBox(hash_bytes) +encrypted = box.encrypt(value_bytes) +``` + +### Setting the Encryption Key + +**Required Environment Variable:** +```bash +export LITELLM_SALT_KEY="your-strong-random-key-here" +``` + +**Important Notes:** +- ⚠️ **Must be set before adding any models** +- ⚠️ **Never change this key** - encrypted data becomes unrecoverable +- ⚠️ Use a strong random key (recommended: https://1password.com/password-generator/) +- If not set, falls back to `LITELLM_MASTER_KEY` + +**Documentation:** `docs/my-website/docs/proxy/prod.md` (section 8, lines 184-196) + +### Documentation Coverage + +**Primary Documentation:** +- `docs/my-website/docs/proxy/prod.md` (section 8) - LITELLM_SALT_KEY setup +- `docs/my-website/docs/secret.md` - Secret management systems +- `docs/my-website/docs/proxy/db_info.md` - Database information + +**Additional References:** +- `security.md` - General security measures +- `docs/my-website/docs/data_security.md` - Data privacy overview +- `schema.prisma` - Database schema with encrypted fields + +--- + +## Summary of Security Features + +### ✅ Provided Out of the Box + +1. **TLS/SSL encryption** for client-to-proxy connections +2. **TLS encryption** for proxy-to-LLM provider connections (with connection pooling) +3. **Encrypted storage** of LLM API keys and credentials +4. **Support for TLS 1.2 and TLS 1.3** +5. **Connection pooling** to reduce TLS handshake overhead + +### ⚠️ Important Limitations + +1. **Cached data is NOT encrypted** (Redis, S3, disk cache) +2. **Log data is NOT encrypted** (spend logs, audit logs) +3. **Request/response payloads in logs are NOT encrypted** +4. **Uses NaCl SecretBox, NOT AES-256** (equivalent security) +5. **TLS version not explicitly configured** - uses Python/system defaults + +### 🔧 Configuration Requirements + +**For Production Deployments:** + +1. **Set LITELLM_SALT_KEY** before adding any models +2. **Configure SSL certificates** for HTTPS client connections +3. **Consider disabling logs** if they contain sensitive data +4. **Use secret managers** for enhanced security (optional) +5. **Configure CA bundles** if using custom certificates + +--- + +## Quick Start Security Checklist + +```bash +# 1. Generate a strong salt key +export LITELLM_SALT_KEY="$(openssl rand -base64 32)" + +# 2. Set up SSL certificates (for HTTPS) +export SSL_KEYFILE_PATH="/path/to/private_key.pem" +export SSL_CERTFILE_PATH="/path/to/certificate.pem" + +# 3. Configure database +export DATABASE_URL="postgresql://user:password@host:port/dbname" + +# 4. (Optional) Disable logs if they contain sensitive data +# Add to config.yaml: +# general_settings: +# disable_spend_logs: True +# disable_error_logs: True + +# 5. Start LiteLLM Proxy +litellm --config config.yaml +``` + +--- + +## Additional Resources + +- **LiteLLM Documentation:** https://docs.litellm.ai/ +- **Security Settings Guide:** https://docs.litellm.ai/docs/guides/security_settings +- **Production Deployment:** https://docs.litellm.ai/docs/proxy/prod +- **Secret Management:** https://docs.litellm.ai/docs/secret + +For security inquiries: support@berri.ai + diff --git a/docs/my-website/sidebars.js b/docs/my-website/sidebars.js index 7acd92240c6..12fb2a681f3 100644 --- a/docs/my-website/sidebars.js +++ b/docs/my-website/sidebars.js @@ -674,6 +674,7 @@ const sidebars = { items: [ "data_security", "data_retention", + "proxy/security_encryption_faq", "migration_policy", { type: "category",