scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-07 02:59:05 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 103

docs(security_encryption_faq.md): answers common questions about data at rest vs. in transit

Browse source
This commit is contained in:
Krrish Dholakia 2025-10-06 17:33:55 -07:00
parent 4c9bd3e0de
commit fbf7ca17ba
2 changed files with 355 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

354
docs/my-website/docs/proxy/security_encryption_faq.md Normal file
View file

@ -0,0 +1,354 @@
# LiteLLM Self-Hosted Security & Encryption FAQ
## Data in Transit Encryption
### Does the product encrypt data in transit?
**Yes**, LiteLLM encrypts data in transit using TLS/SSL.
### Available in both OSS and Enterprise?
**Yes**, TLS encryption is available in both Open Source and Enterprise versions.
### In transit between the calling client and the product?
**Yes**, HTTPS/TLS is supported through SSL certificate configuration.
**Configuration:**
```bash
# CLI
litellm --ssl_keyfile_path /path/to/key.pem --ssl_certfile_path /path/to/cert.pem
# Environment Variables
export SSL_KEYFILE_PATH="/path/to/key.pem"
export SSL_CERTFILE_PATH="/path/to/cert.pem"
```
**Documentation Reference:** `docs/my-website/docs/guides/security_settings.md`
### In transit between the product and the LLM providers?
**Yes**, all connections to LLM providers use TLS encryption by default.
**Implementation Details:**
- Uses Python's `ssl.create_default_context()`
- Leverages HTTPX and aiohttp libraries with SSL/TLS enabled
- Uses certifi CA bundle by default for SSL verification
**Code Reference:** `litellm/llms/custom_httpx/http_handler.py` (lines 43-105)
### Are TCP sessions to the LLM providers shared?
**Yes**, TCP connections are pooled and reused.
**Details:**
- Connection pooling is enabled by default
- Default: 1000 max concurrent connections with keepalive
- Sessions are maintained across requests to the same provider
- Reduces overhead of TLS handshakes
**Code Reference:** `litellm/llms/custom_httpx/http_handler.py` (lines 704-712)
### Or does the product negotiate a new TLS session with the same LLM provider for every sequential call?
**No**, TLS sessions are reused through connection pooling. New TLS handshakes are not performed for every request.
### How is it encrypted?
**TLS 1.2 and TLS 1.3**
Uses Python's default SSL context which supports both TLS 1.2 and TLS 1.3. The specific version negotiated depends on:
- Python version
- System SSL library (typically OpenSSL)
- Server capabilities
**Implementation:** `ssl.create_default_context()` in Python
### How are these added to the product's configuration?
#### x.509 Certificate
**Method 1: CLI Arguments**
```bash
litellm --ssl_certfile_path /path/to/certificate.pem
```
**Method 2: Environment Variable**
```bash
export SSL_CERTFILE_PATH="/path/to/certificate.pem"
```
#### Private Key
**Method 1: CLI Arguments**
```bash
litellm --ssl_keyfile_path /path/to/private_key.pem
```
**Method 2: Environment Variable**
```bash
export SSL_KEYFILE_PATH="/path/to/private_key.pem"
```
#### Certificate Bundle/Chain
**For client-to-proxy connections:**
Use standard SSL certificate setup with intermediate certificates bundled in the certfile.
**For proxy-to-LLM provider connections:**
**Method 1: Config YAML**
```yaml
litellm_settings:
ssl_verify: "/path/to/ca_bundle.pem"
```
**Method 2: Environment Variable**
```bash
export SSL_CERT_FILE="/path/to/ca_bundle.pem"
```
**Method 3: Client Certificate Authentication**
```yaml
litellm_settings:
ssl_certificate: "/path/to/client_certificate.pem"
```
or
```bash
export SSL_CERTIFICATE="/path/to/client_certificate.pem"
```
### Documentation Coverage
**Primary Documentation:**
- `docs/my-website/docs/guides/security_settings.md` - SSL/TLS configuration guide
**Additional References:**
- `litellm/proxy/proxy_cli.py` (lines 455-467) - CLI options
- `docs/my-website/docs/completion/http_handler_config.md` - Custom HTTP handler configuration
---
## Data at Rest Encryption
### Does the product encrypt data at rest?
**Partially**. Only specific sensitive data is encrypted at rest.
### What data is stored in encrypted form?
#### Encrypted Data:
1. **LLM API Keys** - Model credentials in `LiteLLM_ProxyModelTable.litellm_params`
2. **Provider Credentials** - Stored in `LiteLLM_CredentialsTable.credential_values`
3. **Configuration Secrets** - Sensitive config values in `LiteLLM_Config` table
4. **Virtual Keys** - When using secret managers (optional feature)
#### NOT Encrypted:
1. **Spend Logs** - Request/response data in `LiteLLM_SpendLogs`
2. **Audit Logs** - Change history in `LiteLLM_AuditLog`
3. **User/Team/Organization Data** - Metadata and configuration
4. **Cached Prompts and Completions** - Cache data is stored in plaintext
### Cached prompts and completions?
**No**, cached prompts and completions are **NOT encrypted**.
Cache backends (Redis, S3, local disk) store data as plaintext JSON.
**Code References:**
- `litellm/caching/redis_cache.py`
- `litellm/caching/s3_cache.py`
- `litellm/caching/caching.py`
### Configuration data?
**Partially encrypted**.
#### What IS Encrypted:
- LLM API keys and credentials in model configurations
- Sensitive values in `LiteLLM_Config` table
- Credential values in `LiteLLM_CredentialsTable`
#### What is NOT Encrypted:
- Model names and aliases
- Rate limits and budget settings
- User/team/organization metadata
- Non-sensitive configuration parameters
**Code Reference:** `litellm/proxy/management_endpoints/model_management_endpoints.py` (lines 275-308)
### Log data?
**No**, log data is **NOT encrypted**.
Log data stored in database tables is in plaintext:
- `LiteLLM_SpendLogs` - Contains request/response data, tokens, spend
- `LiteLLM_ErrorLogs` - Error information
- `LiteLLM_AuditLog` - Audit trail of changes
**Note:** You can disable logging to avoid storing sensitive data:
```yaml
general_settings:
disable_spend_logs: True # Disable writing spend logs to DB
disable_error_logs: True # Disable writing error logs to DB
```
**Documentation:** `docs/my-website/docs/proxy/db_info.md` (lines 52-60)
### Where is it stored?
#### In the DB?
**Yes**, encrypted data is stored in PostgreSQL database.
**Key Tables with Encrypted Data:**
- `LiteLLM_ProxyModelTable` - Model configurations with encrypted API keys
- `LiteLLM_CredentialsTable` - Credential values
- `LiteLLM_Config` - Configuration secrets
**Schema Reference:** `schema.prisma`
#### In the filesystem?
**No**, encrypted data is not stored in the filesystem by default.
**Note:** If using disk cache (`disk_cache_dir`), cached data is stored unencrypted.
#### Somewhere else?
**Optional:** When using secret managers (AWS Secrets Manager, Azure Key Vault, HashiCorp Vault), encrypted data can be stored externally.
**Configuration:**
```yaml
general_settings:
key_management_system: "aws_secret_manager" # or "azure_key_vault", "hashicorp_vault"
```
**Documentation:** `docs/my-website/docs/secret.md`
### How is it encrypted?
**Algorithm:** NaCl SecretBox (XSalsa20-Poly1305 AEAD)
**NOT AES-256** - LiteLLM uses NaCl (Networking and Cryptography Library) which provides:
- XSalsa20 stream cipher
- Poly1305 MAC for authentication
- Equivalent security to AES-256
**Key Derivation:**
1. Takes `LITELLM_SALT_KEY` (or `LITELLM_MASTER_KEY` if salt key not set)
2. Hashes with SHA-256 to derive 256-bit encryption key
3. Uses NaCl SecretBox for authenticated encryption
**Code Reference:** `litellm/proxy/common_utils/encrypt_decrypt_utils.py` (lines 69-112)
**Implementation:**
```python
import hashlib
import nacl.secret
# Derive 256-bit key from salt
hash_object = hashlib.sha256(signing_key.encode())
hash_bytes = hash_object.digest()
# Create SecretBox and encrypt
box = nacl.secret.SecretBox(hash_bytes)
encrypted = box.encrypt(value_bytes)
```
### Setting the Encryption Key
**Required Environment Variable:**
```bash
export LITELLM_SALT_KEY="your-strong-random-key-here"
```
**Important Notes:**
- ⚠️ **Must be set before adding any models**
- ⚠️ **Never change this key** - encrypted data becomes unrecoverable
- ⚠️ Use a strong random key (recommended: https://1password.com/password-generator/)
- If not set, falls back to `LITELLM_MASTER_KEY`
**Documentation:** `docs/my-website/docs/proxy/prod.md` (section 8, lines 184-196)
### Documentation Coverage
**Primary Documentation:**
- `docs/my-website/docs/proxy/prod.md` (section 8) - LITELLM_SALT_KEY setup
- `docs/my-website/docs/secret.md` - Secret management systems
- `docs/my-website/docs/proxy/db_info.md` - Database information
**Additional References:**
- `security.md` - General security measures
- `docs/my-website/docs/data_security.md` - Data privacy overview
- `schema.prisma` - Database schema with encrypted fields
---
## Summary of Security Features
### ✅ Provided Out of the Box
1. **TLS/SSL encryption** for client-to-proxy connections
2. **TLS encryption** for proxy-to-LLM provider connections (with connection pooling)
3. **Encrypted storage** of LLM API keys and credentials
4. **Support for TLS 1.2 and TLS 1.3**
5. **Connection pooling** to reduce TLS handshake overhead
### ⚠️ Important Limitations
1. **Cached data is NOT encrypted** (Redis, S3, disk cache)
2. **Log data is NOT encrypted** (spend logs, audit logs)
3. **Request/response payloads in logs are NOT encrypted**
4. **Uses NaCl SecretBox, NOT AES-256** (equivalent security)
5. **TLS version not explicitly configured** - uses Python/system defaults
### 🔧 Configuration Requirements
**For Production Deployments:**
1. **Set LITELLM_SALT_KEY** before adding any models
2. **Configure SSL certificates** for HTTPS client connections
3. **Consider disabling logs** if they contain sensitive data
4. **Use secret managers** for enhanced security (optional)
5. **Configure CA bundles** if using custom certificates
---
## Quick Start Security Checklist
```bash
# 1. Generate a strong salt key
export LITELLM_SALT_KEY="$(openssl rand -base64 32)"
# 2. Set up SSL certificates (for HTTPS)
export SSL_KEYFILE_PATH="/path/to/private_key.pem"
export SSL_CERTFILE_PATH="/path/to/certificate.pem"
# 3. Configure database
export DATABASE_URL="postgresql://user:password@host:port/dbname"
# 4. (Optional) Disable logs if they contain sensitive data
# Add to config.yaml:
# general_settings:
# disable_spend_logs: True
# disable_error_logs: True
# 5. Start LiteLLM Proxy
litellm --config config.yaml
```
---
## Additional Resources
- **LiteLLM Documentation:** https://docs.litellm.ai/
- **Security Settings Guide:** https://docs.litellm.ai/docs/guides/security_settings
- **Production Deployment:** https://docs.litellm.ai/docs/proxy/prod
- **Secret Management:** https://docs.litellm.ai/docs/secret
For security inquiries: support@berri.ai

1
docs/my-website/sidebars.js
View file

@ -674,6 +674,7 @@ const sidebars = {
items: [
"data_security",
"data_retention",
"proxy/security_encryption_faq",
"migration_policy",
{
type: "category",