mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-07 08:26:10 +00:00
Update audit document to reflect implemented changes
Co-authored-by: ishaan <ishaan@berri.ai>
This commit is contained in:
parent
0220609ab6
commit
5950fb2d7b
1 changed files with 27 additions and 53 deletions
|
|
@ -177,67 +177,41 @@ export LITELLM_LOCAL_MODEL_COST_MAP=True
|
|||
|
||||
---
|
||||
|
||||
## Proposed Code Changes for Enhanced Safety
|
||||
## Implemented Code Changes
|
||||
|
||||
### 1. Add Validation to `get_model_cost_map.py`
|
||||
### 1. Runtime Validation in `get_model_cost_map.py` ✅ IMPLEMENTED
|
||||
|
||||
```python
|
||||
def validate_model_cost_map(data: dict) -> bool:
|
||||
"""Validate that fetched data is a valid model cost map."""
|
||||
if not isinstance(data, dict):
|
||||
return False
|
||||
if len(data) < 100: # Sanity check - should have many models
|
||||
return False
|
||||
# Check a sample of entries have required fields
|
||||
for key, value in list(data.items())[:10]:
|
||||
if key == "sample_spec":
|
||||
continue
|
||||
if not isinstance(value, dict):
|
||||
return False
|
||||
if "litellm_provider" not in value:
|
||||
return False
|
||||
return True
|
||||
The `get_model_cost_map()` function now:
|
||||
- Validates JSON structure before accepting remote data
|
||||
- Checks for required fields (`litellm_provider`)
|
||||
- Validates mode values against allowed list
|
||||
- Validates numeric cost fields
|
||||
- Falls back to local backup if validation fails
|
||||
- Logs warnings when falling back
|
||||
|
||||
def get_model_cost_map(url: str) -> dict:
|
||||
# ... existing code ...
|
||||
try:
|
||||
response = httpx.get(url, timeout=5)
|
||||
response.raise_for_status()
|
||||
content = response.json()
|
||||
|
||||
# NEW: Validate before accepting
|
||||
if not validate_model_cost_map(content):
|
||||
verbose_logger.warning(
|
||||
"Remote model cost map failed validation, using local backup"
|
||||
)
|
||||
return _load_local_backup()
|
||||
|
||||
return content
|
||||
except Exception:
|
||||
return _load_local_backup()
|
||||
```
|
||||
### 2. Enhanced CI Validation ✅ IMPLEMENTED
|
||||
|
||||
### 2. Enhanced CI Validation
|
||||
`.github/workflows/test-model-map.yaml` now:
|
||||
- Validates JSON syntax (fast fail with `jq`)
|
||||
- Runs comprehensive schema validation via `scripts/validate_model_cost_map.py`
|
||||
- Validates both main file and backup file
|
||||
- Checks backup freshness
|
||||
|
||||
Add comprehensive validation to `.github/workflows/test-model-map.yaml`:
|
||||
### 3. Validation Script ✅ IMPLEMENTED
|
||||
|
||||
```yaml
|
||||
- name: Validate model_prices_and_context_window.json
|
||||
run: |
|
||||
python scripts/validate_model_cost_map.py
|
||||
```
|
||||
`scripts/validate_model_cost_map.py` provides:
|
||||
- Validates all entries have required fields
|
||||
- Checks mode values against allowed modes
|
||||
- Validates numeric fields are actually numeric
|
||||
- Sanity checks for suspiciously high costs
|
||||
- Detailed error reporting with exit codes for CI
|
||||
|
||||
### 3. Add Monitoring/Alerting
|
||||
### 4. Observability ✅ IMPLEMENTED
|
||||
|
||||
Log when falling back to local backup:
|
||||
|
||||
```python
|
||||
if using_local_backup:
|
||||
verbose_logger.warning(
|
||||
"Using local model cost map backup. "
|
||||
"Remote fetch failed or returned invalid data."
|
||||
)
|
||||
```
|
||||
The code now logs warnings when:
|
||||
- Falling back to local backup due to network errors
|
||||
- Falling back due to validation failures
|
||||
- Using local backup mode
|
||||
|
||||
---
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue