mirror of
https://github.com/BerriAI/litellm.git
synced 2026-10-11 03:38:38 +00:00
0220609ab6
This prevents upstream GitHub changes from breaking LLM calls by: 1. Runtime validation in get_model_cost_map.py: - Validates JSON structure before accepting remote data - Checks for required fields (litellm_provider) - Validates mode values against allowed list - Validates numeric cost fields - Falls back to local backup if validation fails - Logs warnings when falling back 2. Enhanced CI validation (.github/workflows/test-model-map.yaml): - JSON syntax validation (fast fail) - Comprehensive schema validation via Python script - Validates both main file and backup file - Checks backup freshness 3. New validation script (scripts/validate_model_cost_map.py): - Validates all entries have required fields - Checks mode values against allowed modes - Validates numeric fields are actually numeric - Sanity checks for suspiciously high costs - Detailed error reporting 4. Unit tests for validation logic The key improvement: if GitHub returns valid JSON but semantically broken data (missing fields, wrong types), we now detect this and fall back to the local backup instead of using the bad data. Customers can also set LITELLM_LOCAL_MODEL_COST_MAP=True to completely disable remote fetching for maximum stability. Co-authored-by: ishaan <ishaan@berri.ai> |
||
|---|---|---|
| .. | ||
| a2a_protocol | ||
| anthropic_interface/exceptions | ||
| caching | ||
| completion_extras | ||
| containers | ||
| enterprise | ||
| experimental_mcp_client | ||
| google_genai | ||
| images | ||
| integrations | ||
| interactions | ||
| litellm_core_utils | ||
| llms | ||
| passthrough | ||
| proxy | ||
| responses | ||
| router_strategy | ||
| router_utils | ||
| secret_managers | ||
| types | ||
| vector_stores | ||
| __init__.py | ||
| conftest.py | ||
| log.txt | ||
| readme.md | ||
| test_acompletion_session_reuse_e2e.py | ||
| test_add_deployment_no_master_key.py | ||
| test_aembedding_session_reuse_e2e.py | ||
| test_azure_video_router.py | ||
| test_claude_haiku_4_5_config.py | ||
| test_constants.py | ||
| test_container_router.py | ||
| test_cost_calculation_log_level.py | ||
| test_cost_calculator.py | ||
| test_eager_tiktoken_load.py | ||
| test_exception_header_preservation.py | ||
| test_exception_mapping_request_attribute.py | ||
| test_filter_out_litellm_params.py | ||
| test_gpt_image_cost_calculator.py | ||
| test_groq_streaming_encoding.py | ||
| test_lazy_imports.py | ||
| test_logging.py | ||
| test_lowest_latency_zero_tokens.py | ||
| test_main.py | ||
| test_model_response_normalization.py | ||
| test_nested_drop_params.py | ||
| test_redis.py | ||
| test_responses_api_bridge_non_stream.py | ||
| test_responses_id_security.py | ||
| test_router.py | ||
| test_router_google_genai.py | ||
| test_router_per_deployment_num_retries.py | ||
| test_router_silent_experiment.py | ||
| test_shared_session_integration.py | ||
| test_ssl_verify_unit.py | ||
| test_system_message_format_bug.py | ||
| test_utils.py | ||
| test_uuid_helper.py | ||
| test_video_generation.py | ||
| test_xai_responses_auto_routing.py | ||
Testing for litellm/
This directory 1:1 maps the the litellm/ directory, and can only contain mocked tests.
The point of this is to:
- Increase test coverage of
litellm/ - Make it easy for contributors to add tests for the
litellm/package and easily run tests without needing LLM API keys.
File name conventions
litellm/proxy/test_caching_routes.pymaps tolitellm/proxy/caching_routes.pytest_<filename>.pymaps tolitellm/<filename>.py