scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-10-05 02:41:56 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 49
litellm/tests/e2e/llm_translation
History
Exact
mateo-berri 09f2a9528a
test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 
Model IDs were hardcoded across the e2e suite, so every provider
deprecation meant a scatter-shot edit and stale models (gemini-2.5-flash,
gpt-4o-mini) kept running in CI. Tests now import role-named pins from
tests/e2e/model_matrix.py, and a bump is a one-file change plus the
compose gateway config.

check_e2e_model_freshness.py (wired into test-code-quality) fails when a
pin is missing from model_prices_and_context_window.json, is within 30
days of its deprecation_date, drifts from the docker-compose gateway
config, or when a test hardcodes a versioned model literal again.

Bumps: gemini-2.5-flash -> gemini-3.5-flash, gpt-4o-mini -> gpt-5.4-mini
2026-07-08 07:15:58 +00:00
..
conftest.py test(e2e): close coverage gaps across chat/responses, provider features, batches, prometheus, and langfuse eviction (#32165) 2026-07-04 18:56:52 -07:00
endpoints_client.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00
LLM_TRANSLATION_COVERAGE_MATRIX.md test(e2e): add vertex_ai passthrough spend-log coverage (#31781) 2026-07-02 17:32:46 -07:00
passthrough_client.py test(e2e): add vertex_ai passthrough spend-log coverage (#31781) 2026-07-02 17:32:46 -07:00
test_audio_speech_e2e.py test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 2026-07-08 07:15:58 +00:00
test_chat_completions_regression_e2e.py test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 2026-07-08 07:15:58 +00:00
test_custom_pricing_e2e.py test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 2026-07-08 07:15:58 +00:00
test_deepseek_reasoning_e2e.py test(e2e): close coverage gaps across chat/responses, provider features, batches, prometheus, and langfuse eviction (#32165) 2026-07-04 18:56:52 -07:00
test_embeddings_endpoint_e2e.py test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 2026-07-08 07:15:58 +00:00
test_image_generation_e2e.py test(e2e): migrate access-control and inference-endpoint regression tests (#32016) 2026-07-05 01:39:10 +00:00
test_messages_e2e.py test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 2026-07-08 07:15:58 +00:00
test_ocr_rust_e2e.py Merge pull request #32166 from BerriAI/litellm_e2e_batches_ocr_model_registration 2026-07-05 02:15:10 +00:00
test_passthrough_e2e.py test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 2026-07-08 07:15:58 +00:00
test_provider_features_e2e.py test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 2026-07-08 07:15:58 +00:00
test_rerank_e2e.py test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 2026-07-08 07:15:58 +00:00
test_responses_e2e.py test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 2026-07-08 07:15:58 +00:00
test_vertex_passthrough_e2e.py test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 2026-07-08 07:15:58 +00:00