scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-13 23:11:40 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 100
litellm/tests/e2e
History
Exact
mateo-berri 09f2a9528a
test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 
Model IDs were hardcoded across the e2e suite, so every provider
deprecation meant a scatter-shot edit and stale models (gemini-2.5-flash,
gpt-4o-mini) kept running in CI. Tests now import role-named pins from
tests/e2e/model_matrix.py, and a bump is a one-file change plus the
compose gateway config.

check_e2e_model_freshness.py (wired into test-code-quality) fails when a
pin is missing from model_prices_and_context_window.json, is within 30
days of its deprecation_date, drifts from the docker-compose gateway
config, or when a test hardcodes a versioned model literal again.

Bumps: gemini-2.5-flash -> gemini-3.5-flash, gpt-4o-mini -> gpt-5.4-mini
2026-07-08 07:15:58 +00:00
..
access_control test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 2026-07-08 07:15:58 +00:00
batches test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 2026-07-08 07:15:58 +00:00
budgets test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 2026-07-08 07:15:58 +00:00
coverage_registry test: add e2e coverage module metrics (#32403) 2026-07-07 19:38:56 -07:00
llm_translation test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 2026-07-08 07:15:58 +00:00
logging test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 2026-07-08 07:15:58 +00:00
management test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 2026-07-08 07:15:58 +00:00
realtime test: add realtime proxy e2e suite across providers (#30960) 2026-06-26 09:36:49 -07:00
spend_tracking test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 2026-07-08 07:15:58 +00:00
CLAUDE.md test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 2026-07-08 07:15:58 +00:00
conftest.py test(e2e): add coverage registry and collector (#32304) 2026-07-07 15:51:20 -04:00
CONTRIBUTING.md test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 2026-07-08 07:15:58 +00:00
docker-compose.yml test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 2026-07-08 07:15:58 +00:00
e2e_config.py fix(ui): scope key models dropdown options to the key's team (#32382) 2026-07-07 18:54:19 -07:00
e2e_gateway.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00
e2e_http.py test(e2e): add live batches suite across providers and routing scenarios (#30958) 2026-07-02 08:05:23 -07:00
lifecycle.py test(e2e): add live batches suite across providers and routing scenarios (#30958) 2026-07-02 08:05:23 -07:00
model_matrix.py test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 2026-07-08 07:15:58 +00:00
models.py test(e2e): add management suite covering key/team/user/org lifecycle and route permissions (#32300) 2026-07-06 19:11:27 -07:00
pytest.ini test: add e2e tests for spend, budgets and llms (#30869) 2026-06-24 15:01:57 -07:00
test_e2e_gateway.py test(e2e): centralize model pins in model_matrix.py and enforce freshness in CI 2026-07-08 07:15:58 +00:00
test_lifecycle.py test: add e2e tests for spend, budgets and llms (#30869) 2026-06-24 15:01:57 -07:00
test_transport.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00
transport.py test(e2e): probe the full spend read surface including schema-hidden routes (#32267) 2026-07-06 14:02:05 -07:00