mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-22 00:31:44 +00:00
d05e45893e
- Switch all per-cell tests from @pytest.mark.parametrize("model", ...)
(3 sequential invocations) to a single test that fans out to all 3
Claude tiers via run_claude_models_parallel. Per-cell wall time is now
bounded by the slowest model rather than the sum.
- Add 5 new v0 feature dirs (5 providers each, 25 new test files):
web_search, pdf_input, prompt_caching_1h,
tool_use_streaming, thinking_with_tool_use
Manifest expanded to match.
- Add cross-process token-bucket rate limiter (rate_limiter.py + tests)
so xdist workers stay under per-provider req/s limits during full-grid
runs. New env knobs: LITELLM_COMPAT_RATE_{ANTHROPIC,AZURE,VERTEX_AI,
BEDROCK_CONVERSE,BEDROCK_INVOKE}.
- conftest.py: write per-worker shards under <artifact>.shards/, merge
in the controller; preserve the "don't write empty artifact" guard so
unit-test runs don't clobber a real compat-results.json.
- Vertex test_config.yaml: route project/location through env so the
cron VM can target a different GCP project than the upstream default.
- Add run_compat.sh wrapper for binary-searching ideal req/s per
provider against compat-rate-limit-summary.json output.
|
||
|---|---|---|
| .. | ||
| _builder_unit_tests | ||
| _driver_unit_tests | ||
| _pr_gate_unit_tests | ||
| basic_messaging_non_streaming | ||
| basic_messaging_streaming | ||
| cron_vm | ||
| extended_thinking | ||
| pdf_input | ||
| prompt_caching_1h | ||
| prompt_caching_5m | ||
| thinking_with_tool_use | ||
| tool_use | ||
| tool_use_streaming | ||
| vision | ||
| web_search | ||
| __init__.py | ||
| cli_driver.py | ||
| conftest.py | ||
| manifest.yaml | ||
| matrix_builder.py | ||
| pr_gate_version_resolver.py | ||
| rate_limiter.py | ||
| run_compat.sh | ||
| sample_compatibility-matrix.json | ||
| test_config.yaml | ||