mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-22 00:31:44 +00:00
d05e45893e
- Switch all per-cell tests from @pytest.mark.parametrize("model", ...)
(3 sequential invocations) to a single test that fans out to all 3
Claude tiers via run_claude_models_parallel. Per-cell wall time is now
bounded by the slowest model rather than the sum.
- Add 5 new v0 feature dirs (5 providers each, 25 new test files):
web_search, pdf_input, prompt_caching_1h,
tool_use_streaming, thinking_with_tool_use
Manifest expanded to match.
- Add cross-process token-bucket rate limiter (rate_limiter.py + tests)
so xdist workers stay under per-provider req/s limits during full-grid
runs. New env knobs: LITELLM_COMPAT_RATE_{ANTHROPIC,AZURE,VERTEX_AI,
BEDROCK_CONVERSE,BEDROCK_INVOKE}.
- conftest.py: write per-worker shards under <artifact>.shards/, merge
in the controller; preserve the "don't write empty artifact" guard so
unit-test runs don't clobber a real compat-results.json.
- Vertex test_config.yaml: route project/location through env so the
cron VM can target a different GCP project than the upstream default.
- Add run_compat.sh wrapper for binary-searching ideal req/s per
provider against compat-rate-limit-summary.json output.
32 lines
1,006 B
Python
32 lines
1,006 B
Python
"""Local conftest for the driver unit tests.
|
|
|
|
Installs a hermetic, no-op rate limiter for every test in this
|
|
subdirectory. Without this, importing `cli_driver` and calling
|
|
`run_claude(..., runner=fake)` would silently consume tokens from the
|
|
shared default limiter (which writes to `$TMPDIR/...`), polluting the
|
|
on-disk state another test run might rely on and adding flakiness if
|
|
the env vars say "rate=0.1/s".
|
|
|
|
A no-op limiter (rate=0 for every provider) returns immediately from
|
|
`acquire(...)`, so unit tests behave exactly as they did before the
|
|
limiter was added.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import pytest
|
|
|
|
from tests.claude_code.rate_limiter import (
|
|
ALL_PROVIDERS,
|
|
ProviderConfig,
|
|
RateLimiter,
|
|
use_limiter,
|
|
)
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def _hermetic_rate_limiter(tmp_path):
|
|
config = {p: ProviderConfig(rate_per_sec=0.0, burst=0.0) for p in ALL_PROVIDERS}
|
|
limiter = RateLimiter(config=config, state_dir=tmp_path)
|
|
with use_limiter(limiter):
|
|
yield
|