scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-14 23:21:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 102
litellm/tests/e2e/claude_code
History
Exact
mateo-berri c4f9688c07 perf(e2e/claude_code): run the compat matrix concurrently end to end 
The matrix had three serial stretches left. `run_compat.sh` only ever
handed six of the sixteen feature directories to `pytest -n`, so the
other ten columns ran one cell at a time; the ten HTTP-probe cells
looped their three Claude tiers in sequence, paying the per-provider
rate limiter's wait three times per cell; and the session-scoped
fixture that registers the fifteen compat deployments POSTed them one
at a time, in every xdist worker process

Registration moves to `pytest_configure` on the controller and goes out
in parallel, so a run registers each deployment once instead of once
per worker. Workers learn the controller owns registration through
`pytest_configure_node`; a run whose controller never loaded this
conftest still registers per worker, so no invocation loses its
deployments. The probe cells share one `run_probe_cell` body that fans
the tiers out and reports rows in declared order, replacing ten copies
of the same loop
2026-07-31 21:14:26 -07:00
..
basic_messaging_non_streaming test(e2e): skip flaky OpenAI GPT cells; raise multi-window max_tokens (#33799) 2026-07-17 19:07:18 -07:00
basic_messaging_streaming test(e2e): skip flaky OpenAI GPT cells; raise multi-window max_tokens (#33799) 2026-07-17 19:07:18 -07:00
count_tokens perf(e2e/claude_code): run the compat matrix concurrently end to end 2026-07-31 21:14:26 -07:00
long_context_1m test(e2e): harness fixes for stage job green (skips + router/UI/budget) (#33634) 2026-07-16 20:30:30 -07:00
passthrough test(e2e): harness fixes for stage job green (skips + router/UI/budget) (#33634) 2026-07-16 20:30:30 -07:00
pdf_input test(e2e): harness fixes for stage job green (skips + router/UI/budget) (#33634) 2026-07-16 20:30:30 -07:00
prompt_caching_1h fix(e2e/claude_code): unblock stage collection, align proxy env names, register compat models (#33433) 2026-07-16 11:05:31 -07:00
prompt_caching_5m fix(e2e/claude_code): unblock stage collection, align proxy env names, register compat models (#33433) 2026-07-16 11:05:31 -07:00
structured_outputs fix(e2e/claude_code): unblock stage collection, align proxy env names, register compat models (#33433) 2026-07-16 11:05:31 -07:00
thinking test(e2e): harness fixes for stage job green (skips + router/UI/budget) (#33634) 2026-07-16 20:30:30 -07:00
thinking_with_tool_use fix(e2e/claude_code): unblock stage collection, align proxy env names, register compat models (#33433) 2026-07-16 11:05:31 -07:00
tool_search perf(e2e/claude_code): run the compat matrix concurrently end to end 2026-07-31 21:14:26 -07:00
tool_use test(e2e): skip flaky OpenAI GPT cells; raise multi-window max_tokens (#33799) 2026-07-17 19:07:18 -07:00
tool_use_streaming test(e2e): skip flaky OpenAI GPT cells; raise multi-window max_tokens (#33799) 2026-07-17 19:07:18 -07:00
vision fix(e2e/claude_code): unblock stage collection, align proxy env names, register compat models (#33433) 2026-07-16 11:05:31 -07:00
web_search fix(e2e/claude_code): unblock stage collection, align proxy env names, register compat models (#33433) 2026-07-16 11:05:31 -07:00
__init__.py test(claude_code): move the Claude Code compatibility matrix under tests/e2e (#32548) 2026-07-14 19:19:03 -07:00
_basic_messaging.py fix(e2e/claude_code): unblock stage collection, align proxy env names, register compat models (#33433) 2026-07-16 11:05:31 -07:00
_compat_models.py perf(e2e/claude_code): run the compat matrix concurrently end to end 2026-07-31 21:14:26 -07:00
_env.py refactor(e2e): fold claude_code HTTP probes onto shared Gateway methods (#33760) 2026-07-18 19:03:01 +00:00
_gpt_cells.py test(e2e): skip flaky OpenAI GPT cells; raise multi-window max_tokens (#33799) 2026-07-17 19:07:18 -07:00
_passthrough.py fix(e2e/claude_code): unblock stage collection, align proxy env names, register compat models (#33433) 2026-07-16 11:05:31 -07:00
_probe_cell.py perf(e2e/claude_code): run the compat matrix concurrently end to end 2026-07-31 21:14:26 -07:00
cli_driver.py test(e2e/claude_code): retry rate-limit-shaped CLI failures with backoff 2026-07-15 18:06:28 -07:00
conftest.py perf(e2e/claude_code): run the compat matrix concurrently end to end 2026-07-31 21:14:26 -07:00
http_probe.py refactor(e2e): fold claude_code HTTP probes onto shared Gateway methods (#33760) 2026-07-18 19:03:01 +00:00
manifest.yaml Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_claude_code_e2e_gpt_big3 2026-07-16 15:04:08 -07:00
matrix_builder.py fix(e2e/claude_code): unblock stage collection, align proxy env names, register compat models (#33433) 2026-07-16 11:05:31 -07:00
pr_gate_version_resolver.py test(claude_code): move the Claude Code compatibility matrix under tests/e2e (#32548) 2026-07-14 19:19:03 -07:00
rate_limiter.py test(e2e/claude_code): add GPT-5.6 Sol/Terra/Luna provider columns for OpenAI, Azure OpenAI, and Bedrock Mantle 2026-07-15 16:25:32 -07:00
run_compat.sh perf(e2e/claude_code): run the compat matrix concurrently end to end 2026-07-31 21:14:26 -07:00
sample_compatibility-matrix.json test(claude_code): move the Claude Code compatibility matrix under tests/e2e (#32548) 2026-07-14 19:19:03 -07:00
test_compat_models.py perf(e2e/claude_code): run the compat matrix concurrently end to end 2026-07-31 21:14:26 -07:00
test_config.yaml test(e2e/claude_code): reuse AZURE_API_BASE/KEY for the azure_openai GPT column 2026-07-16 16:45:18 -07:00
test_probe_cell.py perf(e2e/claude_code): run the compat matrix concurrently end to end 2026-07-31 21:14:26 -07:00