litellm/tests/e2e/claude_code
yuneng-jiang b18188dee2
test(e2e): retire the quiet_stack lock and batch the claude_code model registration (#49984)
* test(e2e): retire the quiet_stack lock

quiet_stack held an exclusive host-wide lock for six tests, draining every
other xdist worker while they ran. On the full e2e lane that left 35-38% of
worker time idle. Each of those tests now lives where it can run without
pausing the suite

- the two memory tests move to perf/ behind the perf marker, so the perf lane
  runs them on its own fresh stack. Their refusing deployments use a literal
  key because the perf stack carries no provider keys
- the stored-request size check from the memory test becomes an integration
  test on an owned proxy that stores prompts
- the discount and margin spend-row tests are dropped from e2e. Their
  breakdown assertions join the existing integration pricing test, which
  already owns the discount and margin repricing flow
- the Bedrock guardrail cost test stays in e2e without the pricing reset
  fixture, since nothing else on the stack writes pricing config now
- the MCP semantic tool filter test turns on a proxy-wide setting, so it gets
  a new proxy_wide_config opt-in marker and runs after the main run

* test(e2e): register the claude_code deployments as one batch

The claude_code session fixture registered its 24 deployments one at a time on every xdist worker, and each one waited out the 15 s propagation window on its own, about 6 minutes per worker. ProxyClient.register_models posts the whole batch, polls each model until every replica serves it, then waits out propagation once from the last write. A model that fails to post or never becomes servable comes back as ModelNotRegistered, and an unservable one is deleted, so the fixture keeps reporting per-deployment failures the way it did
2026-11-04 09:55:53 -08:00
..
basic_messaging_non_streaming test(e2e): tag claude_code cells with Subject metadata and record CLI driver steps (#44961) 2026-10-07 10:31:54 -07:00
basic_messaging_streaming test(e2e): tag claude_code cells with Subject metadata and record CLI driver steps (#44961) 2026-10-07 10:31:54 -07:00
count_tokens test(e2e): tag claude_code cells with Subject metadata and record CLI driver steps (#44961) 2026-10-07 10:31:54 -07:00
cron_vm test(e2e): share the coding-CLI harness between suites (#49370) 2026-11-02 13:50:56 -08:00
long_context_1m test(e2e): tag claude_code cells with Subject metadata and record CLI driver steps (#44961) 2026-10-07 10:31:54 -07:00
passthrough test(e2e): tag claude_code cells with Subject metadata and record CLI driver steps (#44961) 2026-10-07 10:31:54 -07:00
pdf_input test(e2e): tag claude_code cells with Subject metadata and record CLI driver steps (#44961) 2026-10-07 10:31:54 -07:00
prompt_caching_1h test(e2e): tag claude_code cells with Subject metadata and record CLI driver steps (#44961) 2026-10-07 10:31:54 -07:00
prompt_caching_5m test: fix stale and flaky tests across CircleCI, GHA and Buildkite (#45530) 2026-10-09 11:40:14 -07:00
structured_outputs test(e2e): tag claude_code cells with Subject metadata and record CLI driver steps (#44961) 2026-10-07 10:31:54 -07:00
thinking test(e2e): tag claude_code cells with Subject metadata and record CLI driver steps (#44961) 2026-10-07 10:31:54 -07:00
thinking_with_tool_use test(e2e): tag claude_code cells with Subject metadata and record CLI driver steps (#44961) 2026-10-07 10:31:54 -07:00
tool_search test(e2e): tag claude_code cells with Subject metadata and record CLI driver steps (#44961) 2026-10-07 10:31:54 -07:00
tool_use test(e2e): tag claude_code cells with Subject metadata and record CLI driver steps (#44961) 2026-10-07 10:31:54 -07:00
tool_use_streaming test(e2e): tag claude_code cells with Subject metadata and record CLI driver steps (#44961) 2026-10-07 10:31:54 -07:00
vision test(e2e): tag claude_code cells with Subject metadata and record CLI driver steps (#44961) 2026-10-07 10:31:54 -07:00
web_search test(e2e): tag claude_code cells with Subject metadata and record CLI driver steps (#44961) 2026-10-07 10:31:54 -07:00
__init__.py test(claude_code): move the Claude Code compatibility matrix under tests/e2e (#32548) 2026-07-14 19:19:03 -07:00
_basic_messaging.py test(e2e): tag claude_code cells with Subject metadata and record CLI driver steps (#44961) 2026-10-07 10:31:54 -07:00
_compat_models.py fix(e2e/claude_code): unblock stage collection, align proxy env names, register compat models (#33433) 2026-07-16 11:05:31 -07:00
_env.py feat(e2e): read management routes back from the control plane replicas (#43373) 2026-09-26 18:05:55 -07:00
_gpt_cells.py test(e2e): skip flaky OpenAI GPT cells; raise multi-window max_tokens (#33799) 2026-07-17 19:07:18 -07:00
_passthrough.py test(e2e): move the harness self-tests out of tests/e2e (#45172) 2026-10-07 22:20:02 +00:00
cli_driver.py test(e2e): share the coding-CLI harness between suites (#49370) 2026-11-02 13:50:56 -08:00
conftest.py test(e2e): retire the quiet_stack lock and batch the claude_code model registration (#49984) 2026-11-04 09:55:53 -08:00
http_probe.py test(e2e): share the coding-CLI harness between suites (#49370) 2026-11-02 13:50:56 -08:00
manifest.yaml Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_claude_code_e2e_gpt_big3 2026-07-16 15:04:08 -07:00
matrix_builder.py chore(e2e): port the compat-matrix cron publisher to tests/e2e/claude_code 2026-08-10 21:46:56 +00:00
pr_gate_version_resolver.py test(e2e): share the coding-CLI harness between suites (#49370) 2026-11-02 13:50:56 -08:00
run_compat.sh test(e2e): share the coding-CLI harness between suites (#49370) 2026-11-02 13:50:56 -08:00
sample_compatibility-matrix.json test(claude_code): move the Claude Code compatibility matrix under tests/e2e (#32548) 2026-07-14 19:19:03 -07:00
test_config.yaml test(e2e/claude_code): reuse AZURE_API_BASE/KEY for the azure_openai GPT column 2026-07-16 16:45:18 -07:00