litellm/tests/e2e/claude_code
Yuneng Jiang 39acea0754
feat(e2e): make Claude Code send the same bytes every build
The compat cells drove the CLI with a fresh HOME per invocation and the
pytest process's own working directory, and both reach the request body.
The system prompt names a memory directory built from
$CLAUDE_CONFIG_DIR/projects/<cwd slug>, so a per-invocation config
directory rewrote every body, and the CLI adds a git block for its
working directory, so inheriting the checkout rewrote every body once
per candidate. The device id churned for the same reason: the CLI mints
it once and persists it in .claude.json, which we threw away each call.

Nothing here was load-bearing. All three ride in metadata.user_id, whose
job is abuse detection, not quota, caching or continuity. So pin the
config directory and the working directory at fixed paths, seed the
device id, and pin the session id.

HOME stays fresh and empty per invocation, so the isolation is no weaker
than before, and the CLI's own state no longer outlives the pod either.
The working directory is deliberately not the checkout, so a
model-directed Read now sees an empty directory rather than the
repository.

A pinned session id needs --no-session-persistence beside it: the CLI
refuses a session id another live process holds, and the matrix runs its
cells across xdist workers. Without the flag, six of eight concurrent
invocations die on "Session ID is already in use".
2026-09-16 12:42:15 -07:00
..
_builder_unit_tests chore(e2e): port the compat-matrix cron publisher to tests/e2e/claude_code 2026-08-10 21:46:56 +00:00
_driver_unit_tests feat(e2e): make Claude Code send the same bytes every build 2026-09-16 12:42:15 -07:00
_probe_unit_tests test(e2e): replay a real tool-search assistant turn back to Bedrock Invoke (#36856) 2026-08-17 11:59:26 -07:00
basic_messaging_non_streaming test(e2e): skip flaky OpenAI GPT cells; raise multi-window max_tokens (#33799) 2026-07-17 19:07:18 -07:00
basic_messaging_streaming test(e2e): skip flaky OpenAI GPT cells; raise multi-window max_tokens (#33799) 2026-07-17 19:07:18 -07:00
count_tokens refactor(e2e): fold claude_code HTTP probes onto shared Gateway methods (#33760) 2026-07-18 19:03:01 +00:00
cron_vm fix(compat-matrix): keep publish token out of the job-wide process env 2026-08-11 00:57:25 +00:00
long_context_1m test(e2e): harness fixes for stage job green (skips + router/UI/budget) (#33634) 2026-07-16 20:30:30 -07:00
passthrough test(e2e): harness fixes for stage job green (skips + router/UI/budget) (#33634) 2026-07-16 20:30:30 -07:00
pdf_input fix(bedrock): add text block to converse user messages carrying documents 2026-08-11 02:35:49 +00:00
prompt_caching_1h fix(e2e/claude_code): unblock stage collection, align proxy env names, register compat models (#33433) 2026-07-16 11:05:31 -07:00
prompt_caching_5m fix(e2e/claude_code): unblock stage collection, align proxy env names, register compat models (#33433) 2026-07-16 11:05:31 -07:00
structured_outputs fix(e2e/claude_code): unblock stage collection, align proxy env names, register compat models (#33433) 2026-07-16 11:05:31 -07:00
thinking fix(bedrock): preserve adaptive thinking effort through the /v1/messages bridge 2026-08-11 03:23:16 +00:00
thinking_with_tool_use fix(e2e/claude_code): unblock stage collection, align proxy env names, register compat models (#33433) 2026-07-16 11:05:31 -07:00
tool_search test(e2e): replay a real tool-search assistant turn back to Bedrock Invoke (#36856) 2026-08-17 11:59:26 -07:00
tool_use test(e2e): skip flaky OpenAI GPT cells; raise multi-window max_tokens (#33799) 2026-07-17 19:07:18 -07:00
tool_use_streaming test(e2e): skip flaky OpenAI GPT cells; raise multi-window max_tokens (#33799) 2026-07-17 19:07:18 -07:00
vision fix(e2e/claude_code): unblock stage collection, align proxy env names, register compat models (#33433) 2026-07-16 11:05:31 -07:00
web_search fix(e2e/claude_code): unblock stage collection, align proxy env names, register compat models (#33433) 2026-07-16 11:05:31 -07:00
__init__.py test(claude_code): move the Claude Code compatibility matrix under tests/e2e (#32548) 2026-07-14 19:19:03 -07:00
_basic_messaging.py fix(e2e/claude_code): unblock stage collection, align proxy env names, register compat models (#33433) 2026-07-16 11:05:31 -07:00
_compat_models.py fix(e2e/claude_code): unblock stage collection, align proxy env names, register compat models (#33433) 2026-07-16 11:05:31 -07:00
_env.py fix(e2e): wait for every gateway before using a new model and keep the network rerun 2026-09-05 16:10:40 -07:00
_gpt_cells.py test(e2e): skip flaky OpenAI GPT cells; raise multi-window max_tokens (#33799) 2026-07-17 19:07:18 -07:00
_passthrough.py fix(e2e/claude_code): unblock stage collection, align proxy env names, register compat models (#33433) 2026-07-16 11:05:31 -07:00
cli_driver.py feat(e2e): make Claude Code send the same bytes every build 2026-09-16 12:42:15 -07:00
conftest.py fix(e2e): wait for every gateway before using a new model and keep the network rerun 2026-09-05 16:10:40 -07:00
http_probe.py test(e2e): replay a real tool-search assistant turn back to Bedrock Invoke (#36856) 2026-08-17 11:59:26 -07:00
manifest.yaml Merge remote-tracking branch 'origin/litellm_internal_staging' into litellm_claude_code_e2e_gpt_big3 2026-07-16 15:04:08 -07:00
matrix_builder.py chore(e2e): port the compat-matrix cron publisher to tests/e2e/claude_code 2026-08-10 21:46:56 +00:00
pr_gate_version_resolver.py test(claude_code): move the Claude Code compatibility matrix under tests/e2e (#32548) 2026-07-14 19:19:03 -07:00
rate_limiter.py test(e2e/claude_code): add GPT-5.6 Sol/Terra/Luna provider columns for OpenAI, Azure OpenAI, and Bedrock Mantle 2026-07-15 16:25:32 -07:00
run_compat.sh test(e2e/claude_code): update run_compat.sh flag docs to COMPAT_MANTLE_CELLS 2026-07-16 17:26:46 -07:00
sample_compatibility-matrix.json test(claude_code): move the Claude Code compatibility matrix under tests/e2e (#32548) 2026-07-14 19:19:03 -07:00
test_config.yaml test(e2e/claude_code): reuse AZURE_API_BASE/KEY for the azure_openai GPT column 2026-07-16 16:45:18 -07:00