litellm/tests/claude_code
Cursor Agent 0915e87f08
fix(ci): bugbot — persist compat-results artifacts from PR gate
The conftest's pytest_sessionfinish writes per-cell tagged-union JSON
(compat-results.json) and the per-provider rate-limit summary to paths
controlled by COMPAT_RESULTS_PATH / COMPAT_RATE_LIMIT_SUMMARY_PATH.
The PR gate never set either env var, so the artifacts were written to
the working directory — but store_test_results only collects
test-results/junit.xml, leaving the per-cell JSON unreachable from the
CircleCI artifact browser. Reviewers triaging a red PR gate couldn't
pull the cell-level breakdown without re-running.

Point both env vars at a dedicated compat-artifacts/ directory and add
a store_artifacts step so the JSON blobs become downloadable. Also
extend the existing PR-gate wiring test to pin all three pieces:
COMPAT_RESULTS_PATH override, COMPAT_RATE_LIMIT_SUMMARY_PATH override,
and at least one store_artifacts step whose path matches the directory
the exports point at. Without the cross-check, a future refactor could
break the chain (e.g. only export the env vars, or only add the
store_artifacts step) and the wiring would silently drop the artifacts
again.

Co-authored-by: Mateo Wang <mateo-berri@users.noreply.github.com>
2026-05-17 21:39:48 +00:00
..
_builder_unit_tests fix(claude_code): harden parallel runner + de-dup basic_messaging cells 2026-05-17 06:35:15 +00:00
_driver_unit_tests fix(claude_code): harden parallel runner + de-dup basic_messaging cells 2026-05-17 06:35:15 +00:00
_pr_gate_unit_tests fix(ci): bugbot — persist compat-results artifacts from PR gate 2026-05-17 21:39:48 +00:00
_publisher_unit_tests fix(cron): bugbot — paginate all release pages to pick highest-semver stable 2026-05-17 21:39:36 +00:00
basic_messaging_non_streaming fix(claude_code): harden parallel runner + de-dup basic_messaging cells 2026-05-17 06:35:15 +00:00
basic_messaging_streaming claude_code compat: skip skipped reports; drop unreachable stream-events check 2026-05-17 06:51:02 +00:00
count_tokens feat(claude_code): rename thinking row + add 4 feature rows (15 total) 2026-05-16 20:37:01 +00:00
cron_vm fix(cron): bugbot — paginate all release pages to pick highest-semver stable 2026-05-17 21:39:36 +00:00
long_context_1m feat(claude_code): rename thinking row + add 4 feature rows (15 total) 2026-05-16 20:37:01 +00:00
pdf_input compat-matrix: parallel-fanout refactor + 5 new feature dirs + rate limiter 2026-05-06 23:31:19 +00:00
prompt_caching_1h compat-matrix: parallel-fanout refactor + 5 new feature dirs + rate limiter 2026-05-06 23:31:19 +00:00
prompt_caching_5m compat-matrix: parallel-fanout refactor + 5 new feature dirs + rate limiter 2026-05-06 23:31:19 +00:00
structured_outputs feat(claude_code): rename thinking row + add 4 feature rows (15 total) 2026-05-16 20:37:01 +00:00
thinking feat(claude_code): rename thinking row + add 4 feature rows (15 total) 2026-05-16 20:37:01 +00:00
thinking_with_tool_use compat-matrix: fix vision, extended_thinking, web_search test bugs 2026-05-07 02:16:10 +00:00
tool_search feat(claude_code): rename thinking row + add 4 feature rows (15 total) 2026-05-16 20:37:01 +00:00
tool_use compat-matrix: parallel-fanout refactor + 5 new feature dirs + rate limiter 2026-05-06 23:31:19 +00:00
tool_use_streaming compat-matrix: parallel-fanout refactor + 5 new feature dirs + rate limiter 2026-05-06 23:31:19 +00:00
vision compat-matrix: fix vision, extended_thinking, web_search test bugs 2026-05-07 02:16:10 +00:00
web_search compat-matrix: fix vision, extended_thinking, web_search test bugs 2026-05-07 02:16:10 +00:00
__init__.py RALPH: tracer-bullet for Claude Code compatibility matrix (#26477, PRD #26476) 2026-05-06 23:27:05 +00:00
_basic_messaging.py claude_code compat: skip skipped reports; drop unreachable stream-events check 2026-05-17 06:51:02 +00:00
cli_driver.py fix(claude_code): harden parallel runner + de-dup basic_messaging cells 2026-05-17 06:35:15 +00:00
conftest.py fix: clear manifest cache between sessions and align PR gate pytest with cron 2026-05-17 07:04:49 +00:00
http_probe.py fix(claude_code): rate-limit HTTP probe rows alongside CLI rows 2026-05-17 01:40:23 +00:00
manifest.yaml feat(claude_code): rename thinking row + add 4 feature rows (15 total) 2026-05-16 20:37:01 +00:00
matrix_builder.py RALPH: tracer-bullet for Claude Code compatibility matrix (#26477, PRD #26476) 2026-05-06 23:27:05 +00:00
pr_gate_version_resolver.py RALPH: compat matrix slice 3 - wire PR gate in CircleCI (#26479, PRD #26476) 2026-05-06 23:27:05 +00:00
rate_limiter.py compat-matrix: parallel-fanout refactor + 5 new feature dirs + rate limiter 2026-05-06 23:31:19 +00:00
run_compat.sh feat(claude_code): rename thinking row + add 4 feature rows (15 total) 2026-05-16 20:37:01 +00:00
sample_compatibility-matrix.json feat(claude_code): rename thinking row + add 4 feature rows (15 total) 2026-05-16 20:37:01 +00:00
test_config.yaml compat-matrix: parallel-fanout refactor + 5 new feature dirs + rate limiter 2026-05-06 23:31:19 +00:00