scriptoriumadmin/litellm
1
0
Fork 0
mirror of https://github.com/BerriAI/litellm.git synced 2026-09-22 00:31:44 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 51
litellm/tests/claude_code
History
mateo-berri 5d121fa697
fix(matrix-builder): greptile — surface pass over not_applicable when mixed 
When a cell aggregates per-model results across three tiers (Haiku/Sonnet/Opus),
a mix of (pass, not_applicable) used to short-circuit to not_applicable on the
first NA match, discarding the passing tiers from the published matrix.

Treat not_applicable like not_tested when mixed with pass: only return
not_applicable when every observed row is NA. Otherwise any pass surfaces as
pass, so the cell answers 'does this feature work on this provider?' truthfully
when at least one tier passes.

Add two regression tests pinning the new precedence:
  - mixed pass + NA  → pass
  - all NA           → not_applicable (with first reason)
2026-05-19 03:01:19 +00:00
..
_builder_unit_tests fix(matrix-builder): greptile — surface pass over not_applicable when mixed 2026-05-19 03:01:19 +00:00
_driver_unit_tests fix(claude_code): greptile — robust version extraction + narrow pytest.raises 2026-05-19 02:37:48 +00:00
_pr_gate_unit_tests fix(pr-gate): greptile — exclude npm pre-release tags from version resolver 2026-05-19 01:58:23 +00:00
_publisher_unit_tests fix(cron): bugbot — paginate all release pages to pick highest-semver stable 2026-05-17 21:39:36 +00:00
basic_messaging_non_streaming fix(claude_code): harden parallel runner + de-dup basic_messaging cells 2026-05-17 06:35:15 +00:00
basic_messaging_streaming fix(claude_code): verify streaming wire in basic_messaging_streaming cells 2026-05-17 22:47:47 +00:00
count_tokens feat(claude_code): rename thinking row + add 4 feature rows (15 total) 2026-05-16 20:37:01 +00:00
cron_vm fix(claude_code): greptile — robust version extraction + narrow pytest.raises 2026-05-19 02:37:48 +00:00
long_context_1m feat(claude_code): rename thinking row + add 4 feature rows (15 total) 2026-05-16 20:37:01 +00:00
pdf_input compat-matrix: parallel-fanout refactor + 5 new feature dirs + rate limiter 2026-05-06 23:31:19 +00:00
prompt_caching_1h compat-matrix: parallel-fanout refactor + 5 new feature dirs + rate limiter 2026-05-06 23:31:19 +00:00
prompt_caching_5m compat-matrix: parallel-fanout refactor + 5 new feature dirs + rate limiter 2026-05-06 23:31:19 +00:00
structured_outputs feat(claude_code): rename thinking row + add 4 feature rows (15 total) 2026-05-16 20:37:01 +00:00
thinking feat(claude_code): rename thinking row + add 4 feature rows (15 total) 2026-05-16 20:37:01 +00:00
thinking_with_tool_use fix(ci): scrub pytest env + narrow Bash tool-use to exact echo pong 2026-05-18 00:25:43 +00:00
tool_search feat(claude_code): rename thinking row + add 4 feature rows (15 total) 2026-05-16 20:37:01 +00:00
tool_use fix(ci): scrub pytest env + narrow Bash tool-use to exact echo pong 2026-05-18 00:25:43 +00:00
tool_use_streaming fix(ci): scrub pytest env + narrow Bash tool-use to exact echo pong 2026-05-18 00:25:43 +00:00
vision compat-matrix: fix vision, extended_thinking, web_search test bugs 2026-05-07 02:16:10 +00:00
web_search compat-matrix: fix vision, extended_thinking, web_search test bugs 2026-05-07 02:16:10 +00:00
__init__.py RALPH: tracer-bullet for Claude Code compatibility matrix (#26477, PRD #26476) 2026-05-06 23:27:05 +00:00
_basic_messaging.py fix(claude_code): verify streaming wire in basic_messaging_streaming cells 2026-05-17 22:47:47 +00:00
cli_driver.py fix(claude_code): harden parallel runner + de-dup basic_messaging cells 2026-05-17 06:35:15 +00:00
conftest.py fix: clear manifest cache between sessions and align PR gate pytest with cron 2026-05-17 07:04:49 +00:00
http_probe.py fix(claude_code): rate-limit HTTP probe rows alongside CLI rows 2026-05-17 01:40:23 +00:00
manifest.yaml feat(claude_code): rename thinking row + add 4 feature rows (15 total) 2026-05-16 20:37:01 +00:00
matrix_builder.py fix(matrix-builder): greptile — surface pass over not_applicable when mixed 2026-05-19 03:01:19 +00:00
pr_gate_version_resolver.py fix(pr-gate): greptile — exclude npm pre-release tags from version resolver 2026-05-19 01:58:23 +00:00
rate_limiter.py compat-matrix: parallel-fanout refactor + 5 new feature dirs + rate limiter 2026-05-06 23:31:19 +00:00
run_compat.sh feat(claude_code): rename thinking row + add 4 feature rows (15 total) 2026-05-16 20:37:01 +00:00
sample_compatibility-matrix.json feat(claude_code): rename thinking row + add 4 feature rows (15 total) 2026-05-16 20:37:01 +00:00
test_config.yaml compat-matrix: parallel-fanout refactor + 5 new feature dirs + rate limiter 2026-05-06 23:31:19 +00:00