litellm/tests/claude_code/_builder_unit_tests
mateo-berri 5d121fa697
fix(matrix-builder): greptile — surface pass over not_applicable when mixed
When a cell aggregates per-model results across three tiers (Haiku/Sonnet/Opus),
a mix of (pass, not_applicable) used to short-circuit to not_applicable on the
first NA match, discarding the passing tiers from the published matrix.

Treat not_applicable like not_tested when mixed with pass: only return
not_applicable when every observed row is NA. Otherwise any pass surfaces as
pass, so the cell answers 'does this feature work on this provider?' truthfully
when at least one tier passes.

Add two regression tests pinning the new precedence:
  - mixed pass + NA  → pass
  - all NA           → not_applicable (with first reason)
2026-05-19 03:01:19 +00:00
..
fixtures RALPH: tracer-bullet for Claude Code compatibility matrix (#26477, PRD #26476) 2026-05-06 23:27:05 +00:00
__init__.py RALPH: tracer-bullet for Claude Code compatibility matrix (#26477, PRD #26476) 2026-05-06 23:27:05 +00:00
test_matrix_builder.py fix(matrix-builder): greptile — surface pass over not_applicable when mixed 2026-05-19 03:01:19 +00:00
test_v0_layout.py fix(claude_code): bugbot — aggregate all fail errors + structural test every manifest feature 2026-05-18 03:48:23 +00:00