mirror of
https://github.com/BerriAI/litellm.git
synced 2026-09-22 00:31:44 +00:00
When a cell aggregates per-model results across three tiers (Haiku/Sonnet/Opus), a mix of (pass, not_applicable) used to short-circuit to not_applicable on the first NA match, discarding the passing tiers from the published matrix. Treat not_applicable like not_tested when mixed with pass: only return not_applicable when every observed row is NA. Otherwise any pass surfaces as pass, so the cell answers 'does this feature work on this provider?' truthfully when at least one tier passes. Add two regression tests pinning the new precedence: - mixed pass + NA → pass - all NA → not_applicable (with first reason) |
||
|---|---|---|
| .. | ||
| fixtures | ||
| __init__.py | ||
| test_matrix_builder.py | ||
| test_v0_layout.py | ||