Commit graph

3 commits

Author SHA1 Message Date
mateo-berri
f49a3e15a8 test(e2e): read JUnit properties off the real collected pytest Item
tests/e2e/test_junit_properties.py fed a hand-rolled FakeItem to
result_properties and attach_result_properties, both typed pytest.Item,
so uv run basedpyright tests/e2e reported 3 reportArgumentType errors on
litellm_internal_staging and every make check that scopes a litellm/ or
tests/e2e/ Python file failed.

Each test now looks up its own collected Item in request.session.items
and applies the covers marker at run time through request.applymarker,
so the coverage registry's collect-only pass never sees the test ids and
the production functions keep their pytest.Item signatures. No casts, no
ignores.

Resolves LIT-6669
2026-09-01 19:12:07 -07:00
ryan-crabbe-berri
0c2d4c5773 Refuse a source path carrying a colon
`path:line` cannot represent a path that itself contains a colon, and the
one way pytest produces one is a Windows absolute location: separator
normalization turns `C:\app\e2e\a2a\test_x.py` into `C:/app/...`, which
slipped past the leading-slash check and composed the nonsense repo path
`tests/e2e/C:/app/e2e/a2a/test_x.py`.

Reject the colon itself rather than special-casing a drive letter: it is
the character the format reserves, so no path containing one was ever
linkable.

Claude-Session: https://claude.ai/code/session_017dTKXwJkzhtVLzDhePHsKG
2026-09-01 16:02:54 -07:00
ryan-crabbe-berri
00e40c0afe Record each e2e test's source location in the JUnit report
The JUnit report is the only thing that leaves the e2e run, and it says
where a test's results came from but never where its code lives. A reader
looking at `test_cell_claimed_only_by_a_skipped_test_is_uncovered` on the
status page has a name and nothing else -- no file, no line, no way to
reach the source short of grepping the repo by hand.

Pytest knows the location; the report format loses it. The `xunit1` family
wrote `file=` and `line=` onto every `<testcase>`, and the `xunit2` default
this suite runs on drops both. Switching families back would change the
document for every consumer of the same XML -- the Buildkite Test Engine
upload and the Loki pipeline included -- so add the location the way this
suite already adds `package` and `covers`: as a `<property>`, which is
purely additive.

`source` is repo-relative and one-based (`tests/e2e/a2a/test_x.py:41`), so
a consumer can build a link without knowing how pytest was started. That
takes normalizing the two launch shapes -- the runner image runs from its
own copy at /app/e2e, a developer runs from the repo root -- which is the
same normalization `package_from_nodeid` was already doing in reverse, now
factored into `suite_parts` so the two cannot drift apart. Paths that
escape the suite, and tests pytest reports no line for, emit an empty
string: a test with no link beats a link that 404s.

Claude-Session: https://claude.ai/code/session_017dTKXwJkzhtVLzDhePHsKG
2026-09-01 16:02:54 -07:00