Commit graph

3 commits

Author SHA1 Message Date
ryan-crabbe-berri
0c2d4c5773 Refuse a source path carrying a colon
`path:line` cannot represent a path that itself contains a colon, and the
one way pytest produces one is a Windows absolute location: separator
normalization turns `C:\app\e2e\a2a\test_x.py` into `C:/app/...`, which
slipped past the leading-slash check and composed the nonsense repo path
`tests/e2e/C:/app/e2e/a2a/test_x.py`.

Reject the colon itself rather than special-casing a drive letter: it is
the character the format reserves, so no path containing one was ever
linkable.

Claude-Session: https://claude.ai/code/session_017dTKXwJkzhtVLzDhePHsKG
2026-09-01 16:02:54 -07:00
ryan-crabbe-berri
00e40c0afe Record each e2e test's source location in the JUnit report
The JUnit report is the only thing that leaves the e2e run, and it says
where a test's results came from but never where its code lives. A reader
looking at `test_cell_claimed_only_by_a_skipped_test_is_uncovered` on the
status page has a name and nothing else -- no file, no line, no way to
reach the source short of grepping the repo by hand.

Pytest knows the location; the report format loses it. The `xunit1` family
wrote `file=` and `line=` onto every `<testcase>`, and the `xunit2` default
this suite runs on drops both. Switching families back would change the
document for every consumer of the same XML -- the Buildkite Test Engine
upload and the Loki pipeline included -- so add the location the way this
suite already adds `package` and `covers`: as a `<property>`, which is
purely additive.

`source` is repo-relative and one-based (`tests/e2e/a2a/test_x.py:41`), so
a consumer can build a link without knowing how pytest was started. That
takes normalizing the two launch shapes -- the runner image runs from its
own copy at /app/e2e, a developer runs from the repo root -- which is the
same normalization `package_from_nodeid` was already doing in reverse, now
factored into `suite_parts` so the two cannot drift apart. Paths that
escape the suite, and tests pytest reports no line for, emit an empty
string: a test with no link beats a link that 404s.

Claude-Session: https://claude.ai/code/session_017dTKXwJkzhtVLzDhePHsKG
2026-09-01 16:02:54 -07:00
Yassin Kortam
71e0251341
refactor(e2e): replace bespoke result reporter with standard JUnit report (#33758)
* refactor(e2e): replace bespoke result reporter with standard JUnit report

tests/e2e/e2e_result_reporter.py hand-rolled a per-test logfmt emitter that
reimplemented outcome mapping, logfmt escaping, and node-id parsing to print one
E2E_RESULT line per finished test. Outcome, duration, and node id are all things
a standard pytest reporter already produces, so the only genuinely custom data is
the covers marker ids and the normalized package label

Delete the module and emit a standard pytest JUnit XML report (--junitxml)
instead, carrying the two custom signals as user_properties (JUnit <property>
entries) attached at collection time in pytest_collection_modifyitems, so they
land on every test on every outcome including skips and setup errors. The small
package/covers extraction lives in junit_properties.py and is unit tested plus
checked end to end against a real JUnit artifact in test_junit_properties.py

Shipping the JUnit report to Loki is a thin infra-side transform, documented in
grafana/status_history_panels.md

* chore(e2e): remove grafana status history panels doc and junit properties e2e test

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-17 20:53:22 +00:00