Skip to content

pytest: use records to store RIOT version and experiment ID of test#188

Open
miri64 wants to merge 1 commit into
RIOT-OS:masterfrom
miri64:pytest/enh/records
Open

pytest: use records to store RIOT version and experiment ID of test#188
miri64 wants to merge 1 commit into
RIOT-OS:masterfrom
miri64:pytest/enh/records

Conversation

@miri64
Copy link
Copy Markdown
Member

@miri64 miri64 commented Aug 5, 2020

I noticed that I had this still lying around from before my vacation. This should make it easier to track after the fact which

  1. RIOT version was used for a given test and
  2. Which IoT-LAB experiment a test ran in.

Testing procedures

Run

tox  -- -k "(spec03 or spec04) and task01" --non-RC

and after success (given that you have everything set-up.. the tests should not skip ;-))

junit2html test-report.xml test-report.html

and open the test-report.html in a browser (you can of course also read just the XML and skip the junit2html step). The new properties should show up:

exp_id
225847
riot_commit
075b35a71a (HEAD -> gh-actions/enh/release-tests, origin/gh-actions/enh/release-tests) fixup! gh-actions: add release-specs tests

@miri64
Copy link
Copy Markdown
Member Author

miri64 commented Aug 5, 2020

(this is based on an offline suggestion by @MrKevinWeiss)

@miri64
Copy link
Copy Markdown
Member Author

miri64 commented Aug 5, 2020

Mh after re-running the tests I get warnings

03-single-hop-ipv6-icmp/test_spec03.py::test_task01[nodes0]
  03-single-hop-ipv6-icmp/test_spec03.py:25: PytestWarning: record_property is incompatible with junit_family 'xunit2' (use 'legacy' or 'xunit1')
    @pytest.mark.skipif(not bridged(["tap0", "tap1"]),

04-single-hop-6lowpan-icmp/test_spec04.py::test_task01[nodes0]
  04-single-hop-6lowpan-icmp/test_spec04.py:22: PytestWarning: record_property is incompatible with junit_family 'xunit2' (use 'legacy' or 'xunit1')
    @pytest.mark.iotlab_creds

-- Docs: https://docs.pytest.org/en/stable/warnings.html

@MrKevinWeiss do you know of an xunit2 compliant way to do this?

@miri64
Copy link
Copy Markdown
Member Author

miri64 commented Aug 6, 2020

Rebased to current master and also added the Release-Specs version as a property.

@miri64 miri64 force-pushed the pytest/enh/records branch from 8c84918 to efa1c50 Compare August 6, 2020 12:48
@MrKevinWeiss
Copy link
Copy Markdown
Contributor

@MrKevinWeiss do you know of an xunit2 compliant way to do this?

I don't know about this but I like the idea of the PR.

@MrKevinWeiss
Copy link
Copy Markdown
Contributor

Do we now have a way to get this info or should we continue with this?

@miri64
Copy link
Copy Markdown
Member Author

miri64 commented Jul 7, 2021

Do we now have a way to get this info or should we continue with this?

If we can live with the warning, I guess we can continue. Alternatively, we of course also could choose a different output format than xunit2 (which according to my last google search on that topic seems to be pytest specific anyways?)

@MrKevinWeiss
Copy link
Copy Markdown
Contributor

Or we could just use junit_family=legacy, was there a reason that we went with xunit2?

@MrKevinWeiss
Copy link
Copy Markdown
Contributor

I will give it a try and see what breaks.

@miri64
Copy link
Copy Markdown
Member Author

miri64 commented Jul 7, 2021

Or we could just use junit_family=legacy, was there a reason that we went with xunit2?

I think there were warnings to use xunit2 ^^.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants