Python unit testing requirement

[jared321]

The current template repo has two python packages, both of which use python's unittest framework. In mytemplate, this includes two sub-packages that each uses unittest. However, pytest is another framework that others might prefer to use (or are already using).

What are the POptUS requirements for unit testing within a python package, across python packages within a particular repo, and across all python packages in the POptUS organization?

My proposed requirements are:

1. Each python sub-package shall be able to use the the unit testing framework of their choice (e.g., unittest, pytest).
2. Each python package shall be constructed and maintained in such a way that all the tests from the main package and from all sub-packages (regardless of which unit test framework they use) can be effectively grouped into a single test suite that users can run via <package>.test() and with a single command to tox (e.g., tox -r -e nocoverage).

If these are the desired requirements, then this template should be updated so that mytemplate/subA uses unittest and mytemplate/subB uses pytest. This is necessary to prove that these requirements can be satisfied and to provide a concrete demonstration of how to achieve it.

@jmlarson1 @wildsm @mmenickelly Any thoughts on these requirements?

[jmlarson1]

Thank you for the help on testing.

1. I find pytest to be more common, but I have been pleased with what's working with unittest. If unittest works, why support pytest too? (It seems like giving ourselves more work to maintain, but maybe supporting general testing infrastructure will make future dev work easier.)
2. I think the goal of having tests for each sub-package to be separate is logical (and makes my development workflow easier.)

[jared321]

@jmlarson1 I'm glad to hear that you are happy with unittest. I continue to be happy with it as well. One benefit of using unittest that I should mention is that pytest can run tests written in unittest, whereas I do not believe that the opposite is true. This has been one motivation for me to continue using unittest.
https://docs.pytest.org/en/7.1.x/how-to/unittest.html

One possible argument against unittest is that writing a test for unittest that is just a single python function is possible but the function won't have access to full functionality such as the self.assert* family of tests. It looks like you can only use the standard assert.
https://docs.python.org/3/library/unittest.html#re-using-old-test-code
You'll see at that link that they also do not recommend using this part of the unittest interface.

Thoughts?