RFC / RFE - testing python with more than just 'import'

[smoser]

The python/import test is very nice, with 'imports' and makes testing import very easy. I'd like to be able to do some more complex tests, and thought a pipeline would be useful.

The py3-pytest-timeout package currently has a test that uses py3-pytest. Its test stanza is kind of cludgy involving shell grep/exit, which is painful and has high potential for footgun.

I think it would be nice to have at very least something like this:

test:
  pipeline:
    - uses: python/tests
      tests:
       - content: |
           import foo
           assert(foo.version == "${{package.version}}")
       - content: |
           import bar
           ....

My second thought there is that it might be useful to run those with pytest, to allow the sort of things that pytest buys you.

test:
  pipeline:
    - uses: python/tests
      tests:
        - content: |
          import time
          def test_verify_stuff():
             """verify the stuff"""
             ...

Things I like about this:

• it is all in the yaml file - you don't have to go lookiong elsewhere, and changes to the package aren't possibly "hidden" in a / directory.
• one place to execute the things (the python) so each yaml file doesn't have to implement a correct execution

Things I don't like:

• it is all in the yaml file - makes long noisy yaml files and not so easy to just run the same tests manually.

This is a RFC/RFE, I really think we can do better with tests and I think we need to, so that writing them is easier.