Add tests to release tarball (in PyPI)

[jluttine]

Could you include tests in the source tarball in PyPI? I think it's good practice:

• https://docs.python.org/2/distutils/sourcedist.html#specifying-the-files-to-distribute
• https://stackoverflow.com/a/20180000

This way one can run the tests after installing the package. Also, I'm currently packaging this for a Linux distro, and it would be really helpful to be able to run the tests as a part of the automatic package building process while using PyPI for the release tarballs.

Similar request applies to eth-typing at least.

[pipermerriam]

@jluttine can you explain your use case a bit more? We have been intentional about not shipping the tests with the code because:

1. It bloats the size of the package we ship. In some cases this bloat would be significant due to testing fixtures, etc.
2. It ensures that our tests are fully separate from the module code.

If you are packaging up the library for distribution, is there a reason why you cannot do that from the actual source code? We tag every release so you should be able to reliably pull the source tree at the given version for any of our releases.

[jluttine]

As far as I know, it's quite common practice to include tests in PyPI so that it's possible to use the "official" release tarball and check it with unit tests. For instance, NumPy, SciPy, Pandas and many many others ship tests in the release tarball last time I checked. Of course, it is possible to use the corresponding git commit if it's tagged and the release tarball in PyPI has been generated from that commit without any side effects (e.g., dirty working tree or untracked files that get included in the PyPI release tarball). I would prefer to use PyPI tarballs because then it's guaranteed that I have the same package as I would have if I had done just pip install. But if you intentionally don't want to include tests in PyPI, I'll use GitHub source. :+1:

[pipermerriam]

Well, upon further investigation, it looks like you're right (and I'm just stubborn :smile: ). My general guideline has been if django does it then it's probably fine, and it seems django does indeed include the tests. I'm :+1: on this. cc @carver

[carver]

Hm, they also do custom testing with runtests.py and a custom requirements file: https://github.com/django/django/blob/2f7cd7f8ecb01d30c1dfdaefa1c1714db76d2553/tox.ini#L23-L30

When people run tests in sub-projects, are they expected to dig into the package to figure out the extra pip requirements, and the custom testing command?

[pipermerriam]

I guess my thoughts are that I don't have an opinion on how people in sub-projects run the tests, nor is it something I think we should support, only that we'll include them so that they can do this if they wish to do the heavy lifting.

[lemenkov]

Hello! Any news here?