github3.py
github3.py copied to clipboard
Speed up the tests
Currently if you do:
$ tox -e py34 -- tests/unit
The tests take about 12 seconds. If you just run
$ tox -e py34
The tests take about 16 seconds.
We can experiment with pytest-xdist to parallelize the tests but I don't think that will help a great deal. I'm pretty sure this arises from the fact that we're making a bunch of new objects for each test. We create a new instance of each class we're testing before each tests and we create a mocked session before each test. Most of what we care about in these tests is that the request generated by github3.py is correct when it's sent to requests. Keeping that in mind, we can probably create the instance of the class and session exactly once and simply call reset_mockon the mocked session. Not creating and then garbage collecting so many objects will probably improve our test speeds. We just need to make sure we don't lose idempotence because of it (So we should probably start randomizing the ordering of the tests)
Are you seeing this locally? Or Travis? Or ?
(github3.py)➜ github3.py git:(develop) time tox -e py34
GLOB sdist-make: /home/bmorriso/localrepo/github3.py/setup.py
py34 inst-nodeps: /home/bmorriso/localrepo/github3.py/.tox/dist/github3.py-0.9.1.zip
py34 runtests: PYTHONHASHSEED='3779727424'
py34 runtests: commands[0] | python setup.py test
running test
running egg_info
writing github3.py.egg-info/PKG-INFO
writing dependency_links to github3.py.egg-info/dependency_links.txt
writing requirements to github3.py.egg-info/requires.txt
writing top-level names to github3.py.egg-info/top_level.txt
reading manifest file 'github3.py.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
no previously-included directories found matching '*.pyc'
no previously-included directories found matching 'docs/_build'
writing manifest file 'github3.py.egg-info/SOURCES.txt'
running build_ext
..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
494 passed in 4.40 seconds
___________________________________ summary ____________________________________
py34: commands succeeded
congratulations :)
tox -e py34 6.33s user 0.32s system 96% cpu 6.876 total
(github3.py)➜ github3.py git:(develop) time tox -e py34 -- tests/unit
GLOB sdist-make: /home/bmorriso/localrepo/github3.py/setup.py
py34 inst-nodeps: /home/bmorriso/localrepo/github3.py/.tox/dist/github3.py-0.9.1.zip
py34 runtests: PYTHONHASHSEED='1862512541'
py34 runtests: commands[0] | python setup.py test
running test
running egg_info
writing github3.py.egg-info/PKG-INFO
writing requirements to github3.py.egg-info/requires.txt
writing top-level names to github3.py.egg-info/top_level.txt
writing dependency_links to github3.py.egg-info/dependency_links.txt
reading manifest file 'github3.py.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
no previously-included directories found matching '*.pyc'
no previously-included directories found matching 'docs/_build'
writing manifest file 'github3.py.egg-info/SOURCES.txt'
running build_ext
..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
494 passed in 3.71 seconds
____________________________________________________________ summary _____________________________________________________________
py34: commands succeeded
congratulations :)
tox -e py34 -- tests/unit 5.60s user 0.30s system 99% cpu 5.898 total
Edit: bad copy & paste
I'm seeing this locally... I wonder why mine seems so much slower than yours
FWIW, It looks like Travis experiences much longer test runs than either of us. (20 seconds for the whole suite)
@omgjlk Although these tests are running 35 secs for now, the number of tests are ~3x more than at that time. Also this 35 secs contains integration/cassette tests as well. WDYT?
Probably not worth chasing down at the moment. I'd like to see us move off of Travis, which has been getting slower and slower, and instead use GitHub Actions. Once moved, or during the move, we can explore optimizations.