github3.py icon indicating copy to clipboard operation
github3.py copied to clipboard

Speed up the tests

Open sigmavirus24 opened this issue 11 years ago • 5 comments

Currently if you do:

$ tox -e py34 -- tests/unit

The tests take about 12 seconds. If you just run

$ tox -e py34

The tests take about 16 seconds.

We can experiment with pytest-xdist to parallelize the tests but I don't think that will help a great deal. I'm pretty sure this arises from the fact that we're making a bunch of new objects for each test. We create a new instance of each class we're testing before each tests and we create a mocked session before each test. Most of what we care about in these tests is that the request generated by github3.py is correct when it's sent to requests. Keeping that in mind, we can probably create the instance of the class and session exactly once and simply call reset_mockon the mocked session. Not creating and then garbage collecting so many objects will probably improve our test speeds. We just need to make sure we don't lose idempotence because of it (So we should probably start randomizing the ordering of the tests)

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/6621137-speed-up-the-tests?utm_campaign=plugin&utm_content=tracker%2F183477&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F183477&utm_medium=issues&utm_source=github).

sigmavirus24 avatar Nov 29 '14 22:11 sigmavirus24

Are you seeing this locally? Or Travis? Or ?

(github3.py)➜  github3.py git:(develop) time tox -e py34
GLOB sdist-make: /home/bmorriso/localrepo/github3.py/setup.py
py34 inst-nodeps: /home/bmorriso/localrepo/github3.py/.tox/dist/github3.py-0.9.1.zip
py34 runtests: PYTHONHASHSEED='3779727424'
py34 runtests: commands[0] | python setup.py test
running test
running egg_info
writing github3.py.egg-info/PKG-INFO
writing dependency_links to github3.py.egg-info/dependency_links.txt
writing requirements to github3.py.egg-info/requires.txt
writing top-level names to github3.py.egg-info/top_level.txt
reading manifest file 'github3.py.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
no previously-included directories found matching '*.pyc'
no previously-included directories found matching 'docs/_build'
writing manifest file 'github3.py.egg-info/SOURCES.txt'
running build_ext
..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
494 passed in 4.40 seconds
___________________________________ summary ____________________________________
  py34: commands succeeded
  congratulations :)
tox -e py34  6.33s user 0.32s system 96% cpu 6.876 total
(github3.py)➜  github3.py git:(develop) time tox -e py34 -- tests/unit
GLOB sdist-make: /home/bmorriso/localrepo/github3.py/setup.py
py34 inst-nodeps: /home/bmorriso/localrepo/github3.py/.tox/dist/github3.py-0.9.1.zip
py34 runtests: PYTHONHASHSEED='1862512541'
py34 runtests: commands[0] | python setup.py test
running test
running egg_info
writing github3.py.egg-info/PKG-INFO
writing requirements to github3.py.egg-info/requires.txt
writing top-level names to github3.py.egg-info/top_level.txt
writing dependency_links to github3.py.egg-info/dependency_links.txt
reading manifest file 'github3.py.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
no previously-included directories found matching '*.pyc'
no previously-included directories found matching 'docs/_build'
writing manifest file 'github3.py.egg-info/SOURCES.txt'
running build_ext
..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
494 passed in 3.71 seconds
____________________________________________________________ summary _____________________________________________________________
  py34: commands succeeded
  congratulations :)
tox -e py34 -- tests/unit  5.60s user 0.30s system 99% cpu 5.898 total

Edit: bad copy & paste

esacteksab avatar Nov 30 '14 19:11 esacteksab

I'm seeing this locally... I wonder why mine seems so much slower than yours

sigmavirus24 avatar Nov 30 '14 20:11 sigmavirus24

FWIW, It looks like Travis experiences much longer test runs than either of us. (20 seconds for the whole suite)

sigmavirus24 avatar Nov 30 '14 20:11 sigmavirus24

@omgjlk Although these tests are running 35 secs for now, the number of tests are ~3x more than at that time. Also this 35 secs contains integration/cassette tests as well. WDYT?

gabor-boros avatar Aug 11 '20 07:08 gabor-boros

Probably not worth chasing down at the moment. I'd like to see us move off of Travis, which has been getting slower and slower, and instead use GitHub Actions. Once moved, or during the move, we can explore optimizations.

omgjlk avatar Aug 12 '20 20:08 omgjlk