python_mozetl
python_mozetl copied to clipboard
Add incremental testing to reduce CI time
Currently, every PR requires running tests for all modules if even a single line is changed. A blog post by digitalocean describing their go monorepo showcases the benefits of an incremental test tool.
There is a tool called pytest-testmon that can monitor changes in a python package using the coverage tool. This could be used in conjunction with environment variables packaged with Travis to reduce CI time significantly.
In order for a system like this to work, coverage would have to immensely improve in this project. In addition, some code paths like python rdd functions do not get executed locally and may fail to be covered in coverage tests.