grove icon indicating copy to clipboard operation
grove copied to clipboard

improve testing functionality

Open vontell opened this issue 7 years ago • 0 comments

While building tests for shor.py, I realized that there are a few issues with the testing infrastructure in grove.

  1. All tests import the modules that they would like to test in the following way (example taken from test_maxcut.py:
from grove.pyqaoa.maxcut_qaoa import maxcut_qaoa
from grove.pyqaoa.qaoa import QAOA
from grove.pyqaoa.utils import compare_progs
...

However, these imports are grabbing the module from the pip installed grove dependency, rather than the actual current pyqaoa file one directory above. This means that in order to accurately test the newly updated pyqaoa module, one needs to install their grove directory with pip install -e ., which may have broken changes due to updates you have made.

Additionally, that means if you try to create a new module with tests, those tests won't work until you actually update grove with the new module, which doesn't make too much sense from a test-first perspective. I think that these grove imports should instead use relative imports, which will allow tests to use the currently-defined grove files on your local machine without having to install the entire library using pip.

  1. Originally, testing was being done by comparing the created Quil programs with the expected program. However, there are an increasing number of features in Grove that are abstracted away from the Quil code directly, and instead need to run multiple programs on Forest to get a final result (for example, the factor() function in shor.py simply returns a final answer after multiple calls to Forest). Pytest provides some tools to ignore certain tests. I think it would be useful if we had tests that were run conditional on whether or not the testing machine could make a valid connection to the Forest service.

vontell avatar Jun 18 '17 20:06 vontell