kubeyaml icon indicating copy to clipboard operation
kubeyaml copied to clipboard

test_kubeyaml.py::test_update_image_apply indefinite loop

Open brezerk opened this issue 5 years ago • 1 comments

it seems like recent changes https://github.com/squaremo/kubeyaml/commit/1f64738ef6e3ea26058dc191bbcb441c2f61c828 broke the tests:

circleci log:

============================= test session starts ==============================
platform linux -- Python 3.6.1, pytest-4.4.1, py-1.8.0, pluggy-0.9.0
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/circleci/repo/.hypothesis/examples')
rootdir: /home/circleci/repo
plugins: hypothesis-4.21.0
collected 9 items                                                              

test_kubeyaml.py ........Makefile:18: recipe for target 'test' failed
make: *** [test] Terminated
Too long with no output (exceeded 10m0s)

test_kubeyaml.py::test_update_image_apply test is running forever now:

(.venv) [ himera ] brezerk@pts/0:36 :( 1 ~/develop/kubeyaml $
 05/05/19 18:06:24 EEST > pytest --hypothesis-show-statistics -v
=========================================================================================================== test session starts ============================================================================================================
platform linux -- Python 3.6.8, pytest-4.4.1, py-1.8.0, pluggy-0.9.0 -- /home/brezerk/develop/kubeyaml/.venv/bin/python3.6
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/brezerk/develop/kubeyaml/.hypothesis/examples')
rootdir: /home/brezerk/develop/kubeyaml
plugins: hypothesis-4.21.0
collected 9 items                                                                                                                                                                                                                          

test_kubeyaml.py::test_includes_all_containers PASSED                                                                                                                                                                                [ 11%]
test_kubeyaml.py::test_extract_custom_containers PASSED                                                                                                                                                                              [ 22%]
test_kubeyaml.py::test_set_custom_container_preserves_structure PASSED                                                                                                                                                               [ 33%]
test_kubeyaml.py::test_match_self PASSED                                                                                                                                                                                             [ 44%]
test_kubeyaml.py::test_find_container PASSED                                                                                                                                                                                         [ 55%]
test_kubeyaml.py::test_manifests PASSED                                                                                                                                                                                              [ 66%]
test_kubeyaml.py::test_image_update PASSED                                                                                                                                                                                           [ 77%]
test_kubeyaml.py::test_ident_apply PASSED                                                                                                                                                                                            [ 88%]
test_kubeyaml.py::test_update_image_apply

brezerk avatar May 05 '19 15:05 brezerk

I don't think it was that change. I can run the tests to completion on HEAD:

$ pytest --hypothesis-show-statistics -v                                                                                                                                                                           
=============================================================================================== test session starts ===============================================================================================
platform linux2 -- Python 2.7.15rc1, pytest-4.6.3, py-1.8.0, pluggy-0.12.0 -- /usr/bin/python                                                                                                                      
cachedir: .pytest_cache                                                                                                                                                                                            
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/mikeb/space/kubeyaml/.hypothesis/examples')                                                                                          
rootdir: /home/mikeb/space/kubeyaml                                                                                                                                                                                
plugins: hypothesis-4.24.3                                                                                                                                                                                         
collected 9 items                                                                                                                                                                                                  
                                                                                                                                                                                                                   
test_kubeyaml.py::test_includes_all_containers PASSED                                                                                                                                                       [ 11%]
test_kubeyaml.py::test_extract_custom_containers PASSED                                                                                                                                                     [ 22%]
test_kubeyaml.py::test_set_custom_container_preserves_structure PASSED                                                                                                                                      [ 33%]
test_kubeyaml.py::test_match_self PASSED                                                                                                                                                                    [ 44%]
test_kubeyaml.py::test_find_container PASSED                                                                                                                                                                [ 55%]
test_kubeyaml.py::test_manifests PASSED                                                                                                                                                                     [ 66%]
test_kubeyaml.py::test_image_update PASSED                                                                                                                                                                  [ 77%]
test_kubeyaml.py::test_ident_apply PASSED                                                                                                                                                                   [ 88%]
test_kubeyaml.py::test_update_image_apply PASSED                                                                                                                                                            [100%]

But then, ever after it seems, they fail to complete! And if I git checkout HEAD^, they still fail. If I downgrade Hypothesis to 3.88.3 (the last of the v3 line), it passes reliably. So possibly there's something in the way the tests interact with recent releases of Hypothesis, that means it goes into a death spiral on that last test case.

squaremo avatar Jun 12 '19 11:06 squaremo