Regular failure of test_http_connection_range

[embray]

I get consistent failures in test_http_connection_range when I run the tests locally, on both Python 2 and 3. Obviously since this failure doesn't seem to be occurring on the CI builds it may be an issue local to me. But I'm opening an issue to remember to investigate:

tree = {'more': array([[[ 0.16936457,  0.04898563,  0.68901559, ...,  0.97004914,
          0.....46327186,  0.78642262, ...,...123, ...,  0.8856184 ,
         0.13... 0.00274184,  0.78529121, ...,  0.6853034 ,
         0.08646289,  0.77335592]])}
rhttpserver = <pyasdf.conftest.RangeHTTPServer object at 0x428ea50>

    @pytest.mark.skipif(sys.platform.startswith('win'),
                        reason="Windows firewall prevents test")
    def test_http_connection_range(tree, rhttpserver):
        path = os.path.join(rhttpserver.tmpdir, 'test.asdf')
        connection = [None]

        def get_write_fd():
            return generic_io.get_file(open(path, 'wb'), mode='w')

        def get_read_fd():
            fd = generic_io.get_file(rhttpserver.url + "test.asdf")
            assert isinstance(fd, generic_io.HTTPConnection)
            connection[0] = fd
            return fd

        with _roundtrip(tree, get_write_fd, get_read_fd) as ff:
            if len(tree) == 4:
                assert connection[0]._nreads == 0
            else:
>               assert connection[0]._nreads == 6
E               assert 5 == 6
E                +  where 5 = <pyasdf.generic_io.HTTPConnection object at 0x431e910>._nreads

../../../.virtualenvs/13aecf6e-83d7-40c6-86f5-713fad8a4373/lib/python2.7/site-packages/pyasdf/tests/test_generic_io.py:306: AssertionError
------------------------------------------------- Captured stdout call -------------------------------------------------
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 53619)
----------------------------------------
------------------------------------------------- Captured stderr call -------------------------------------------------
127.0.0.1 - - [08/Jan/2016 15:04:07] "GET /test.asdf HTTP/1.1" 206 -
Traceback (most recent call last):
  File "/internal/1/root/usr/local/lib/python2.7/SocketServer.py", line 295, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/internal/1/root/usr/local/lib/python2.7/SocketServer.py", line 321, in process_request
    self.finish_request(request, client_address)
  File "/internal/1/root/usr/local/lib/python2.7/SocketServer.py", line 334, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/internal/1/root/usr/local/lib/python2.7/SocketServer.py", line 651, in __init__
    self.finish()
  File "/internal/1/root/usr/local/lib/python2.7/SocketServer.py", line 710, in finish
    self.wfile.close()
  File "/internal/1/root/usr/local/lib/python2.7/socket.py", line 279, in close
    self.flush()
  File "/internal/1/root/usr/local/lib/python2.7/socket.py", line 303, in flush
    self._sock.sendall(view[write_offset:write_offset+buffer_size])
error: [Errno 32] Broken pipe
127.0.0.1 - - [08/Jan/2016 15:04:07] "GET /test.asdf HTTP/1.1" 206 -
127.0.0.1 - - [08/Jan/2016 15:04:07] "GET /test.asdf HTTP/1.1" 206 -
127.0.0.1 - - [08/Jan/2016 15:04:07] "GET /test.asdf HTTP/1.1" 206 -
127.0.0.1 - - [08/Jan/2016 15:04:07] "GET /test.asdf HTTP/1.1" 206 -
127.0.0.1 - - [08/Jan/2016 15:04:07] "GET /test.asdf HTTP/1.1" 206 -

[drdavella]

I have seen this issue as well. It only seems to occur when running in a python-2.7 environment.

[drdavella]

Since we are removing support for Py2, maybe this can be closed.

[drdavella]

Closing since Py2 is no longer supported. Will reopen if the issue reappears.

[jdavies-st]

Reopening this as I see this failure locally when running under MacOS and Python 3.9. Seems to pass fine on Travis. Clearly, the test is flaky.

$ pytest asdf/tests/test_generic_io.py::test_http_connection_range --remote-data -v
================================================== test session starts ==================================================
platform darwin -- Python 3.9.0, pytest-6.1.2, py-1.9.0, pluggy-0.13.1 -- /Users/jdavies/miniconda3/envs/asdf/bin/python
cachedir: .pytest_cache
rootdir: /Users/jdavies/dev/asdf, configfile: setup.cfg
plugins: doctestplus-0.8.0, remotedata-0.3.2, openfiles-0.5.0, asdf-2.8.0.dev103+g89947dd.d20201112
collected 2 items                                                                                                       

asdf/tests/test_generic_io.py::test_http_connection_range[create_small_tree] PASSED                               [ 50%]
asdf/tests/test_generic_io.py::test_http_connection_range[create_large_tree] FAILED                               [100%]

======================================================= FAILURES ========================================================
_____________________________________ test_http_connection_range[create_large_tree] _____________________________________

tree = {'more': array([[[0.46697203, 0.26663249, 0.31253938, ..., 0.42559718,
         0.05273264, 0.19073801],
        [0.61...1,
        0.08548561],
       [0.20447504, 0.64278415, 0.44272929, ..., 0.53209797, 0.99239067,
        0.73564929]])}
rhttpserver = <asdf.tests.httpserver.RangeHTTPServer object at 0x7f8b08477250>

    @pytest.mark.remote_data
    def test_http_connection_range(tree, rhttpserver):
        path = os.path.join(rhttpserver.tmpdir, 'test.asdf')
        connection = [None]
    
        def get_write_fd():
            return generic_io.get_file(open(path, 'wb'), mode='w')
    
        def get_read_fd():
            fd = generic_io.get_file(rhttpserver.url + "test.asdf")
            assert isinstance(fd, generic_io.HTTPConnection)
            connection[0] = fd
            return fd
    
        with _roundtrip(tree, get_write_fd, get_read_fd) as ff:
            if len(tree) == 4:
                assert connection[0]._nreads == 0
            else:
>               assert connection[0]._nreads == 6
E               assert 5 == 6
E                 +5
E                 -6

asdf/tests/test_generic_io.py:296: AssertionError
------------------------------------------------- Captured stderr call --------------------------------------------------
127.0.0.1 - - [12/Nov/2020 15:20:58] "GET /test.asdf HTTP/1.1" 206 -
127.0.0.1 - - [12/Nov/2020 15:20:58] "GET /test.asdf HTTP/1.1" 206 -
127.0.0.1 - - [12/Nov/2020 15:20:58] "GET /test.asdf HTTP/1.1" 206 -
127.0.0.1 - - [12/Nov/2020 15:20:58] "GET /test.asdf HTTP/1.1" 206 -
127.0.0.1 - - [12/Nov/2020 15:20:58] "GET /test.asdf HTTP/1.1" 206 -
127.0.0.1 - - [12/Nov/2020 15:20:58] "GET /test.asdf HTTP/1.1" 206 -
================================================ short test summary info ================================================
FAILED asdf/tests/test_generic_io.py::test_http_connection_range[create_large_tree] - assert 5 == 6
============================================== 1 failed, 1 passed in 0.87s ==============================================

[braingram]

Closing as it appears this test no longer exists.

[jdavies-st]

The test was removed in #979.