elastic-transport-python icon indicating copy to clipboard operation
elastic-transport-python copied to clipboard

`httpbin` tests are failing if `zstandard` installed

Open mgorny opened this issue 1 month ago • 2 comments

It seems that a few httpbin-based tests are hardcoding the Accept-Encoding list and therefore failing if zstandard is installed, and its support is being declared as well, e.g.:

$ uv venv
$ uv pip install -e .[develop] zstandard
============================= test session starts ==============================
platform linux -- Python 3.13.9, pytest-8.4.2, pluggy-1.6.0 -- /tmp/elastic-transport-python/.venv/bin/python3
cachedir: .pytest_cache
rootdir: /tmp/elastic-transport-python
configfile: setup.cfg
plugins: anyio-4.11.0, mock-3.15.1, httpbin-2.1.0, respx-0.22.0, pytest_httpserver-1.1.3, cov-7.0.0, asyncio-1.2.0
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=function, asyncio_default_test_loop_scope=function
collecting ... collected 4 items

tests/async_/test_async_transport.py::test_async_transport_httpbin[asyncio] FAILED [ 25%]
tests/async_/test_async_transport.py::test_async_transport_httpbin[trio] FAILED [ 50%]
tests/async_/test_httpbin.py::test_node[asyncio] FAILED                  [ 75%]
tests/async_/test_httpbin.py::test_node[trio] FAILED                     [100%]

=================================== FAILURES ===================================
____________________ test_async_transport_httpbin[asyncio] _____________________

httpbin_node_config = NodeConfig(scheme='http', host='127.0.0.1', port=45419, path_prefix='', headers={}, connections_per_node=10, request_timeout=10.0, http_compress=False, verify_certs=False, ca_certs=None, client_cert=None, client_key=None, ssl_assert_hostname=None, ssl_assert_fingerprint=None, ssl_version=None, ssl_context=None, ssl_show_warn=False, _extras={})
httpbin = <pytest_httpbin.serve.Server object at 0x7f51fb9ca660>

    @pytest.mark.anyio
    async def test_async_transport_httpbin(httpbin_node_config, httpbin):
        t = AsyncTransport(
            [httpbin_node_config], meta_header=False, node_class=HttpxAsyncHttpNode
        )
        resp, data = await t.perform_request("GET", "/anything?key=value")
    
        assert resp.status == 200
        assert data["method"] == "GET"
        assert data["url"] == f"{httpbin.url}/anything?key=value"
        assert data["args"] == {"key": "value"}
    
        data["headers"].pop("X-Amzn-Trace-Id", None)
>       assert data["headers"] == {
            "Accept": "*/*",
            "Accept-Encoding": "gzip, deflate, br",
            "User-Agent": DEFAULT_USER_AGENT,
            "Connection": "keep-alive",
            "Host": f"{httpbin.host}:{httpbin.port}",
        }
E       AssertionError: assert {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate, br, zstd', 'Connection': 'keep-alive', 'Host': '127.0.0.1:45419', 'User-Agent': 'elastic-transport-python/9.2.0'} == {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate, br', 'User-Agent': 'elastic-transport-python/9.2.0', 'Connection': 'keep-alive', 'Host': '127.0.0.1:45419'}
E         
E         Common items:
E         {'Accept': '*/*',
E          'Connection': 'keep-alive',
E          'Host': '127.0.0.1:45419',
E          'User-Agent': 'elastic-transport-python/9.2.0'}
E         Differing items:
E         {'Accept-Encoding': 'gzip, deflate, br, zstd'} != {'Accept-Encoding': 'gzip, deflate, br'}
E         
E         Full diff:
E           {
E               'Accept': '*/*',
E         -     'Accept-Encoding': 'gzip, deflate, br',
E         +     'Accept-Encoding': 'gzip, deflate, br, zstd',
E         ?                                          ++++++
E               'Connection': 'keep-alive',
E               'Host': '127.0.0.1:45419',
E               'User-Agent': 'elastic-transport-python/9.2.0',
E           }

tests/async_/test_async_transport.py:63: AssertionError
----------------------------- Captured stderr call -----------------------------
127.0.0.1 - - [18/Oct/2025 07:31:06] "GET /anything?key=value HTTP/1.1" 200 322
______________________ test_async_transport_httpbin[trio] ______________________

httpbin_node_config = NodeConfig(scheme='http', host='127.0.0.1', port=45419, path_prefix='', headers={}, connections_per_node=10, request_timeout=10.0, http_compress=False, verify_certs=False, ca_certs=None, client_cert=None, client_key=None, ssl_assert_hostname=None, ssl_assert_fingerprint=None, ssl_version=None, ssl_context=None, ssl_show_warn=False, _extras={})
httpbin = <pytest_httpbin.serve.Server object at 0x7f51fb9ca660>

    @pytest.mark.anyio
    async def test_async_transport_httpbin(httpbin_node_config, httpbin):
        t = AsyncTransport(
            [httpbin_node_config], meta_header=False, node_class=HttpxAsyncHttpNode
        )
        resp, data = await t.perform_request("GET", "/anything?key=value")
    
        assert resp.status == 200
        assert data["method"] == "GET"
        assert data["url"] == f"{httpbin.url}/anything?key=value"
        assert data["args"] == {"key": "value"}
    
        data["headers"].pop("X-Amzn-Trace-Id", None)
>       assert data["headers"] == {
            "Accept": "*/*",
            "Accept-Encoding": "gzip, deflate, br",
            "User-Agent": DEFAULT_USER_AGENT,
            "Connection": "keep-alive",
            "Host": f"{httpbin.host}:{httpbin.port}",
        }
E       AssertionError: assert {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate, br, zstd', 'Connection': 'keep-alive', 'Host': '127.0.0.1:45419', 'User-Agent': 'elastic-transport-python/9.2.0'} == {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate, br', 'User-Agent': 'elastic-transport-python/9.2.0', 'Connection': 'keep-alive', 'Host': '127.0.0.1:45419'}
E         
E         Common items:
E         {'Accept': '*/*',
E          'Connection': 'keep-alive',
E          'Host': '127.0.0.1:45419',
E          'User-Agent': 'elastic-transport-python/9.2.0'}
E         Differing items:
E         {'Accept-Encoding': 'gzip, deflate, br, zstd'} != {'Accept-Encoding': 'gzip, deflate, br'}
E         
E         Full diff:
E           {
E               'Accept': '*/*',
E         -     'Accept-Encoding': 'gzip, deflate, br',
E         +     'Accept-Encoding': 'gzip, deflate, br, zstd',
E         ?                                          ++++++
E               'Connection': 'keep-alive',
E               'Host': '127.0.0.1:45419',
E               'User-Agent': 'elastic-transport-python/9.2.0',
E           }

tests/async_/test_async_transport.py:63: AssertionError
----------------------------- Captured stderr call -----------------------------
127.0.0.1 - - [18/Oct/2025 07:31:06] "GET /anything?key=value HTTP/1.1" 200 322
______________________________ test_node[asyncio] ______________________________

httpbin_node_config = NodeConfig(scheme='http', host='127.0.0.1', port=45419, path_prefix='', headers={}, connections_per_node=10, request_timeout=10.0, http_compress=False, verify_certs=False, ca_certs=None, client_cert=None, client_key=None, ssl_assert_hostname=None, ssl_assert_fingerprint=None, ssl_version=None, ssl_context=None, ssl_show_warn=False, _extras={})
httpbin = <pytest_httpbin.serve.Server object at 0x7f51fb9ca660>

    @pytest.mark.anyio
    async def test_node(httpbin_node_config, httpbin):
        def new_node(**kwargs):
            return HttpxAsyncHttpNode(dataclasses.replace(httpbin_node_config, **kwargs))
    
        node = new_node()
        resp, data = await node.perform_request("GET", "/anything")
        assert resp.status == 200
        parsed = parse_httpbin(data)
>       assert parsed == {
            "headers": {
                "Accept": "*/*",
                "Accept-Encoding": "gzip, deflate, br",
                "Connection": "keep-alive",
                "Host": f"{httpbin.host}:{httpbin.port}",
                "User-Agent": DEFAULT_USER_AGENT,
            },
            "method": "GET",
            "url": f"{httpbin.url}/anything",
        }
E       AssertionError: assert {'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate, br, zstd', 'Connection': 'keep-alive', 'Host': '127.0.0.1:45419', 'User-Agent': 'elastic-transport-python/9.2.0'}, 'method': 'GET', 'url': 'http://127.0.0.1:45419/anything'} == {'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate, br', 'Connection': 'keep-alive', 'Host': '127.0.0.1:45419', 'User-Agent': 'elastic-transport-python/9.2.0'}, 'method': 'GET', 'url': 'http://127.0.0.1:45419/anything'}
E         
E         Common items:
E         {'method': 'GET', 'url': 'http://127.0.0.1:45419/anything'}
E         Differing items:
E         {'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate, br, zstd', 'Connection': 'keep-alive', 'Host': '127.0.0.1:45419', ...}} != {'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate, br', 'Connection': 'keep-alive', 'Host': '127.0.0.1:45419', ...}}
E         
E         Full diff:
E           {
E               'headers': {
E                   'Accept': '*/*',
E         -         'Accept-Encoding': 'gzip, deflate, br',
E         +         'Accept-Encoding': 'gzip, deflate, br, zstd',
E         ?                                              ++++++
E                   'Connection': 'keep-alive',
E                   'Host': '127.0.0.1:45419',
E                   'User-Agent': 'elastic-transport-python/9.2.0',
E               },
E               'method': 'GET',
E               'url': 'http://127.0.0.1:45419/anything',
E           }

tests/async_/test_httpbin.py:71: AssertionError
----------------------------- Captured stderr call -----------------------------
127.0.0.1 - - [18/Oct/2025 07:31:06] "GET /anything HTTP/1.1" 200 299
_______________________________ test_node[trio] ________________________________

httpbin_node_config = NodeConfig(scheme='http', host='127.0.0.1', port=45419, path_prefix='', headers={}, connections_per_node=10, request_timeout=10.0, http_compress=False, verify_certs=False, ca_certs=None, client_cert=None, client_key=None, ssl_assert_hostname=None, ssl_assert_fingerprint=None, ssl_version=None, ssl_context=None, ssl_show_warn=False, _extras={})
httpbin = <pytest_httpbin.serve.Server object at 0x7f51fb9ca660>

    @pytest.mark.anyio
    async def test_node(httpbin_node_config, httpbin):
        def new_node(**kwargs):
            return HttpxAsyncHttpNode(dataclasses.replace(httpbin_node_config, **kwargs))
    
        node = new_node()
        resp, data = await node.perform_request("GET", "/anything")
        assert resp.status == 200
        parsed = parse_httpbin(data)
>       assert parsed == {
            "headers": {
                "Accept": "*/*",
                "Accept-Encoding": "gzip, deflate, br",
                "Connection": "keep-alive",
                "Host": f"{httpbin.host}:{httpbin.port}",
                "User-Agent": DEFAULT_USER_AGENT,
            },
            "method": "GET",
            "url": f"{httpbin.url}/anything",
        }
E       AssertionError: assert {'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate, br, zstd', 'Connection': 'keep-alive', 'Host': '127.0.0.1:45419', 'User-Agent': 'elastic-transport-python/9.2.0'}, 'method': 'GET', 'url': 'http://127.0.0.1:45419/anything'} == {'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate, br', 'Connection': 'keep-alive', 'Host': '127.0.0.1:45419', 'User-Agent': 'elastic-transport-python/9.2.0'}, 'method': 'GET', 'url': 'http://127.0.0.1:45419/anything'}
E         
E         Common items:
E         {'method': 'GET', 'url': 'http://127.0.0.1:45419/anything'}
E         Differing items:
E         {'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate, br, zstd', 'Connection': 'keep-alive', 'Host': '127.0.0.1:45419', ...}} != {'headers': {'Accept': '*/*', 'Accept-Encoding': 'gzip, deflate, br', 'Connection': 'keep-alive', 'Host': '127.0.0.1:45419', ...}}
E         
E         Full diff:
E           {
E               'headers': {
E                   'Accept': '*/*',
E         -         'Accept-Encoding': 'gzip, deflate, br',
E         +         'Accept-Encoding': 'gzip, deflate, br, zstd',
E         ?                                              ++++++
E                   'Connection': 'keep-alive',
E                   'Host': '127.0.0.1:45419',
E                   'User-Agent': 'elastic-transport-python/9.2.0',
E               },
E               'method': 'GET',
E               'url': 'http://127.0.0.1:45419/anything',
E           }

tests/async_/test_httpbin.py:71: AssertionError
----------------------------- Captured stderr call -----------------------------
127.0.0.1 - - [18/Oct/2025 07:31:06] "GET /anything HTTP/1.1" 200 299
=========================== short test summary info ============================
FAILED tests/async_/test_async_transport.py::test_async_transport_httpbin[asyncio]
FAILED tests/async_/test_async_transport.py::test_async_transport_httpbin[trio]
FAILED tests/async_/test_httpbin.py::test_node[asyncio] - AssertionError: ass...
FAILED tests/async_/test_httpbin.py::test_node[trio] - AssertionError: assert...
============================== 4 failed in 0.60s ===============================

Reproduced on bfe6acc4ddf280b3dc4c40309b44ee1409b4e138. I can make a pull request if you tell me what solution you'd prefer.

mgorny avatar Oct 18 '25 05:10 mgorny