ImportError on Github Actions

[schlegelp]

Hi! I just hit an issue with my test CI for navis running on GH actions.

    >>> import cloudvolume as cv
UNEXPECTED EXCEPTION: ImportError('zstd C API versions mismatch; Python bindings were not compiled/linked against expected zstd version (10404 returned by the lib, 10501 hardcoded in zstd headers, 10501 hardcoded in the cext)')
raceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest navis.utils.cv.patch_cloudvolume[1]>", line 1, in <module>
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/cloudvolume/__init__.py", line 49, in <module>
    from .cloudvolume import CloudVolume, register_plugin
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/cloudvolume/cloudvolume.py", line 8, in <module>
    from .exceptions import UnsupportedFormatError, DimensionError
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/cloudvolume/exceptions.py", line 1, in <module>
    from cloudfiles.exceptions import CompressionError, DecompressionError
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/cloudfiles/__init__.py", line 12, in <module>
    from .cloudfiles import CloudFiles, dl
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/cloudfiles/cloudfiles.py", line 27, in <module>
    from . import compression, paths
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/cloudfiles/compression.py", line 13, in <module>
    import zstandard as zstd
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/zstandard/__init__.py", line 39, in <module>
    from .backend_c import *  # type: ignore
ImportError: zstd C API versions mismatch; Python bindings were not compiled/linked against expected zstd version (10404 returned by the lib, 10501 hardcoded in zstd headers, 10501 hardcoded in the cext)
/home/runner/work/navis/navis/navis/utils/cv.py:32: UnexpectedException

I haven't dug into the issue - hard to debug the GH action runners - but does that ring a bell?

For reference: this is the offending run and this is the workflow file.

[schlegelp]

Also just realised that this actually points to cloud-files, not cloud-volume.

[william-silversmith]

Interesting. I looked into zstandard's code and saw this. I'm not sure why zstandard's c library would have been unbundled though.

    /* python-zstandard relies on unstable zstd C API features. This means
       that changes in zstd may break expectations in python-zstandard.
       python-zstandard is distributed with a copy of the zstd sources.
       python-zstandard is only guaranteed to work with the bundled version
       of zstd.
       However, downstream redistributors or packagers may unbundle zstd
       from python-zstandard. This can result in a mismatch between zstd
       versions and API semantics. This essentially "voids the warranty"
       of python-zstandard and may cause undefined behavior.
       We detect this mismatch here and refuse to load the module if this
       scenario is detected.
    */```

https://github.com/indygreg/python-zstandard/blob/964141349479070486ea25253bd4cc106929b2fc/c-ext/backend_c.c

[william-silversmith]

It looks like this error can occur if zstandard is installed as python setup.py build --system-zstd. Not sure why that would be relevant for a standard pip install.

https://github.com/indygreg/python-zstandard/issues/143

[william-silversmith]

In the GHA logs, I don't see anything report of "Successfully installed ... cloud-volume" or cloud-files or their dependencies. Weird.

[schlegelp]

Yeah I noticed that too. Nevertheless the tests used to pass 🤷‍♂️