zipline-reloaded icon indicating copy to clipboard operation
zipline-reloaded copied to clipboard

Fresh install zipline 3.0.3 - issue with h5py - includes workaround

Open sammmyc opened this issue 1 year ago • 2 comments

Dear Zipline Maintainers,

Before I tell you about my issue, let me describe my environment:

Environment

  • Operating System: Windows 10
  • Python Version: 3.10.13
  • Python Bitness: 64 bit
  • How did you install Zipline: conda - details below
  • Python packages:
(zipline_test) C:\Users\admin>conda list
alembic                   1.13.1             pyhd8ed1ab_0    conda-forge
appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
bcolz-zipline             1.2.6           py310h3e78b6c_1    conda-forge
beautifulsoup4            4.12.2             pyha770c72_0    conda-forge
blosc                     1.21.3               h6c2663c_0
bottleneck                1.3.7           py310h3e78b6c_1    conda-forge
brotli-python             1.1.0           py310h00ffb61_1    conda-forge
bzip2                     1.0.8                he774522_0
c-blosc2                  2.12.0               h183a6f4_0    conda-forge
ca-certificates           2023.12.12           haa95532_0
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
certifi                   2023.11.17         pyhd8ed1ab_0    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
click                     8.1.7           win_pyh7428d3b_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
empyrical-reloaded        0.5.9              pyhd8ed1ab_0    conda-forge
exchange-calendars        4.2.8              pyhd8ed1ab_1    conda-forge
frozendict                2.4.0           py310h8d17308_0    conda-forge
greenlet                  3.0.3           py310h00ffb61_0    conda-forge
h5py                      3.7.0           nompi_py310h00cbb18_100    conda-forge
hdf5                      1.12.1               h51c971a_3
html5lib                  1.1                pyh9f0ad1d_0    conda-forge
icc_rt                    2022.1.0             h6049295_2
idna                      3.6                pyhd8ed1ab_0    conda-forge
importlib-metadata        7.0.1              pyha770c72_0    conda-forge
importlib_resources       6.1.1              pyhd8ed1ab_0    conda-forge
intervaltree              3.1.0              pyhd8ed1ab_1    conda-forge
iso3166                   2.1.1              pyhd8ed1ab_0    conda-forge
iso4217                   1.9.20220401       pyhd8ed1ab_0    conda-forge
korean_lunar_calendar     0.3.1              pyhd8ed1ab_0    conda-forge
libblas                   3.9.0           20_win64_openblas    conda-forge
libcblas                  3.9.0           20_win64_openblas    conda-forge
libffi                    3.4.4                hd77b12b_0
libflang                  5.0.0           h6538335_20180525    conda-forge
libiconv                  1.17                 hcfcfb64_2    conda-forge
liblapack                 3.9.0           20_win64_openblas    conda-forge
libopenblas               0.3.25          pthreads_hc140b1d_0    conda-forge
libsqlite                 3.44.2               hcfcfb64_0    conda-forge
libta-lib                 0.4.0                hcfcfb64_2    conda-forge
libxml2                   2.10.4               h0ad7f3c_1
libxslt                   1.1.37               h0192164_0    conda-forge
llvm-meta                 5.0.0                         0    conda-forge
lru-dict                  1.3.0           py310h8d17308_0    conda-forge
lxml                      4.9.1           py310he2412df_0    conda-forge
lz4-c                     1.9.4                hcfcfb64_0    conda-forge
lzo                       2.10              he774522_1000    conda-forge
mako                      1.3.0              pyhd8ed1ab_0    conda-forge
markupsafe                2.1.3           py310h8d17308_1    conda-forge
multipledispatch          0.6.0                      py_0    conda-forge
multitasking              0.0.9              pyhd8ed1ab_0    conda-forge
networkx                  3.2.1              pyhd8ed1ab_0    conda-forge
nomkl                     1.0                  h5ca1d4c_0    conda-forge
numexpr                   2.8.8           py310h27d5182_100    conda-forge
numpy                     1.26.3          py310hf667824_0    conda-forge
openmp                    5.0.0                    vc14_1    conda-forge
openssl                   3.2.0                hcfcfb64_1    conda-forge
packaging                 23.2               pyhd8ed1ab_0    conda-forge
pandas                    2.1.4           py310hecd3228_0    conda-forge
pandas-datareader         0.10.0             pyh6c4a22f_0    conda-forge
patsy                     0.5.5              pyhd8ed1ab_0    conda-forge
peewee                    3.17.0          py310h8b95c24_0    conda-forge
pip                       23.3.1          py310haa95532_0
py-cpuinfo                9.0.0              pyhd8ed1ab_0    conda-forge
pyluach                   2.2.0              pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1              pyh0701188_6    conda-forge
pytables                  3.8.0           py310h4671533_3
python                    3.10.13              he1021f5_0
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-interface          1.6.0                      py_0    conda-forge
python-tzdata             2023.4             pyhd8ed1ab_0    conda-forge
python_abi                3.10                    2_cp310    conda-forge
pytz                      2023.3.post1       pyhd8ed1ab_0    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
scipy                     1.11.4          py310hf667824_0    conda-forge
setuptools                68.2.2          py310haa95532_0
six                       1.16.0             pyh6c4a22f_0    conda-forge
sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
soupsieve                 2.5                pyhd8ed1ab_1    conda-forge
sqlalchemy                2.0.25          py310h8d17308_0    conda-forge
sqlite                    3.41.2               h2bbff1b_0
statsmodels               0.14.1          py310h3e78b6c_0    conda-forge
ta-lib                    0.4.19          py310h9b08ddd_4    conda-forge
tk                        8.6.12               h2bbff1b_0
toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
typing-extensions         4.9.0                hd8ed1ab_0    conda-forge
typing_extensions         4.9.0              pyha770c72_0    conda-forge
tzdata                    2023d                h04d1e81_0
ucrt                      10.0.22621.0         h57928b3_0    conda-forge
urllib3                   2.1.0              pyhd8ed1ab_0    conda-forge
vc                        14.2                 h21ff451_1
vc14_runtime              14.38.33130         h82b7239_18    conda-forge
vs2015_runtime            14.38.33130         hcb4865c_18    conda-forge
webencodings              0.5.1              pyhd8ed1ab_2    conda-forge
wheel                     0.41.2          py310haa95532_0
win_inet_pton             1.1.0              pyhd8ed1ab_6    conda-forge
xz                        5.4.5                h8cc25b3_0
yfinance                  0.2.33             pyhd8ed1ab_0    conda-forge
zipline-reloaded          3.0.3           py310h9b08ddd_0    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               h8cc25b3_0
zlib-ng                   2.0.7                hcfcfb64_0    conda-forge
zstd                      1.5.5                hd43e919_0

Now that you know a little about me, let me tell you about the issue I am having:

Description of Issue

  • What did you expect to happen? I load zipline-reloaded into a fresh conda environment, and expect ziplnie to start cleanly.

  • What happened instead? I run zipline and get the following error:

  File "F:\Python\anaconda3\envs\zipline_test\lib\site-packages\zipline\data\fx\__init__.py", line 4, in <module>
    from .hdf5 import HDF5FXRateReader, HDF5FXRateWriter
  File "F:\Python\anaconda3\envs\zipline_test\lib\site-packages\zipline\data\fx\hdf5.py", line 96, in <module>
    import h5py
  File "F:\Python\anaconda3\envs\zipline_test\lib\site-packages\h5py\__init__.py", line 33, in <module>
    from . import version
  File "F:\Python\anaconda3\envs\zipline_test\lib\site-packages\h5py\version.py", line 15, in <module>
    from . import h5 as _h5
  File "h5py\h5.pyx", line 1, in init h5py.h5
ImportError: DLL load failed while importing defs: The specified procedure could not be found.

Here is how you can reproduce this issue on your machine:

Reproduction Steps

  1. $ conda create -y -n zipline_test python=3.10
  2. $ conda activate zipline_test
  3. $ conda install -y -c conda-forge zipline-reloaded
  4. $ zipline (-> look for error lines quoted above...) ...

What steps have you taken to resolve this already?

I have found a workaround.

  1. $ conda create -y -n zipline_test python=3.10
  2. $ conda activate zipline_test
  3. $ conda install -y -c conda-forge h5py
  4. $ conda install -y -c conda-forge zipline-reloaded
  5. $ zipline (-> starts correctly) ...

Anything else?

I found the workaround by observing this behaviour:

  1. $ conda create -y -n zipline_test python=3.10
  2. $ conda activate zipline_test
  3. $ conda install -y -c conda-forge zipline-reloaded
  4. $ zipline (-> look for error line quoted above...)
  5. $ conda uninstall h5py
  6. $ conda install -y -c conda-forge h5py
  7. $ conda install -y -c conda-forge zipline-reloaded
  8. $ zipline (-> starts correctly) ie: remove and reload the h5py package followed by re-install of zipline-reloaded worked

Whilst this may be an obvious workaround, I thought it may be useful to capture as I did not see a specific instruction on Windows to load h5py prior to zipline-reloaded. ...

Sincerely, SamC

sammmyc avatar Jan 05 '24 14:01 sammmyc

using pip this requires to install debian/ubuntu package libhdf5-dev

gnzsnz avatar Apr 13 '24 15:04 gnzsnz

Ran into it as well. On a Mac, running py3.12 (via pyenv), you need a couple of dependencies before it installs successfully with pip:

brew install hdf5
brew install ta-lib
pip install zipline-reloaded

nucflash avatar May 31 '24 21:05 nucflash

Closing for lack of activity, feel free to reopen if the issue persists.

stefan-jansen avatar Sep 26 '24 14:09 stefan-jansen