bcolz icon indicating copy to clipboard operation
bcolz copied to clipboard

Update older dependencies

Open CarstVaartjes opened this issue 7 years ago • 2 comments

Hi @FrancescAlted!

I hope you had a nice Christmas. I'm currently looking into some warnings that bcolz is spiking in our tests, namely:

/srv/python/venv/local/lib/python2.7/site-packages/bcolz/ctable.py:808:
FutureWarning: from_items is deprecated. Please use DataFrame.from_dict(dict(items), ...) instead. DataFrame.from_dict(OrderedDict(items)) may be used to preserve the key order.

bcolz/carray_ext.c:613:
/srv/python/venv/local/lib/python2.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
 #warning "Using deprecated NumPy API, disable it by " \
  ^~~~~~~

Also the c-blosc version used in bcolz was rather behind the latest version, which means we're missing out on nice zstd improvements etc.

I've made a PR (https://github.com/Blosc/bcolz/pull/386); it addresses the pandas and blosc versions, i have to look into the numpy depreciation still, but that's more complex.

I'm running into some issues with blosc though. I can build it with cmake without issues, but when installing bcolz with python setup.py build_ext --inplace, i get these errors:

c-blosc/internal-complibs/zstd-1.3.7/compress/zstd_compress.c: In function ‘ZSTD_minCLevel’:
c-blosc/internal-complibs/zstd-1.3.7/compress/zstd_compress.c:3901:41: error: ‘ZSTD_TARGETLENGTH_MAX’ undeclared (first use in this function); did you mean ‘ZSTD_TARGETLENGTH_MIN’?
 int ZSTD_minCLevel(void) { return (int)-ZSTD_TARGETLENGTH_MAX; }
                                         ^~~~~~~~~~~~~~~~~~~~~
                                         ZSTD_TARGETLENGTH_MIN
c-blosc/internal-complibs/zstd-1.3.7/compress/zstd_compress.c: In function ‘ZSTD_getCParams’:
c-blosc/internal-complibs/zstd-1.3.7/compress/zstd_compress.c:4020:38: error: ‘ZSTD_CLEVEL_DEFAULT’ undeclared (first use in this function); did you mean ‘HUF_TABLELOG_DEFAULT’?
     if (compressionLevel == 0) row = ZSTD_CLEVEL_DEFAULT;   /* 0 == default */
                                      ^~~~~~~~~~~~~~~~~~~
                                      HUF_TABLELOG_DEFAULT

Travis also fails: https://travis-ci.org/Blosc/bcolz/jobs/473336491

Could you perhaps give me a hint of what i'm doing wrong here?

CarstVaartjes avatar Dec 29 '18 15:12 CarstVaartjes

Yeah, that was much needed; thanks for addressing this. Regarding the Zstd errors, well, it looks like somehow, the Python extension is not finding the zstd.h header, mmmhhh...

FrancescAlted avatar Dec 31 '18 08:12 FrancescAlted

I will look into it this weekend!

CarstVaartjes avatar Jan 01 '19 22:01 CarstVaartjes