bcolz
bcolz copied to clipboard
Published
20 hours ago •
Blosc
Blosc
Failed to install 1.2.1 by pip on Dell Optiplex 5050 («error: conflicting types for ‘_xgetbv’»). Probably problem in compiling if "avx2" in cpu_info["flags"]
Failed to install 1.2.1 (also 1.2.0) by pip (pip3, python 3.7) on Fedora 30. — «c-blosc/blosc/shuffle.c:170:1: error: conflicting types for ‘_xgetbv’»
ternal-complibs/zstd-1.3.4/common -Ic-blosc/internal-complibs/zstd-1.3.4/legacy -Ic-blosc/internal-complibs/zstd-1.3.4/decompress -Ic-blosc/internal-complibs/zstd-1.3.4/dictBuilder -Ic-blosc/internal-complibs/zstd-1.3.4/compress -I/usr/local/lib64/python3.7/site-packages/numpy/core/include -I/usr/include/python3.7m -c c-blosc/blosc/shuffle.c -o build/temp.linux-x86_64-3.7/c-blosc/blosc/shuffle.o -DSHUFFLE_SSE2_ENABLED -msse2 -DSHUFFLE_AVX2_ENABLED -mavx2
c-blosc/blosc/shuffle.c:170:1: error: conflicting types for ‘_xgetbv’
170 | _xgetbv(uint32_t xcr) {
| ^~~~~~~
In file included from /usr/lib/gcc/x86_64-redhat-linux/9/include/immintrin.h:43,
from c-blosc/blosc/blosc-common.h:77,
from c-blosc/blosc/shuffle.h:18,
from c-blosc/blosc/shuffle.c:10:
/usr/lib/gcc/x86_64-redhat-linux/9/include/xsaveintrin.h:60:1: note: previous definition of ‘_xgetbv’ was here
60 | _xgetbv (unsigned int __A)
| ^~~~~~~
In file included from c-blosc/blosc/shuffle.c:11:
c-blosc/blosc/shuffle-generic.h:61:13: warning: ‘unshuffle_generic_inline’ defined but not used [-Wunused-function]
61 | static void unshuffle_generic_inline(const size_t type_size,
| ^~~~~~~~~~~~~~~~~~~~~~~~
c-blosc/blosc/shuffle-generic.h:32:13: warning: ‘shuffle_generic_inline’ defined but not used [-Wunused-function]
32 | static void shuffle_generic_inline(const size_t type_size,
| ^~~~~~~~~~~~~~~~~~~~~~
error: command 'gcc' failed with exit status 1
Actually, the problem not in Linux Distro, but in target architecture.
I tried to install on Dell Optiplex 5050, and the problem in supporting "avx2" in cpu_info["flags"].
I cannot fix the bug, but found workaround (after enumeration all possible flags) → add to setup.py
after the block
# For guessing the capabilities of the CPU for C-Blosc
try:
# Currently just Intel and some ARM archs are supported by cpuinfo module
import cpuinfo
cpu_info = cpuinfo.get_cpu_info()
except:
cpu_info = {'flags': []}
something like this:
flags_ = cpu_info["flags"]
flags_.remove("avx2")
I confirm facing the same issue on a computer running Ubuntu 19.10 with Core™ i7-10510U CPU. Note that bcolz v1.1.1 do compile without issue on the very same system.
I confirm the same issue with 1.2.2 (also all the other versions) by pip (pip3, python 3.7) on Fedora 31. «c-blosc/blosc/shuffle.c:170:1: error: conflicting types for ‘_xgetbv’»
Is there any way to bypass the error and get installed the package?
Collecting bcolz
Using cached bcolz-1.2.1.tar.gz (1.5 MB)
Requirement already satisfied: numpy>=1.7 in /usr/local/lib64/python3.7/site-packages (from bcolz) (1.18.1)
Building wheels for collected packages: bcolz
Building wheel for bcolz (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-_dm1gcei/bcolz/setup.py'"'"'; __file__='"'"'/tmp/pip-install-_dm1gcei/bcolz/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-ckm5pl6i
cwd: /tmp/pip-install-_dm1gcei/bcolz/
Complete output (86 lines):
SSE2 detected
AVX2 detected
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.7
creating build/lib.linux-x86_64-3.7/bcolz
copying bcolz/arrayprint.py -> build/lib.linux-x86_64-3.7/bcolz
copying bcolz/py2help.py -> build/lib.linux-x86_64-3.7/bcolz
copying bcolz/attrs.py -> build/lib.linux-x86_64-3.7/bcolz
copying bcolz/defaults.py -> build/lib.linux-x86_64-3.7/bcolz
copying bcolz/chunked_eval.py -> build/lib.linux-x86_64-3.7/bcolz
copying bcolz/utils.py -> build/lib.linux-x86_64-3.7/bcolz
copying bcolz/__init__.py -> build/lib.linux-x86_64-3.7/bcolz
copying bcolz/ctable.py -> build/lib.linux-x86_64-3.7/bcolz
copying bcolz/toplevel.py -> build/lib.linux-x86_64-3.7/bcolz
copying bcolz/version.py -> build/lib.linux-x86_64-3.7/bcolz
copying bcolz/py2help_tests.py -> build/lib.linux-x86_64-3.7/bcolz
creating build/lib.linux-x86_64-3.7/bcolz/tests
copying bcolz/tests/test_carray.py -> build/lib.linux-x86_64-3.7/bcolz/tests
copying bcolz/tests/test_ctable.py -> build/lib.linux-x86_64-3.7/bcolz/tests
copying bcolz/tests/test_queries.py -> build/lib.linux-x86_64-3.7/bcolz/tests
copying bcolz/tests/common.py -> build/lib.linux-x86_64-3.7/bcolz/tests
copying bcolz/tests/all.py -> build/lib.linux-x86_64-3.7/bcolz/tests
copying bcolz/tests/__init__.py -> build/lib.linux-x86_64-3.7/bcolz/tests
copying bcolz/tests/test_attrs.py -> build/lib.linux-x86_64-3.7/bcolz/tests
copying bcolz/tests/test_carray_objects.py -> build/lib.linux-x86_64-3.7/bcolz/tests
copying bcolz/tests/test_ndcarray.py -> build/lib.linux-x86_64-3.7/bcolz/tests
copying bcolz/carray_ext.pxd -> build/lib.linux-x86_64-3.7/bcolz
running build_ext
skipping 'bcolz/carray_ext.c' Cython extension (up-to-date)
building 'bcolz.carray_ext' extension
creating build/temp.linux-x86_64-3.7
creating build/temp.linux-x86_64-3.7/bcolz
creating build/temp.linux-x86_64-3.7/c-blosc
creating build/temp.linux-x86_64-3.7/c-blosc/blosc
creating build/temp.linux-x86_64-3.7/c-blosc/internal-complibs
creating build/temp.linux-x86_64-3.7/c-blosc/internal-complibs/lz4-1.8.1.2
creating build/temp.linux-x86_64-3.7/c-blosc/internal-complibs/snappy-1.1.1
creating build/temp.linux-x86_64-3.7/c-blosc/internal-complibs/zlib-1.2.8
creating build/temp.linux-x86_64-3.7/c-blosc/internal-complibs/zstd-1.3.4
creating build/temp.linux-x86_64-3.7/c-blosc/internal-complibs/zstd-1.3.4/deprecated
creating build/temp.linux-x86_64-3.7/c-blosc/internal-complibs/zstd-1.3.4/common
creating build/temp.linux-x86_64-3.7/c-blosc/internal-complibs/zstd-1.3.4/legacy
creating build/temp.linux-x86_64-3.7/c-blosc/internal-complibs/zstd-1.3.4/decompress
creating build/temp.linux-x86_64-3.7/c-blosc/internal-complibs/zstd-1.3.4/dictBuilder
creating build/temp.linux-x86_64-3.7/c-blosc/internal-complibs/zstd-1.3.4/compress
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DHAVE_LZ4=1 -DHAVE_SNAPPY=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Ibcolz -Ic-blosc/blosc -Ic-blosc/internal-complibs/zlib-1.2.8 -Ic-blosc/internal-complibs/snappy-1.1.1 -Ic-blosc/internal-complibs/lz4-1.8.1.2 -Ic-blosc/internal-complibs/zstd-1.3.4-Ic-blosc/internal-complibs/zstd-1.3.4/deprecated -Ic-blosc/internal-complibs/zstd-1.3.4/dll -Ic-blosc/internal-complibs/zstd-1.3.4/common -Ic-blosc/internal-complibs/zstd-1.3.4/legacy -Ic-blosc/internal-complibs/zstd-1.3.4/decompress -Ic-blosc/internal-complibs/zstd-1.3.4/dictBuilder -Ic-blosc/internal-complibs/zstd-1.3.4/compress -I/usr/local/lib64/python3.7/site-packages/numpy/core/include -I/usr/include/python3.7m -c bcolz/carray_ext.c -o build/temp.linux-x86_64-3.7/bcolz/carray_ext.o -DSHUFFLE_SSE2_ENABLED -msse2 -DSHUFFLE_AVX2_ENABLED -mavx2
In file included from /usr/local/lib64/python3.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1832,
from /usr/local/lib64/python3.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:12,
from /usr/local/lib64/python3.7/site-packages/numpy/core/include/numpy/arrayobject.h:4,
from bcolz/carray_ext.c:525:
/usr/local/lib64/python3.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
17 | #warning "Using deprecated NumPy API, disable it with " \
| ^~~~~~~
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DHAVE_LZ4=1 -DHAVE_SNAPPY=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Ibcolz -Ic-blosc/blosc -Ic-blosc/internal-complibs/zlib-1.2.8 -Ic-blosc/internal-complibs/snappy-1.1.1 -Ic-blosc/internal-complibs/lz4-1.8.1.2 -Ic-blosc/internal-complibs/zstd-1.3.4-Ic-blosc/internal-complibs/zstd-1.3.4/deprecated -Ic-blosc/internal-complibs/zstd-1.3.4/dll -Ic-blosc/internal-complibs/zstd-1.3.4/common -Ic-blosc/internal-complibs/zstd-1.3.4/legacy -Ic-blosc/internal-complibs/zstd-1.3.4/decompress -Ic-blosc/internal-complibs/zstd-1.3.4/dictBuilder -Ic-blosc/internal-complibs/zstd-1.3.4/compress -I/usr/local/lib64/python3.7/site-packages/numpy/core/include -I/usr/include/python3.7m -c c-blosc/blosc/shuffle-generic.c -o build/temp.linux-x86_64-3.7/c-blosc/blosc/shuffle-generic.o -DSHUFFLE_SSE2_ENABLED -msse2 -DSHUFFLE_AVX2_ENABLED -mavx2
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DHAVE_LZ4=1 -DHAVE_SNAPPY=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Ibcolz -Ic-blosc/blosc -Ic-blosc/internal-complibs/zlib-1.2.8 -Ic-blosc/internal-complibs/snappy-1.1.1 -Ic-blosc/internal-complibs/lz4-1.8.1.2 -Ic-blosc/internal-complibs/zstd-1.3.4-Ic-blosc/internal-complibs/zstd-1.3.4/deprecated -Ic-blosc/internal-complibs/zstd-1.3.4/dll -Ic-blosc/internal-complibs/zstd-1.3.4/common -Ic-blosc/internal-complibs/zstd-1.3.4/legacy -Ic-blosc/internal-complibs/zstd-1.3.4/decompress -Ic-blosc/internal-complibs/zstd-1.3.4/dictBuilder -Ic-blosc/internal-complibs/zstd-1.3.4/compress -I/usr/local/lib64/python3.7/site-packages/numpy/core/include -I/usr/include/python3.7m -c c-blosc/blosc/blosclz.c -o build/temp.linux-x86_64-3.7/c-blosc/blosc/blosclz.o -DSHUFFLE_SSE2_ENABLED -msse2 -DSHUFFLE_AVX2_ENABLED -mavx2
c-blosc/blosc/blosclz.c: In function ‘get_run_32’:
c-blosc/blosc/blosclz.c:148:35: warning: comparison of integer expressions of different signedness:‘int’ and ‘unsigned int’ [-Wsign-compare]
148 | if (_mm256_movemask_epi8(cmp) != 0xFFFFFFFF) {
| ^~
c-blosc/blosc/blosclz.c: In function ‘get_match_32’:
c-blosc/blosc/blosclz.c:219:35: warning: comparison of integer expressions of different signedness:‘int’ and ‘unsigned int’ [-Wsign-compare]
219 | if (_mm256_movemask_epi8(cmp) != 0xFFFFFFFF) {
| ^~
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DHAVE_LZ4=1 -DHAVE_SNAPPY=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Ibcolz -Ic-blosc/blosc -Ic-blosc/internal-complibs/zlib-1.2.8 -Ic-blosc/internal-complibs/snappy-1.1.1 -Ic-blosc/internal-complibs/lz4-1.8.1.2 -Ic-blosc/internal-complibs/zstd-1.3.4-Ic-blosc/internal-complibs/zstd-1.3.4/deprecated -Ic-blosc/internal-complibs/zstd-1.3.4/dll -Ic-blosc/internal-complibs/zstd-1.3.4/common -Ic-blosc/internal-complibs/zstd-1.3.4/legacy -Ic-blosc/internal-complibs/zstd-1.3.4/decompress -Ic-blosc/internal-complibs/zstd-1.3.4/dictBuilder -Ic-blosc/internal-complibs/zstd-1.3.4/compress -I/usr/local/lib64/python3.7/site-packages/numpy/core/include -I/usr/include/python3.7m -c c-blosc/blosc/fastcopy.c -o build/temp.linux-x86_64-3.7/c-blosc/blosc/fastcopy.o -DSHUFFLE_SSE2_ENABLED -msse2 -DSHUFFLE_AVX2_ENABLED -mavx2
gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DHAVE_LZ4=1 -DHAVE_SNAPPY=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -Ibcolz -Ic-blosc/blosc -Ic-blosc/internal-complibs/zlib-1.2.8 -Ic-blosc/internal-complibs/snappy-1.1.1 -Ic-blosc/internal-complibs/lz4-1.8.1.2 -Ic-blosc/internal-complibs/zstd-1.3.4-Ic-blosc/internal-complibs/zstd-1.3.4/deprecated -Ic-blosc/internal-complibs/zstd-1.3.4/dll -Ic-blosc/internal-complibs/zstd-1.3.4/common -Ic-blosc/internal-complibs/zstd-1.3.4/legacy -Ic-blosc/internal-complibs/zstd-1.3.4/decompress -Ic-blosc/internal-complibs/zstd-1.3.4/dictBuilder -Ic-blosc/internal-complibs/zstd-1.3.4/compress -I/usr/local/lib64/python3.7/site-packages/numpy/core/include -I/usr/include/python3.7m -c c-blosc/blosc/shuffle.c -o build/temp.linux-x86_64-3.7/c-blosc/blosc/shuffle.o -DSHUFFLE_SSE2_ENABLED -msse2 -DSHUFFLE_AVX2_ENABLED -mavx2
c-blosc/blosc/shuffle.c:170:1: error: conflicting types for ‘_xgetbv’
170 | _xgetbv(uint32_t xcr) {
| ^~~~~~~
In file included from /usr/lib/gcc/x86_64-redhat-linux/9/include/immintrin.h:43,
from c-blosc/blosc/blosc-common.h:77,
from c-blosc/blosc/shuffle.h:18,
from c-blosc/blosc/shuffle.c:10:
/usr/lib/gcc/x86_64-redhat-linux/9/include/xsaveintrin.h:60:1: note: previous definition of ‘_xgetbv’ was here
60 | _xgetbv (unsigned int __A)
| ^~~~~~~
In file included from c-blosc/blosc/shuffle.c:11:
c-blosc/blosc/shuffle-generic.h:61:13: warning: ‘unshuffle_generic_inline’ defined but not used [-Wunused-function]
61 | static void unshuffle_generic_inline(const size_t type_size,
| ^~~~~~~~~~~~~~~~~~~~~~~~
c-blosc/blosc/shuffle-generic.h:32:13: warning: ‘shuffle_generic_inline’ defined but not used [-Wunused-function]
32 | static void shuffle_generic_inline(const size_t type_size,
| ^~~~~~~~~~~~~~~~~~~~~~
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for bcolz
Is there any way to bypass the error and get installed the package?
My workaround → https://github.com/Blosc/bcolz/issues/398#issuecomment-536650569
@belonesox I tried using your workaround. However that results in the following error
cythoning bcolz/carray_ext.pyx to bcolz/carray_ext.c
/tmp/pip-req-build-8dm5a1w2/.eggs/Cython-3.0a5-py3.6.egg/Cython/Compiler/Main.py:344: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: /tmp/pip-req-build-8dm5a1w2/bcolz/carray_ext.pxd
tree = Parsing.p_module(s, pxd, full_module_name)
Error compiling Cython file:
------------------------------------------------------------
...
# Create the final container and fill it
out = carray([], dtype=newdtype, cparams=self.cparams,
expectedlen=newlen,
rootdir=rootdir, mode='w')
if newlen < ilen:
rsize = isize / newlen
^
------------------------------------------------------------
bcolz/carray_ext.pyx:1685:26: Cannot assign type 'double' to 'npy_intp'
I had the same issue on Dell Latitude 4790 (Ubuntu 20.04). I was able to install the package v1.2.1 with the following commands:
export DISABLE_BCOLZ_AVX2=true
python setup.py build_ext --inplace
I had the same issue on Dell Latitude 4790 (Ubuntu 20.04). I was able to install the package v1.2.1 with the following commands:
export DISABLE_BCOLZ_AVX2=true python setup.py build_ext --inplace
Thank you. It works
I had the same issue on Dell Latitude 4790 (Ubuntu 20.04). I was able to install the package v1.2.1 with the following commands:
export DISABLE_BCOLZ_AVX2=true python setup.py build_ext --inplace
@nevezhyn thanks. this solved it for me in an MSI laptop (Pop OS 20.10)
