faiss-wheels
faiss-wheels copied to clipboard
Add support to release linux aarch64 wheels
Problem
On aarch64, ‘pip install faiss-gpu’ builds the wheels from source code and gives the below error-
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-mjddlp5q/faiss-gpu/setup.py'"'"'; __file__='"'"'/tmp/pip-install-mjddlp5q/faiss-gpu/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-r241ub16
cwd: /tmp/pip-install-mjddlp5q/faiss-gpu/
Complete output (75 lines):
running bdist_wheel
running build
running build_py
running build_ext
building 'faiss._swigfaiss' extension
swigging faiss/faiss/python/swigfaiss.i to faiss/faiss/python/swigfaiss_wrap.cpp
swig -python -c++ -Doverride= -I/usr/local/include -Ifaiss -DSWIGWORDSIZE64 -o faiss/faiss/python/swigfaiss_wrap.cpp faiss/faiss/python/swigfaiss.i
faiss/faiss/python/swigfaiss.i:250: Error: Unable to find 'faiss/impl/platform_macros.h'
faiss/faiss/python/swigfaiss.i:254: Error: Unable to find 'faiss/utils/ordered_key_value.h'
faiss/faiss/python/swigfaiss.i:255: Error: Unable to find 'faiss/utils/Heap.h'
faiss/faiss/python/swigfaiss.i:260: Error: Unable to find 'faiss/utils/AlignedTable.h'
faiss/faiss/python/swigfaiss.i:261: Error: Unable to find 'faiss/utils/partitioning.h'
faiss/faiss/python/swigfaiss.i:262: Error: Unable to find 'faiss/utils/hamming.h'
faiss/faiss/python/swigfaiss.i:365: Error: Unable to find 'faiss/utils/utils.h'
faiss/faiss/python/swigfaiss.i:366: Error: Unable to find 'faiss/utils/distances.h'
faiss/faiss/python/swigfaiss.i:367: Error: Unable to find 'faiss/utils/random.h'
faiss/faiss/python/swigfaiss.i:369: Error: Unable to find 'faiss/MetricType.h'
faiss/faiss/python/swigfaiss.i:372: Error: Unable to find 'faiss/Index.h'
faiss/faiss/python/swigfaiss.i:373: Error: Unable to find 'faiss/Clustering.h'
faiss/faiss/python/swigfaiss.i:375: Error: Unable to find 'faiss/utils/extra_distances.h'
faiss/faiss/python/swigfaiss.i:379: Error: Unable to find 'faiss/impl/ProductQuantizer.h'
faiss/faiss/python/swigfaiss.i:380: Error: Unable to find 'faiss/impl/AdditiveQuantizer.h'
faiss/faiss/python/swigfaiss.i:381: Error: Unable to find 'faiss/impl/ResidualQuantizer.h'
Resolution
On aarch64, ‘pip install faiss-gpu’ should download the wheels from PyPI
@kyamagu Please let me know your interest in releasing aarch64 wheels.
@odidev the current CI build environment cannot build GPU binaries due to the extremely long build time (easily timeout). Unfortunately I have no plan on adding external CI specifically for aarch64 gpu build
Relevant info: https://github.blog/changelog/2023-10-30-accelerate-your-ci-cd-with-arm-based-hosted-runners-in-github-actions/