faiss-wheels icon indicating copy to clipboard operation
faiss-wheels copied to clipboard

Add support to release linux aarch64 wheels

Open odidev opened this issue 2 years ago • 2 comments

Problem

On aarch64, ‘pip install faiss-gpu’ builds the wheels from source code and gives the below error-

ERROR: Command errored out with exit status 1: 
   command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-mjddlp5q/faiss-gpu/setup.py'"'"'; __file__='"'"'/tmp/pip-install-mjddlp5q/faiss-gpu/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-r241ub16 
       cwd: /tmp/pip-install-mjddlp5q/faiss-gpu/ 
  Complete output (75 lines): 
  running bdist_wheel 
  running build 
  running build_py 
  running build_ext 
  building 'faiss._swigfaiss' extension 
  swigging faiss/faiss/python/swigfaiss.i to faiss/faiss/python/swigfaiss_wrap.cpp 
  swig -python -c++ -Doverride= -I/usr/local/include -Ifaiss -DSWIGWORDSIZE64 -o faiss/faiss/python/swigfaiss_wrap.cpp faiss/faiss/python/swigfaiss.i 
  faiss/faiss/python/swigfaiss.i:250: Error: Unable to find 'faiss/impl/platform_macros.h' 
  faiss/faiss/python/swigfaiss.i:254: Error: Unable to find 'faiss/utils/ordered_key_value.h' 
  faiss/faiss/python/swigfaiss.i:255: Error: Unable to find 'faiss/utils/Heap.h' 
  faiss/faiss/python/swigfaiss.i:260: Error: Unable to find 'faiss/utils/AlignedTable.h' 
  faiss/faiss/python/swigfaiss.i:261: Error: Unable to find 'faiss/utils/partitioning.h' 
  faiss/faiss/python/swigfaiss.i:262: Error: Unable to find 'faiss/utils/hamming.h' 
  faiss/faiss/python/swigfaiss.i:365: Error: Unable to find 'faiss/utils/utils.h' 
  faiss/faiss/python/swigfaiss.i:366: Error: Unable to find 'faiss/utils/distances.h' 
  faiss/faiss/python/swigfaiss.i:367: Error: Unable to find 'faiss/utils/random.h' 
  faiss/faiss/python/swigfaiss.i:369: Error: Unable to find 'faiss/MetricType.h' 
  faiss/faiss/python/swigfaiss.i:372: Error: Unable to find 'faiss/Index.h' 
  faiss/faiss/python/swigfaiss.i:373: Error: Unable to find 'faiss/Clustering.h' 
  faiss/faiss/python/swigfaiss.i:375: Error: Unable to find 'faiss/utils/extra_distances.h' 
  faiss/faiss/python/swigfaiss.i:379: Error: Unable to find 'faiss/impl/ProductQuantizer.h' 
  faiss/faiss/python/swigfaiss.i:380: Error: Unable to find 'faiss/impl/AdditiveQuantizer.h' 
  faiss/faiss/python/swigfaiss.i:381: Error: Unable to find 'faiss/impl/ResidualQuantizer.h'

Resolution

On aarch64, ‘pip install faiss-gpu’ should download the wheels from PyPI

@kyamagu Please let me know your interest in releasing aarch64 wheels.

odidev avatar Dec 24 '21 05:12 odidev

@odidev the current CI build environment cannot build GPU binaries due to the extremely long build time (easily timeout). Unfortunately I have no plan on adding external CI specifically for aarch64 gpu build

kyamagu avatar Dec 26 '21 04:12 kyamagu

Relevant info: https://github.blog/changelog/2023-10-30-accelerate-your-ci-cd-with-arm-based-hosted-runners-in-github-actions/

kyamagu avatar Oct 31 '23 04:10 kyamagu