Retrieval-based-Voice-Conversion-WebUI icon indicating copy to clipboard operation
Retrieval-based-Voice-Conversion-WebUI copied to clipboard

MacOS Apple M1 Pro faiss-cpu==1.7.0 报错

Open ourines opened this issue 1 year ago • 9 comments

修改requirements.txt中的

faiss-cpu==1.7.0; sys_platform == "darwin"
faiss-cpu==1.7.2; sys_platform == "darwin"

则不会报错,顺利安装成功。

python版本3.10 原安装错误信息

pip3 install faiss-cpu==1.7.0 --no-cache-dir

Collecting faiss-cpu==1.7.0
  Downloading faiss-cpu-1.7.0.tar.gz (34 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: faiss-cpu
  Building wheel for faiss-cpu (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [67 lines of output]
      running bdist_wheel
      running build
      running build_py
      running build_ext
      building 'faiss._swigfaiss' extension
      swigging faiss/faiss/python/swigfaiss.i to faiss/faiss/python/swigfaiss_wrap.cpp
      swig -python -c++ -Doverride= -I/usr/local/include -Ifaiss -o faiss/faiss/python/swigfaiss_wrap.cpp faiss/faiss/python/swigfaiss.i
      faiss/faiss/python/swigfaiss.i:241: Error: Unable to find 'faiss/impl/platform_macros.h'
      faiss/faiss/python/swigfaiss.i:245: Error: Unable to find 'faiss/utils/ordered_key_value.h'
      faiss/faiss/python/swigfaiss.i:246: Error: Unable to find 'faiss/utils/Heap.h'
      faiss/faiss/python/swigfaiss.i:251: Error: Unable to find 'faiss/utils/AlignedTable.h'
      faiss/faiss/python/swigfaiss.i:252: Error: Unable to find 'faiss/utils/partitioning.h'
      faiss/faiss/python/swigfaiss.i:253: Error: Unable to find 'faiss/utils/hamming.h'
      faiss/faiss/python/swigfaiss.i:356: Error: Unable to find 'faiss/utils/utils.h'
      faiss/faiss/python/swigfaiss.i:357: Error: Unable to find 'faiss/utils/distances.h'
      faiss/faiss/python/swigfaiss.i:358: Error: Unable to find 'faiss/utils/random.h'
      faiss/faiss/python/swigfaiss.i:360: Error: Unable to find 'faiss/MetricType.h'
      faiss/faiss/python/swigfaiss.i:363: Error: Unable to find 'faiss/Index.h'
      faiss/faiss/python/swigfaiss.i:364: Error: Unable to find 'faiss/Clustering.h'
      faiss/faiss/python/swigfaiss.i:366: Error: Unable to find 'faiss/utils/extra_distances.h'
      faiss/faiss/python/swigfaiss.i:370: Error: Unable to find 'faiss/impl/ProductQuantizer.h'
      faiss/faiss/python/swigfaiss.i:372: Error: Unable to find 'faiss/VectorTransform.h'
      faiss/faiss/python/swigfaiss.i:373: Error: Unable to find 'faiss/IndexPreTransform.h'
      faiss/faiss/python/swigfaiss.i:374: Error: Unable to find 'faiss/IndexFlat.h'
      faiss/faiss/python/swigfaiss.i:375: Error: Unable to find 'faiss/IndexRefine.h'
      faiss/faiss/python/swigfaiss.i:376: Error: Unable to find 'faiss/IndexLSH.h'
      faiss/faiss/python/swigfaiss.i:377: Error: Unable to find 'faiss/impl/PolysemousTraining.h'
      faiss/faiss/python/swigfaiss.i:378: Error: Unable to find 'faiss/IndexPQ.h'
      faiss/faiss/python/swigfaiss.i:379: Error: Unable to find 'faiss/impl/io.h'
      faiss/faiss/python/swigfaiss.i:381: Error: Unable to find 'faiss/invlists/InvertedLists.h'
      faiss/faiss/python/swigfaiss.i:382: Error: Unable to find 'faiss/invlists/InvertedListsIOHook.h'
      faiss/faiss/python/swigfaiss.i:384: Error: Unable to find 'faiss/invlists/BlockInvertedLists.h'
      faiss/faiss/python/swigfaiss.i:385: Error: Unable to find 'faiss/invlists/DirectMap.h'
      faiss/faiss/python/swigfaiss.i:388: Error: Unable to find 'faiss/IndexIVF.h'
      faiss/faiss/python/swigfaiss.i:393: Error: Unable to find 'faiss/IVFlib.h'
      faiss/faiss/python/swigfaiss.i:394: Error: Unable to find 'faiss/impl/ScalarQuantizer.h'
      faiss/faiss/python/swigfaiss.i:395: Error: Unable to find 'faiss/IndexScalarQuantizer.h'
      faiss/faiss/python/swigfaiss.i:396: Error: Unable to find 'faiss/IndexIVFSpectralHash.h'
      faiss/faiss/python/swigfaiss.i:397: Error: Unable to find 'faiss/impl/HNSW.h'
      faiss/faiss/python/swigfaiss.i:398: Error: Unable to find 'faiss/IndexHNSW.h'
      faiss/faiss/python/swigfaiss.i:399: Error: Unable to find 'faiss/IndexIVFFlat.h'
      faiss/faiss/python/swigfaiss.i:404: Error: Unable to find 'faiss/invlists/OnDiskInvertedLists.h'
      faiss/faiss/python/swigfaiss.i:407: Error: Unable to find 'faiss/impl/lattice_Zn.h'
      faiss/faiss/python/swigfaiss.i:408: Error: Unable to find 'faiss/IndexLattice.h'
      faiss/faiss/python/swigfaiss.i:411: Error: Unable to find 'faiss/IndexIVFPQ.h'
      faiss/faiss/python/swigfaiss.i:412: Error: Unable to find 'faiss/IndexIVFPQR.h'
      faiss/faiss/python/swigfaiss.i:413: Error: Unable to find 'faiss/Index2Layer.h'
      faiss/faiss/python/swigfaiss.i:415: Error: Unable to find 'faiss/IndexPQFastScan.h'
      faiss/faiss/python/swigfaiss.i:416: Error: Unable to find 'faiss/IndexIVFPQFastScan.h'
      faiss/faiss/python/swigfaiss.i:417: Error: Unable to find 'faiss/utils/quantize_lut.h'
      faiss/faiss/python/swigfaiss.i:419: Error: Unable to find 'faiss/IndexBinary.h'
      faiss/faiss/python/swigfaiss.i:420: Error: Unable to find 'faiss/IndexBinaryFlat.h'
      faiss/faiss/python/swigfaiss.i:421: Error: Unable to find 'faiss/IndexBinaryIVF.h'
      faiss/faiss/python/swigfaiss.i:422: Error: Unable to find 'faiss/IndexBinaryFromFloat.h'
      faiss/faiss/python/swigfaiss.i:423: Error: Unable to find 'faiss/IndexBinaryHNSW.h'
      faiss/faiss/python/swigfaiss.i:424: Error: Unable to find 'faiss/IndexBinaryHash.h'
      faiss/faiss/python/swigfaiss.i:426: Error: Unable to find 'faiss/impl/ThreadedIndex.h'
      faiss/faiss/python/swigfaiss.i:430: Error: Unable to find 'faiss/IndexShards.h'
      faiss/faiss/python/swigfaiss.i:434: Error: Unable to find 'faiss/IndexReplicas.h'
      faiss/faiss/python/swigfaiss.i:438: Error: Unable to find 'faiss/MetaIndexes.h'
      faiss/faiss/python/swigfaiss.i:453: Error: Unable to find 'faiss/impl/AuxIndexStructures.h'
      faiss/faiss/python/swigfaiss.i:639: Error: Unable to find 'faiss/index_io.h'
      faiss/faiss/python/swigfaiss.i:640: Error: Unable to find 'faiss/clone_index.h'
      faiss/faiss/python/swigfaiss.i:644: Error: Unable to find 'faiss/AutoTune.h'
      faiss/faiss/python/swigfaiss.i:645: Error: Unable to find 'faiss/index_factory.h'
      faiss/faiss/python/swigfaiss.i:646: Error: Unable to find 'faiss/MatrixStats.h'
      error: command '/opt/homebrew/bin/swig' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for faiss-cpu
  Running setup.py clean for faiss-cpu
Failed to build faiss-cpu
ERROR: Could not build wheels for faiss-cpu, which is required to install pyproject.toml-based projects

使用brew install swig后重新可安装1.7.2

ourines avatar May 08 '23 13:05 ourines

Because we didn't have it validated with 1.7.2 at the time. I will issue a PR for mps support in the near future.

Tps-F avatar May 08 '23 15:05 Tps-F

英特尔的MAC使用1.7.0之后的faiss推理会报错,不知道M1的情况。如果M1没有问题,可能需要在README作特别说明了。

fumiama avatar May 09 '23 06:05 fumiama

m1 同样报错 ps:现在对apple sillicon支持的如何,我使用的时候要调整好多位置,但是最后还是错误

iamqk avatar May 09 '23 06:05 iamqk

m1 同样报错

ps:现在对apple sillicon支持的如何,我使用的时候要调整好多位置,但是最后还是错误

m1 可以按照readme使用brew安装试下

ourines avatar May 09 '23 17:05 ourines

m1 同样报错 ps:现在对apple sillicon支持的如何,我使用的时候要调整好多位置,但是最后还是错误

m1 可以按照readme使用brew安装试下

后续出了问题 mat1 mat2类型不符合 我看了一下相关py文件,好像都是用cuda实现的,又不是很懂 所以等更新了

iamqk avatar May 10 '23 00:05 iamqk

关于faiss,可以尝试使用conda安装,pip即使安装成功也可能会有问题。

fumiama avatar May 10 '23 05:05 fumiama

// macOS 13.2.1, Conda with Python 3.10, 0428完整包, M2Pro

一键推理时出现 1.FileNotFoundError: [Errno 2] No such file or directory: '<PathTo>/3_feature256' :当使用v1模型时出现(256维) (当前master branch)前导出现AttributeError: module 'numpy' has no attribute 'int'错误 2.ValueError: need at least one array to concatenate : 当使用v2模型时出现(训练正常,Training is done,The program is closed.) .pth在推理页面可见,特征索引与特征文件生成未生成。 此后异常处理cleanup似乎存在问题:multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 20 leake d semaphore objects to clean up at shutdown 3.RuntimeError: mat1 and mat2 must have the same dtype

https://eyatu-vrc.hatenablog.com/entry/2023/04/06/193512 指出 FileError 大约是GPU-related ?( https://child-programmer.com/ai-voice-change-tutorial-q-a/#RVC_WebUI038_8211_ValueError_need_at_least_one_array_to_concatenate 好像ValueError也跟GPU有关...

conda + python 3.9 / 3.10, 通过pip/poetry 安装, faiss使用pip/conda安装, 都会遇到上面这三幻神(

如果pip uninstall faiss-cpu然后使用conda conda install -c "pytorch/" faiss-cpu, 启动即出现SystemError: initialization of _internal failed without raising an exception : conda install librosa解决

在折腾环境的同时httpx还会抛锚: ValueError: invalid literal for int() with base 10: ':1' , 重新安装httpx解决

shaotz avatar May 19 '23 14:05 shaotz

Different in here. conda install -c "pytorch/" faiss-cpu gave me another error:

(svc01) naozumi@Naozumis-MBP RVC-beta % python infer-web.py                  
Traceback (most recent call last):
  File "infer-web.py", line 7, in <module>
    import faiss
  File "/usr/local/anaconda3/envs/svc01/lib/python3.8/site-packages/faiss/__init__.py", line 16, in <module>
    from .loader import *
  File "/usr/local/anaconda3/envs/svc01/lib/python3.8/site-packages/faiss/loader.py", line 65, in <module>
    from .swigfaiss import *
  File "/usr/local/anaconda3/envs/svc01/lib/python3.8/site-packages/faiss/swigfaiss.py", line 13, in <module>
    from . import _swigfaiss
ImportError: dlopen(/usr/local/anaconda3/envs/svc01/lib/python3.8/site-packages/faiss/_swigfaiss.so, 0x0002): Library not loaded: @rpath/libmkl_intel_lp64.1.dylib
  Referenced from: <4A025BD7-6F1F-3CD5-B319-DE475BB38238> /usr/local/anaconda3/envs/svc01/lib/libfaiss.dylib
  Reason: tried: '/usr/local/anaconda3/envs/svc01/lib/libmkl_intel_lp64.1.dylib' (no such file), '/usr/local/anaconda3/envs/svc01/lib/python3.8/site-packages/faiss/../../../libmkl_intel_lp64.1.dylib' (no such file), '/usr/local/anaconda3/envs/svc01/lib/python3.8/site-packages/faiss/../../../libmkl_intel_lp64.1.dylib' (no such file), '/usr/local/anaconda3/envs/svc01/bin/../lib/libmkl_intel_lp64.1.dylib' (no such file), '/usr/local/anaconda3/envs/svc01/bin/../lib/libmkl_intel_lp64.1.dylib' (no such file), '/usr/local/lib/libmkl_intel_lp64.1.dylib' (no such file), '/usr/lib/libmkl_intel_lp64.1.dylib' (no such file, not in dyld cache)

I'm really confusing is this repo really support mps...

Naozumi520 avatar May 20 '23 01:05 Naozumi520

Different in here. conda install -c "pytorch/" faiss-cpu gave me another error:

(svc01) naozumi@Naozumis-MBP RVC-beta % python infer-web.py                  
Traceback (most recent call last):
  File "infer-web.py", line 7, in <module>
    import faiss
  File "/usr/local/anaconda3/envs/svc01/lib/python3.8/site-packages/faiss/__init__.py", line 16, in <module>
    from .loader import *
  File "/usr/local/anaconda3/envs/svc01/lib/python3.8/site-packages/faiss/loader.py", line 65, in <module>
    from .swigfaiss import *
  File "/usr/local/anaconda3/envs/svc01/lib/python3.8/site-packages/faiss/swigfaiss.py", line 13, in <module>
    from . import _swigfaiss
ImportError: dlopen(/usr/local/anaconda3/envs/svc01/lib/python3.8/site-packages/faiss/_swigfaiss.so, 0x0002): Library not loaded: @rpath/libmkl_intel_lp64.1.dylib
  Referenced from: <4A025BD7-6F1F-3CD5-B319-DE475BB38238> /usr/local/anaconda3/envs/svc01/lib/libfaiss.dylib
  Reason: tried: '/usr/local/anaconda3/envs/svc01/lib/libmkl_intel_lp64.1.dylib' (no such file), '/usr/local/anaconda3/envs/svc01/lib/python3.8/site-packages/faiss/../../../libmkl_intel_lp64.1.dylib' (no such file), '/usr/local/anaconda3/envs/svc01/lib/python3.8/site-packages/faiss/../../../libmkl_intel_lp64.1.dylib' (no such file), '/usr/local/anaconda3/envs/svc01/bin/../lib/libmkl_intel_lp64.1.dylib' (no such file), '/usr/local/anaconda3/envs/svc01/bin/../lib/libmkl_intel_lp64.1.dylib' (no such file), '/usr/local/lib/libmkl_intel_lp64.1.dylib' (no such file), '/usr/lib/libmkl_intel_lp64.1.dylib' (no such file, not in dyld cache)

Not sure if it is for libmkl, because libmkl is also not present here on my Mac. Maybe trying conda install swig or conda update --all?

I'm really confusing is this repo really support mps...

Yeah, FileNotFoundError: [Errno 2] No such file or directory: '<PathTo>/3_feature256' comes in at step3a: training the model, and some Japanese developers have mentioned this as a GPU-related issue in their blogs (here probably MPS).

shaotz avatar May 20 '23 03:05 shaotz

Now using faiss-cpu==1.7.3 and it's work https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI/blob/main/requirements.txt#LL8C1-L8C1

Tps-F avatar Jun 05 '23 11:06 Tps-F

Had the same problem on M1, faiss-cpu==1.7.3 made pip install -r requirements.txt work for me as well.

geonnave avatar Jul 22 '23 11:07 geonnave

好像使用 faiss-cpu==1.7.3 还是不行,Apple M1 Pro 然后 swig 是 4.2.0 不知道和这个有没有关系

jkryanchou avatar Feb 14 '24 03:02 jkryanchou