Unable to Run Python Tests and Segementation Fault Error on Being run on a Flask Server on M1

[xinslu]

Summary

Platform

OS:

macOS Monterey 12.0.1 (M1, 2020)

Faiss version: 1.7.2

Installed from:

Compiled from Source

Faiss compilation options:

cmake -B build . -DFAISS_ENABLE_PYTHON=ON, -DBUILD_TESTING=ON, -DFAISS_ENABLE_GPU=OFF

Running on:

• [x] CPU
• [ ] GPU

Interface:

• [ ] C++
• [x] Python

Reproduction instructions

I build the code using CMake and saved it on the directory I am working on and running all other related code from. Test Session Result:

============================= test session starts ==============================
platform darwin -- Python 3.9.7, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /Users/kinshukphalke/Documents/Notitia/faiss
collecting 5 items                                                             Fatal Python error: Aborted

Current thread 0x0000000100bd8580 (most recent call first):
  File "/Users/kinshukphalke/Documents/Notitia/faiss/build/faiss/python/build/lib/faiss/swigfaiss.py", line 9236 in omp_set_num_threads
  File "/Users/kinshukphalke/Documents/Notitia/faiss/tests/common_faiss_tests.py", line 15 in <module>
  File "<frozen importlib._bootstrap>", line 228 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 850 in exec_module
  File "<frozen importlib._bootstrap>", line 680 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 986 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1007 in _find_and_load
  File "/Users/kinshukphalke/Documents/Notitia/faiss/tests/test_binary_hashindex.py", line 10 in <module>
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/_pytest/assertion/rewrite.py", line 170 in exec_module
  File "<frozen importlib._bootstrap>", line 680 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 986 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1007 in _find_and_load
  File "<frozen importlib._bootstrap>", line 1030 in _gcd_import
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/importlib/__init__.py", line 127 in import_module
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/_pytest/pathlib.py", line 524 in import_path
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/_pytest/python.py", line 578 in _importtestmodule
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/_pytest/python.py", line 500 in _getobj
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/_pytest/python.py", line 291 in obj
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/_pytest/python.py", line 516 in _inject_setup_module_fixture
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/_pytest/python.py", line 503 in collect
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/_pytest/runner.py", line 341 in <lambda>
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/_pytest/runner.py", line 311 in from_call
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/_pytest/runner.py", line 341 in pytest_make_collect_report
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/_pytest/runner.py", line 458 in collect_one_node
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/_pytest/main.py", line 808 in genitems
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/_pytest/main.py", line 634 in perform_collect
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/_pytest/main.py", line 333 in pytest_collection
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/_pytest/main.py", line 322 in _main
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/_pytest/main.py", line 269 in wrap_session
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/_pytest/main.py", line 316 in pytest_cmdline_main
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/_pytest/config/__init__.py", line 162 in main
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/lib/python3.9/site-packages/_pytest/config/__init__.py", line 185 in console_main
  File "/opt/homebrew/Caskroom/miniforge/base/envs/Notitia/bin/pytest", line 11 in <module>
[1]    17637 abort      PYTHONPATH="$(ls -d ./build/faiss/python/build/lib*/)" pytest tests/test_*.py

On the other hand, when I run the C++ Tests using the make command I pass every single of them. Also when I run the Flask Server calling some data on it gives a segmentation Fault Error.

2021-12-25 22:01:34.424121: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
2021-12-25 22:01:34.549980: W tensorflow/core/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz
Done loading embeddings model.
 * Serving Flask app 'app' (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
WARNING:werkzeug: * Running on all addresses.
   WARNING: This is a development server. Do not use it in a production deployment.
INFO:werkzeug: * Running on http://192.168.1.247:105/ (Press CTRL+C to quit)
WARNING clustering 146 points to 6 centroids: please provide at least 234 training points
[1]    17899 segmentation fault  python3 app.py

This is the relevant code from the Flask Server:

import faiss
test_kmeans = faiss.Kmeans(k=min(250, int(math.sqrt(len(np_embeddings))) // 2), d=len(np_embeddings[0]), niter = 20)
test_kmeans.train(np_embeddings)
D, Y = test_kmeans.index.search(np_embeddings, 1)

I can confirm that the code works on intel processor. My Code does successfully import faiss.

EDIT: Forgot to Mention I've been building and doing everything on a Conda Env

EDIT 2: I've been able to "Run" the test once i quit the Conda Env. But they all result in an error.

This is the Summary of one of the Errors, They're all the same.

ImportError while importing test module '/Users/kinshukphalke/Documents/Notitia/faiss/tests/test_referenced_objects.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_referenced_objects.py:11: in <module>
    import faiss
build/faiss/python/build/lib/faiss/__init__.py:18: in <module>
    from .loader import *
build/faiss/python/build/lib/faiss/loader.py:65: in <module>
    from .swigfaiss import *
build/faiss/python/build/lib/faiss/swigfaiss.py:13: in <module>
    from . import _swigfaiss
E   ImportError: dlopen(/Users/kinshukphalke/Documents/Notitia/faiss/build/faiss/python/build/lib/faiss/_swigfaiss.so, 0x0002): tried: '/Users/kinshukphalke/Documents/Notitia/faiss/build/faiss/python/build/lib/faiss/_swigfaiss.so' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/usr/lib/_swigfaiss.so' (no such file)

EDIT 3: On deeper dive into the CMake commands i noticed that while running make -C build -j swigfaiss, I get weak link errors mostly in the index part of the code could these be related? (leaving a small set of those here)

[100%] Linking CXX shared library _swigfaiss.so
ld: warning: direct access in function '_wrap_downcast_index(_object*, _object*)' from file 'CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx.o' to global weak symbol 'typeinfo for faiss::IndexReplicasTemplate<faiss::Index>' from file '../libfaiss.a(IndexReplicas.cpp.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
ld: warning: direct access in function '_wrap_downcast_index(_object*, _object*)' from file 'CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx.o' to global weak symbol 'typeinfo for faiss::IndexReplicasTemplate<faiss::Index>' from file '../libfaiss.a(IndexReplicas.cpp.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
ld: warning: direct access in function '_wrap_downcast_index(_object*, _object*)' from file 'CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx.o' to global weak symbol 'typeinfo for faiss::IndexShardsTemplate<faiss::Index>' from file '../libfaiss.a(IndexShards.cpp.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
ld: warning: direct access in function '_wrap_downcast_index(_object*, _object*)' from file 'CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx.o' to global weak symbol 'typeinfo for faiss::IndexShardsTemplate<faiss::Index>' from file '../libfaiss.a(IndexShards.cpp.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
ld: warning: direct access in function '_wrap_downcast_index(_object*, _object*)' from file 'CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx.o' to global weak symbol 'typeinfo for faiss::IndexIDMap2Template<faiss::Index>' from file '../libfaiss.a(MetaIndexes.cpp.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
ld: warning: direct access in function '_wrap_downcast_index(_object*, _object*)' from file 'CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx.o' to global weak symbol 'typeinfo for faiss::IndexIDMap2Template<faiss::Index>' from file '../libfaiss.a(MetaIndexes.cpp.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
ld: warning: direct access in function '_wrap_downcast_index(_object*, _object*)' from file 'CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx.o' to global weak symbol 'typeinfo for faiss::IndexIDMapTemplate<faiss::Index>' from file '../libfaiss.a(MetaIndexes.cpp.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.
ld: warning: direct access in function '_wrap_downcast_index(_object*, _object*)' from file 'CMakeFiles/swigfaiss.dir/CMakeFiles/swigfaiss.dir/swigfaissPYTHON_wrap.cxx.o' to global weak symbol 'typeinfo for faiss::IndexIDMapTemplate<faiss::Index>' from file '../libfaiss.a(MetaIndexes.cpp.o)' means the weak symbol cannot be overridden at runtime. This was likely caused by different translation units being compiled with different visibility settings.

[xinslu]

On Putting Multiple Print Statements I was able to single out the error:

I ran out this code:

import faiss
import numpy as np
d = 64                           # dimension
nb = 100000                      # database size
nq = 10000                       # nb of queries
np.random.seed(1234)             # make reproducible
xb = np.random.random((nb, d)).astype('float32')
xb[:, 0] += np.arange(nb) / 1000.
xq = np.random.random((nq, d)).astype('float32')
xq[:, 0] += np.arange(nq) / 1000.
index = faiss.IndexFlatL2(d)   # build the index
print(index.is_trained)
index.add(xb)                  # add vectors to the index
print(index.ntotal)
print("Done")
k = 4                          # we want to see 4 nearest neighbors
D, I = index.search(xb[:5], k) # sanity check
print("Done???")
print(I)
print(D)
D, I = index.search(xq, k)     # actual search
print(I[:5])                   # neighbors of the 5 first queries
print(I[-5:])                  # neighbors of the 5 last queries

The Second Done Statement doesn't get printed and there is a Segmentation Fault Error so it fails on searching.

[yanhuqing666]

any update?