Installation errors apple m1

[khomchyk]

Hello!

I tried to install via pip install nmslib There are following logs and errors:

Collecting nmslib

Using cached nmslib-2.1.1.tar.gz (188 kB)

Requirement already satisfied: pybind11<2.6.2 in /opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages (from nmslib) (2.6.1)

Requirement already satisfied: psutil in /opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages (from nmslib) (5.8.0)

Requirement already satisfied: numpy>=1.10.0 in /opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages (from nmslib) (1.19.5)

Building wheels for collected packages: nmslib

Building wheel for nmslib (setup.py) ... error

ERROR: Command errored out with exit status 1:

command: /opt/homebrew/Caskroom/miniforge/base/envs/cndv8/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/pip-install-xph8mukd/nmslib_455b1b59d1ad43a58c444254f181ce3b/setup.py'"'"'; __file__='"'"'/private/var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/pip-install-xph8mukd/nmslib_455b1b59d1ad43a58c444254f181ce3b/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/pip-wheel-imwlifeo

cwd: /private/var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/pip-install-xph8mukd/nmslib_455b1b59d1ad43a58c444254f181ce3b/

Complete output (23 lines):

Dependence list: ['pybind11<2.6.2', 'psutil', "numpy>=1.10.0,<1.17 ; python_version=='2.7'", "numpy>=1.10.0 ; python_version>='3.5'"]

running bdist_wheel

running build

running build_ext

creating var

creating var/folders

creating var/folders/lk

creating var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn

creating var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T

gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include/python3.8 -c /var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/tmpqhrb07ue.cpp -o var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/tmpqhrb07ue.o -std=c++14

gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include/python3.8 -c /var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/tmp8tdl7mo4.cpp -o var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/tmp8tdl7mo4.o -fvisibility=hidden

Extra compilation arguments: ['-O3', '-march=native', '-stdlib=libc++', '-mmacosx-version-min=10.7', '-DVERSION_INFO="2.1.1"', '-std=c++14', '-fvisibility=hidden']

building 'nmslib' extension

creating build

creating build/temp.macosx-11.0-arm64-3.8

creating build/temp.macosx-11.0-arm64-3.8/tensorflow

creating build/temp.macosx-11.0-arm64-3.8/similarity_search

creating build/temp.macosx-11.0-arm64-3.8/similarity_search/src

creating build/temp.macosx-11.0-arm64-3.8/similarity_search/src/space

creating build/temp.macosx-11.0-arm64-3.8/similarity_search/src/method

gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I./similarity_search/include -Itensorflow -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages/pybind11/include -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages/pybind11/include -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages/pybind11/include -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages/pybind11/include -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages/numpy/core/include -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include/python3.8 -c nmslib.cc -o build/temp.macosx-11.0-arm64-3.8/nmslib.o -O3 -march=native -stdlib=libc++ -mmacosx-version-min=10.7 -DVERSION_INFO="2.1.1" -std=c++14 -fvisibility=hidden

clang: error: the clang compiler does not support '-march=native'

error: command 'gcc' failed with exit status 1

----------------------------------------

ERROR: Failed building wheel for nmslib

Running setup.py clean for nmslib

Failed to build nmslib

Installing collected packages: nmslib

Running setup.py install for nmslib ... error

ERROR: Command errored out with exit status 1:

command: /opt/homebrew/Caskroom/miniforge/base/envs/cndv8/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/pip-install-xph8mukd/nmslib_455b1b59d1ad43a58c444254f181ce3b/setup.py'"'"'; __file__='"'"'/private/var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/pip-install-xph8mukd/nmslib_455b1b59d1ad43a58c444254f181ce3b/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/pip-record-ccpn80un/install-record.txt --single-version-externally-managed --compile --install-headers /opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include/python3.8/nmslib

cwd: /private/var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/pip-install-xph8mukd/nmslib_455b1b59d1ad43a58c444254f181ce3b/

Complete output (18 lines):

Dependence list: ['pybind11<2.6.2', 'psutil', "numpy>=1.10.0,<1.17 ; python_version=='2.7'", "numpy>=1.10.0 ; python_version>='3.5'"]

running install

running build

running build_ext

gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include/python3.8 -c /var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/tmpci3coski.cpp -o var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/tmpci3coski.o -std=c++14

gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include/python3.8 -c /var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/tmpfun71y94.cpp -o var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/tmpfun71y94.o -fvisibility=hidden

Extra compilation arguments: ['-O3', '-march=native', '-stdlib=libc++', '-mmacosx-version-min=10.7', '-DVERSION_INFO="2.1.1"', '-std=c++14', '-fvisibility=hidden']

building 'nmslib' extension

creating build

creating build/temp.macosx-11.0-arm64-3.8

creating build/temp.macosx-11.0-arm64-3.8/tensorflow

creating build/temp.macosx-11.0-arm64-3.8/similarity_search

creating build/temp.macosx-11.0-arm64-3.8/similarity_search/src

creating build/temp.macosx-11.0-arm64-3.8/similarity_search/src/space

creating build/temp.macosx-11.0-arm64-3.8/similarity_search/src/method

gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include -arch arm64 -I./similarity_search/include -Itensorflow -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages/pybind11/include -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages/pybind11/include -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages/pybind11/include -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages/pybind11/include -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/lib/python3.8/site-packages/numpy/core/include -I/opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include/python3.8 -c nmslib.cc -o build/temp.macosx-11.0-arm64-3.8/nmslib.o -O3 -march=native -stdlib=libc++ -mmacosx-version-min=10.7 -DVERSION_INFO="2.1.1" -std=c++14 -fvisibility=hidden

clang: error: the clang compiler does not support '-march=native'

error: command 'gcc' failed with exit status 1

----------------------------------------

ERROR: Command errored out with exit status 1: /opt/homebrew/Caskroom/miniforge/base/envs/cndv8/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/pip-install-xph8mukd/nmslib_455b1b59d1ad43a58c444254f181ce3b/setup.py'"'"'; __file__='"'"'/private/var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/pip-install-xph8mukd/nmslib_455b1b59d1ad43a58c444254f181ce3b/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/lk/b1b1kdvn2437gjdz_cslsj_40000gn/T/pip-record-ccpn80un/install-record.txt --single-version-externally-managed --compile --install-headers /opt/homebrew/Caskroom/miniforge/base/envs/cndv8/include/python3.8/nmslib Check the logs for full command output.

Also I tried to install via pip install --no-binary :all: nmslib but the error still the same.

Versions: Python 3.8.8 pip 21.0.1

Could you please help me to figure out what are the ways to install nmslib on Apple Silicon M1? Or maybe I'm missing something.

[searchivarius]

Hi @khomchyk this is apparently b/c clang on M1 doesn't support march=native yet: https://stackoverflow.com/a/66333485/2120401

It's not clear what I should use instead, but I probably have to disable it for the Apple M1 platform, could you run the following command:

python -c 'import sys ; print(sys.platform)'

and post the results here?

Thank you!

[khomchyk]

Hi @searchivarius

thanks for an answer.

Yes, sure! The result of the python -c 'import sys ; print(sys.platform)' command: <<<darwin

[searchivarius]

mehh, it's the same as for the regular mac book and what does the following line print:

python -c 'import platform ; print(platform.processor())'

[khomchyk]

mehh, it's the same as for the regular mac book and what does the following line print:

python -c 'import platform ; print(platform.processor())'

Yeah, looks the same.

The output for print(platform.processor()) differs: <<< arm for m1. <<< i386 for a regular intel one.

[azhx]

Also ran into this issue on the M1 chip. Installing on a conda environment with pip install --no-binary :all: nmslib resolved it

[searchivarius]

@azhx strage it helped, maybe the modify the setup. The problem is that one C flag doesn't work on the new apple platform. I really need to remove it but only for Mac OS and arm CPUs.

[azhx]

actually, initially for me normal anaconda install of the precompiled binary worked without errors, but actually importing inside the python shell gave me a illegal hardware instruction which probably was caused by the C flag. Not sure either what installing from source did differently. Oh also, im not familiar with how Rosetta works specifically, but I was running terminal inside rosetta2

[searchivarius]

@azhx I don't maintain a conda installation, but apparently it doesn't distinguish between Intel and m1 MAC. So, then it simply crashes. I have heard M1 can run Intel code, but, apparently, this does not work ideally. They may also remove that flag from their installation code. That is one reason why you can install from sources. When you install from sources, clearly, there is no issue with running Intel binary on M1, b/c you compile into native N1 code.

[drkeoni]

I got here after trying to install nmslib on a MacBookPro (Big Sur, M1 chip).

What finally worked for me was a variation on a few of these recommendations:

• I'm using conda (miniforge) to manage my environment
• Neither pip install nmslib nor pip install nmslib --no-binary :all: worked
• Installing from source did work but
  • I needed to override the clang setting from -march=native to -mcpu=apple-a14. I did this by adding set(SIMD_FLAGS " -mcpu=apple-a14") into CMakeLists.txt but I'm sure there's an environment variable way to do this...
  • clang was still complaining about a nested macro in a #pragma message in similarity_search/src/distcomp_scalar.cc. Commenting out those two warnings (innocent) permitted everything to build (cmake .; make)
  • Verified everything by running bunit (n.b. bunit looks for data in ../sample_data by default but that's not the correct relative path from its default build location ... by soft-linking it into ../ and then running there it worked)
• After the library build was successful I built the python bindings and package
  • In python_bindings/setup.py change c_opts['unix'].append('-march=native') to c_opts['unix'].append('-mcpu=apple-a14')
  • At this point python setup.py install worked perfectly fine and installed nmslib==2.1.1 into my conda environment. Verified everything by running the tutorial example in the docs.

[searchivarius]

@searchivarius sorry, this clearly requires a setup.py fix as new Mac compilers don't support some of the flags. There's even a pull request for this. I will get back to this.

[8W9aG]

I can get this to install using the following command: CFLAGS="-mavx -DWARN(a)=(a)" pip install nmslib Note that this seems to disable a lot of acceleration, so I wouldn't recommend it for production.

[cfloressuazo]

The only solution that worked for me was the one pointed by @8W9aG . If you are installing with Poetry also, simply write CFLAGS="-mavx -DWARN(a)=(a)" poetry add .....

[lifepillar]

I could install it following @drkeoni's instructions. I hope they will be taken into account for a future release out-of-the-box compatible with M1.

[Berlyli866]

did you solve this issue finally?

[searchivarius]

there's a PR, I will merge it soon.

[cwelton]

Hit the same issue. Tried the various fixes in the open PRs, which helped, but I'm still having additional issues beyond that.

E.g.

./similarity_search/src/distcomp_lp.cc:59:15: error: no matching function for call to 'max'
        res = max(res, fabs(*pVect1++ - *pVect2++));

Which seems to be getting confused and getting a float for res but a double for the fabs result.

Got past that by adding an explicit:

        res = max(res, (T) fabs(*pVect1++ - *pVect2++));

(though unclear why that was needed)

Still hit other issues:

/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/v1/cmath:642:26: error: no template named 'numeric_limits'
    bool _FloatBigger = (numeric_limits<_FloatT>::digits > numeric_limits<_IntT>::digits),

[searchivarius]

@cwelton that's for the new Mac and using the master branch? Which compiler do you use?

[cwelton]

@searchivarius Apple M1 max, OSX 12.5

$ uname -a

Darwin Calebs-MBP.attlocal.net 21.6.0 Darwin Kernel Version 21.6.0: Sat Jun 18 17:07:22 PDT 2022; root:xnu-8020.140.41~1/RELEASE_ARM64_T6000 arm64.

$ clang --version

Apple clang version 13.1.6 (clang-1316.0.21.2.5)
Target: arm64-apple-darwin21.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

I tried building directly on the fix-build-on-macos-arm64 PR branch, and on a v2.1.1 branch with the changes from the PR branch pulled in.

[searchivarius]

@cwelton do you get same errors with GCC?

[cwelton]

Let me give it a try...

[cwelton]

Yes, same errors with gcc:

Example:

/usr/bin/gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/include -I./similarity_search/include -Itensorflow -I/Users/cwelton/.pyenv/versions/3.10.2/lib/python3.10/site-packages/pybind11/include -I/Users/cwelton/.pyenv/versions/3.10.2/lib/python3.10/site-packages/pybind11/include -I/Users/cwelton/.pyenv/versions/3.10.2/lib/python3.10/site-packages/pybind11/include -I/Users/cwelton/.pyenv/versions/3.10.2/lib/python3.10/site-packages/pybind11/include -I/Users/cwelton/.pyenv/versions/3.10.2/lib/python3.10/site-packages/numpy/core/include -I/Users/cwelton/.pyenv/versions/3.10.2/include/python3.10 -c ./similarity_search/src/distcomp_lp.cc -o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/distcomp_lp.o -O3 -mcpu=apple-a14 -stdlib=libc++ -mmacosx-version-min=10.7 -DVERSION_INFO="2.1.1" -std=c++14 -fvisibility=hidden
./similarity_search/src/distcomp_lp.cc:40:13: error: no matching function for call to 'max'
      res = max(res, fabs(p1[i]-p2[i]));
            ^~~

[cwelton]

Oh wait...

gcc --version
Apple clang version 13.1.6 (clang-1316.0.21.2.5)
Target: arm64-apple-darwin21.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

That's still clang.

[searchivarius]

@cwelton thank you, it looks like more fixes are needed. Apparently, there are some minor differences with respect to default floating point types. Apparently, all the previous platforms were different from M1 in this respect.

[searchivarius]

@cwelton didn't notice your last message. Could you try actual gcc?

it should work with clang as well, but ...

thank you!

[cwelton]

Using: g++-12 (Homebrew GCC 12.1.0) 12.1.0

With the outstanding PRs to support osx I had pulled in the following to setup.py

    if sys.platform == 'darwin':
        if platform.processor() in ('arm', 'arm64'):
            c_opts['unix'].remove('-march=native')
            c_opts['unix'].append('-mcpu=apple-a14')
        c_opts['unix'] += ['-stdlib=libc++', '-mmacosx-version-min=10.7']
        link_opts['unix'] += ['-stdlib=libc++', '-mmacosx-version-min=10.7']

Using gcc instead of clang makes -stdlib=libc++ and -mcpu options not applicable, so some extra changes are needed there. Removing the offending options and compiling... get's further than before, but ultimately errors with:

/opt/homebrew/bin/g++-12 -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/include -I./similarity_search/include -Itensorflow -I/Users/cwelton/.pyenv/versions/3.10.2/lib/python3.10/site-packages/pybind11/include -I/Users/cwelton/.pyenv/versions/3.10.2/lib/python3.10/site-packages/pybind11/include -I/Users/cwelton/.pyenv/versions/3.10.2/lib/python3.10/site-packages/pybind11/include -I/Users/cwelton/.pyenv/versions/3.10.2/lib/python3.10/site-packages/pybind11/include -I/Users/cwelton/.pyenv/versions/3.10.2/lib/python3.10/site-packages/numpy/core/include -I/Users/cwelton/.pyenv/versions/3.10.2/include/python3.10 -c ./similarity_search/src/memory.cc -o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/memory.o -O3 -DVERSION_INFO="2.1.1" -std=c++14 -fvisibility=hidden
./similarity_search/src/memory.cc:37:1: error: 'MemUsage' does not name a type
   37 | MemUsage::MemUsage() {
      | ^~~~~~~~

[cwelton]

Wrapping that in a class statement

class MemUsage {

MemUsage() {
...

completes the compile and gets to the link, which fails.

creating build/lib.macosx-12.1-arm64-3.10
/opt/homebrew/bin/g++-12 -bundle -undefined dynamic_lookup -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/opt/readline/lib -L/Users/cwelton/.pyenv/versions/3.10.2/lib -L/opt/homebrew/lib -L/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/lib -L/opt/homebrew/opt/readline/lib -L/opt/homebrew/opt/readline/lib -L/Users/cwelton/.pyenv/versions/3.10.2/lib -L/opt/homebrew/lib -L/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/lib build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/distcomp_bregman.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/distcomp_diverg.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/distcomp_edist.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/distcomp_js.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/distcomp_l2sqr_sift.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/distcomp_lp.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/distcomp_overlap.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/distcomp_rankcorr.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/distcomp_scalar.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/distcomp_sparse_scalar_fast.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/experimentconf.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/global.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/init.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/knnquery.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/logging.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/memory.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/method/dummy.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/method/hnsw.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/method/hnsw_distfunc_opt.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/method/pivot_neighb_invindx.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/method/seqsearch.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/method/simple_inverted_index.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/method/small_world_rand.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/method/vptree.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/params.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/params_cmdline.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/query.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/rangequery.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/searchoracle.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/space.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/space/space_ab_diverg.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/space/space_bregman.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/space/space_dummy.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/space/space_js.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/space/space_l2sqr_sift.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/space/space_lp.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/space/space_renyi_diverg.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/space/space_scalar.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/space/space_sparse_dense_fusion.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/space/space_sparse_jaccard.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/space/space_sparse_lp.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/space/space_sparse_scalar_bin_fast.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/space/space_sparse_scalar_fast.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/space/space_sparse_vector.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/space/space_sparse_vector_inter.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/space/space_string.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/space/space_vector.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/space/space_word_embed.o build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/utils.o build/temp.macosx-12.1-arm64-3.10/nmslib.o build/temp.macosx-12.1-arm64-3.10/tensorflow/cpu_feature_guard.o build/temp.macosx-12.1-arm64-3.10/tensorflow/cpu_info.o -o build/lib.macosx-12.1-arm64-3.10/nmslib.cpython-310-darwin.so
duplicate symbol '__ZTSN10similarity4HnswIiEE' in:
    build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/method/hnsw.o
    build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/method/hnsw_distfunc_opt.o
duplicate symbol '__ZTIN10similarity4HnswIiEE' in:
    build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/method/hnsw.o
    build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/method/hnsw_distfunc_opt.o
duplicate symbol '__ZTSN10similarity4HnswIfEE' in:
    build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/method/hnsw.o
    build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/method/hnsw_distfunc_opt.o
duplicate symbol '__ZTIN10similarity4HnswIfEE' in:
    build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/method/hnsw.o
    build/temp.macosx-12.1-arm64-3.10/./similarity_search/src/method/hnsw_distfunc_opt.o
ld: 4 duplicate symbols for architecture arm64
collect2: error: ld returned 1 exit status
error: command '/opt/homebrew/bin/g++-12' failed with exit code 1

[cwelton]

One last update:

=> I got it to build!

1. Switched back to clang, and worked through debugging the build errors. a) Main issue was include paths were a bit messed up and it was using a weird mix of header files across a couple different clang install directories. b) Root cause of that seems to be that the default clang version I was using was different from the clang version I had at the time my python environment was installed. E.g. -I/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/usr/include [what was on the command line] vs -I/Library/Developer/CommandLineTools/SDKs/MacOSX12.3.sdk/usr/include [default in my env]
2. Installed a new python environment which then setup the build extension paths based on my current env
3. Used the changes in https://github.com/nmslib/nmslib/pull/516

[searchivarius]

Thank you for all the investigation, I will need to work on this soon. So, as a summary, is it an issue with the code or with a setup? I a bit surprised it didn't compile, b/c previously we did not have such problems (and we compile on Windows, which uses a very different compiler).

[PiercarloSlavazza]

Tried to follow @drkeoni instructions: everything was ok, managed to install successfully the package, but, when I run the python program, I get this error:

 Traceback (most recent call last):
  File "<…>/main.py", line 13, in <module>
    import nmslib
ImportError: dlopen(/Users/<…>/lib/python3.8/site-packages/nmslib-2.1.2-py3.8-macosx-10.9-x86_64.egg/nmslib.cpython-38-darwin.so, 0x0002): symbol not found in flat namespace '__ZTI6Logger'

Does anyone have an idea how to solve?

Apple M1 pro, Apple clang version 14.0.0 (clang-1400.0.29.202), using master at commit ade4bcdc9dd3719990de2503871450b8a62df4a5.

[sanjosh]

option described here also works

CFLAGS="-mavx -DWARN(a)=(a)" pip install nmslib

https://dev.to/adityakanekar/installing-scispacy-on-apple-m1-laptops-3cn6