[Bug] (macOS) Linking problem? symbol not found in flat namespace '__ZN2at3mps9MPSStream6commitEb'

[debugmenot]

Prerequisite

• [X] I have searched Issues and Discussions but cannot get the expected help.
• [X] The bug has not been fixed in the latest version(https://github.com/open-mmlab/mmcv).

Environment

MacOS 13.6 (Same issue on 14.0 Sonoma) Apple Silicon M1 Ultra / 64G PyTorch 2.1 (release)

Impossible to get info from mmcv because of symbol not found error.

Reproduces the problem - code sample

Installed by official instructions.

If building MMCV from source with PyTorch 2.1 problem still exists...

There are no such issues with PyTorch 2.0

Reproduces the problem - command or script

~/o/mmdetection (main|✔) $ mim download mmdet --config rtmdet_tiny_8xb32-300e_coco --dest . (openmmlab) processing rtmdet_tiny_8xb32-300e_coco... downloading ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.9/54.9 MiB 851.5 kB/s eta 0:00:00 Successfully downloaded rtmdet_tiny_8xb32-300e_coco_20220902_112414-78e30dcc.pth to /Users/madcode/openmmlab/mmdetection Successfully dumped rtmdet_tiny_8xb32-300e_coco.py to /Users/madcode/openmmlab/mmdetection ~/o/mmdetection (main|✔) $ python demo/image_demo.py demo/demo.jpg rtmdet_tiny_8xb32-300e_coco.py --weights rtmdet_tiny_8xb32-300e_coco_20220902_112414-78e30dcc.pth --device cpu

Reproduces the problem - error message

~/o/mmdetection (main|✔) $ mim download mmdet --config rtmdet_tiny_8xb32-300e_coco --dest . (openmmlab) processing rtmdet_tiny_8xb32-300e_coco... downloading ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 54.9/54.9 MiB 851.5 kB/s eta 0:00:00 Successfully downloaded rtmdet_tiny_8xb32-300e_coco_20220902_112414-78e30dcc.pth to /Users/madcode/openmmlab/mmdetection Successfully dumped rtmdet_tiny_8xb32-300e_coco.py to /Users/madcode/openmmlab/mmdetection ~/o/mmdetection (main|✔) $ python demo/image_demo.py demo/demo.jpg rtmdet_tiny_8xb32-300e_coco.py --weights rtmdet_tiny_8xb32-300e_coco_20220902_112414-78e30dcc.pth --device cpu Traceback (most recent call last): File "demo/image_demo.py", line 43, in from mmdet.apis import DetInferencer File "/Users/madcode/openmmlab/mmdetection/mmdet/apis/init.py", line 2, in from .det_inferencer import DetInferencer File "/Users/madcode/openmmlab/mmdetection/mmdet/apis/det_inferencer.py", line 21, in from mmdet.evaluation import INSTANCE_OFFSET File "/Users/madcode/openmmlab/mmdetection/mmdet/evaluation/init.py", line 3, in from .metrics import * # noqa: F401,F403 File "/Users/madcode/openmmlab/mmdetection/mmdet/evaluation/metrics/init.py", line 5, in from .coco_metric import CocoMetric File "/Users/madcode/openmmlab/mmdetection/mmdet/evaluation/metrics/coco_metric.py", line 16, in from mmdet.datasets.api_wrappers import COCO, COCOeval File "/Users/madcode/openmmlab/mmdetection/mmdet/datasets/init.py", line 26, in from .utils import get_loading_pipeline File "/Users/madcode/openmmlab/mmdetection/mmdet/datasets/utils.py", line 5, in from mmdet.datasets.transforms import LoadAnnotations, LoadPanopticAnnotations File "/Users/madcode/openmmlab/mmdetection/mmdet/datasets/transforms/init.py", line 6, in from .formatting import (ImageToTensor, PackDetInputs, PackReIDInputs, File "/Users/madcode/openmmlab/mmdetection/mmdet/datasets/transforms/formatting.py", line 11, in from mmdet.structures.bbox import BaseBoxes File "/Users/madcode/openmmlab/mmdetection/mmdet/structures/bbox/init.py", line 2, in from .base_boxes import BaseBoxes File "/Users/madcode/openmmlab/mmdetection/mmdet/structures/bbox/base_boxes.py", line 9, in from mmdet.structures.mask.structures import BitmapMasks, PolygonMasks File "/Users/madcode/openmmlab/mmdetection/mmdet/structures/mask/init.py", line 3, in from .structures import (BaseInstanceMasks, BitmapMasks, PolygonMasks, File "/Users/madcode/openmmlab/mmdetection/mmdet/structures/mask/structures.py", line 12, in from mmcv.ops.roi_align import roi_align File "/opt/homebrew/Caskroom/miniconda/base/envs/openmmlab/lib/python3.8/site-packages/mmcv/ops/init.py", line 2, in from .active_rotated_filter import active_rotated_filter File "/opt/homebrew/Caskroom/miniconda/base/envs/openmmlab/lib/python3.8/site-packages/mmcv/ops/active_rotated_filter.py", line 10, in ext_module = ext_loader.load_ext( File "/opt/homebrew/Caskroom/miniconda/base/envs/openmmlab/lib/python3.8/site-packages/mmcv/utils/ext_loader.py", line 13, in load_ext ext = importlib.import_module('mmcv.' + name) File "/opt/homebrew/Caskroom/miniconda/base/envs/openmmlab/lib/python3.8/importlib/init.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) ImportError: dlopen(/opt/homebrew/Caskroom/miniconda/base/envs/openmmlab/lib/python3.8/site-packages/mmcv/_ext.cpython-38-darwin.so, 0x0002): symbol not found in flat namespace '__ZN2at3mps9MPSStream6commitEb'

Additional information

No response

[debugmenot]

Building from source:

~/openmmlab $ conda create --name openmmlab_2 python=3.8 -y
Collecting package metadata (current_repodata.json): done Solving environment: done

(...)

ca-certificates pkgs/main/osx-arm64::ca-certificates-2023.08.22-hca03da5_0 libcxx pkgs/main/osx-arm64::libcxx-14.0.6-h848a8c0_0 libffi pkgs/main/osx-arm64::libffi-3.4.4-hca03da5_0 ncurses pkgs/main/osx-arm64::ncurses-6.4-h313beb8_0 openssl pkgs/main/osx-arm64::openssl-3.0.11-h1a28f6b_2 pip pkgs/main/osx-arm64::pip-23.2.1-py38hca03da5_0 python pkgs/main/osx-arm64::python-3.8.18-hb885b13_0 readline pkgs/main/osx-arm64::readline-8.2-h1a28f6b_0 setuptools pkgs/main/osx-arm64::setuptools-68.0.0-py38hca03da5_0 sqlite pkgs/main/osx-arm64::sqlite-3.41.2-h80987f9_0 tk pkgs/main/osx-arm64::tk-8.6.12-hb8d0fd4_0 wheel pkgs/main/osx-arm64::wheel-0.41.2-py38hca03da5_0 xz pkgs/main/osx-arm64::xz-5.4.2-h80987f9_0 zlib pkgs/main/osx-arm64::zlib-1.2.13-h5a0b063_0

(...)

~/openmmlab $ conda activate openmmlab_2 ~/openmmlab $ conda install pytorch torchvision cpuonly -c pytorch (openmmlab_2) Collecting package metadata (current_repodata.json): done Solving environment: done

(...)

The following packages will be downloaded:

package                    |            build
---------------------------|-----------------
cpuonly-2.0                |                0           2 KB  pytorch
pytorch-mutex-1.0          |              cpu           3 KB  pytorch
------------------------------------------------------------
                                       Total:           5 KB

The following NEW packages will be INSTALLED:

blas pkgs/main/osx-arm64::blas-1.0-openblas brotlipy pkgs/main/osx-arm64::brotlipy-0.7.0-py38h1a28f6b_1002 bzip2 pkgs/main/osx-arm64::bzip2-1.0.8-h620ffc9_4 certifi pkgs/main/osx-arm64::certifi-2023.7.22-py38hca03da5_0 cffi pkgs/main/osx-arm64::cffi-1.15.1-py38h80987f9_3 charset-normalizer pkgs/main/noarch::charset-normalizer-2.0.4-pyhd3eb1b0_0 cpuonly pytorch/noarch::cpuonly-2.0-0 cryptography pkgs/main/osx-arm64::cryptography-41.0.3-py38hd4332d6_0 ffmpeg pkgs/main/osx-arm64::ffmpeg-4.2.2-h04105a8_0 filelock pkgs/main/osx-arm64::filelock-3.9.0-py38hca03da5_0 freetype pkgs/main/osx-arm64::freetype-2.12.1-h1192e45_0 gettext pkgs/main/osx-arm64::gettext-0.21.0-h13f89a0_1 giflib pkgs/main/osx-arm64::giflib-5.2.1-h80987f9_3 gmp pkgs/main/osx-arm64::gmp-6.2.1-hc377ac9_3 gmpy2 pkgs/main/osx-arm64::gmpy2-2.1.2-py38h8c48613_0 gnutls pkgs/main/osx-arm64::gnutls-3.6.15-h887c41c_0 icu pkgs/main/osx-arm64::icu-73.1-h313beb8_0 idna pkgs/main/osx-arm64::idna-3.4-py38hca03da5_0 jinja2 pkgs/main/osx-arm64::jinja2-3.1.2-py38hca03da5_0 jpeg pkgs/main/osx-arm64::jpeg-9e-h80987f9_1 lame pkgs/main/osx-arm64::lame-3.100-h1a28f6b_0 lcms2 pkgs/main/osx-arm64::lcms2-2.12-hba8e193_0 lerc pkgs/main/osx-arm64::lerc-3.0-hc377ac9_0 libdeflate pkgs/main/osx-arm64::libdeflate-1.17-h80987f9_1 libgfortran pkgs/main/osx-arm64::libgfortran-5.0.0-11_3_0_hca03da5_28 libgfortran5 pkgs/main/osx-arm64::libgfortran5-11.3.0-h009349e_28 libiconv pkgs/main/osx-arm64::libiconv-1.16-h1a28f6b_2 libidn2 pkgs/main/osx-arm64::libidn2-2.3.4-h80987f9_0 libjpeg-turbo pkgs/main/osx-arm64::libjpeg-turbo-2.0.0-h1a28f6b_0 libopenblas pkgs/main/osx-arm64::libopenblas-0.3.21-h269037a_0 libopus pkgs/main/osx-arm64::libopus-1.3-h1a28f6b_1 libpng pkgs/main/osx-arm64::libpng-1.6.39-h80987f9_0 libtasn1 pkgs/main/osx-arm64::libtasn1-4.19.0-h80987f9_0 libtiff pkgs/main/osx-arm64::libtiff-4.5.1-h313beb8_0 libunistring pkgs/main/osx-arm64::libunistring-0.9.10-h1a28f6b_0 libvpx pkgs/main/osx-arm64::libvpx-1.10.0-hc377ac9_0 libwebp pkgs/main/osx-arm64::libwebp-1.3.2-ha3663a8_0 libwebp-base pkgs/main/osx-arm64::libwebp-base-1.3.2-h80987f9_0 libxml2 pkgs/main/osx-arm64::libxml2-2.10.4-h0dcf63f_1 llvm-openmp pkgs/main/osx-arm64::llvm-openmp-14.0.6-hc6e5704_0 lz4-c pkgs/main/osx-arm64::lz4-c-1.9.4-h313beb8_0 markupsafe pkgs/main/osx-arm64::markupsafe-2.1.1-py38h1a28f6b_0 mpc pkgs/main/osx-arm64::mpc-1.1.0-h8c48613_1 mpfr pkgs/main/osx-arm64::mpfr-4.0.2-h695f6f0_1 mpmath pkgs/main/osx-arm64::mpmath-1.3.0-py38hca03da5_0 nettle pkgs/main/osx-arm64::nettle-3.7.3-h84b5d62_1 networkx pkgs/main/osx-arm64::networkx-3.1-py38hca03da5_0 numpy pkgs/main/osx-arm64::numpy-1.24.3-py38h1398885_0 numpy-base pkgs/main/osx-arm64::numpy-base-1.24.3-py38h90707a3_0 openh264 pkgs/main/osx-arm64::openh264-1.8.0-h98b2900_0 pillow pkgs/main/osx-arm64::pillow-9.4.0-py38h313beb8_1 pycparser pkgs/main/noarch::pycparser-2.21-pyhd3eb1b0_0 pyopenssl pkgs/main/osx-arm64::pyopenssl-23.2.0-py38hca03da5_0 pysocks pkgs/main/osx-arm64::pysocks-1.7.1-py38hca03da5_0 pytorch pytorch/osx-arm64::pytorch-2.1.0-py3.8_0 pytorch-mutex pytorch/noarch::pytorch-mutex-1.0-cpu pyyaml pkgs/main/osx-arm64::pyyaml-6.0-py38h80987f9_1 requests pkgs/main/osx-arm64::requests-2.31.0-py38hca03da5_0 sympy pkgs/main/osx-arm64::sympy-1.11.1-py38hca03da5_0 torchvision pytorch/osx-arm64::torchvision-0.16.0-py38_cpu typing_extensions pkgs/main/osx-arm64::typing_extensions-4.7.1-py38hca03da5_0 urllib3 pkgs/main/osx-arm64::urllib3-1.26.16-py38hca03da5_0 x264 pkgs/main/osx-arm64::x264-1!152.20180806-h1a28f6b_0 yaml pkgs/main/osx-arm64::yaml-0.2.5-h1a28f6b_0 zstd pkgs/main/osx-arm64::zstd-1.5.5-hd90d995_0

(...)

~/openmmlab $ python -c 'import torch;print(torch.version)' (openmmlab_2) 2.1.0 ~/openmmlab $ git clone https://github.com/open-mmlab/mmcv.git ~/openmmlab $ cd mmcv ~/o/mmcv (main|✔) $ pip install -r requirements/optional.txt (openmmlab_2) Collecting ninja (from -r requirements/optional.txt (line 1)) Downloading ninja-1.11.1-py2.py3-none-macosx_10_9_universal2.macosx_10_9_x86_64.macosx_11_0_arm64.macosx_11_0_universal2.whl (270 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 270.7/270.7 kB 1.2 MB/s eta 0:00:00 Collecting psutil (from -r requirements/optional.txt (line 2)) Downloading psutil-5.9.5-cp38-abi3-macosx_11_0_arm64.whl (246 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 246.1/246.1 kB 7.0 MB/s eta 0:00:00 Installing collected packages: ninja, psutil Successfully installed ninja-1.11.1 psutil-5.9.5 ~/o/mmcv (main|✔) $ gcc --version (openmmlab_2) Apple clang version 15.0.0 (clang-1500.0.40.1) Target: arm64-apple-darwin22.6.0 Thread model: posix InstalledDir: /Library/Developer/CommandLineTools/usr/bin ~/o/mmcv (main|✔) $ pip install -e . -v

(…)

building output omitted. Only warnings are like:

ld: warning: object file (/Users/madcode/openmmlab/mmcv/build/temp.macosx-11.1-arm64-cpython-38/mmcv/ops/csrc/pytorch/voxelization.o) was built for newer 'macOS' version (13.0) than being linked (11.1) and so one...

Installed /Users/madcode/openmmlab/mmcv

Successfully installed addict-2.4.0 contourpy-1.1.1 cycler-0.12.0 fonttools-4.43.0 importlib-metadata-6.8.0 importlib-resources-6.1.0 kiwisolver-1.4.5 markdown-it-py-3.0.0 matplotlib-3.7.3 mdurl-0.1.2 mmcv-2.0.1 mmengine-0.8.5 opencv-python-4.8.1.78 packaging-23.2 platformdirs-3.11.0 pygments-2.16.1 pyparsing-3.1.1 python-dateutil-2.8.2 rich-13.6.0 six-1.16.0 termcolor-2.3.0 tomli-2.0.1 yapf-0.40.2 zipp-3.17.0

~/o/mmcv (main|✔) $ python .dev_scripts/check_installation.py (openmmlab_2) Traceback (most recent call last): File ".dev_scripts/check_installation.py", line 4, in from mmcv.ops import box_iou_rotated File "/Users/madcode/openmmlab/mmcv/mmcv/ops/init.py", line 3, in from .active_rotated_filter import active_rotated_filter File "/Users/madcode/openmmlab/mmcv/mmcv/ops/active_rotated_filter.py", line 10, in ext_module = ext_loader.load_ext( File "/Users/madcode/openmmlab/mmcv/mmcv/utils/ext_loader.py", line 13, in load_ext ext = importlib.import_module('mmcv.' + name) File "/opt/homebrew/Caskroom/miniconda/base/envs/openmmlab_2/lib/python3.8/importlib/init.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) ImportError: dlopen(/Users/madcode/openmmlab/mmcv/mmcv/_ext.cpython-38-darwin.so, 0x0002): symbol not found in flat namespace '__ZN2at3mps9MPSStream6commitEb'

[fazhang-master]

[zhouzaida]

Hi @debugmenot , as a workaround, you can pull the main branch to fix this error (#2958) or downgrade your torch to 2.0.0 and we will fix this error for 2.1.0 in the future.

[debugmenot]

Hi! Any schedule to return back MPS support? 2.1.0 is already released but MPS is disabled :( There are lots of new MPS functions supported in Torch 2.1...

[zhouzaida]

Hi @debugmenot, the support for MPS backend is in our schedule but with a not high priority. By the way, you can still use the op supported in torch.

[kojipy]

@debugmenot I encountered sama error when I use python 3.10.1. but swithing to python3.9.6 resolved the issue. I'm not sure but using python3.9.x might potentialy fix this issue.

[LeiChen9]

2.0.0, still don't work

[zhiwuz]

2.0.0, still don't work

+1

[RoderickQiu]

When I use mmcv 2.0.x it failed. But after changing to both using mmcv 2.1.0 and torch 2.0.x it get fixed.