[Bug] Unable to run BEVFusion Demo

[Torreskai0722]

Prerequisite

• [X] I have searched Issues and Discussions but cannot get the expected help.
• [X] I have read the FAQ documentation but cannot get the expected help.
• [X] The bug has not been fixed in the latest version (dev-1.x) or latest version (dev-1.0).

Task

I'm using the official example scripts/configs for the officially supported tasks/models/datasets.

Branch

main branch https://github.com/open-mmlab/mmdetection3d

Environment

sys.platform: linux Python: 3.8.13 (default, Mar 28 2022, 11:38:47) [GCC 7.5.0] CUDA available: True numpy_random_seed: 2147483648 GPU 0: NVIDIA GeForce RTX 3080 CUDA_HOME: /usr/local/cuda-12 NVCC: Cuda compilation tools, release 12.1, V12.1.66 GCC: gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0 PyTorch: 2.1.0+cu121 PyTorch compiling details: PyTorch built with:

• GCC 9.3
• C++ Version: 201703
• Intel(R) oneAPI Math Kernel Library Version 2022.2-Product Build 20220804 for Intel(R) 64 architecture applications
• Intel(R) MKL-DNN v3.1.1 (Git Hash 64f6bcbcbab628e96f33a62c3e975f8535a7bde4)
• OpenMP 201511 (a.k.a. OpenMP 4.5)
• LAPACK is enabled (usually provided by MKL)
• NNPACK is enabled
• CPU capability usage: AVX2
• CUDA Runtime 12.1
• NVCC architecture flags: -gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90
• CuDNN 8.9.2
• Magma 2.6.1
• Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=8.9.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -fabi-version=11 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-invalid-partial-specialization -Wno-unused-private-field -Wno-aligned-allocation-unavailable -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=ON, TORCH_VERSION=2.1.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,

TorchVision: 0.16.0+cu121 OpenCV: 4.8.1 MMEngine: 0.9.1 MMDetection: 3.2.0 MMDetection3D: 1.2.0+12b595c spconv2.0: False

Reproduces the problem - code sample

(openmmlab) hydrapc@hydrapc:~/projects/DROI-BEV/mmdetection3d$ python projects/BEVFusion/demo/multi_modality_demo.py demo/data/nuscenes/n015-2018-07-24-11-22-45+0800__LIDAR_TOP__1532402927647951.pcd.bin demo/data/nuscenes/ demo/data/nuscenes/n015-2018-07-24-11-22-45+0800.pkl projects/BEVFusion/configs/bevfusion_lidar-cam_voxel0075_second_secfpn_8xb4-cyclic-20e_nus-3d.py bevfusion_converted.pth --cam-type all --score-thr 0.2 --show

Reproduces the problem - command or script

(openmmlab) hydrapc@hydrapc:~/projects/DROI-BEV/mmdetection3d$ python projects/BEVFusion/demo/multi_modality_demo.py demo/data/nuscenes/n015-2018-07-24-11-22-45+0800__LIDAR_TOP__1532402927647951.pcd.bin demo/data/nuscenes/ demo/data/nuscenes/n015-2018-07-24-11-22-45+0800.pkl projects/BEVFusion/configs/bevfusion_lidar-cam_voxel0075_second_secfpn_8xb4-cyclic-20e_nus-3d.py bevfusion_converted.pth --cam-type all --score-thr 0.2 --show

Reproduces the problem - error message

Traceback (most recent call last):
  File "/home/hydrapc/anaconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/utils/misc.py", line 77, in import_modules_from_strings
    imported_tmp = import_module(imp)
  File "/home/hydrapc/anaconda3/envs/openmmlab/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/hydrapc/projects/DROI-BEV/mmdetection3d/projects/BEVFusion/bevfusion/__init__.py", line 1, in <module>
    from .bevfusion import BEVFusion
  File "/home/hydrapc/projects/DROI-BEV/mmdetection3d/projects/BEVFusion/bevfusion/bevfusion.py", line 16, in <module>
    from .ops import Voxelization
  File "/home/hydrapc/projects/DROI-BEV/mmdetection3d/projects/BEVFusion/bevfusion/ops/__init__.py", line 1, in <module>
    from .bev_pool import bev_pool
  File "/home/hydrapc/projects/DROI-BEV/mmdetection3d/projects/BEVFusion/bevfusion/ops/bev_pool/__init__.py", line 1, in <module>
    from .bev_pool import bev_pool
  File "/home/hydrapc/projects/DROI-BEV/mmdetection3d/projects/BEVFusion/bevfusion/ops/bev_pool/bev_pool.py", line 3, in <module>
    from . import bev_pool_ext
ImportError: /home/hydrapc/projects/DROI-BEV/mmdetection3d/projects/BEVFusion/bevfusion/ops/bev_pool/bev_pool_ext.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZNSt15__exception_ptr13exception_ptr9_M_addrefEv

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/hydrapc/anaconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/config/config.py", line 462, in fromfile
    import_modules_from_strings(**cfg_dict['custom_imports'])
  File "/home/hydrapc/anaconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/utils/misc.py", line 84, in import_modules_from_strings
    raise ImportError(f'Failed to import {imp}')
ImportError: Failed to import projects.BEVFusion.bevfusion

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "projects/BEVFusion/demo/multi_modality_demo.py", line 78, in <module>
    main(args)
  File "projects/BEVFusion/demo/multi_modality_demo.py", line 42, in main
    model = init_model(args.config, args.checkpoint, device=args.device)
  File "/home/hydrapc/projects/DROI-BEV/mmdetection3d/mmdet3d/apis/inference.py", line 59, in init_model
    config = Config.fromfile(config)
  File "/home/hydrapc/anaconda3/envs/openmmlab/lib/python3.8/site-packages/mmengine/config/config.py", line 474, in fromfile
    raise ImportError(err_msg) from e
ImportError: Failed to import custom modules from {'imports': ['projects.BEVFusion.bevfusion'], 'allow_failed_imports': False}, the current sys.path is: 
    /home/hydrapc/projects/DROI-BEV/mmdetection3d/projects/BEVFusion/demo
    /home/hydrapc/anaconda3/envs/openmmlab/lib/python38.zip
    /home/hydrapc/anaconda3/envs/openmmlab/lib/python3.8
    /home/hydrapc/anaconda3/envs/openmmlab/lib/python3.8/lib-dynload
    /home/hydrapc/anaconda3/envs/openmmlab/lib/python3.8/site-packages
    /home/hydrapc/projects/DROI-BEV/mmdetection3d
    /tmp/tmp_f0g2gch
You should set `PYTHONPATH` to make `sys.path` include the directory which contains your custom module

Additional information

I follow the installation of mmdetection3d but cannot run the demo for BEVFusion.

[ranai-srivastav]

I have the same issue as well.

Tried adding init.py to some folders and added the BEVFusion path to system path as well

[Machine-NO-Learning]

any solutions?

[Machine-NO-Learning]

there is one possible sol, config.py raise error is based on your import_module, but import_module raise error is not reason of wrong sys path, maybe you need install einops, so just pip install einops.

[FraMarotta]

I have the same problem, anyone has solved?

[qiliang72]

Possible problem might be compatibility of gcc, g++, cuda version. Maybe you can try to use GCC 9.3 which is your pytorch compiler version. Also make sure to delete all your temporary files of previous builds. (This takes me too much time ><)

[razgzy]

I have the same problem

[sss-stu-stupid]

I have import error too, when from . import bev_pool_ext; but it describes the problem as {ImportError: cannot import name 'bev_pool_ext' from partially initialized module 'projects.BEVFusion.bevfusion.ops.bev_pool' (most likely due to a circular import) (mmdetection3d/projects/BEVFusion/bevfusion/ops/bev_pool/init.py)} This is the only difference in our logs.

[BeMuCa]

I have import error too, when from . import bev_pool_ext; but it describes the problem as {ImportError: cannot import name 'bev_pool_ext' from partially initialized module 'projects.BEVFusion.bevfusion.ops.bev_pool' (most likely due to a circular import) (mmdetection3d/projects/BEVFusion/bevfusion/ops/bev_pool/init.py)} This is the only difference in our logs.

This Solution from the OG BEVFusion Git solved it for me:

https://github.com/mit-han-lab/bevfusion/issues/504

[ranai-srivastav]

@BeMuCa I don't have a mmdetection3d/mmdet3d/ops folder. This is my folder structure

~/software/mmdetection3d/mmdet3d$ ls
apis  configs  datasets  engine  evaluation  __init__.py  models  __pycache__  registry.py  structures  testing  utils  version.py  visualization

[Torreskai0722]

My issue is resolved after I uninstall and reinstall numba:

pip uninstall numba
pip install numba

Besides, I also installed spconv which causes another cuda runtime error:

pip install spconv-cu120

[seven-linglx]

cd mmdetection3d python projects/BEVFusion/setup.py develop

[Gigalomanicx]

因为在https://mmdetection3d.readthedocs.io/en/latest/get_started.html 安装pytorch时直接使用了 condainstall pytorch torchvision -c pytorch 然而环境的cuda版本低于pytorch2.3.0的11.8，所以自己安装了cpu版（后续检查发现）。 解决方法是对着pytorch官网调整cuda和pytorch版本，然后python projects/BEVFusion/setup.py develop 编译bev_pool_ext