3d-multi-resolution-rcnn icon indicating copy to clipboard operation
3d-multi-resolution-rcnn copied to clipboard

mmcv and mmdet compatibility?

Open jiapei100 opened this issue 3 years ago • 1 comments

➜  3d-multi-resolution-rcnn git:(master) ✗ CUDA_VISIBLE_DEVICES=0 ./tools/dist_train.sh configs/3d-multi-resolution-rcnn.py 1 --validate

~/.local/lib/python3.8/site-packages/torch/distributed/launch.py:178: FutureWarning: The module torch.distributed.launch is deprecated
and will be removed in future. Use torchrun.
Note that --use_env is set by default in torchrun.
If your script expects `--local_rank` argument to be set, please
change it to read from `os.environ['LOCAL_RANK']` instead. See 
https://pytorch.org/docs/stable/distributed.html#launch-utility for 
further instructions

  warnings.warn(
~/.local/lib/python3.8/site-packages/numpy/core/getlimits.py:499: UserWarning: The value of the smallest subnormal for <class 'numpy.float64'> type is zero.
  setattr(self, word, getattr(machar, word).flat[0])
~/.local/lib/python3.8/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for <class 'numpy.float64'> type is zero.
  return self._float_to_str(self.smallest_subnormal)
~/.local/lib/python3.8/site-packages/numpy/core/getlimits.py:499: UserWarning: The value of the smallest subnormal for <class 'numpy.float32'> type is zero.
  setattr(self, word, getattr(machar, word).flat[0])
~/.local/lib/python3.8/site-packages/numpy/core/getlimits.py:89: UserWarning: The value of the smallest subnormal for <class 'numpy.float32'> type is zero.
  return self._float_to_str(self.smallest_subnormal)
Traceback (most recent call last):
  File "./tools/train.py", line 7, in <module>
    from mmdet.datasets import get_dataset
  File "~/.local/lib/python3.8/site-packages/mmdet/datasets/__init__.py", line 3, in <module>
    from .cityscapes import CityscapesDataset
  File "~/.local/lib/python3.8/site-packages/mmdet/datasets/cityscapes.py", line 17, in <module>
    from .coco import CocoDataset
  File "~/.local/lib/python3.8/site-packages/mmdet/datasets/coco.py", line 16, in <module>
    from mmdet.core import eval_recalls
  File "~/.local/lib/python3.8/site-packages/mmdet/core/__init__.py", line 3, in <module>
    from .bbox import *  # noqa: F401, F403
  File "~/.local/lib/python3.8/site-packages/mmdet/core/bbox/__init__.py", line 8, in <module>
    from .samplers import (BaseSampler, CombinedSampler,
  File "~/.local/lib/python3.8/site-packages/mmdet/core/bbox/samplers/__init__.py", line 10, in <module>
    from .score_hlr_sampler import ScoreHLRSampler
  File "~/.local/lib/python3.8/site-packages/mmdet/core/bbox/samplers/score_hlr_sampler.py", line 3, in <module>
    from mmcv.ops import nms_match
  File "~/.local/lib/python3.8/site-packages/mmcv-1.4.2-py3.8.egg/mmcv/ops/__init__.py", line 2, in <module>
    from .assign_score_withk import assign_score_withk
  File "~/.local/lib/python3.8/site-packages/mmcv-1.4.2-py3.8.egg/mmcv/ops/assign_score_withk.py", line 5, in <module>
    ext_module = ext_loader.load_ext(
  File "~/.local/lib/python3.8/site-packages/mmcv-1.4.2-py3.8.egg/mmcv/utils/ext_loader.py", line 13, in load_ext
    ext = importlib.import_module('mmcv.' + name)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'mmcv._ext'
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 3285279) of binary: /usr/bin/python
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "~/.local/lib/python3.8/site-packages/torch/distributed/launch.py", line 193, in <module>
    main()
  File "~/.local/lib/python3.8/site-packages/torch/distributed/launch.py", line 189, in main
    launch(args)
  File "~/.local/lib/python3.8/site-packages/torch/distributed/launch.py", line 174, in launch
    run(args)
  File "~/.local/lib/python3.8/site-packages/torch/distributed/run.py", line 710, in run
    elastic_launch(
  File "~/.local/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 131, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "~/.local/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 259, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
============================================================
./tools/train.py FAILED
------------------------------------------------------------
Failures:
  <NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2022-01-06_12:52:47
  host      : lvision-MS-7C59
  rank      : 0 (local_rank: 0)
  exitcode  : 1 (pid: 3285279)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================

jiapei100 avatar Jan 06 '22 20:01 jiapei100

Hi, did you ever successfully run training and inference with custom dataset?

alibool avatar Feb 24 '22 03:02 alibool