MinkowskiEngine icon indicating copy to clipboard operation
MinkowskiEngine copied to clipboard

Cannot use standard pytorch multi gpu feature

Open Xinghui-Li opened this issue 4 years ago • 2 comments

Problem Minkowski Engine is unable to incorporate pytorch multi gpu features. I train the model using standard pytorch multi gpu feature: model = torch.nn.DataParallel(model)

and the error occur when using multi gpu:

File "/home/xinghui/miniconda3/envs/mink/lib/python3.7/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/xinghui/miniconda3/envs/mink/lib/python3.7/site-packages/MinkowskiEngine-0.5.4-py3.7-linux-x86_64.egg/MinkowskiEngine/MinkowskiConvolution.py", line 302, in forward
    assert input.D == self.dimension
  File "/home/xinghui/miniconda3/envs/mink/lib/python3.7/site-packages/torch/nn/modules/module.py", line 779, in __getattr__
    type(self).__name__, name))
torch.nn.modules.module.ModuleAttributeError: 'MinkowskiConvolution' object has no attribute 'dimension'
  • Please complete all sections of this template if applicable. For installation, you must report the environment. Otherwise, your issue will be closed automatically.

To Reproduce

import torch
import MinkowskiEngine as ME

class Test(torch.nn.Module):
    def __init__(self):
        super(Test, self).__init__()
        nn_modules = list()
        nn_modules.append(
            ME.MinkowskiConvolution(1, 2, kernel_size=3, dimension=2))
        nn_modules.append(ME.MinkowskiConvolution(2, 2, kernel_size=3, dimension=2))
        self.conv = torch.nn.Sequential(*nn_modules)
    def forward(self, x):
        return self.conv(x)

model = Test()
model = model.cuda()
model = torch.nn.DataParallel(model)

data = [
    [0, 0, 2.1, 0, 0, 2],
    [0, 1, 1.4, 3, 0, 1],
    [0, 0, 4.0, 0, 0, 0.5],
    [1, 0, 0, 2, 1],
    [0, 0, 0, 0, 0]
]

def to_sparse_coo(data):
    # An intuitive way to extract coordinates and features
    coords, feats = [], []
    for i, row in enumerate(data):
        for j, val in enumerate(row):
            if val != 0:
                coords.append([i, j])
                feats.append([val])

    return torch.IntTensor(coords), torch.FloatTensor(feats)

coords, feats = to_sparse_coo(data)
coords, feats = ME.utils.sparse_collate(
    coords=[coords], feats=[feats])

x = ME.SparseTensor(features=feats.cuda(), coordinates=coords.cuda())
x = model(x)

Expected behavior The code should not encounter any problem


Desktop (please complete the following information):

  • OS: [e.g. Ubuntu 18.04]
  • Python version: [e.g. 3.7]
  • Pytorch version: [e.g. 1.7.1]
  • CUDA version: [e.g. 10.2]
  • NVIDIA Driver version: [e.g. 450.11]
  • Minkowski Engine version [e.g. 0.5.0]
  • Output of the following command. (If you installed the latest MinkowskiEngine, paste the output of python -c "import MinkowskiEngine as ME; ME.print_diagnostics()". Otherwise, paste the output of the following command.)
wget -q https://raw.githubusercontent.com/NVIDIA/MinkowskiEngine/master/MinkowskiEngine/diagnostics.py ; python diagnostics.py

Additional context Add any other context about the problem here.

Xinghui-Li avatar Jun 26 '21 03:06 Xinghui-Li

Hi @Xinghui-Li , Any news about how to solve this error?

capoommain avatar Dec 05 '22 09:12 capoommain

I got similar issue with:

  1. Ubuntu 18.04.6
  2. Python 3.8.16
  3. ME 0.5.4
  4. Pytorch 0.13.1 +cu117
  5. Cuda 11.7
  6. GPU Quadro RTX 6000

Is there any new regarding it?

yuvalH9 avatar Jun 29 '23 17:06 yuvalH9