Federated-Learning-in-PyTorch icon indicating copy to clipboard operation
Federated-Learning-in-PyTorch copied to clipboard

AssertionError: Invalid device id

Open brainie opened this issue 2 years ago • 1 comments

So while trying to set up my Cuda GPU and installing pytorch so as to run main.py. I came across this error

  File "main.py", line 57, in <module>
    central_server.setup()
  File "/content/Federated-Averaging-PyTorch/src/server.py", line 85, in setup
    init_net(self.model, **self.init_config)
  File "/content/Federated-Averaging-PyTorch/src/utils.py", line 77, in init_net
    model = nn.DataParallel(model, gpu_ids)
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/parallel/data_parallel.py", line 142, in __init__
    _check_balance(self.device_ids)
  File "/usr/local/lib/python3.7/dist-packages/torch/nn/parallel/data_parallel.py", line 23, in _check_balance
    dev_props = _get_devices_properties(device_ids)
  File "/usr/local/lib/python3.7/dist-packages/torch/_utils.py", line 487, in _get_devices_properties
    return [_get_device_attr(lambda m: m.get_device_properties(i)) for i in device_ids]
  File "/usr/local/lib/python3.7/dist-packages/torch/_utils.py", line 487, in <listcomp>
    return [_get_device_attr(lambda m: m.get_device_properties(i)) for i in device_ids]
  File "/usr/local/lib/python3.7/dist-packages/torch/_utils.py", line 470, in _get_device_attr
    return get_member(torch.cuda)
  File "/usr/local/lib/python3.7/dist-packages/torch/_utils.py", line 487, in <lambda>
    return [_get_device_attr(lambda m: m.get_device_properties(i)) for i in device_ids]
  File "/usr/local/lib/python3.7/dist-packages/torch/cuda/__init__.py", line 361, in get_device_properties
    raise AssertionError("Invalid device id")
AssertionError: Invalid device id 

How can i fix the device ID issue.

brainie avatar May 14 '22 11:05 brainie

the default gpu_ids from config.yaml is [0,1,2], check if you have at least 3 available GPUs.

MaoPopovich avatar Oct 15 '22 03:10 MaoPopovich

Sorry for super late reply. as @MaoPopovich kindly said, you should align the device ids with the number of your physical GPU cards.

vaseline555 avatar Dec 16 '22 04:12 vaseline555