GPU available: True, used: True
TPU available: False, using: 0 TPU cores
D:\anaconda3\envs\torch_1_11\lib\site-packages\numpy\core\shape_base.py:420: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
arrays = [asanyarray(arr) for arr in arrays]
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
==> image down scale: 1.0
==> image down scale: 1.0
D:\anaconda3\envs\torch_1_11\lib\site-packages\pytorch_lightning\utilities\distributed.py:69: UserWarning: The dataloader, val dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the num_workers
argument(try 16 which is the number of cpus on this machine) in the
DataLoader` init to improve performance.
warnings.warn(*args, **kwargs)
Validation sanity check: 0%| | 0/1 [00:00<?, ?it/s]Traceback (most recent call last):
File "G:/sc/mvsnerf-main/train_mvs_nerf_pl.py", line 320, in
trainer.fit(system)
File "D:\anaconda3\envs\torch_1_11\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 458, in fit
self._run(model)
File "D:\anaconda3\envs\torch_1_11\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 756, in _run
self.dispatch()
File "D:\anaconda3\envs\torch_1_11\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 797, in dispatch
self.accelerator.start_training(self)
File "D:\anaconda3\envs\torch_1_11\lib\site-packages\pytorch_lightning\accelerators\accelerator.py", line 96, in start_training
self.training_type_plugin.start_training(trainer)
File "D:\anaconda3\envs\torch_1_11\lib\site-packages\pytorch_lightning\plugins\training_type\training_type_plugin.py", line 144, in start_training
self._results = trainer.run_stage()
File "D:\anaconda3\envs\torch_1_11\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 807, in run_stage
return self.run_train()
File "D:\anaconda3\envs\torch_1_11\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 842, in run_train
self.run_sanity_check(self.lightning_module)
File "D:\anaconda3\envs\torch_1_11\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 1107, in run_sanity_check
self.run_evaluation()
File "D:\anaconda3\envs\torch_1_11\lib\site-packages\pytorch_lightning\trainer\trainer.py", line 962, in run_evaluation
output = self.evaluation_loop.evaluation_step(batch, batch_idx, dataloader_idx)
File "D:\anaconda3\envs\torch_1_11\lib\site-packages\pytorch_lightning\trainer\evaluation_loop.py", line 174, in evaluation_step
output = self.trainer.accelerator.validation_step(args)
File "D:\anaconda3\envs\torch_1_11\lib\site-packages\pytorch_lightning\accelerators\accelerator.py", line 226, in validation_step
return self.training_type_plugin.validation_step(args)
File "D:\anaconda3\envs\torch_1_11\lib\site-packages\pytorch_lightning\plugins\training_type\training_type_plugin.py", line 161, in validation_step
return self.lightning_module.validation_step(args, **kwargs)
File "G:/sc/mvsnerf-main/train_mvs_nerf_pl.py", line 195, in validation_step
volume_feature, img_feat, _ = self.MVSNet(imgs[:, :3], proj_mats[:, :3], near_fars[0], pad=args.pad)
File "D:\anaconda3\envs\torch_1_11\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
return forward_call(input, **kwargs)
File "G:\sc\mvsnerf-main\models.py", line 904, in forward
feats = self.feature(imgs) # (BV, 8, H, W), (BV, 16, H//2, W//2), (BV, 32, H//4, W//4)
File "D:\anaconda3\envs\torch_1_11\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "G:\sc\mvsnerf-main\models.py", line 717, in forward
x = self.conv0(x) # (B, 8, H, W)
File "D:\anaconda3\envs\torch_1_11\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "D:\anaconda3\envs\torch_1_11\lib\site-packages\torch\nn\modules\container.py", line 141, in forward
input = module(input)
File "D:\anaconda3\envs\torch_1_11\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "G:\sc\mvsnerf-main\models.py", line 672, in forward
return self.bn(self.conv(x))
File "D:\anaconda3\envs\torch_1_11\lib\site-packages\torch\nn\modules\module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "D:\anaconda3\envs\torch_1_11\lib\site-packages\inplace_abn\abn.py", line 237, in forward
return inplace_abn(
File "D:\anaconda3\envs\torch_1_11\lib\site-packages\inplace_abn\functions.py", line 241, in inplace_abn
return InPlaceABN.apply(
File "D:\anaconda3\envs\torch_1_11\lib\site-packages\inplace_abn\functions.py", line 100, in forward
mean, var, count = _backend.statistics(x)
TypeError: cannot unpack non-iterable NoneType object
This problem occurred at the beginning of training. Does anyone know how to solve it?
I encountered the same problem. I solved it by simple repalcing all the normact
in models.py to [torch.nn.BatchNorm3d(out_channels), nn.ReLU()]