D3D
D3D copied to clipboard
请问如何解决训练时 CUDA out of memory
训练时使用了LRW-1000,数据、标签路径均无误。即使改小了batch_size=10也是无法解决
Traceback (most recent call last): File "D:\Anaconda3\envs\py36\lib\multiprocessing\process.py", line 258, in _bootstrap self.run() File "D:\Anaconda3\envs\py36\lib\multiprocessing\process.py", line 93, in run self._target(*self._args, **self._kwargs) File "E:\Git_Lip\D3D\main.py", line 74, in run trn_epoch(model=model, data_loader=trn_loader, optimizer=optimizer, epoch=epoch) File "E:\Git_Lip\D3D\util.py", line 76, in trn_epoch outputs = model(inputs) File "D:\Anaconda3\envs\py36\lib\site-packages\torch\nn\modules\module.py", line 547, in call result = self.forward(*input, **kwargs) File "E:\Git_Lip\D3D\model\D3D.py", line 125, in forward f2 = self.features(x) File "D:\Anaconda3\envs\py36\lib\site-packages\torch\nn\modules\module.py", line 547, in call result = self.forward(*input, **kwargs) File "D:\Anaconda3\envs\py36\lib\site-packages\torch\nn\modules\container.py", line 92, in forward input = module(input) File "D:\Anaconda3\envs\py36\lib\site-packages\torch\nn\modules\module.py", line 547, in call result = self.forward(*input, **kwargs) File "D:\Anaconda3\envs\py36\lib\site-packages\torch\nn\modules\pooling.py", line 210, in forward self.return_indices) File "D:\Anaconda3\envs\py36\lib\site-packages\torch_jit_internal.py", line 134, in fn return if_false(*args, **kwargs) File "D:\Anaconda3\envs\py36\lib\site-packages\torch\nn\functional.py", line 519, in _max_pool3d input, kernel_size, stride, padding, dilation, ceil_mode) RuntimeError: CUDA out of memory. Tried to allocate 1.44 GiB (GPU 0; 11.00 GiB total capacity; 7.05 GiB already allocated; 1.37 GiB free; 10.02 MiB cached)