torch.multiprocessing.spawn.ProcessRaisedException: & TypeError: cannot pickle '_io.BufferedReader' object

[premgadwal]

Command: CUDA_VISIBLE_DEVICES=0,1,2,3 python tasks/run.py --config=egs/datasets/lrs3/lm3d_syncnet.yaml --exp_name=lrs3/syncnet

Error: Traceback (most recent call last): File "/home/deepk/Desktop/GeneFace/tasks/run.py", line 21, in run_task() File "/home/deepk/Desktop/GeneFace/tasks/run.py", line 16, in run_task task_cls.start() File "/home/deepk/Desktop/GeneFace/utils/commons/base_task.py", line 251, in start trainer.fit(cls) File "/home/deepk/Desktop/GeneFace/utils/commons/trainer.py", line 118, in fit mp.spawn(self.ddp_run, nprocs=self.num_gpus, args=(task_cls, copy.deepcopy(hparams))) File "/opt/conda/envs/face/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 240, in spawn return start_processes(fn, args, nprocs, join, daemon, start_method='spawn') File "/opt/conda/envs/face/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 198, in start_processes while not context.join(): File "/opt/conda/envs/face/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 160, in join raise ProcessRaisedException(msg, error_index, failed_process.pid) torch.multiprocessing.spawn.ProcessRaisedException:

-- Process 2 terminated with the following error: Traceback (most recent call last): File "/opt/conda/envs/face/lib/python3.9/site-packages/torch/multiprocessing/spawn.py", line 69, in _wrap fn(i, *args) File "/home/deepk/Desktop/GeneFace/utils/commons/trainer.py", line 138, in ddp_run self.run_single_process(task) File "/home/deepk/Desktop/GeneFace/utils/commons/trainer.py", line 187, in run_single_process self.train() File "/home/deepk/Desktop/GeneFace/utils/commons/trainer.py", line 263, in train self.evaluate(self.task, False, 'Sanity Val', max_batches=self.num_sanity_val_steps) File "/home/deepk/Desktop/GeneFace/utils/commons/trainer.py", line 225, in evaluate for batch_idx, batch in enumerate(pbar): File "/opt/conda/envs/face/lib/python3.9/site-packages/tqdm/std.py", line 1166, in iter for obj in iterable: File "/opt/conda/envs/face/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 444, in iter return self._get_iterator() File "/opt/conda/envs/face/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 390, in _get_iterator return _MultiProcessingDataLoaderIter(self) File "/opt/conda/envs/face/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1077, in init w.start() File "/opt/conda/envs/face/lib/python3.9/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) File "/opt/conda/envs/face/lib/python3.9/multiprocessing/context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/opt/conda/envs/face/lib/python3.9/multiprocessing/context.py", line 284, in _Popen return Popen(process_obj) File "/opt/conda/envs/face/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in init super().init(process_obj) File "/opt/conda/envs/face/lib/python3.9/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/opt/conda/envs/face/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 47, in _launch reduction.dump(process_obj, fp) File "/opt/conda/envs/face/lib/python3.9/multiprocessing/reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) TypeError: cannot pickle '_io.BufferedReader' object

[Goldcat1996]

Sir, I have encountered the same problem. Have you resolved the problem？ @premgadwal

[premgadwal]

No i havent