cytoself icon indicating copy to clipboard operation
cytoself copied to clipboard

PicklingError: Can't pickle <function DataManagerOpenCell.<lambda>

Open gitclipp opened this issue 7 months ago • 3 comments

Dear Author,

I tried reproducing the steps from the readme file, but I received an error. at step 2 the line trainer.fit(datamanager, tensorboard_path='tb_logs') throws the following error:

PicklingError: Can't pickle <function DataManagerOpenCell. at 0x000001070AC03550>: attribute lookup DataManagerOpenCell. on cytoself.datamanager.opencell failed.

I get the following Traceback:

PicklingError Traceback (most recent call last) Cell In[16], line 20 12 train_args = { 13 'lr': 1e-3, 14 'max_epoch': 1, (...) 17 'earlystop_patience': 6, 18 } 19 trainer = CytoselfFullTrainer(train_args, homepath='demo_output', model_args=model_args) ---> 20 trainer.fit(datamanager, tensorboard_path='tb_logs')

File D:\MA\CytoSelf\cytoself-main\cytoself-main\cytoself\trainer\basetrainer.py:427, in BaseTrainer.fit(self, datamanager, initial_epoch, tensorboard_path, **kwargs) 425 # Train the model 426 self.model.train(True) --> 427 train_metrics = self.run_one_epoch(datamanager, 'train', **kwargs) 428 self.model.train(False) 430 # Validate the model

File D:\MA\CytoSelf\cytoself-main\cytoself-main\cytoself\trainer\vqvae_trainer.py:192, in VQVAETrainer.run_one_epoch(self, datamanager, phase, **kwargs) 190 raise ValueError('phase only accepts train, val or test.') 191 _metrics = [] --> 192 for _batch in tqdm(data_loader, desc=f'{phase.capitalize():>5}'): 193 loss = self.run_one_batch( 194 _batch, var, zero_grad=is_train, backward=is_train, optimize=is_train, **kwargs 195 ) 196 _metrics.append(loss)

File D:\Programme\Miniconda\envs\cytoself\lib\site-packages\tqdm\std.py:1182, in tqdm.iter(self) 1179 time = self._time 1181 try: -> 1182 for obj in iterable: 1183 yield obj 1184 # Update and possibly print the progressbar. 1185 # Note: does not call self.update(1) for speed optimisation.

File D:\Programme\Miniconda\envs\cytoself\lib\site-packages\torch\utils\data\dataloader.py:438, in DataLoader.iter(self) 436 return self._iterator 437 else: --> 438 return self._get_iterator()

File D:\Programme\Miniconda\envs\cytoself\lib\site-packages\torch\utils\data\dataloader.py:386, in DataLoader._get_iterator(self) 384 else: 385 self.check_worker_number_rationality() --> 386 return _MultiProcessingDataLoaderIter(self)

File D:\Programme\Miniconda\envs\cytoself\lib\site-packages\torch\utils\data\dataloader.py:1039, in _MultiProcessingDataLoaderIter.init(self, loader) 1032 w.daemon = True 1033 # NB: Process.start() actually take some time as it needs to 1034 # start a process and pass the arguments over via a pipe. 1035 # Therefore, we only add a worker to self._workers list after 1036 # it started, so that we do not call .join() if program dies 1037 # before it starts, and del tries to join but will get: 1038 # AssertionError: can only join a started process. -> 1039 w.start() 1040 self._index_queues.append(index_queue) 1041 self._workers.append(w)

File D:\Programme\Miniconda\envs\cytoself\lib\multiprocessing\process.py:121, in BaseProcess.start(self) 118 assert not _current_process._config.get('daemon'),
119 'daemonic processes are not allowed to have children' 120 _cleanup() --> 121 self._popen = self._Popen(self) 122 self._sentinel = self._popen.sentinel 123 # Avoid a refcycle if the target function holds an indirect 124 # reference to the process object (see bpo-30775)

File D:\Programme\Miniconda\envs\cytoself\lib\multiprocessing\context.py:224, in Process._Popen(process_obj) 222 @staticmethod 223 def _Popen(process_obj): --> 224 return _default_context.get_context().Process._Popen(process_obj)

File D:\Programme\Miniconda\envs\cytoself\lib\multiprocessing\context.py:327, in SpawnProcess._Popen(process_obj) 324 @staticmethod 325 def _Popen(process_obj): 326 from .popen_spawn_win32 import Popen --> 327 return Popen(process_obj)

File D:\Programme\Miniconda\envs\cytoself\lib\multiprocessing\popen_spawn_win32.py:93, in Popen.init(self, process_obj) 91 try: 92 reduction.dump(prep_data, to_child) ---> 93 reduction.dump(process_obj, to_child) 94 finally: 95 set_spawning_popen(None)

File D:\Programme\Miniconda\envs\cytoself\lib\multiprocessing\reduction.py:60, in dump(obj, file, protocol) 58 def dump(obj, file, protocol=None): 59 '''Replacement for pickle.dump() using ForkingPickler.''' ---> 60 ForkingPickler(file, protocol).dump(obj)

PicklingError: Can't pickle <function DataManagerOpenCell. at 0x000001070AC03550>: attribute lookup DataManagerOpenCell. on cytoself.datamanager.opencell failed

Thank You for Your time,

gitclipp avatar Nov 14 '23 13:11 gitclipp