Mark Rustad comments

Results 3 comments of


                                            Mark Rustad

`batchgenerators` worker dies silently during `nnUNetv2_train` on Slurm HPC (Persists with Low Workers / High Memory)

> Hi, > try to run it with deactivated compilation, this can cause these errors. > You can deactivate it by setting the environment variable `nnUNet_compile` to False before starting...

`batchgenerators` worker dies silently during `nnUNetv2_train` on Slurm HPC (Persists with Low Workers / High Memory)

[run_training.py#L277](https://github.com/MIC-DKFZ/nnUNet/blob/ea8baa328fba93dd41f8001675557174c8923c41/nnunetv2/run/run_training.py#L277): `os.environ['TORCHINDUCTOR_COMPILE_THREADS'] = 1` type error ```python if __name__ == '__main__': os.environ['OMP_NUM_THREADS'] = '1' os.environ['MKL_NUM_THREADS'] = '1' os.environ['OPENBLAS_NUM_THREADS'] = '1' # reduces the number of threads used for compiling. More...

`batchgenerators` worker dies silently during `nnUNetv2_train` on Slurm HPC (Persists with Low Workers / High Memory)

Update: Issue resolved after commit 0d04234 I can confirm that after updating my environment to include commit [0d04234](https://github.com/MIC-DKFZ/nnUNet/commit/0d042347d9587b8a12b40377066b3eeb5df102bd), I no longer experience the immediate death of background workers on SLURM....