MIC-DKFZ
training time is toooo long
Dear Author, I noticed that the training time is quite long while using the code you recommended. Do you have any suggestions for optimization? Thank you for your help!
2024-09-24 21:29:45.962188: epoch: 1 2024-09-25 03:54:18.427683: train loss : -0.3505 2024-09-25 04:18:11.552264: validation loss: -0.4416 2024-09-25 04:18:11.552535: Average global foreground Dice: [np.float32(0.682), np.float32(0.6089)] 2024-09-25 04:18:11.552575: (interpret this as an estimate for the Dice of the different classes. This is not exact.) 2024-09-25 04:18:11.796514: lr: 0.000998 2024-09-25 04:18:11.797650: saving checkpoint... 2024-09-25 04:18:11.866219: done, saving took 0.07 seconds 2024-09-25 04:18:11.877072: This epoch took 24505.914864 s
2024-09-25 04:18:11.877121: epoch: 2 2024-09-25 10:42:24.207050: train loss : -0.4211 2024-09-25 11:06:42.930458: validation loss: -0.5107 2024-09-25 11:06:42.930737: Average global foreground Dice: [np.float32(0.7205), np.float32(0.5943)] 2024-09-25 11:06:42.930787: (interpret this as an estimate for the Dice of the different classes. This is not exact.) 2024-09-25 11:06:43.180424: lr: 0.000997 2024-09-25 11:06:43.181376: saving checkpoint... 2024-09-25 11:06:43.621577: done, saving took 0.44 seconds 2024-09-25 11:06:43.761684: This epoch took 24511.884540 s
