Yannick Kirchhoff
Yannick Kirchhoff
Hi @RoloSmajda, the input and CPU output both look reasonable. So the issue is indeed in the forward pass on the GPU. It is definitely worth a try setting up...
Hi @RoloSmajda, I just thought of something, if you are setting up a new environment also check that your NVIDIA driver is up-to-date. We recently had some issues with incompatibilities...
Hi @RoloSmajda, did you already find the time to setup a new environment and test it? Best, Yannick
No worries, just wanted to check :)
@FabianIsensee yes I was not able to reproduce the issue so far. But I also only tested with the most recent version, so if the pypi version is not the...
I could build a docker image and send it to you @RoloSmajda. That way we could rule out any software related issues. Does that make sense @FabianIsensee?
Hi Rui, could you please post the lines before that error message? You can usually find the problem somewhere in there. Best, Yannick
Hi Rui, thanks for the complete error. There is a KeyboardInterrupt in your error, which if you didn't trigger it yourself is probably triggered by Azure for some reason. As...
Hi Rui, yeah training times become a pain when setting low `nnUNet_n_proc_DA`. It is kind of weird that the error appears only after a few epochs. It seems to be...
Hi Rui, no problem at all and sorry that I couldn't help more. I will try to find out more about your problem. Regarding specs, everything with at least 10GB...