DPT
DPT copied to clipboard
segmentation failure on Titan RTX
I tested segmentation on Titan RTX and it outputs black mask with some white noises.
It seems that "optimize" option caused that problem.
The problem is solved by deleting 2 lines about "memory_format=torch.channels_last"
but I did not figure out the reason.
Thanks for reporting. You can also disable the optimization by calling the script with the --no-optimize
flag. The optimizations refer to common speed improvements and are completely optional.
As for the reason for this issue: I wasn't able to find a reason either. I don't have a Titan RTX to debug myself, unfortunately. If you really need the optimization, my only recommendation is to make sure to have the latest drivers/CUDA/Pytorch installed.
We'd be also happy to hear about any solutions that you might find.
It seems that half accuracy method ( "model.half()" & "sample.half()" ) is helpful for optimization, but the "torch.channels_last" stuff will cause nonsense output on TITAN RTX or just slow it down on Geforce 1080.
By the way, I tested on CUDA 11.3, Pytorch 1.8.1, Python 3.9