Thomas Viehmann
Thomas Viehmann
Hi, so I had this same issue with ROCm 2.10, but it is gone with 3.1. If that's the other people's experience, too, we might consider this fixed. Best regards...
I'm not entirely sure, it could be that the tolerance is somewhat tight, but I have not checked in great detail what I'd expect for bf16.
Hi, is there anything I can do to move this along? my impression is that there are other things suffering from their objects not being deleted, too (e.g. the tqdm...
Hi Stas! Yeah, I feel your pain. Memory leaks are a very real problem for those using Jupyter notebooks for long-running experimentation (I usually keep my stuff open for days...
Thanks for reporting! I'll fix this after 1.0 comes out soon. Please note that I recommend to use PyTorch's native CTC Loss instead, I'll put up a wrapper providing compatibility...
Basically, don't use this library. > As of today, PyTorch (master) comes with its own implementation of the CTC loss. It has CuDNN support and fast CUDA and reasonably fast...
If we are to look into it, we would need a minimal reproducing example. Can you give the exact tensors you passed to the loss function? Best regards Thomas
Thanks for the script! - The "unknown error" is the ctc return code, so the wrapper handling is OK. The thing that is unfortunate is that there is a C++...
Yeah, it's broken and needs fixing (CPU works, too).
You can use GDB or valgrind if it is CPU memory (I think setting an environment DEBUG=1 works for PyTorch itself). PyTorch can report GPU memory consumption pretty well, or...