pocketfft
pocketfft copied to clipboard
mreineck
Avoid using thread_local to work-around mingw bug
Ref #1, #2, msys2/MINGW-packages#10164
The mingw issue suggests that thread-local storage is buggy in mingw, so avoiding thread_local might help. I don't have a mingw setup to test though.
cc @carlkl, @rgommers
Thanks a lot for this, @peterbell10 ! I don't have access to a MinGW installation either, so let's see what @carlkl says.
If I read the changes correctly, the evolution of this multi-threading code in https://github.com/mreineck/ducc/blob/ducc0/src/ducc0/infra/threading.cc should be unaffected by this problem; do you agree @peterbell10 ?
Pity we can't switch to this implementation yet, at least for scipy.
Yes, because you use pass scheduler explicitly instead of via thread-local storage it should be equivalent.
As a data point, I tried naively copying pocketfft_hdronly.h here to the Scipy source tree, and compiling with
-DPOCKETFFT_PTHREADSand pthreads linking - https://github.com/matthew-brett/scipy/runs/4447908937- Without - https://github.com/matthew-brett/scipy/runs/4440857238
but in both cases, Scipy.fft still hangs.
Can I ask - what is the best way of replicating the hang without compiling Scipy? I have not observed a hang on my machine when using mingw-w64 to compile the demo from this repo : https://github.com/mreineck/pocketfft/issues/1#issuecomment-976423438 - even running the demo hundreds of times.
I now tested this PR with scipy. Sorry for the long time ... With MT scipy still hangs unfortunately. I extracted a stacktrace during the hang: pocketffthang.log.txt (reformatted and commented). However, I don't really get any more insight from it though. It seems to hang on shutdown of the thread pool, but why? Maybe the experts?
This stacktrace looks like issue LLD had with mingw-w64 GCC: https://reviews.llvm.org/D102944
Replacing exit with _Exit might help.
To clarify:
The mingw issue suggests that thread-local storage is buggy in mingw
Emutls that GCC uses is buggy on Windows but it's totally unrelated to mingw-w64, TLS in Clang+mingw-w64 works just fine.
I don't think that this hang is related to DLL unloading but I may be wrong. In my opinion it has to do with threads shutdown. And there is no exit in pocketfft code.
EDIT: I'm using the latest and greatest UCRT64 gcc-11.2 (msys2)
I have to correct myself: The hang happens at process termination. @mati865, is this related to https://sourceforge.net/p/mingw-w64/bugs/859/?
as reference: https://github.com/msys2/MINGW-packages/issues/10164#issuecomment-978033320. Seems to be the __cxa_thread_atexit hell... Still investigating.
Sorry foe the late reply, this notification got lost in my backlog.
I have to correct myself: The hang happens at process termination. @mati865, is this related to https://sourceforge.net/p/mingw-w64/bugs/859/?
I think this is similar issue that might require another fix.