Libtask.jl
Libtask.jl copied to clipboard
TuringLang
Significantly improve the Libtask library using ideas from Mooncake / ReverseDiff
TODO
Edit this issue with more details about what needs to be done.
Known issues
- Examples of a segfault with current Libtask: here and https://github.com/TuringLang/Turing.jl/issues/2379#issue-2616058378
- Performance degrades on Julia 1.11: https://github.com/TuringLang/Turing.jl/pull/2328#issuecomment-2553425976
- ~~TracedRNG isn't deterministic even on the same architecture https://github.com/TuringLang/AdvancedPS.jl/issues/108~~ -- this has been resolved independently of this issue.
@willtebbutt and I have discussed this a few times during meetings. Will, can you add your plan here?
Regarding the issues above linked above.
Examples of a segfault with current Libtask: here and https://github.com/TuringLang/Turing.jl/issues/2379#issue-2616058378
The first example isn't a segfault, it's an OOM error on x86. Additionally, https://github.com/TuringLang/Turing.jl/issues/2379#issue-2616058378 appears to be a Windows specific issue.
In both cases, it's quite unclear to me what relationship we believe that these have to the introduction of ideas from Mooncake / ReverseDiff . @yebai could you explain how you think these are related?
In both cases, it's quite unclear to me what relationship we believe that these have to the introduction of ideas from Mooncake / ReverseDiff . @yebai could you explain how you think these are related?
No, they are not directly related -- ideas from Mooncake and ReverseDiff are mainly to improve the robustness and maintainability of the Libtask implementation.
Re https://github.com/TuringLang/Turing.jl/issues/2379 -- I've just spoken with @mhauru and @penelopeysm and apparently we've not seen it happen again since it was initially reported. If it pops up again we can take a look, but otherwise I'm going to assume that I'm unlikely to see it.
