TuringLang
Can DifferentiationInterface be useful for Turing?
Hi there!
@adrhill and I recently started https://github.com/gdalle/DifferentiationInterface.jl to provide a common interface for automatic differentiation in Julia. We're currently chatting with Lux.jl, Flux.jl and Optimization.jl to see how they can benefit from it, and so my mind went to Turing.jl as another AD power user :)
DifferentiationInterface.jl only guarantees support for functions of the type f(x) = y or f!(y, x) with standard numbers or arrays in and out. Within these restrictions, we are compatible with 13 different AD backends, including the cool kids like Enzyme.jl and even the hipsters like Tapir.jl. Do you think it could come in handy?
Ping @yebai @willtebbutt
Some issues that seem related to AD support (not sure they are all relevant here):
- https://github.com/TuringLang/Turing.jl/issues/2186
- https://github.com/TuringLang/Turing.jl/issues/2141
- https://github.com/TuringLang/Turing.jl/issues/2000
- https://github.com/TuringLang/Turing.jl/issues/1988
- https://github.com/TuringLang/Turing.jl/issues/1835
- https://github.com/TuringLang/Turing.jl/issues/1754
- https://github.com/TuringLang/Turing.jl/issues/1481
- https://github.com/TuringLang/Turing.jl/issues/1419
Turing's current interface to autodiff backends is based on LogDensityProblemsAD -- it is a fairly lightweight package for sharing glue code to various autodiff backends. You can take a look there to see how things might work.
Looks great!
Is there a summary of how this is different from AbstractDifferentiation.jl somewhere?:)
I updated the summary in this issue: https://github.com/JuliaDiff/AbstractDifferentiation.jl/issues/131
Hi, I think DifferentiationInterface could be perfectly useful for AdvancedVI. However, for that, I think it would be useful to add an interface where auxiliary information can be passed to the target function. For example, like this. This should be very useful for any statistical application that uses minibatch subsampling. Apparently, Tapir supports passing auxiliary input to the target function. So it would be nice to have a unifying interface that provides access to that. (It would also be really nice if ReverseDiff precompiled tapes provided this feature, but unfortunately doesn't.)
Sounds reasonable, I have opened this issue to keep track:
- https://github.com/gdalle/DifferentiationInterface.jl/issues/311
Closing this in favour of https://github.com/tpapp/LogDensityProblemsAD.jl/pull/29; Turing will automatically use DI when https://github.com/tpapp/LogDensityProblemsAD.jl/pull/29 is merged.