ChainRules.jl icon indicating copy to clipboard operation
ChainRules.jl copied to clipboard
verified publisher icon JuliaDiff

Add Checkpointing

Open oxinabox opened this issue 6 years ago • 2 comments

Since checkpointing can be implemented as a sensitivity, it may very well belong here.

See:

  • Nabla: https://github.com/invenia/Nabla.jl/pull/171
  • Zygote: https://fluxml.ai/Zygote.jl/dev/adjoints/#Checkpointing-1
  • Tracker: https://github.com/FluxML/Tracker.jl/blob/bd98dab59026eb753c296a5f19367a408c2cb8b6/src/Tracker.jl#L82-L89

oxinabox avatar May 30 '19 16:05 oxinabox

I'm not sure it makes sense for that to be in ChainRules; as far as I understand, checkpointing is fairly specific to the underlying implementation. Take for example Will's Nabla PR, wherein the defined sensitivity has to know about the tape and all of that, which is specific to Nabla.

ararslan avatar May 30 '19 18:05 ararslan

This is true, though conceptually it is completely identical to Trackers.

oxinabox avatar May 30 '19 19:05 oxinabox