EnzymeAD
Add Unknown/NoShadow activity
In a session with @boriskaus, I had the issue that I wasn't sure if a gradient was zero, because it was zero, -- or because it was propagated through Memory annotated as Const. IIUC user provided annotation are strong and as such Enzyme will do as the user said.
For the default in Enzyme.jl I would like to say: No information provided, if you need to store gradient information to the shadow of this input, throw an error and the user must annotate it either Const or Duplicated
Semi-related. If we are already touching the Activity enum and looking towards a 0.1 release, wouldn't that be a good opportunity to make the C-Api somewhat stricter, e.g. having different Activity enums for fwd and rev? Would also allow us to have things like Duplicated vs Dual not only in the Rust and Julia frontends, but also in Enzyme proper. Inside of Enzyme we can still map them back to the current large enum, s.t. we don't have to adjust the inner logic.
