Mike J Innes

`init` should just return whatever state would normally be stored in the `IdDict`, eg a zero array of some kind. Our functional training loop has the broad structure of ```julia...

While you're looking at it, it may make sense to try Optimisers.jl on the [rnn example](https://github.com/FluxML/XLA.jl/blob/fbd35b73517845eaf5f4d8430a5d2305cc5739e7/examples/char-rnn.jl). Zygote returns `Ref`s for mutable struct gradients, which I don't think the optimisers support...

Honestly, I can't remember – it's been a while since I worked on this. If you want to patch up these issues through a PR you'd be very welcome – ...

Awesome, this will help us get rid of the last bits of Objective-C code – thanks for providing the example. One tricky part is that currently method signatures are loaded...

Yes, if you're referring to module scoping. e.g. typing while inside module `Foo` gets you functions defined in Base + Foo, and typing `Foo.` gives you completions defined in Foo...

I still have a bunch of concerns about this kind of approach, _but_ since you're going to figure it out anyway, I may as well point out that this is...

In general, it seems extremely unlikely to me that there's something hybrid/OO can do that pure SCT can't. Mutations, including global ones, caches etc. are all fine; you can also...

If you can get the current design working fast you can certainly get an SCT version to work fast too. I suspect it's significantly easier than putting effort to working...

I think you're technically right that you could twist Cassette's metadata system to behaving like (forward mode) SCT AD with some effort, but this would look very different from using...

That's true, though only if you choose to represent `Zero` that way. You could instead make use of the field type and use a natural dual for that (if it's...