Kyle Daruwalla

True, sorry I missed that. Let's just move ahead with this one since it has a docstring and Felix put in some time recently.

Yeah any help achieving these goals is greatly appreciated. Take a look at FluxBench.jl. It's probably a good starting point for this. Feel free to ping us on Slack or...

One improvement for Flux development would be integration with a comment bot so that the benchmarks can be invoked on PRs to see performance differences. This would require being able...

Probably due to https://github.com/JuliaDiff/ChainRules.jl/issues/598 The main issue seems to be finding a reliable reproducer. cc @mcabbott

I like four spaces because it matches docstrings/markdown, but I'm happy with Dhairya's suggestion.

> fall out of the general set of guidelines around iteration and so on MLDataPattern is just `getindex`/`length`/`iterate` but with observation dimensions added on. It _is_ what falls out. As...

FWIW I have been thinking about refactoring how observation dimensions are specified in MLDataPattern which might reduce the interface down to literally the Base indexing/iteration interface.

I would prefer we remove the hooks from `Flux.train!` in this PR and save those for a later commit. This PR can simply remove the optimizers in favor of Optimisers.jl.

Cool, looks good. Should we be calling `Optimisers.update` here then? Calling `apply` directly bypasses the mutability check.

Ref https://github.com/FluxML/Flux.jl/pull/1613#issuecomment-863274729 In my mind, this PR is basically good to go. What I would suggest is establishing a `optimizers.jl` branch off `master` (similar to how we had a `zygote`...