Brian Chen

Related: https://github.com/FluxML/Optimisers.jl/pull/57. We have proposals for `map` and `reduce`, but does it make sense to try for a `mapreduce`?

I don't claim to know what the right answer is, so I posted those examples because it's not clear _if_ they'd be considered consistent enough to pass muster. Another one...

A simpler version of this came up in conversation over RL models on Slack today. The current incantation for updating one model's parameters based on the moving average of another...

We could always make `reset!` take the parameter tree of `x`s too, but that may come at the cost of sacrificing type stability.

Could the in-place version be tripping some aliasing heuristic and hitting a slow path? I guess a profile would be illuminating.

One idea for making this differentiable is to take the non-diff part (caching) out of `fmap` and instead creating a memoized callback wrapper type. Then `fmap` itself is trivially differentiable...

`fmap` itself is not sufficient for this. My thought was that you could use it as the `map` part of a `mapreduce`, where cached leaves are replaced with some neutral...

`fmap` is the only intrinsically type unstable part of Functors. Part of that is due to the IdDict cache, but another is the fragility of a chain of mutually recursive...

Now that people are starting to use explicit params, we've seen a few instances where it would be nice to have a easy method for adding regularization terms. I believe...

It's not so straightforward, unfortunately. See https://github.com/FluxML/Flux.jl/issues/1991#issuecomment-1148627732 and https://github.com/FluxML/Optimisers.jl/pull/33.