Optimisers.jl
Optimisers.jl copied to clipboard
FluxML
Optimisers.jl defines many standard optimisers and utilities for learning loops.
ArrowTypes is a [light package](https://github.com/apache/arrow-julia/blob/main/src/ArrowTypes/Project.toml) used for defining how to serialize objects to arrow format with Arrow.jl. Arrow is a heavy dependency that actually does the serialization. We could add...
Zygote compile times make what ordinarily should be a pretty fast test suite rather sluggish. Perhaps we could borrow some functionality from CRTU: https://github.com/JuliaDiff/ChainRulesTestUtils.jl/blob/v1.8.1/src/testers.jl#L224.
It would be nice to be able to temporarily exclude some parameters from training. (Edit: I forgot that there is https://github.com/FluxML/Flux.jl/issues/1931, now folded in here.) 1. One mechanism is to...
This function makes a good-faith effort to mutate, but provides no guarantee. That's unlike `freeze!`, and unlike Base functions, which *guarantee* they will mutate. There's some convention that `!!` means...
If you print the status of the example in the docs at the readme, this makes it about 4 screenfuls, instead of 20something. But shows a few numbers so that...
There are many different design on how the update function/api should be implemented, but the application rules of each optimizer instances are the same. I wonder if we can separate...
Example: ```julia using Optimisers, Functors, Zygote struct TwoThirds a; b; c; end # from the tests Functors.@functor TwoThirds (a, c) Optimisers.trainable(x::TwoThirds) = (a = x.a,) mtt = TwoThirds([1.0, 2.0], [3.0],...
This adds a couple small changes on top of [this draft PR](https://github.com/FluxML/Optimisers.jl/pull/63) in order to fix #62: 1. Wrap offset indices in a dummy struct `Offset` to fix the issue...
This tries to fix #62 by using `functor(typeof(x), dx)` only for gradients, and `functor(aux)` for the tree of offsets made by `destructure`. But it doesn't quite work, as for an...
This adds tests which should detect problems with CuArrays, which can run on your laptop (and on CI) using a fake GPUArray. Similar to https://github.com/FluxML/OneHotArrays.jl/pull/9 ~~WIP as it should still...