Avik Pal
## WHY? Serves a dual purpose: - Provides a cheap way to verify that the model works. Compiling the model for several minutes only to realize that the dimensions were...
The original idea was to perform some kinds of primitive layer fusion, but that is not implemented currently. Instead, it has led to a quite a few unexpected issues like...
An extremely WIP attempt to facilitate using Lux with MLIR optimizations via https://github.com/JuliaLabs/brutus. Currently introduces `BruLuxArray`, which removes BLAS calls.
`track_stats` was added for compatibility with Flux. But there is no reference in literature to this. See https://github.com/FluxML/Flux.jl/issues/2006.
Testing out the Immutable Arrays from https://github.com/JuliaLang/julia/pull/44381 with #7 **TLDR:** Performance is a slight pain (seems broadcasting) right now, but it is very straightforward to support these once the functionality...
Currently, we have very rudimentary handling of stochastic layers. Initialization of RNGs for stochastic layers is done as: ```julia randn(rng, 1) return (rng=replicate(rng), training=true) ``` This makes stochastic layers start...
As observed in https://github.com/SciML/FastDEQ.jl/pull/45#issuecomment-1107662055 we get a 2x speedup by moving from GroupNorm to BatchNorm which uses CUDNN kernels.
- [ ] Remove Requires.jl dependency with Flux.jl - [x] Introduce a new package Flux2Lux.jl and store the transformation there.
```julia using Lux, ComponentArrays, ReverseDiff, Random, Zygote c = Chain(Dense(3, 128), Dense(128, 1024)) x = randn(Float32, 3, 1) ps, st = Lux.setup(Random.default_rng(), c) ps_c = ps |> Lux.ComponentArray @benchmark ReverseDiff.gradient(ps...
The current change in docs #56 broke nested lists. 