Lux.jl
Lux.jl copied to clipboard
Improve broadcasting via `FastBroadcast,jl`
FastBroadcast is really good on CPUs (on GPUs it doesn't really matter a lot). We should migrate to use that in the case where it can give speedups like __apply_activation.
Need to figure out a way for AD packages to play nice with it though because scalar --> scalar we would want to use forward mode ad.