Neural Network optimizers implemented from scratch in numpy (Adam, Adadelta, RMSProp, SGD, etc.)
A collection of optimizers, some arcane others well known, for Flax.
🧑🏫 60 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(c...
Instantly improve your training performance of TensorFlow models with just 2 lines of code!
Keras/TF implementation of AdamW, SGDW, NadamW, Warm Restarts, and Learning Rate multipliers
RAdam implemented in Keras & TensorFlow
Nx-powered Neural Networks
FrostNet: Towards Quantization-Aware Network Architecture Search
A New Optimization Technique for Deep Neural Networks