MyNN
MyNN copied to clipboard
davidmascharka
Pure Python/NumPy neural network library extending MyGrad
Add a basic vision transformer (ViT) model for classification. Sample definition for training on MNIST: ```python >>> model = ViT( ... input_shape=(28, 28), ... input_dim=1, ... patch_shape=(4, 4), ... feature_dim=128,...
Similar to: https://pytorch.org/docs/master/_modules/torch/nn/utils/clip_grad.html#clip_grad_norm_
The current solution looks like iterating through the model parameters and `np.save`ing all the weights, then `np.load`ing them in a new model. We should support actually serializing a model
to finish: add some text about what we're doing, ~wait on the `repeat` PR from MyGrad to be merged~ Closes #49
Again could easily do an MNIST or CIFAR example with some neat visualizations. Need to do #47 first.
Easy enough to put together for MNIST once [MyGrad#169](https://github.com/rsokl/MyGrad/issues/169) is added
The paper is available [here](https://openreview.net/forum?id=ryQu7f-RZ).
The paper is availabe [here](http://jmlr.org/papers/volume12/duchi11a/duchi11a.pdf) (PDF link).
A description of the algorithm is available [here on wiki](https://en.wikipedia.org/wiki/Broyden%E2%80%93Fletcher%E2%80%93Goldfarb%E2%80%93Shanno_algorithm).
Metadata
Owner
Metadata
Pure Python/NumPy neural network library extending MyGrad