Sean Moriarity
Sean Moriarity
We should work out a common API and implementation for working with attention layers. Right now they are easily implemented as custom layers, but we should have conveniences to save...
For now, we'll only consider how this should work in the model creation and execution API, but it will touch the training API as well. Consider the models in a...
Combinators are used in the model creation API for model composition. Based on my research from other libraries, we should include the following combinators to provide maximum expressivity: - [x]...
Here is an (incomplete) list of guides that should be added to the documentation: - [ ] Basic Axon models - [ ] Complex Models (combinators, weight sharing, etc.) -...
Currently we abstract a lot of what goes on within RNNs away to a single layer call with some parameters. Our API is similar to Keras. I am considering making...
See also https://pytorch.org/tutorials/intermediate/parametrizations.html and https://keras.io/api/layers/constraints/
As I go through a major rewrite of the Axon compiler in #260, I've realized it will be very easy to make breaking changes if we do not include versions...
Willing to accept examples on different datasets and models to demonstrate different parts of the Axon API and to demonstrate Axon's viability in the ecosystem. The TensorFlow guides are a...
As one example `validate` forwards metrics to the outer loop state and thus after it runs the loop step needs to be recompiled. This is possible in other handlers as...
There are some stability issues during training that I am investigating. Hopefully not due to grad :D Performance wise this isn't bad!