axon

axon copied to clipboard

Custom layer serialization implementation

1

Currently custom layers will fail when they are imported if the implementation is not an MFA function (`&Module.function/arity`). We should consider raising in these cases, or support some other way...

seanmor5

msluszniak

Reinforcement Learning?

3

Hey I'm quite new to Axon and have only experimented with reinforcement learning a tiny bit, so please excuse if I haven't understood something. From what I can gather there...

HurricanKai

We should consider unwrapping containers by default in Axon custom layers. So for example if you have a custom layer: ```elixir defn custom_layer(foo, bar, _opts) do {foo, bar} end ```...

seanmor5

There are cases where the input to a model is an integer type, e.g. an attention mask or token IDs. Axon currently does not respect input types, and aggressively casts...

seanmor5

Add weight sharing

2

There is currently no explicit way to use the same weights/layer for separate inputs. For example, in Keras I could do: ```python dense = tf.keras.layers.Dense(32) d_x1 = dense(x1) d_x2 =...

seanmor5

Add attention layers

2

We should work out a common API and implementation for working with attention layers. Right now they are easily implemented as custom layers, but we should have conveniences to save...

seanmor5

Create mechanism for easy model composition

1

For now, we'll only consider how this should work in the model creation and execution API, but it will touch the training API as well. Consider the models in a...

seanmor5

Combinators are used in the model creation API for model composition. Based on my research from other libraries, we should include the following combinators to provide maximum expressivity: - [x]...

seanmor5

Add guides to documentation

1

Here is an (incomplete) list of guides that should be added to the documentation: - [ ] Basic Axon models - [ ] Complex Models (combinators, weight sharing, etc.) -...

seanmor5