Lukasz Stafiniak

Reversing `Assignments.to_string_hum` is non-trivial because for readability `to_string_hum` omits much information; but would be great for usability.

Introduce training checkpointing

1

I.e. saving a restoring all global state: from `Node`, `NodeUI`, `Formula`, `Session`, and `Random` (from the standard library).

Implement support for quantization

3

The part that's a bit confusing is dynamic indexing. For consistency, the underlying integers still need to be scaled before becoming the indexing integers.

Extend virtual node inlining to non-scalar constants including ranges

1

Initially we only inline scalar constants, because that's what's supported by the `fetch_op` and (leaf) `low_level` types.

Make small-code/computation nodes virtual even if they have many accesses (optionally)

1

Currently the heuristic to make nodes virtual only considers the usage, not the cost of single use. We should at least have the option to make getter-only (axis permutation: `einsum1`...

Support bfloat16

2

OCaml 5.2 will have float16, we will still need bfloat16...

My first attempt at it in equational shape inference migration has had lots of bugs so I'm removing it for now. Let's first add fully-featured convolution support to indices and...

For example, the comment breaks the flow here: ```ocaml let sgd_one ?(lr = 0.001) ?(momentum = 0.0) ?(weight_decay = 0.0) ?(nesterov = false) p = if not @@ is_param p...

Inspirations: * [Guidelines from OCaml.org](https://ocaml.org/docs/guidelines) * [Guidelines from the Dune project](https://github.com/ocaml/dune/blob/main/doc/hacking.rst#general-guidelines) Related: issue [#34](https://github.com/lukstafi/ocannl/issues/34).

I'll remove `__constant__` for now to simplify code. It requires the constant arrays to be copied for each module. Beware -- `__constant__` is not necessarily faster than global memory when...