Trainer API

[dfdx]

High-level API for model training similar to PyTorch Lightning.

One advantage over Python is that we don't need a separate module type to add methods to a model class (e.g. train_step() and similar).

[RoyiAvital]

Are you still developing this? Since it seems KNet.jl isn't developed anymore I was thinking about using this one.

[dfdx]

More no than yes. Initial goals of Avalon are now fulfilled with other libraries - Flux (which is now stable enough), Yota (graph-based AD) and ONNX (interop with existing DL frameworks). Avalon may still see another iteration of development as an alternative top-level package of the same tools, but at the moment it's not on the radar.

[RoyiAvital]

I thought that in order to use Yota.jl for DL one need to use Avalon.jl.

[dfdx]

Not really. Let me explain how some packages communicate with each other.

• ChainRules - a large collection of autodiff rules
• Yota, Zygote, Diffractor, etc. - autodiff packages based on ChainRules
• Optimisers - implementation of optimisation algorithms such as SGD, Adam, etc.
• NNlib provides low-level functional primitives for deep learning
• Flux - top-level package containing neural network layers (mostly based on NNlib) and various utilities for training deep learning models. In tutorials, you can usually see Flux used with Zygote and Optimisers.

All these libraries are built with the idea of open ecosystem where you can combine tools in whatever way you want. For example, you can avoid any top-level deep learning libraries at all and instead use NNlib + Optimisers + any of the mentioned AD packages directly. Or use can use Flux, but replace Zygote with Yota and Optimisers with your own gradient descent implementation, etc.

Unfortunately, flexibility of the open ecosystem and wide range of competing tools also means that proportionally less time is spent on testing each of the combinations. So don't expect flawless experience, yet the maintainers are usually very responsive.

Some other packages you may want to know about:

• Umlaut - tracer + tape (computational graph) implementation which Yota is based on.
• ONNX - reading and writing ONNX graphs for interop with other frameworks. Currently is based on Umlaut and supports conversion of ONNX subset to/from NNlib operations.

[RoyiAvital]

I am aware of the structure of a Deep Learning framework and the approach in the Julia Eco System to build each element in its own with the vision of being able to do mix & match.

This is a great vision, yet in practice, engineering guidelines says the tight development yields faster iterations and more robust results in short time while separation means independent development at the cost of challenging integration.

Since we're in practice, I was after a combination which works well. I thought Avalon.jl + Yota.jl would be just that :-).

Too bad not enough momentum was behind KNet.jl. If it was backed by community, its simple policy, let's recreate PyTorch in Julia would mean having something working by now :-).

As you wrote, there is no syncing in the efforts. Too bad.