Feature: Backend

[elbow-jason]

Summary

This issue is intended for discussion and planning around developing a Backend abstraction for Annex similar to Keras's Backend abstraction.

This description from the Keras docs capture the intent of the Annex Backend abstraction.

Rather than picking one single tensor library and making the implementation of Keras tied to that library, Keras handles the problem in a modular way, and several different backend engines can be plugged seamlessly into Keras.

Keras Backend Documentation

A Backend for Annex should be pluggable, swappable, and not affect the high-level usage of Annex.

Previous work

The current implementation of Annex.Data was implemented with a similar intent to that of the Backend, though the current implementation of Annex.Data leaves a lot to be desired.

Questions

• What is the minimal interface that a backend must implement to be functional?

• Are there sets of functions that could/should be broken into separate interfaces/behaviours? (e.g. in Keras the functions for conv1d and conv2d (and the like) seem like they could each be their own behaviour. Whether or not a Backend supports those operations is up the to implementer of the backend.

• What are the nice-to-haves/optional callbacks for the Backend interface.

Requirements

• A Backend must be runtime configurable and swappable.
• A Backend should be memory safe (either provided by the low-level lib itself, or the lib's idiomatic usage via the Backend).
• A Backend must be able to dump its data (Arrays, Lists, Vectors, Tensors, etc.) into an Elixir term (e.g. a binary or list or something like {:u32, <data_here>}).
• A Backend must be able to load its data from an Elixir term (e.g. list or binary or something like {:u32, <data_here>})

Interested parties

@arpieb

[elbow-jason]

Questions about a common low-level data structure

In Keras, and many of Python's number crunching libs there exists a common datastructure for manipulating collections of numbers. This common data structure is the NumpPy ndarray.

Should Annex define or use (TODO: find if this exists in the Erlang ecosystem) a common, C-native data structure for manipulating a collection of numbers that the Backend can target?

Thoughts?

[arpieb]

I've been toying with that - a simplified NumPy type interface with:

• Support for opaque, NIF-based n-dimensional tensors in float16, float32 to ubiquitously support off-CPU processing
• Basic BLAS type functions required for manipulation of said tensors in an ML model for feedforward and backprop operations as well as things like ones, zeros, identity, transpose, etc
• Support for a common set of required activation functions that can be effectively parallelized by a backend, dictated by Annex for consistency
• Let all layer config and coordination happen at the Annex core level, unless we want to support "compiling" the LayerConfig into an opaque backend-specific model

That being said, we probably don't want to leave functionality baked into some backends (e.g. OpenCL, CUDA) on the table since they've already been highly optimized, so maybe a "driver" approach that could do things like query the backend for high-level feature support, and fall back to simpler minimum required implementations (like the above). This kind of functionality is common in graphics drivers (not talking CUDA etc, actual display):

• Can you render a circle?
  • Great! Here's yer circle params...
  • No? Can you render an ellipse?
    • Good deal - here's your ellipse params...
    • No? Can you render a polygon?
      • OK, we can decompose the circle to a list vertices if we have to...
      • No? Can you render a series of lines?
        • This is becoming a hassle, but we can manage...
        • No? How about a single line...?
          • I am going to be calling LINE a looooot...
          • Seriously? How about POINTS???
            • I didn't sign up for rasterization, but wth...
            • No??? I give up...

Actually, come to think of it, OpenCL provides this kind of query as well to find out what functionality a device provides, as some could be non-GPU devices or have special vendor-specific extensions...

Along the lines of a common NIF-based tensor library - maybe Matrex would be a solid, performant option that the backends could leverage for host-based matrix manipulation that would be way faster than Elixir native lists for assembling data to be passed into a backend? I'm not even sure how well supported NIFs talking to each other is, I've never tried to do it... Assuming with the right includes and linking directives we could do it, just a big unknown for me.

[arpieb]

Not to suggest we knock off the backend interface that Keras defines verbatim, but here's how they layered things out:

https://github.com/keras-team/keras/tree/master/keras/backend