Arraymancer icon indicating copy to clipboard operation
Arraymancer copied to clipboard
verified publisher icon mratsim

Image Classification example?

Open UNIcodeX opened this issue 5 years ago • 4 comments

Is there an example of having arraymancer look at a folder containing multiple classes in their own folders, specifying a train / test split, and generating a model based on that? Finally saving the model to a file, loading it and predicting based on image fed to it?

UNIcodeX avatar May 20 '20 22:05 UNIcodeX

Unfortunately no,

In particular the train_valid_split KFold/StratifiedKFold don't exist https://github.com/mratsim/Arraymancer/issues/324

The NN models also are not serializable/deserializable at the moment https://github.com/mratsim/Arraymancer/issues/163#issuecomment-557034573

And some work is needed to support classic pretrained models like Resnet50, in particular creating the bottleneck layers as separate network and then combining them may work as TrainableLayer and the SGD/Adam optimizers where designed with this use-case in mind https://github.com/mratsim/Arraymancer/blob/974efe7a57481c06d4fc6995629e99603e33c017/src/nn_dsl/dsl_types.nim#L63-L68 but that was not tested yet (or at least not that I am aware of.)

mratsim avatar May 21 '20 17:05 mratsim

I understand. I'd be happy to help where I can, but I can't help feeling like I'm not at your level of genius.

Some context for my interest / question: I've been working on a project to classify some images and return a set of probabilities (top-5) to the user.

After months of tweaking and learning and tweaking some more with Keras, TensorFlow, and PyTorch, I finally ended up just using FastAI to get what I needed and it's working well enough for now, but I have my eye on optimization.

Currently Quart and Hypercorn are being used to serve the REST back-end for the mobile app (currently under development). I've looked at Japronto (since FastAI is Python based and easier to import everything). I know there's also Nimporter and YGlukhov's amazingly useful and well-done Nimpy package, but I'd really like to use an entirely Nim stack (arraymancer / NimTorch and Jester), as having to import and run with the overhead of the Python interpreter would probably negate performance gains of serving the back-end with Jester. And in any case, certainly still leaves the Python dependency chain to deal with. This is made better with Anaconda, but I believe Nim and Nimble can handle this better.

TLDR: I'm definitely interested in further optimizing the entire process, since the intention is for my current project to be a commercial one, and therefore will have all the accompanying demands on efficiency and availability. If that means working on a project to assist with that, then I am interested in doing so, or even, should I be so fortunate, allocating employee time / funding to the task.

UNIcodeX avatar May 21 '20 21:05 UNIcodeX

The less risky this to do would be to generate bindings to the new libtorch API (which is different from the backend nimtorch uses ATen). And then write a high-level API on top.

cc @sinkingsugar

The reason why is that writing kernels, say:

  • Convolution 1D
  • Bilinear
  • Instance segmentation
  • Deconvolution
  • LSTM
  • ....

Is a very involved task and writing optimized backpropagation is a huge pain as well (it took me lots of time and sources to debug GRU https://github.com/mratsim/Arraymancer/issues/228)

That's also why I want to write a linear algebra compiler. But that requires a lot of time as well (just writing the multithreading backend took me 6 months but thankfully it's super useful).

Why libtorch instead of ATen

  1. libtorch didn't exist when @sinkingsugar wrote his library.
  2. ATen is being phased out for C10 and so maintaining low-level bindings is costly.
  3. PyTorch now has a JIT for their computation DAG while ATen only manages the data buffer.
  4. PyTorch has dedicated tensor allocators, they are actually simple but it's probably easier to just use them.
  5. AFAIK ATen doesn't have serialization either (like Arraymancer, missing Protobuf for ONNX, Tensorflow model cc @KayabaNerve and HDF5 -> NN model to write)
  6. Compiling takes a long long time.
  7. Distribution: it's already painful to get people on Windows a BLAS, but libtorch is prepackaged and we can automate a download to precompiled binaries

The way forward

  1. The bindings may be autogenerated via nimterop (cc @genotrance), or maybe fragments. The result can be stored in the new collaborative SciNim organization

  2. A high-API can be written to regain the Python look and feel. This can reuse Arraymancer macros for:

  • slicing, fancy indexing, ...
  • neural network declaration
  • Einstein Summation
  • ... Those macros are already written so it may actually be easier (at least for me) then the bindings.

mratsim avatar May 22 '20 08:05 mratsim

The fragments library is very interesting. I don't so much understand the vector or linalg parts but could after some study and the FFI, DSL, and Serialization tools are especially cool. Seems like one could model an object off the HDF5 spec and then use that to ease saving and loading of an h5 file.

I have attempted to use Nimterop in the past to try to translate the cx_Oracle Python package to use with Nim, but didn't get very far, but I've picked up Dom's book and have been reading it. Perhaps the section on FFI will help my understanding of and skill in the process.

Nimline seems it would be very handy for something like this also.

UNIcodeX avatar May 22 '20 17:05 UNIcodeX