nitroml How to use it?

How can I use this benchmark suite? I'm working on AutoML https://github.com/mljar/mljar-supervised and would love to have a quick way to run tests.

Does it integrate with GitHub Actions?

Sep 14 '20 13:09 pplonski

Note: We're still in the pre-release stage, so caveat emptor. :)

@pplonski: Thanks for reaching out. If you want to try running our benchmark suite, follow the instructions at https://github.com/google/nitroml/tree/master/examples. This suite is designed to be run on GCP using KubeFlow.

Integration is straightforward if you are already using TensorFlow, i.e. accepting TensorFlow Examples as inputs, and exporting a TensorFlow SavedModel. Does MLJar support that? If so I can point you in the right direction. Otherwise, we are planning to make this this framework more ML framework agnostic in Q4 2020, i.e. accepting raw CSV data as inputs, and exporting numpy predictions.

Sep 14 '20 22:09 cweill

@pplonski: You mentioned integrating with GitHub actions which sounds like a good idea: could you explain what you're envisioning?

Sep 15 '20 21:09 cweill