Unable to convert RandomForestModel into tflite form

[datasciencemachine]

I was able to fit my RandomForest model, however when I try to convert it into tflite format it throws error. The error is : InvalidArgumentError: Cannot convert a Tensor of dtype resource to a NumPy array.

[janpfeifer]

Thanks for bringing this up and creating the issue.

I'll repeat the message on the TF Forum, for others to have an alternative for the time being.

Unfortunately TFLite does not yet implement TF-DF models. It's in our radar, but it's not top priority (not likely this quarter).

In the short term, for a very fast/cheap inference for a purely decision forest models, consider doing inference using the TF-DF C++ library called Yggdrasil. There is an example that one can use to get started. It will read the TF-DF saved model that one trained in TensorFlow directly.

The Decision Forest models served in this fashion are often incredibly low-latency / low-cost. You can measure the serving speed without writing code using the benchmark inference tool.

[paro-]

Hi @janpfeifer, Any update on TFLite supporting TF-DF models? We're interserted in it as well. Thanks, good day, Paul

[janpfeifer]

hey @paro- , sorry no good news here.

TFLite integration, like TensorFlow.js (or something similar) integration, are features we would really love to have/work on, but currently we have a list of higher priority things we are are working on.

At least there is an alternative for serving on anything that runs C++: use the Yggdrasil implementation direction, see links on previous email -- I hope it works for your use case.

We will post here, as soon as we have more news.

[arg0]

Hey guys，any further updates on this topic？

Convert the TFDF model to TF Lite model structure, support inference and training on device

[janpfeifer]

hi @arg0 , no news on our side, sorry. :(

Any chances you could post this on TensorFlow Forum with the tflite tag ? You could also include the decision_forests tag, which would include us from TF-DF.

This is because it's something that needs to be of interest for TF Lite team as well (we actually would like to work on it).

[ys31jp]

Hi, anyone make it?

[tgpsantos]

hi @arg0 , no news on our side, sorry. :(

Any chances you could post this on TensorFlow Forum with the tflite tag ? You could also include the decision_forests tag, which would include us from TF-DF.

This is because it's something that needs to be of interest for TF Lite team as well (we actually would like to work on it).

Hi @janpfeifer , I'm also interested in deploying this through tlfite.

Does the following post on TF Forum suffice or do you require a new one? It's from 2021 and it's about boosted tree classifier (even though it can generalize for the decision trees as a whole): https://discuss.tensorflow.org/t/gradientboostedtreeclassifier-model-in-tensorflow-lite/2899

[dipanjans16]

Hi @janpfeifer, Any update on TFLite supporting TF-DF models?

[rstz]

Unfortunately not 😞

[dipanjans16]

Are model trained using yggdrasil-decision-forests lighter and faster than TF-DF

[rstz]

Assuming you're referring to this repo, either C++ or Python: Yes, in many cases they are. A TF-DF model is essentially an Yggdrasil Decision Forests model with a TensorFlow hat on. During inference and (often) training, there is significant overhead for going through the TensorFlow layers. There's more about this in the documentation, including a guide to make your model faster and smaller.

Finally, the YDF inference engine is much smaller than full Tensorflow - probably less than 10 MB before optimizing for size via compiler flags / excluding unnecessary parts.

[HudsonGraeme]

Also looking for this feature, hopefully it can be prioritized sometime in the near future :smile: