rankeval
rankeval copied to clipboard
hpclab
wishlist: TF-Ranking support
It would be great to have support for the (fairly new) TF-Ranking library by Google: https://github.com/tensorflow/ranking
I'm playing right now with tf-ranking in order to compare its performance with that of state-of-the-art ensemble-based models.
That said, how do you think RankEval can support a neural network model given it is based on ensemble of regression trees? The simplest way to "integrate" it would be to extend the tf model with a score method with the same signature of the score method in the RTEnsemble class in such a way the analysis methods can be used without incurring in errors. Obviously, it means we will have a dependency with tf and tf-ranking for such models.
If you have any other suggestion, please let me know.
Another option could be to fork the RTEnsemble class, creating a NNModel, with a common Model base class. All the utility methods (the one of course for which it makes sense to accept different models, e.g., not the topological analysis) will accept a base Model instance, and the signatures of the methods belonging to the two classes should be identical. This way everything will be transparent to the user, but still we need the dependency with tf and tf-ranking.
This is probably a cleaner solution, but with more code to change.
I like the common base class approach. The dependency does not have to be a hard one, the import could happen on demand.
Let's start thus implementing this solution. I'm not sure about the timing of the implementation, I'm busy till next week then could start working on it.
Update: I'm quite overwhelmed by the work this week and the next one. The implementation will not start before the middle of February...