Using shared models/modules across inference pipelines

[saeid93]

Academic systems like Rim and grandslam have the ability to share a model across multiple pipelines. As there are use cases in which a single model could be used as part of two separate pipelines in a company submitted by multiple users, it would be a good idea to just reuse the already deployed model and autoscale it based on need rather than have two separate versions of it. However, as it needs some custom policy on top of K8S scheduler it might not be an urgent change.

An implementation idea for the user interface, would be to first deploy models as separate services and then just make the connection between them through the YAML file. This needs to decouple the physical placement of models from the graph representation.

[ukclivecox]

We will be addressing this is v2 of our APIs