multi model graphs request propagation

[saeid93]

Follow up to Slack channel question with @adriangonz . As part of a project, I have a set of interconnected nodes (each node is a Triton server) in which each of them has a number of models (with the same goal) and I want each request to pass through a different set of models. For example, I want request 1 to pass through model 1 in node 1 -> model 1 in node 2 -> model 1 in node 3, and another path R2. The path for each request is determined prior to run the request and can be possibly sent through metadata to the Triton servers and Seldon core graph.

P.S A possible use case is that I have requests with different latency-accuracy requirements and I want to choose the path of the request based on that.

[ukclivecox]

In general you could achieve this by

1. Extending your graph to have a Router before each item. Although this is a challenge with Triton as that is using the V2 protocol.
2. Utilize Triton's business logic scripting to call the models as needed

In general we will be investigating these use cases in v2 of our APIs.

[ukclivecox]

Please test in v2