metaflow icon indicating copy to clipboard operation
metaflow copied to clipboard

Trigger flows on different kubernetes clusters

Open omriel1 opened this issue 2 years ago • 2 comments

Hi, I'm currently exploring Metaflow, specifically its functionality related to Triggering Flows Based on External Events, and I'm curious about its applicability to the following use case:

Assume I have two flows F1 and F2 and kubernetes clusters A and B. Ideally I'd want to be able to execute F1 on cluster A and once it finishes, make it trigger the execution of F2 on cluster B.

Can it be done in some way?

omriel1 avatar Nov 02 '23 14:11 omriel1

@omriel1 we now have support for multi-cluster execution within a single flow - where different steps can run on different clusters. that would be one option to achieve your use case - where flow B runs all the steps in a different cluster but is still deployed in the original kubernetes cluster. yet another option would be to configure a shared event bus for both the clusters . there are pros and cons of each approach (as well as few other approaches). let us know which direction you would like to explore.

savingoyal avatar Aug 16 '24 06:08 savingoyal

Thank you for your response!

omriel1 avatar Aug 18 '24 19:08 omriel1