[BUG] dynamic number of branch with Fork/Join

[eduardsjermaks]

Version: Elsa 2.13

Hello,

I am searching for the way to implement a dynamic number of branches based on context variables using the activities Fork and Join. The goal is to run multiple activities in parallel and join them later once all are complete. I've tried placing if conditions in each fork branch and performing the if checks externally before initiating the switch, but neither approach has worked for me as expected.

1. Option

            builder
                .Then(ac =>
                {
                    ac.SetVariable<bool>("Branch1Flag", v => true); // can be set as false
                    ac.SetVariable<bool>("Branch2Flag", v => true); // can be set as false
                })
                .Then<Fork>(fork => fork.WithBranches("Branch0", "Branch1", "Branch2"), fork =>
                {
                    fork.When("Branch0")
                        .SignalReceived("Branch0Signal")
                        .ThenNamed("Join1");

                    fork.When("Branch1")
                        .If(context => context.GetVariable<bool?>("Branch1Flag").GetValueOrDefault(), @if =>
                        {
                            @if
                                .When(OutcomeNames.True)
                                .SignalReceived("Branch1Signal")
                                .ThenNamed("Join1");

                            @if
                                .When(OutcomeNames.False)
                                .ThenNamed("Join1");
                        })
                        .ThenNamed("Join1");

                    fork.When("Branch2")
                       .If(context => context.GetVariable<bool?>("Branch2Flag").GetValueOrDefault(), @if =>
                       {
                           @if
                               .When(OutcomeNames.True)
                               .SignalReceived("Branch2Signal")
                               .ThenNamed("Join1");

                           @if
                               .When(OutcomeNames.False)
                               .ThenNamed("Join1");
                       })
                       .ThenNamed("Join1");
                })
                .Then<Join>(join => join.WithMode(Join.JoinMode.WaitAll)).WithName("Join1")
                .WriteLine("Workflow finished.");

When all flags are true Test sends sequentially these signals to the workflow: Branch0Signal, Branch1Signal, Branch2Signal.

This test works as expected - Join waits until all of the Signals are received, then workflow transition to Finish.

When only one of the flags is true e.g. Branch1Flag is true, Branch2Flag is false Test sends sequentially these signals to the workflow: Branch0Signal, Branch1Signal.

This test does not finish. The reason I found is Join activity implementation, in IsDone(context) method. It waits when all inboundConnections are done (are in recordedInboundTransitions list), meaning it waits when all if branches are done, including the one which is turned off by having value Branch2Flag set to false. This branch will never receive a signal by design, therefore workflow never finishes.

2. Option

            builder
                .Then(ac =>
                {
                    ac.SetVariable<bool>("Branch1Flag", v => true);  // can be set as false
                    ac.SetVariable<bool>("Branch2Flag", v => true);  // can be set as false
                })
                .If(context =>
                    {
                        return context.GetVariable<bool?>("Branch1Flag").GetValueOrDefault() == true && context.GetVariable<bool?>("Branch2Flag").GetValueOrDefault() == true;
                    }, 
                    whenTrue =>
                    {
                        whenTrue
                          .Then<Fork>(fork => fork.WithBranches("Branch0", "Branch1", "Branch2"), fork =>
                          {
                              fork.When("Branch0")
                                  .SignalReceived("Branch0Signal")
                                  .ThenNamed("Join1");

                              fork.When("Branch1")
                                  .SignalReceived("Branch0Signal")
                                  .ThenNamed("Join1");

                              fork.When("Branch2")
                                  .SignalReceived("Branch2Signal")
                                  .ThenNamed("Join1");
                          })
                          .Add<Join>(join => join.WithMode(Join.JoinMode.WaitAll)).WithName("Join1")
                          .ThenNamed("EndOfIfs");
                    }, 
                    whenFalse => { whenFalse.ThenNamed("EndOfIfs"); }
                )
                .If(context =>
                    {
                        return context.GetVariable<bool?>("Branch1Flag").GetValueOrDefault() == true && context.GetVariable<bool?>("Branch2Flag").GetValueOrDefault() == false;
                    },
                    whenTrue =>
                    {
                        whenTrue
                          .Then<Fork>(fork => fork.WithBranches("Branch0", "Branch1"), fork =>
                          {
                              fork.When("Branch0")
                                  .SignalReceived("Branch0Signal")
                                  .ThenNamed("Join2");

                              fork.When("Branch1")
                                  .SignalReceived("Branch0Signal")
                                  .ThenNamed("Join2");
                          })
                          .Add<Join>(join => join.WithMode(Join.JoinMode.WaitAll)).WithName("Join2")
                          .ThenNamed("EndOfIfs");
                    },
                    whenFalse => { whenFalse.ThenNamed("EndOfIfs"); }
                )
                .WriteLine("Workflow finished.").WithName("EndOfIfs")
                .Finish();

Same test sends sequentially these signals to the workflow: Branch0Signal, Branch1Signal, Branch2Signal. This workflow setup does not work as expected in both cases.

When all flags are true When I put If statement around a Fork and Join, then when first Signal comes - Branch0Signal, then workflow finishes immediately. However, Join activity's IsDone method returns False.

When only one of the flags is true e.g. Branch1Flag is true, Branch2Flag is false Same behavior.

Is building a dynamic Fork/Joins a supported scenario in Elsa 2.x or 3.x? Is there any other alternative how to build this?

I am considering overriding Join activity where it would be possible to specify which branches to wait, or running each combination of Fork/Joins as separate workflow, if these are plausible ways how to implement this.

Thank you

[glime-ai[bot]]

Based on the context provided and the Elsa Workflow documentation, it seems that you are trying to implement a dynamic forking and joining mechanism where the execution of branches is determined by certain conditions. Elsa Workflows does support dynamic forking and joining, but it requires careful design to ensure that the workflow behaves as expected.

From the information provided, it appears that the first option you tried is closer to a solution, but the issue is with the Join activity waiting for all branches, including those that should not execute due to their conditions. The second option seems to be prematurely completing the workflow when the first signal is received.

To address the issue with the first option, you might consider modifying the Join activity's logic to account for conditional branches. This could involve customizing the Join activity to ignore certain branches based on the context variables or to dynamically determine which branches to wait for.

Another approach could be to use a combination of Fork, If, and Join activities to create separate paths for each condition and then join them at the end. This would involve creating a separate Join activity for each conditional path and then joining those paths at a final Join activity.

Here's a simplified example of how you might structure the workflow:

builder
    .Then<Fork>(fork => fork.WithBranches("Branch0", "ConditionalBranch"), fork =>
    {
        fork.When("Branch0")
            .SignalReceived("Branch0Signal")
            .ThenNamed("FinalJoin");

        fork.When("ConditionalBranch")
            .If(context => context.GetVariable<bool>("Branch1Flag"), ifBranch1 =>
            {
                ifBranch1.When(OutcomeNames.True)
                    .SignalReceived("Branch1Signal")
                    .ThenNamed("JoinBranch1");
            })
            .If(context => context.GetVariable<bool>("Branch2Flag"), ifBranch2 =>
            {
                ifBranch2.When(OutcomeNames.True)
                    .SignalReceived("Branch2Signal")
                    .ThenNamed("JoinBranch2");
            })
            .Then<Join>(join => join.WithMode(Join.JoinMode.WaitAny)).WithName("ConditionalJoin")
            .ThenNamed("FinalJoin");
    })
    .Add<Join>(join => join.WithMode(Join.JoinMode.WaitAll)).WithName("FinalJoin")
    .WriteLine("Workflow finished.");

In this example, "Branch0" always runs and waits for its signal. "ConditionalBranch" checks the flags and only runs the branches that are true. Each conditional branch has its own Join activity ("JoinBranch1" and "JoinBranch2"), and they all converge at "ConditionalJoin". Finally, "FinalJoin" waits for "Branch0" and "ConditionalJoin" to complete before finishing the workflow.

Please note that this is a simplified example and may need to be adjusted based on the specifics of your workflow. Additionally, if you find that the built-in activities do not meet your needs, you may need to create custom activities or extend existing ones.

If you are considering upgrading to Elsa 3.x, you should review the documentation and migration guides to understand any changes in the activity API and behavior that may affect your workflow design.

This comment was generated by Glime.

[sfmskywalker]

I understand the issue, and you are right with your observations. The heart of the issue lies with the way the Join activity is waiting for all inbound paths to complete, even those that will never be activated. What it should do instead, is only wait for "activated" inbound paths.

This is a similar, if not, the same, issue as #4433 Going through the thread, you'll see it takes me a while to understand the core issue, and that this comment clearly visualizes the desired behavior and what is lacking in Elsa at the moment (both Elsa 2 & 3).

#4433 describes the issue and will be used to explore potential solutions for Elsa 3, which may or may not translate easily to Elsa 2.