Local run with existed sources (without repo download)

[a-osipov-toxa]

Describe the desirable behavior of the program

It would be wonderful if you add, the ability to do local launch simultaneously without

• Download sources from repository
• Copy source code in temporary folder

Describe alternatives you've considered

Now I can use:

1. --vcs-type none option. In this case, sources will be copied to the temporary dir. If I have already have temp dir (for example from the previous Universum run) I will get an error
2. Run Universum locally (Non-CI mode) via run command. This pretty close to my request, except:
   • It is strange to use Universum in Non-CI mode on CI (Teamcity/Jenkins)
   • In my request I want the same Universum behavior regardless --vcs-type value. In Non-CI mode we have the other behavior

Additional context

This feature is needed to do several launches of Universum within the same job

[i-keliukh]

I see this feature as a part of more configurable execution, with the ability to split one run into multiple smaller ones. This is like the one needed for distributed steps. Another, much simpler option, is to just add parameter to skip copying files into "temp". As of now, this looks reasonable to only do simpler option, but consider complex one as well. Maybe we can gather some requirements for complex one from this request.

[i-keliukh]

The usage scenario behind this request is the following:

1. The config is executed once with all usual steps – downloading sources, applying patches, etc
2. The results of the execution are checked in some project-specific way. This is done outside of the config execution.
3. If the results are good, the execution finishes.
4. If the results are not good, the same config is executed over already downloaded sources with some changes in options.

Considering the scenario, it looks like the best option is to implement conditional execution. I propose to outline possible ways of implementing conditional steps execution in comments.

[i-keliukh]

Proposal of conditional execution based on new flag set in step.

1. The step is marked with new flag called “return”
2. The step is executed as regular step
3. During the execution, the condition is calculated by this step [Option 1]: the condition is reported to the universum by the step exit code. Success means the condition is satisfied, failure means the condition is not satisfied. The step failure is not considered as a run failure – it does not report to code review and does not affect the final result. [Option 2]: the new api call is implemented to mark the condition as satisfied. If the call is not made during the step execution, the condition is considered as not satisfied.
4. If the condition is satisfied, the execution is stopped, no further steps are executed and the run result is reported as “success”.
5. If the condition is not satisfied, the execution continues as usual.

All of the steps above work this way on the top level of the config. In case the step with "return" flag is placed inside the group, the behavior is changed for the current group only. Group is like function, and step with "return" flag is like return statement inside that function.

[miltolstoy]

Here is another proposal, which will allow building more complex scenarios. Pseudo-code follows:

class ConditionalConfig:
    self.success_branch_step
    self.failed_branch_step

regular_step = Step(...)
success_branch_step = Step(...)
failed_branch_step = Step(...)  // can be steps block instead of a single test
conditional_step = Step(..., cond_config=ConditionalConfig(success_branch_step, failed_branch_step))

configs = regular_step + conditional_step

• Such implementation allows:
  • creating more than one "if" condition during an execution
  • place a conditional step in any position inside the config
• Similar to Ivan's proposal:
  • the conditional step will always have the "success" status
  • options from Ivan's p.3 can be also applied

[miltolstoy]

Minimalistic test config:

condition_step = Configuration([dict(name="Condition step", command=["./test.sh"])])
false_branch_step = Configuration([dict(name="False branch step", command=["echo $?"])])

configs = condition_step + false_branch_step

Desired scenario: launch false_branch_step only if the condition_step failed

Ivan's solution (as I understood it):

condition_step = Configuration([dict(name="Condition step", command=["./test.sh"], return=True)])
false_branch_step = Configuration([dict(name="False branch step", command=["echo $?"])])

configs = condition_step + false_branch_step

My solution:

false_branch_step = Configuration([dict(name="False branch step", command=["echo $?"])])
condition_step = If([dict(name="Condition step", command=["./test.sh"]), else=false_branch_step])

configs = condition_step

Inline version:

condition_step = If([dict(name="Condition step", command=["./test.sh"]), 
    else=Configuration([dict(name="False branch step", command=["echo $?"])])])

configs = condition_step

More complex scenario, with both "true" and "false" branches.

true_branch_step = Configuration([dict(name="True branch step", command=["ll"])])
false_branch_step = Configuration([dict(name="False branch step", command=["echo $?"])])
condition_step = If([dict(name="Condition step", command=["./test.sh"]), then=true_branch_step, else=false_branch_step])

configs = condition_step

Inline version:

condition_step = If([dict(name="Condition step", command=["./test.sh"]), 
    then=Configuration([dict(name="True branch step", command=["ll"])]), 
    else=Configuration([dict(name="False branch step", command=["echo $?"])])])

configs = condition_step

[miltolstoy]

Let's continue conditional steps discussion under https://github.com/Samsung/Universum/issues/709.