[Feature Request] DABs option to configure restart of streaming jobs and pipelines during bundle deploy

[kimberlyma]

Request Description:

Some developers would like a way to configure an automatic restart of their streaming jobs and pipelines upon deployment. This could change based on the environment (ex - always restart in production).

What is available today:

Today this can be done with thebundle run --restart subcommand after you deploy. In order to get all the relevant resources you need to write a custom bash script to run the command for each.

Terraform has control_run_state for continuous DLT pipelines (docs), but this only captures DLT configuration changes (not non-DLT jobs, not code changes).

[pietern]

Thanks for the suggestion.

This has come up before and the reason we haven't done this yet is that the right thing is ambiguous. Some users expect only changed pipelines or jobs to be restarted, whereas others expect all pipelines or jobs to be restarted. Whether or not they have changed depends on the configuration, the code they reference, etc.

We can consider capturing this at the pipeline or job level as a parameter to always restart on redeploy. But given that composition with bundle run --restart makes this possible today, it does not feel super high impact.

Can you elaborate on how you would use this? For example, would you always restart everything under management?

[github-actions[bot]]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.