Add approve stage to deploys?

[Fizzadar]

pyinfra's two-stage execution model means it could behave like other tools in the infrastructure space by not automatically executing. Thinking of terraform & pulumi in particular.

So my question: should pyinfra have an approval pause before execution that shows the operation summary as currently, but waits for the user to confirm, before execution. Combined with a -y/--yes argument, of course.

[themanifold]

Yes, I think this is a good idea. You could even make it two completely separate steps: pyinfra plan and pyinfra apply just like those tools you mentioned.

[Fizzadar]

I like the two step idea, currently thinking of having the following variants:

# Run plan & apply, with a prompt to approve (or -y/--yes)
pyinfra INVENTORY COMMANDS...

# Run plan, with optional -o/--output [filename.json], default stdout
pyinfra plan INVENTORY COMMANDS...

# Run apply, based on output from plan
pyinfra apply operations.json

# Should also work:
pyinfra plan INVENTORY COMMANDS | pyinfra apply -

The first variant is "classic pyinfra" and the plan/apply are more specific. I also think apply should always take an input file of operations upfront. This could enable some interesting workflows where the JSON could be used to diff remote hosts and/or plan and apply could be done on a different machines (is that useful?).

It's actually similar to a enhanced version of https://github.com/Fizzadar/pyinfra/issues/509.

[Fizzadar]

For context: I'm thinking this is pyinfra v3 territory, and v2 will be breaking for removing python2 + deprecated features.

[Fizzadar]

Another note on this (per https://github.com/Fizzadar/pyinfra/issues/686) - not everything is simple to output into JSON - in particular Python functions. It is possible with pickle, but that is potentially a dangerous road to go down. Perhaps a simple solution to that is to allow serializing them, but upon deserialize have some kind of --allow-unsafe-pickled-opeartions flag or similar.

[themanifold]

This could enable some interesting workflows where the JSON could be used to diff remote hosts and/or plan and apply could be done on a different machines (is that useful?).

Yes, this is how some modern "gitops" stuff is done with terraform.

not everything is simple to output into JSON - in particular Python functions.

In what cases will we have to pickle python functions?

[Fizzadar]

In what cases will we have to pickle python functions?

Callback functions from the python.* operations are the only place I believe, but I think with a suitably scary sounding flag this is acceptable. It's a pretty rare operation call so for the most part unlikely to be any issue.

[skutter-de]

How about implementing a switch in the deploy where the deploy developer can ask for explicit approval. Also, I think all cases where data has to be interactively entered into the program/deploy should be asked before asking for approval so you can just type in what you need, then approve, then go to something else while the deploy is running.