Import resources in parallel

[magodo]

Currently in resource group mode, the resources are imported one by one. This is because terraform locks the state file during importing a single resource.

Since terraform v1.1.0, the terraform state mv command can be used to move resources among different state file. With that, we can import resources in parallel to different working directories. After that, using terraform state mv to move them into a single state file. This applies to both local and remote backends.

There is a experimental implementation about the workflow in https://github.com/magodo/tfmerge (though it only supports local backend for now). After it goes stable, we can move on integrating it into aztfy.

[stemaMSFT]

What thinking is there around how many resources do you think can be imported in parallel at once? For example, for 16 resources, 16 different state files seem a lot but 1 seems a lot less. Curious as I'm sure there's an algorithmic way to optimize/determine the number of parallel resource imports. Another thought is recording historical telemetry around resource import time to better optimize, and also provide insights into bottlenecking around speed/performance.

[magodo]

@stemaMSFT We can expose an option -parallelism to allow users to change the parallelism, which defaults to 10. This would align with terraform's CLI.