terraform
terraform copied to clipboard
Concurrency-safe unpacking of TF providers.
The original implementation unpacked the downloaded provider .zip files directly in the Terraform plugin cache directory. This made the terraform init
command prone to race conditions when multiple Terraform modules using the same cache were downloading, checksumming, and validating the provider files at the same time. This is a serious problem in CI workflows, as it forces developers to choose between two unattractive options:
- initializing modules serially and using the cache, or
- initializing modules in parallel, but download the same modules every time, increasing the consumed bandwidth.
This change unpacks the .zip files in a temporary directory with a unique name inside the plugin cache directory, and only then moves the files to the expected location.
It also takes a file lock on the target cache directory to prevent two Terraform processes from installing the same provider and conflicting with each other.
This change is a simpler version of this change by @mplzik, rebased on the latest main
branch: https://github.com/hashicorp/terraform/pull/33479
Fixes https://github.com/hashicorp/terraform/issues/31964
Target Release
1.11.x
Draft CHANGELOG entry
BUG FIXES
- Concurrent write access to plugin cache directory has been fixed.