tflint
tflint copied to clipboard
Recursive inspection in parallel
Fixes https://github.com/terraform-linters/tflint/issues/1943
As explained in #1943, this PR parallelizes recursive inspection by multi-processes.
The process that runs tflint --recursive is called the coordinator, and it starts a worker process for each directory. The worker process executes with --chdir in each directory and outputs issues in the dedicated JSON format, just like a normal TFLint command. The coordinator aggregates JSON output from each process via stdout, and provides feedback to the user.
flowchart TD
User -- "tflint --recursive" --> Coordinator
Coordinator -- "tflint --act-as-worker --chdir=subdir1" --> Worker1
Coordinator -- "tflint --act-as-worker --chdir=subdir2" --> Worker2
Coordinator -- "tflint --act-as-worker --chdir=subdir3" --> Worker3
The default parallelism is the number of CPUs. This can be controlled with the --max-workers flag, but should not need to be changed in most cases. In my tested repository (over 200 root modules with calls to over 40 child modules), 4 cores cut the time in half.
Before
$ time tflint --recursive
real 5m10.744s
user 5m53.837s
sys 2m36.451s
After
$ time tflint --recursive
real 2m24.805s
user 6m50.139s
sys 2m33.531s