cfn-lint icon indicating copy to clipboard operation
cfn-lint copied to clipboard

feature request: distribute work over different cpu cores

Open OlafConijn opened this issue 4 years ago • 4 comments

cfn-lint version: (0.39)

Description of issue.

I have a build process that i made cfn-lint part of. the build process outputs a rather large number of cloudformation files. as a result running cfn-lint takes quite a bit of time.

OlafConijn avatar Nov 03 '20 12:11 OlafConijn

agreed. Started looking into this.

kddejong avatar Nov 04 '20 22:11 kddejong

Started doing some performance checks and determining where I would want to split out over cores. My going in assumption was spreading the rule executions against the cores was the way to go. However after evaluating some performance measures I noticed some inefficiencies in the rule loading process in relation to executing cfn-lint against many templates. Going to work on that first. From there I will work to determine where it makes sense to split across the cores.

kddejong avatar Nov 18 '20 17:11 kddejong

Great! Would be happy to help by providing performance measurements or otherwise 👍

OlafConijn avatar Nov 18 '20 19:11 OlafConijn

@OlafConijn The first part of this has been merged. Now we are on to figuring out how to spread across many cores. Originally I was thinking of doing this per rule but now I think it would be easier to do per template.

kddejong avatar Jan 07 '21 20:01 kddejong