cfn-lint
cfn-lint copied to clipboard
aws-cloudformation
--update-specs is taking ~30 minutes to run
CloudFormation Lint Version
cfn-lint 0.76.2
What operating system are you using?
Linux
Describe the bug
For the past few weeks, running cfn-lint --update-specs has taken roughly 30 minutes every time.
Expected behavior
--update-specs should not take 30 minutes
Reproduction template
N/A
My update took about 5 minutes but it had all 8 of my CPUs pegged at 100%.
OS: Ubuntu 22.04 LTS
Linux deepthought 5.19.0-42-generic #43~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Apr 21 16:51:08 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
Same issue... At 30 minutes now and climbing... What does -u actually do? I can't imagine it's a heavy process, this should be pulling in template/json files that live in a shared S3 bucket or are behind CloudFront.
Someone please look into.
It looks like the cloudspecs locations are here:
https://github.com/aws-cloudformation/cfn-lint/blob/main/src/cfnlint/helpers.py
So I suppose a quick and dirty hack could be to just write a script that grabs those, and shoves them into src/cfn-lint/data/CloudSpecs/ folder
Don't try this on a production server, as I just glanced and didn't dig deep enough to know if this would harm the app.
Although, it looks like the specs in this repo were updated 3 days ago, so if you just want to delete and re-install cfn-lint that might be quickest path at the moment.
I install it into a docker container for my build agents, and I update that docker once a month, so I'll just for now, trust that these files get updated on the install and stop wasting my time on running -u ...
Yea, been thinking of some ideas. I think the short term solution would be to restrict what regions we are updating.
I will say I'm seeing a significant improvement in timing with v1 (currently pre-release).
i restricted regions to us-east-2 and it didnt help our team. excited for the v1 fix
Yea, sorry currently the region parameter doesn't apply to the --update-specs. I'm curious about how often you are running this? Specs are updated weekly and we try to release cfn-lint weekly with those updates.
A short term fix would be to account for the region parameter into the --update-specs but we would have to implement it.
We have it in our CI/CD pipeline to run on deployments to Dev/QA/Prod.. short term fix would be to turn it off for dev, of course.
We have it in our CI/CD pipeline to run on deployments to Dev/QA/Prod.. short term fix would be to turn it off for dev, of course.
One option, as I do in my pipelines, I update my code build image weekly via a cron job or manually run the image builder project as needed.
