Very high RAM usage for big configuration passing from 4.28.0 to 4.29.0 [Bug]:

[Roxyrob]

Terraform Core Version

1.3.9

AWS Provider Version

4.29.0

Affected Resource(s)

• machine RAM

Expected Behavior

In the same environment (devops machine, ...) and with the same terraform config I expect same behavior of terraform apply in terms of time and devops machine resource usage

Actual Behavior

On the same terraform configuration (big tfvar for hybrid DNS - lot of r53 resolver rules, private zones, etc.) upgrading from aws provider 4.19.0 to 4.55.0 introduce a large increase of devops machine resource especially of RAM (from 6GB with 4.19.0 to 11GB) in the early stage of "terraform apply -input=false", before refreshing resourses:

MiB Mem :  30392.6 total,  14788.6 free,  13362.5 used,   2241.5 buff/cache
MiB Swap:    820.0 total,    112.9 free,    707.1 used.  16572.8 avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
2108532 roberto+  20   0 2998152   2.2g  41812 S  13.0   7.4   2:03.78 terraform
2110315 roberto+  20   0 1109828 217780  98076 S   0.0   0.7   0:01.40 terraform-provi
2110341 roberto+  20   0 1110532 216624 102476 S   0.0   0.7   0:02.23 terraform-provi
2110168 roberto+  20   0 1110084 209324  97740 S   3.0   0.7   0:01.70 terraform-provi
2110352 roberto+  20   0 1110036 209196 100108 S   0.0   0.7   0:01.29 terraform-provi
2110224 roberto+  20   0 1110100 206932 100428 S   0.0   0.7   0:01.39 terraform-provi
2110279 roberto+  20   0 1109716 205060  97676 S   0.0   0.7   0:01.18 terraform-provi
2110087 roberto+  20   0 1109652 204668  98696 S   0.0   0.7   0:01.02 terraform-provi
2110461 roberto+  20   0 1109332 204308  99532 S   0.0   0.7   0:00.87 terraform-provi
...

High terraform process memory usage and lot of terraform-provi consuming all together a lot of memory. I tried different aws provider versions and identified as breaking changes 4.29.0 with 4.55.0 further worsened.

Relevant Error/Panic Output Snippet

I was forced to increase devops machine RAM to 30GB to avoid crash with version 4.55.0
aws versions >=4.29.0 are slower then <=4.28.0

Terraform Configuration Files

In attach terraform configuration file encrypted with hashicorp gpg pub key foun here HashiCorp's PGP Public Key. I hope this is the right one.

Terraform configuration uses private modules, but the huge part described above can be isolated for testing (route53)

I get:

run with aws provider version 4.28.0
real    2m26.538s
user    4m30.834s
sys     0m30.367s

run with aws provider version 4.55.0
real    2m54.931s
user   5m18.609s
sys     0m37.354s

(attached - is encrypted so remove last .zip) tf config: account-mgnt.zip.gpg.zip

Steps to Reproduce

terraform init terraform apply -input=false

Debug Output

(attached - is encrypted so remove last .zip) trace aws 4.28.0: account-mgnt_4.28.0.log.zip.gpg.zip

(attached - is encrypted so remove last .zip) trace aws 4.55.0: account-mgnt.4.55.0.log.zip.gpg.zip

Panic Output

n/a

Important Factoids

n/a

References

n/a

Would you like to implement a fix?

No

[github-actions[bot]]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[gavinelder]

Hi @Roxyrob could you look at and upvote https://github.com/hashicorp/terraform-provider-aws/issues/26742 which I believe is the same issue

[Roxyrob]

Hi @gavinelder voted same issue #26742 thanks

[Roxyrob]

Closed as same issue of https://github.com/hashicorp/terraform-provider-aws/issues/26742

[github-actions[bot]]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.