terraform validate fails in pre-commit but not in docker container

[tolidano]

Describe the bug

Terraform validate.......................................................Failed
- hook id: terraform_validate
- exit code: 1

Validation failed: tf/modules/core/vpc

Error: Module source has changed

  on endpoints.tf line 2, in module "vpc_endpoints":
   2:   source  = "terraform-aws-modules/vpc/aws//modules/vpc-endpoints"

The source address was changed since this module was installed. Run "terraform
init" to install all modules required by this configuration.


Error: Module source has changed

  on vpc.tf line 4, in module "vpc":
   4:   source  = "terraform-aws-modules/vpc/aws"

The source address was changed since this module was installed. Run "terraform
init" to install all modules required by this configuration.


Lock terraform provider versions.....................(no files to check)Skipped

/lint/tf/modules/core/vpc > terraform validate
╷
│ Warning: Argument is deprecated
│
│   with module.s3.aws_s3_bucket.bucket,
│   on ../../../modules/core/s3/main.tf line 2, in resource "aws_s3_bucket" "bucket":
│    2: resource "aws_s3_bucket" "bucket" {
│
│ Use the aws_s3_bucket_acl resource instead
│
│ (and 17 more similar warnings elsewhere)
╵
Success! The configuration is valid, but there were some validation warnings as shown above.

How can we reproduce it?

Create a module that uses the terraform-aws-vpc module? It's not clear to me it will happen on each run of such.

Environment information

• OS: MacOS

• docker info:

docker for mac

• Docker image tag/git commit:

custom, based on your dockerfile with a small edit to pre-install the hooks

• Tools versions. Don't forget to specify right tag in command -
  TAG=latest && docker run --entrypoint cat pre-commit:$TAG /usr/bin/tools_versions_info

/lint/tf/modules/core/vpc > cat  /usr/bin/tools_versions_info
pre-commit 2.20.0
Terraform v1.2.6
checkov 2.1.82
infracost SKIPPED
terraform-docs version v0.16.0 1f686b1 linux/amd64
terragrunt SKIPPED
terrascan version: v1.15.2
TFLint version 0.39.1
tfsec v1.26.3
tfupdate 0.6.5
hcledit SKIPPED

• .pre-commit-config.yaml:

- repo: https://github.com/antonbabenko/pre-commit-terraform
   rev: v1.74.1
   hooks:
     - id: terraform_fmt
       name: Terraform fmt
       description: Rewrites all Terraform configuration files to a canonical format.
       entry: hooks/terraform_fmt.sh
       language: script
       files: (\.tf|\.tfvars)$
       exclude: \.terraform\/.*$

     - id: terraform_docs
       name: Terraform docs
       description: Inserts input and output documentation into README.md (using terraform-docs).
       require_serial: true
       entry: hooks/terraform_docs.sh
       language: script
       files: (\.tf|\.terraform\.lock\.hcl)$
       exclude: \.terraform\/.*$
       args:
         - --hook-config=--path-to-file=VARIABLES.md
         - --hook-config=--create-file-if-not-exist=true

     - id: terraform_validate
       name: Terraform validate
       description: Validates all Terraform configuration files.
       require_serial: true
       entry: hooks/terraform_validate.sh
       language: script
       files: (\.tf|\.tfvars)$
       exclude: \.terraform\/.*$

- id: terraform_providers_lock
       name: Lock terraform provider versions
       description: Updates provider signatures in dependency lock files.
       require_serial: true
       entry: hooks/terraform_providers_lock.sh
       language: script
       files: (\.terraform\.lock\.hcl)$
       exclude: \.terraform\/.*$

     - id: terraform_tflint
       name: Terraform validate with tflint
       description: Validates all Terraform configuration files with TFLint.
       require_serial: true
       entry: hooks/terraform_tflint.sh
       language: script
       files: (\.tf|\.tfvars)$
       exclude: \.terraform\/.*$

     - id: terraform_tfsec
       name: Terraform validate with tfsec
       description: Static analysis of Terraform templates to spot potential security issues.
       require_serial: true
       entry: hooks/terraform_tfsec.sh
       language: script

     - id: terraform_checkov
       name: Checkov
       description: Runs checkov on Terraform templates.
       entry: hooks/terraform_checkov.sh
       language: script
       always_run: false
       files: \.tf$
       exclude: \.terraform\/.*$
       require_serial: true
       args:
         - --args=--quiet
         - --args=--download-external-modules true
         - --args=--skip-check CKV2_AWS_11,CKV2_AWS_12,CKV2_AWS_19
   
     - id: tfupdate
       name: tfupdate
       description: Runs tfupdate on Terraform templates.
       language: script
       entry: hooks/tfupdate.sh
       args:
         - --args=terraform
       files: \.tf$
       require_serial: true

Waiting for someone to look at my terrascan PR before re-enabling that.

[MaxymVlasov]

Waiting for someone to look at my terrascan PR before re-enabling that.

Sorry, which PR and how does it relates issue described here about terraform_validate ?

So, we expect that terraform_validate will work the same on Docker and host machine if all deps are the same. I see deps for Docker image, but not for your MacOS. Please provide them by:

bash << EOF
bash --version | head -n 1                2>/dev/null || echo "bash SKIPPED"
pre-commit --version                      2>/dev/null || echo "pre-commit SKIPPED"
terraform --version | head -n 1           2>/dev/null || echo "terraform SKIPPED"
python --version                          2>/dev/null || echo "python SKIPPED"
python3 --version                         2>/dev/null || echo "python3 SKIPPED"
echo -n "checkov " && checkov --version   2>/dev/null || echo "checkov SKIPPED"
terraform-docs --version                  2>/dev/null || echo "terraform-docs SKIPPED"
terragrunt --version                      2>/dev/null || echo "terragrunt SKIPPED"
echo -n "terrascan " && terrascan version 2>/dev/null || echo "terrascan SKIPPED"
tflint --version                          2>/dev/null || echo "tflint SKIPPED"
echo -n "tfsec " && tfsec --version       2>/dev/null || echo "tfsec SKIPPED"
echo -n "tfupdate " && tfupdate --version 2>/dev/null || echo "tfupdate SKIPPED"
echo -n "hcledit " && hcledit version     2>/dev/null || echo "hcledit SKIPPED" 
EOF

Also, you didn't specify which installation, docker or local, you think are works not as expected

[tolidano]

I think the configuration is valid, I did not move the configuration source. What you are looking at is ALWAYS in docker. When you see the Failed line and the Module Source message, that's the pre-commit hook firing the docker container.

When you see the terraform_validate, it's also inside the container (I exec'ed into it).

My local installation of any tools is never used.

This was the PR I was referencing: https://github.com/tenable/terrascan/pull/1343

[MaxymVlasov]

Can't reproduce