pre-commit-terraform icon indicating copy to clipboard operation
pre-commit-terraform copied to clipboard
verified publisher icon antonbabenko

`terraform_providers_lock` produces error when `terraform_validate` is not called before

Open jaygridley opened this issue 1 month ago • 9 comments

With introduction of mode support to terraform_providers_lock hooks it should be possible to run the hook without terraform_validate when mode=only-check-is-current-lockfile-cross-platform, but it is not working as advertised.

It works as advertised only when lockfile_contains_all_needed_sha returns 0 = when the lock file is valid. If it NOT valid, for example it is missing hash for some platform, the condition https://github.com/antonbabenko/pre-commit-terraform/blob/cba87736ffe2c479523d9d69919c1215084c7acc/hooks/terraform_providers_lock.sh#L150 fails and code continues and runs terraform providers lock on line https://github.com/antonbabenko/pre-commit-terraform/blob/cba87736ffe2c479523d9d69919c1215084c7acc/hooks/terraform_providers_lock.sh#L158.

IMHO the condition could be rewritten to:

if [ "$mode" == "only-check-is-current-lockfile-cross-platform" ] ; then
    lockfile_contains_all_needed_sha "$platforms_count"

    exit_code=$?
    return $exit_code
  fi

In this case it could be executed in mode=only-check-is-current-lockfile-cross-platform without running terraform_validate which runs terraform init under the hood.

jaygridley avatar Nov 21 '25 15:11 jaygridley

but it is not working as advertised.

Could you please share: 1) your config of this particular hook, 2) expected behavior and 3) actual behavior? Thanks.

yermulnik avatar Nov 21 '25 17:11 yermulnik

  1. Config
  - repo: https://github.com/antonbabenko/pre-commit-terraform
    rev: v1.96.3
    hooks:
      - id: terraform_providers_lock
        args:
          - --hook-config=--mode=only-check-is-current-lockfile-cross-platform
          - --args=-platform=darwin_amd64
          - --args=-platform=darwin_arm64
          - --args=-platform=linux_amd64
  1. Expected behaviour
Lock terraform provider versions.........................................Failed
- hook id: terraform_providers_lock
- exit code: 152

| Error: <PATH TO THE LOCK FILE> missing checksum for platform X, Y, Z
  1. Actual behaviour
Lock terraform provider versions.........................................Failed
- hook id: terraform_providers_lock
- exit code: 1

╷
│ Error: Module not installed
│ 
│   on grafana.tf line 1:
│    1: module "grafana" {
│ 
│ This module is not yet installed. Run "terraform init" to install all modules required by this configuration.
╵

jaygridley avatar Nov 21 '25 19:11 jaygridley

Thanks for details.

IMHO the condition could be rewritten to:

I guess I see the point and I'm inclined to agree that the condition should look like:

if [ "$mode" == "only-check-is-current-lockfile-cross-platform" ]; then
  lockfile_contains_all_needed_sha "$platforms_count"
  exit_code=$?
  return $exit_code
fi

@MaxymVlasov Could you please take a look into this as I can't say for sure whether there is no other use case for lockfile_contains_all_needed_sha() here except for when $mode equals only-check-is-current-lockfile-cross-platform). Thanks.

yermulnik avatar Nov 23 '25 16:11 yermulnik 

3. Actual behaviour

Lock terraform provider versions.........................................Failed
- hook id: terraform_providers_lock
- exit code: 1

╷
│ Error: Module not installed
│ 
│   on grafana.tf line 1:
│    1: module "grafana" {
│ 
│ This module is not yet installed. Run "terraform init" to install all modules required by this configuration.
╵

It's terraform behavior. You must run t init if you want to validate checksums as long as there at least 1 child module imported inside your root module

2. Expected behaviour

Lock terraform provider versions.........................................Failed
- hook id: terraform_providers_lock
- exit code: 152

| Error: <PATH TO THE LOCK FILE> missing checksum for platform X, Y, Z

Error: <PATH TO THE LOCK FILE> missing checksum for platform - that's can be done, but only after running t init

Before - that all you can get:

TF_LOG=trace terraform providers lock \
  -platform=darwin_amd64 \
  -platform=darwin_arm64
2025-11-26T16:46:55.656+0200 [INFO]  Terraform version: 1.10.5
2025-11-26T16:46:55.656+0200 [DEBUG] using github.com/hashicorp/go-tfe v1.70.0
2025-11-26T16:46:55.656+0200 [DEBUG] using github.com/hashicorp/hcl/v2 v2.23.0
2025-11-26T16:46:55.656+0200 [DEBUG] using github.com/hashicorp/terraform-svchost v0.1.1
2025-11-26T16:46:55.656+0200 [DEBUG] using github.com/zclconf/go-cty v1.16.2
2025-11-26T16:46:55.656+0200 [INFO]  Go runtime version: go1.23.3
2025-11-26T16:46:55.656+0200 [INFO]  CLI args: []string{"/home/vm/.asdf/installs/terraform/1.10.5/bin/terraform", "providers", "lock", "-platform=darwin_amd64", "-platform=darwin_arm64"}
2025-11-26T16:46:55.656+0200 [TRACE] Stdout is a terminal of width 96
2025-11-26T16:46:55.656+0200 [TRACE] Stderr is a terminal of width 96
2025-11-26T16:46:55.656+0200 [TRACE] Stdin is a terminal
2025-11-26T16:46:55.656+0200 [DEBUG] Attempting to open CLI config file: /home/vm/.terraformrc
2025-11-26T16:46:55.656+0200 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2025-11-26T16:46:55.656+0200 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2025-11-26T16:46:55.656+0200 [DEBUG] ignoring non-existing provider search directory /home/vm/.terraform.d/plugins
2025-11-26T16:46:55.656+0200 [DEBUG] ignoring non-existing provider search directory /home/vm/snap/code/214/.local/share/terraform/plugins
2025-11-26T16:46:55.656+0200 [DEBUG] ignoring non-existing provider search directory /home/vm/snap/code/214/.local/share/terraform/plugins
2025-11-26T16:46:55.656+0200 [DEBUG] ignoring non-existing provider search directory /home/vm/snap/code/214/terraform/plugins
2025-11-26T16:46:55.656+0200 [DEBUG] ignoring non-existing provider search directory /snap/code/214/usr/share/terraform/plugins
2025-11-26T16:46:55.656+0200 [DEBUG] ignoring non-existing provider search directory /home/vm/.local/share/flatpak/exports/share/terraform/plugins
2025-11-26T16:46:55.656+0200 [DEBUG] ignoring non-existing provider search directory /var/lib/flatpak/exports/share/terraform/plugins
2025-11-26T16:46:55.656+0200 [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins
2025-11-26T16:46:55.656+0200 [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins
2025-11-26T16:46:55.656+0200 [DEBUG] ignoring non-existing provider search directory /var/lib/snapd/desktop/terraform/plugins
2025-11-26T16:46:55.657+0200 [INFO]  CLI command args: []string{"providers", "lock", "-platform=darwin_amd64", "-platform=darwin_arm64"}
╷
│ Error: Module not installed
│ 
│   on main.tf line 1:
│    1: module "environment" {
│ 
│ This module is not yet installed. Run "terraform init" to install all modules required by
│ this configuration.

If by X, Y, Z you want to have specific values, not generic extraction of provided args - it's not possible before t init too, but after - it could be extracted from regular logs

$ t init
  ...
$terraform providers lock \ 
  -platform=darwin_amd64 \
  -platform=linux_amd64
- Fetching hashicorp/aws 5.36.0 for darwin_amd64...
- Retrieved hashicorp/aws 5.36.0 for darwin_amd64 (signed by HashiCorp)
- Fetching hashicorp/template 2.2.0 for darwin_amd64...
- Retrieved hashicorp/template 2.2.0 for darwin_amd64 (signed by HashiCorp)
- Fetching hashicorp/random 3.6.0 for darwin_amd64...
- Retrieved hashicorp/random 3.6.0 for darwin_amd64 (signed by HashiCorp)
- Fetching hashicorp/null 3.2.2 for darwin_amd64...
- Retrieved hashicorp/null 3.2.2 for darwin_amd64 (signed by HashiCorp)
- Fetching hashicorp/archive 2.4.2 for darwin_amd64...
- Retrieved hashicorp/archive 2.4.2 for darwin_amd64 (signed by HashiCorp)
- Fetching hashicorp/random 3.6.0 for linux_amd64...
- Retrieved hashicorp/random 3.6.0 for linux_amd64 (signed by HashiCorp)
- Fetching hashicorp/null 3.2.2 for linux_amd64...
- Retrieved hashicorp/null 3.2.2 for linux_amd64 (signed by HashiCorp)
- Fetching hashicorp/archive 2.4.2 for linux_amd64...
- Retrieved hashicorp/archive 2.4.2 for linux_amd64 (signed by HashiCorp)
- Fetching hashicorp/aws 5.36.0 for linux_amd64...
- Retrieved hashicorp/aws 5.36.0 for linux_amd64 (signed by HashiCorp)
- Fetching hashicorp/template 2.2.0 for linux_amd64...
- Retrieved hashicorp/template 2.2.0 for linux_amd64 (signed by HashiCorp)
- Obtained hashicorp/null checksums for darwin_amd64; Additional checksums for this platform are now tracked in the lock file
- Obtained hashicorp/null checksums for linux_amd64; All checksums for this platform were already tracked in the lock file
- Obtained hashicorp/archive checksums for darwin_amd64; Additional checksums for this platform are now tracked in the lock file
- Obtained hashicorp/archive checksums for linux_amd64; All checksums for this platform were already tracked in the lock file
- Obtained hashicorp/aws checksums for darwin_amd64; Additional checksums for this platform are now tracked in the lock file
- Obtained hashicorp/aws checksums for linux_amd64; All checksums for this platform were already tracked in the lock file
- Obtained hashicorp/template checksums for darwin_amd64; All checksums for this platform were already tracked in the lock file
- Obtained hashicorp/template checksums for linux_amd64; All checksums for this platform were already tracked in the lock file
- Obtained hashicorp/random checksums for linux_amd64; All checksums for this platform were already tracked in the lock file
- Obtained hashicorp/random checksums for darwin_amd64; Additional checksums for this platform are now tracked in the lock file

Success! Terraform has updated the lock file.

Review the changes in .terraform.lock.hcl and then commit to your
version control system to retain the new checksums.

Do you actually asking about returning t init run inside terraform_providers_lock hook for cases when there is no valid ./terraform/ content AND do it w/o checking is terraform code is actually valid to produce valid ./terraform/ (done by terraform_validate)? Or do you suggest improve docs about this behavior?

MaxymVlasov avatar Nov 26 '25 14:11 MaxymVlasov

I think if terraform_providers_lock requires t init in some cases there should be a way how to run it. Possibly adding terraform_init hook that can be used internally by terraform_validate and optionally with terraform_providers_lock?

Or just improving lockfile_contains_all_needed_sha function to return "more rich error" containing at least the lock file, it was checking? This is probably related to https://github.com/antonbabenko/pre-commit-terraform/issues/54

jaygridley avatar Nov 27 '25 09:11 jaygridley

Improving lockfile_contains_all_needed_sha function to return "more rich error" containing at least the lock file

We can get it from

https://github.com/antonbabenko/pre-commit-terraform/blob/cba87736ffe2c479523d9d69919c1215084c7acc/hooks/terraform_providers_lock.sh#L94-L95

and post under if exist_code != 0 right before return https://github.com/antonbabenko/pre-commit-terraform/blob/cba87736ffe2c479523d9d69919c1215084c7acc/hooks/terraform_providers_lock.sh#L158-L162

Possibly adding terraform_init hook that can be used internally by terraform_validate and optionally with terraform_providers_lock?

That's already in place, rn called function common::terraform_init

I think if terraform_providers_lock requires t init in some cases there should be a way how to run it.

Using terraform_validate hook? Can you please describe use case in which anyone will want to skip terraform validation step?

Technically, errors are the same, but it will be little weird to folks who don't know how TF works to see such errors from provider locks operation

14:38 Oslo/environment/qa git:(test $✘!+?) 
➜ terraform providers lock \                                         
  -platform=darwin_amd64 \
  -platform=linux_amd64
╷
│ Error: Unsupported block type
│ 
│   on providers.tf line 1:
│    1: provi der "aws" {
│ 
│ Blocks of type "provi" are not expected here.
╵


14:38 Oslo/environment/qa git:(test $✘!+?) 
✘1 ➜ t validate                                                         
╷
│ Error: Unsupported block type
│ 
│   on providers.tf line 1:
│    1: provi der "aws" {
│ 
│ Blocks of type "provi" are not expected here.
╵

14:38 Oslo/environment/qa git:(test $✘!+?) 
✘1 ➜ t init                                                             
Initializing the backend...
Initializing modules...
╷
│ Error: Unsupported block type
│ 
│   on providers.tf line 1:
│    1: provi der "aws" {
│ 
│ Blocks of type "provi" are not expected here.
╵

MaxymVlasov avatar Nov 27 '25 12:11 MaxymVlasov

Using terraform_validate hook? Can you please describe use case in which anyone will want to skip terraform validation step?

Actually, we skip it because it needs t init which is slow for our use-case.

I am fine with my original suggestion and add extra output with lock file being checked as you suggested.

jaygridley avatar Nov 27 '25 14:11 jaygridley

@jaygridley please check is next works for you

- repo: https://github.com/antonbabenko/pre-commit-terraform
  rev: f74b9d750396da7ac2b40b99d72d80759576e1ba

PR - https://github.com/antonbabenko/pre-commit-terraform/pull/950

MaxymVlasov avatar Nov 28 '25 17:11 MaxymVlasov

Hi @MaxymVlasov, the https://github.com/antonbabenko/pre-commit-terraform/pull/950 solves only part of the issue. I am still missing a way how to run lockfile_contains_all_needed_sha function without invoking Terraform whatsoever.

I could be achieved by changing the logic

if [ "$mode" == "only-check-is-current-lockfile-cross-platform" ] ; then
    lockfile_contains_all_needed_sha "$platforms_count"

    exit_code=$?
 
    if [[ $exit_code -ne 0 ]]; then
      common::colorify "red" "$dir_path/.terraform.lock.hcl missing all specified platforms"
    fi

    return $exit_code
  fi

jaygridley avatar Dec 02 '25 07:12 jaygridley