Cannot locate module locally

[solarmosaic-kflorence]

Prerequisites

• [x] Ensure you have latest version of plugin installed
• [x] Search for possible issue duplicates - related to closed issue https://github.com/VladRassokhin/intellij-hcl/issues/170

Installation details

• [x] IDE version (Help->About->Copy to Clipboard)

  IntelliJ IDEA 2021.2 (Ultimate Edition)
  Build #IU-212.4746.92, built on July 27, 2021
  Runtime version: 11.0.11+9-b1504.13 x86_64
  VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
  macOS 10.15.7
  GC: ParNew, ConcurrentMarkSweep
  Memory: 4029M
  Cores: 8
  Registry: scala.erase.compiler.process.jdk.once=false
  Non-Bundled Plugins: org.intellij.plugins.hcl (0.7.14), mobi.hsz.idea.gitignore (4.2.0), name.kropp.intellij.makefile (212.4746.52), org.toml.lang (0.2.152.4026-212), org.jetbrains.plugins.hocon (2021.1.0), intellij.prettierJS (212.4746.57), Pythonid (212.4746.96), org.jetbrains.plugins.go (212.4746.92), org.intellij.scala (2021.2.15), pro.bashsupport (2.0.4.212)
  Kotlin: 212-1.5.10-release-IJ4746.92
  

• [x] intellij-hcl plugin version (Settings->Plugins) - 0.7.14
• [x] Terraform version (terraform -v) - v1.0.10 (darwin_amd64)

Terraform Configuration Files

module "iam" {
  source = "s3::https://s3.amazonaws.com/path/to/module.zip"
}

And here is the project folder (including .terraform cache):

.
├── .terraform
│   ├── modules
│   │   ├── iam
│   │   │   ├── Makefile
│   │   │   ├── main.tf
│   │   │   ├── modules
│   │   │   │   ├── iam
│   │   │   │   │   ├── .terraform.lock.hcl
│   │   │   │   │   ├── main.tf
│   │   │   │   │   ├── outputs.tf
│   │   │   │   │   └── variables.tf
│   │   │   │   ├── local-data
│   │   │   │   │   └── outputs.tf
│   │   │   │   └── path
│   │   │   │       ├── outputs.tf
│   │   │   │       └── variables.tf
│   │   │   ├── output.tf
│   │   │   ├── policies
│   │   │   │   └── assume-role
│   │   │   │       └── codebuild.json
│   │   │   ├── variables.tf
│   │   │   └── versions.tf
│   │   ├── iam.path.path
│   │   │   ├── go.mod
│   │   │   ├── go.sum
│   │   │   ├── infra
│   │   │   ├── output.tf
│   │   │   ├── readme.md
│   │   │   └── variables.tf
│   │   ├── iam.session
│   │   │   ├── Makefile
│   │   │   ├── main.tf
│   │   │   ├── modules
│   │   │   │   └── local-data
│   │   │   │       └── outputs.tf
│   │   │   ├── outputs.tf
│   │   │   └── versions.tf
│   │   └── modules.json
│   └── providers
│       └── registry.terraform.io
│           └── hashicorp
│               └── aws
│                   └── 3.58.0
│                       └── darwin_amd64
│                           └── terraform-provider-aws_v3.58.0_x5
├── .terraform.lock.hcl
└── main.tf

Exception

N/A

Expected Behavior

The HCL plugin should be able to reference any module cached in the ./.terraform in the current directory. We are using Terraform inside of a monorepo, so we have several directories scattered around with Terraform configuration. I have noticed in some project folders we do not have this issue and in others we do. It seems to happen most often within sub-modules (e.g. ./modules/bar, relative to the project root), or when referencing sub-folders of a module, for example:

module "bar" {
  source = "s3::https://s3.amazonaws.com/path/to/module.zip//path/to/sub-module"
}

Actual Behavior

 Cannot locate module locally: Terraform Module 'iam' with source 's3::https://s3.amazonaws.com/path/to/module.zip' directory not found locally, use `terraform get` to fetch modules.

The module exists, of course, and is downloaded and cached in the .terraform folder. Performing Terraform CLI commands in the directory will succeed. It's just a problem with the plugin cache.

. <-- in this folder the module was resolved, and it also resolved it in the sub-folder
├── Makefile
├── main.tf
├── modules
│   ├── iam <-- the folder where the module cannot be resolved
│   │   └── main.tf
│   └── meta
│       └── outputs.tf
├── output.tf
├── readme.md
├── variables.tf
└── versions.tf

In the example we are downloading the module from S3, but it shouldn't matter what the source is. Notably, though, I haven't seen this happen with a local module reference (e.g. source = "../../path/to/module").

Steps to Reproduce

See the comment below https://github.com/VladRassokhin/intellij-hcl/issues/365#issuecomment-981972598

[solarmosaic-kflorence]

Here is a more thorough example. This is the project structure (this project exists in our monorepo repository at location: ./libraries/terraform/aws/bucket-object):

.
├── main.tf
├── modules
│   └── policies
│       ├── main.tf
│       ├── outputs.tf
│       └── variables.tf
├── outputs.tf
├── readme.md
├── variables.tf
└── versions.tf

Inside main.tf there is a module reference:

module "session" {
  source = "s3::https://s3.amazonaws.com/<REDACTED>/terraform/terraform-aws-session/terraform-aws-session_v0.2.2.zip"
}

Starting with no .terraform cache, I run terraform init in the root of the project...

➜  monorepo git:(feature/foo) cd libraries/terraform/aws/bucket-object
➜  bucket-object git:(feature/foo) terraform init
Initializing modules...
Downloading s3::https://s3.amazonaws.com/<REDACTED>/terraform/terraform-aws-session/terraform-aws-session_v0.2.2.zip for session...
- session in .terraform/modules/session

Initializing the backend...

Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Installing hashicorp/aws v3.61.0...
- Installed hashicorp/aws v3.61.0 (signed by HashiCorp)

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

The project .terraform cache folder now has the following structure:

.
├── modules
│   ├── modules.json
│   └── session
│       ├── main.tf
│       ├── modules
│       │   ├── meta
│       │   │   └── outputs.tf
│       │   └── policies
│       │       ├── main.tf
│       │       └── outputs.tf
│       ├── outputs.tf
│       └── versions.tf
└── providers
    └── registry.terraform.io
        └── hashicorp
            └── aws
                └── 3.61.0
                    └── darwin_amd64
                        └── terraform-provider-aws_v3.61.0_x5

The modules/modules.json file has the following content:

{"Modules":[{"Key":"","Source":"","Dir":"."},{"Key":"session","Source":"s3::https://s3.amazonaws.com/<REDACTED>/terraform/terraform-aws-session/terraform-aws-session_v0.2.2.zip","Dir":".terraform/modules/session"}]}

IntelliJ now correctly resolves this module from the cache, so the modules cache at the root of the project work fine. Now let's resolve the module references for the ./modules/policies sub-module, which contains the following in it's main.tf file:

module "session" {
  source = "s3::https://s3.amazonaws.com/<REDACTED>/terraform/terraform-aws-session/terraform-aws-session_v0.2.2.zip//modules/policies"
}

Again starting from having no .terraform cache, I navigate into the sub-module folder and run terraform init:

➜  bucket-object git:(feature/foo) cd modules/policies
➜  policies git:(feature/foo) terraform init
Initializing modules...
Downloading s3::https://s3.amazonaws.com/<REDACTED>/terraform/terraform-aws-session/terraform-aws-session_v0.2.2.zip for session...
- session in .terraform/modules/session/modules/policies

Initializing the backend...

Initializing provider plugins...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

The ./modules/policies/.terraform cache directory now has the following structure:

.
└── modules
    ├── modules.json
    └── session
        ├── main.tf
        ├── modules
        │   ├── meta
        │   │   └── outputs.tf
        │   └── policies
        │       ├── main.tf
        │       └── outputs.tf
        ├── outputs.tf
        └── versions.tf

The modules/modules.json file has the following content:

{"Modules":[{"Key":"","Source":"","Dir":"."},{"Key":"session","Source":"s3::https://s3.amazonaws.com/<REDACTED>/terraform/terraform-aws-session/terraform-aws-session_v0.2.2.zip//modules/policies","Dir":".terraform/modules/session/modules/policies"}]}

However, IntelliJ cannot resolve the module:

[ambis]

Exactly the same problem.

terraform plan works A-OK but PHPStorm doesn't find the module files

EDIT: This got fixed by just restarting the IDE.

[Xerkus]

I believe it worked with multiple terraform modules in same project before and now it can't locate modules for unknown reason for all but one module.

[mohamed-haidara-cko]

Not sure if it's the same cause but had the same issue with Terraform 1.1.0. They changed how they store the Source in .terraform/modules/modules.json causing my issue. See https://github.com/hashicorp/terraform/pull/28854. Looks like the plugin reads that file.

Fixed it by making sure the Source field in .terraform/modules/modules.json matches exactly the source in the TF file. If doesn't match, modify it for all the modules in that file and restart the IDE (or close and open the file).

Example for an official module: change registry.terraform.io/terraform-aws-modules/vpc/aws to terraform-aws-modules/vpc/aws

I'm using PyCharm 2021.3 and version 0.7.14 of the plugin. I have also a monorepo with multiple Terraform projects!

[jzburns]

Same issue. This seems to have started recently:

Webstorm: Build #WS-213.5744.224, built on November 27, 2021 Plugin 0.7.14 OS: Arch Linux Error Cannot locate module locally: Unknown reason

module "gce-container-c1" {
  source = "terraform-google-modules/container-vm/google"
  version = "~> 2.0"
  container = {
    image = "eu.gcr.io/XXXX"
  }
  restart_policy = "Always"
}

Any workarounds?

[Xerkus]

Not sure if it's the same cause but had the same issue with Terraform 1.1.0. They changed how they store the Source in .terraform/modules/modules.json causing my issue. See hashicorp/terraform#28854. Looks like the plugin reads that file.

I can confirm, this solved the issue for me. Since the original reporter used 1.0.10, what we experienced here is a separate issue

[danpoland]

Alternativlity, you can prepend the source URI in your main.tf with registry.terraform.io/. Not including it is just a short hand way of indicating that the module is hosted on the Terraform registry. By prepending the registry in your source reference you don't have to touch the files in the cache.

module "vpc" {
  source  = "registry.terraform.io/terraform-aws-modules/vpc/aws"
  version = "3.11.0"
}

[lonegunmanb]

Thanks @danpoland , it's really helpful!

[shuliakovsky]

looks like plugin successfully can find a remote modules after terraform get with terraform version 1.0.11 and below, unfortunately can't find modules with latest terraform versions.

[mohamed-haidara-cko]

@shuliakovsky see my reponse above in case you missed it for a workaround: https://github.com/VladRassokhin/intellij-hcl/issues/365#issuecomment-996019841.

You will to need to update the .terraform/modules/modules.json for the plugin to work properly when using Terraform >= 1.1.0.

[gchamon]

We can use sed to quickly fix a repository with multiple workspaces:

sed -i --regexp-extended "s/registry.terraform.io\///g" **/.terraform/modules/modules.json && sed -i --regexp-extended "s/git::https:\/\/(.*)\.git/\1/" **/.terraform/modules/modules.json

I set an alias to this command so I can easily rerun it.

[solarmosaic-kflorence]

Please see https://github.com/VladRassokhin/intellij-hcl/issues/373 for issues related to the module paths introduced in Terraform 1.1.x -- the issue in this ticket only relates to sub-modules.

[rashidnhm]

Would there be a way to update the plugin to look for the new Source address alongside the exact match?

From glancing at hashicorp/terraform#28854 I see the following:

• registry/module --> registry.terraform.io/registry/module
• github.com/some/module --> git::https://github.com/some/module
• [email protected]:some/module --> git::ssh://[email protected]/some/module

^By no means is this an exhaustive list, I compiled it from looking at some of the updated unit tests.

The fixes mentioned above do work for me, but it is a lot more verbose than I'd like. If the plugin can search modules.json for both an exact match OR an expected transformed match, wouldn't it handle the cases for both Terraform 1.1.x and below that as well?

[userbradley]

I've just recently had this issue, again despite checking that the values under source = and values under the file modules.json are the same

Resolution

1. Try restarting Intelli-j, or
2. See if there are any updates to the plugin, or
3. Update terraform (Personally I use tfswitch, or
4. Reboot ( ͡° ͜ʖ ͡°)

I'm not sure why this happened, but the above steps seem to solve it temporarily

[fabio-filho]

Not sure if it's the same cause but had the same issue with Terraform 1.1.0. They changed how they store the Source in .terraform/modules/modules.json causing my issue. See hashicorp/terraform#28854. Looks like the plugin reads that file.

Fixed it by making sure the Source field in .terraform/modules/modules.json matches exactly the source in the TF file. If doesn't match, modify it for all the modules in that file and restart the IDE (or close and open the file).

Example for an official module: change registry.terraform.io/terraform-aws-modules/vpc/aws to terraform-aws-modules/vpc/aws

I'm using PyCharm 2021.3 and version 0.7.14 of the plugin. I have also a monorepo with multiple Terraform projects!

I have same issue using:

• terraform v1.2.4;
• WebStorm v2022.1.3
• Terraform Plugin v22.5921.12

This answer still works, thanks!

[lozuwa]

In my case I had to switch from terraform-aws-modules/vpc/aws to registry.terraform.io/terraform-aws-modules/vpc/aws.

• terraform 1.2.4
• Intellij 2022.1.2

[psantus]

In my case I had to switch from terraform-aws-modules/vpc/aws to registry.terraform.io/terraform-aws-modules/vpc/aws.

• terraform 1.2.4
• Intellij 2022.1.2

That tip was life-saving for me, thanks @lozuwa

[mirzazeyrek]

I can confirm the issue is still there and the solution mentioned above works.

https://plugins.jetbrains.com/plugin/7808-terraform-and-hcl v222.4345.14 PhpStorm 2022.2.3 Terraform v1.3.3 on linux_amd64