The ".git" folder is not present in each instantiation of modules pulled from a git repository

[erwan-simon]

Terraform Version

1.9.8 (and all before, at least from v1.5.7)

Terraform Configuration Files

module "module_one" {
  source            = "git::https://github.com/repo-name?ref=vX.Y.Z"
}

module "module_two" {
  source            = "git::https://github.com/repo-name?ref=vX.Y.Z"
}

# inside "https://github.com/repo-name?ref=vX.Y.Z" module code
# should allow me to get the version of the module ("vX.Y.Z")
data "external" "job_factory_version" {
  program     = ["bash", "-c", "echo '{\"module_version\": \"'$(git tag --points-at HEAD)'\"}'"]
  working_dir = abspath(path.module)
}

Debug Output

No relevant debug is necessary as far as I know

Expected Behavior

data.external.job_factory_version.result.version should work (e.g. showing vX.Y.Z) both for module_one and module_two.

I use the module version as resources tag to know which version of the module I use directly in the AWS console, which is a "feature" I really like to have.

Actual Behavior

data.external.job_factory_version.result.version works (shows vX.Y.Z) only for module_one, not for module_two.

For module_two, the git command returns nothing because no .git folder is present in this module (.terraform/modules/module_two/). This can be explained by the fact that the getmodule part (https://github.com/hashicorp/terraform/blob/main/internal/getmodules/getter.go#L119) uses the the terraform internal copy function (https://github.com/hashicorp/terraform/blob/main/internal/copy/copy_dir.go#L38) which specifies :

// Any "dot files" it encounters along the way are skipped, even on platforms // that do not normally ascribe special meaning to files with names starting // with dots.

I think I understand the reason why dot files are not copied, especially in the scenario where you are copying a git repository. ".git" folders can be especially heavy and I understand that in the event of having to copy the module a lot, it would occupy a lot of space in the host machine.

I was thinking that maybe we can find a middle ground and I am willing to do a PR if required but I must admit I am not sure about how to implement that fix. So I thought about multiple scenarios :

1. remove the lines from internal copy functions which prevent dot files to be copied (https://github.com/hashicorp/terraform/blob/main/internal/copy/copy_dir.go#L53). I think it is a somehow dangerous solution because it would silently change the way this function behaves in the whole project.
2. add a boolean parameter in the CopyDir function specifying if you want to copy the dot files or not. But that would require to add this parameter everywhere this function is used in the project (which involves a cumbersome but feasible PR).
3. stop using the internal copy function and copy paste its code directly in the getter.go module. I do not like this solution as it involves code duplication
4. disable the copy altogether and clone the git repository for each instantiation. Which induces bandwidth over-usage.

Even if we implement a way to allow the dot files to be copied or pulled, there is still the problem that this behavior would impact every user even though it would be useful to only a small portion of them. So you would want the default behavior unchanged, and probably find a way to only apply this behavior in a declarative fashion, probably by adding a flag to the terraform init or by a configuration environment variable (https://developer.hashicorp.com/terraform/cli/config/environment-variables) which would allow the dot files to be included in each module folder for each instantiation.

In my opinion, I think the scenario 2 coupled with a configuration environment variable is the best combination in term of technical elegance and user experience, but I am eager for your inputs on this matter.

Steps to Reproduce

terraform init terraform apply

Additional Context

No response

References

No response

[crw]

Hi @erwan-simon, thanks for this feature request and the thoughtful implementation suggestions!

If you are viewing this issue and would like to indicate your interest, please use the 👍 reaction on the issue description to upvote this issue. We also welcome additional use case descriptions. Thanks again!

[rmcolbert]

We are having a similar issue (but not with the whole .git directory). We are using GitHub's recommendation for handling metadata and use several . prefixed files to hold data that the TF code actually uses. For example, we inject tags w/ the module's version in the cloud resource for lifecycle tracking. To make this easier to manage and less error prone, we moved that value into a metadata file that the GitHub Action updates and the module reads that file into a local.

This works fine for the 1st copy (download) of module in a workspace but fails on any other instances of that module because all dot prefixed files are skipped in the copy phase.

I would propose that if the copy function's exclusion of dot prefixed files/folders was specifically for .git, then it should only skip the .git folder and copy all other dot prefixed entities.