Consider local_directories Data Source

[bflad]

Terraform CLI and Provider Versions

N/A

Use Cases or Problem Statement

The Terraform configuration language has the fileset() function which returns a set of all files matching a glob pattern given a path, however it does not currently handle directories. Even if it was adjusted to support directories (or created as a separate function), as with most configuration language functions, the result is evaluated before the rest of the graph. This can cause issues for practitioners wishing to use a collection of directories generated/updated during the graph for purposes such as calculating checksums to explicitly cause resource replacement.

While today we generally rely on the function documentation to warn practitioners about this potential issue, it may make sense to provide a graph-aware solution for the use case.

Proposal

Create a new local_directories data source which operates similarly to the fileset() function or the proposed local_files data source. It can pair nicely with the proposed local_directory data source as well. As a design sketch of potential usage:

data "local_directories" "example" {
  path    = path.module
  pattern = "{one,three,five}" # optional
}

data "local_directory" "example" {
  for_each = data.local_files.example.directory_paths

  filename = each.value
}

# further logic on data.local_directory.example[*]

Proposed data source configurable attributes:

• path - String, required, the starting path for matching
• pattern - String, optional, defaults to *, the glob pattern (using the same doublestar dependency as Terraform CLI)

Proposed data source computed attributes:

• directory_paths - Set of strings, the paths to all directories found by the glob pattern

How much impact is this issue causing?

Medium

Additional Information

• Upstream Terraform feature request: https://github.com/hashicorp/terraform/issues/32654
• local_directory data source proposal: #216

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[bflad]

I'm going to split this feature request since local_directory is likely less controversial and has immediate use cases with checks.

[olivers-xaxis]

One concern is the proliferation of such data sources based on the file type: regular files, directories, single directory etc, and in time more types like symbolic links, sockets etc (however unlikely they appear to be at this time). So I've created an alternative proposal (#217) as it is sufficiently different from this one and local_directory and covers (AFAICT) both sets of use cases.