Support for go-getter's subdirectory glob patterns

[sybereal]

go-getter supports using glob patterns in the subdirectory part after the double-slash, as long as there is only one match. This is incredibly useful when dealing with repository tar balls, as these usually have a top-level directory wrapping the repository contents.

My use case is the following: to reduce external dependencies (Git), we wanted to configure Terragrunt to download our module sources as a tar ball from the GitHub API using the https://api.github.com/repos/:org/:repo/tarball/:ref API, using a fine-grained PAT as the basic auth password to limit access to only the modules repository.

The problem is that the tar balls this API produces have a top-level directory name following the pattern ${org}-${repo}-${commit-sha}, even when the ref is a branch or tag. For go-getter itself, this isn't an issue. I ran go-getter https://dummy:[email protected]/repos/org/repo/tarball/vX.Y.Z//org-repo-*/submodule?archive=tgz submodule and it worked perfectly fine; the submodule directory contained the correct files from that tag.

However, when attempting to use an URL like this with Terragrunt, it fails with Working dir org-repo-*/submodule from source https://dummy:[email protected]/repos/org/repo/tarball/vX.Y.Z//org-repo-*/submodule?archive=tgz does not exist.

Minimal repro

I'm using the https://github.com/hashicorp/learn-terraform-modules-create repo because it was the first example repository I found. I've used comments to represent the command outputs to simplify pasting.

go-getter 'https://api.github.com/repos/hashicorp/learn-terraform-modules-create/tarball/main//hashicorp-learn-terraform-modules-create-*?archive=tgz' test
# 2022/11/17 12:50:54 success!
ls test
# LICENSE  main.tf  modules  outputs.tf  README.md  variables.tf
cat <<EOF > terragrunt.hcl
terraform {
    source = "https://api.github.com/repos/hashicorp/learn-terraform-modules-create/tarball/main//hashicorp-learn-terraform-modules-create-*?archive=tgz"
}
EOF
terragrunt apply
# ERRO[0001] Working dir hashicorp-learn-terraform-modules-create-* from source https://api.github.com/repos/hashicorp/learn-terraform-modules-create/tarball/main?archive=tgz does not exist
# ERRO[0001] Unable to determine underlying exit code, so Terragrunt will exit with error code 1