atmos icon indicating copy to clipboard operation
atmos copied to clipboard

`atmos vendor pull` behaves differently than `terraform init` for subdirectory based modules

Open mss opened this issue 8 months ago • 10 comments

Describe the Bug

I must admit that this is a slightly esoteric use case and thus maybe some documentation at https://atmos.tools/cli/commands/vendor/pull#description would be sufficient.

Let's assume that we have some module source with the URL git::ssh://[email protected]/tf/tf-modules.git//modules/account-vpc?ref=v0.1.0 (note the double-slash syntax to use the given subdirectory).

For some reason are certain users or processes not able to access the repository via SSH but need to use HTTPS instead. Since it only affects this one server they add some config to ~/.gitconfig like this:

[includeIf "gitdir:~/code/work/"]
    path = ~/.config/git/work.config

And that work.config file contains something like this:

[url "https://git.example.com/scm/"]
    insteadOf = ssh://[email protected]/

This configuration works and the modules are pulled via HTTPS instead of SSH if one creates a plain old Terraform root module and one calls terraform init.

Now we want to use Atmos vendoring and add vendor.yaml:

apiVersion: atmos/v1
kind: AtmosVendorConfig
metadata:
  name: account-vpc
  description: account components
spec:
  sources:
    - component: 'account-vpc-v0.1.0'
      source: 'git::ssh://[email protected]/tf/tf-modules.git//modules/account-vpc?ref=v0.1.0'
      targets:
        - 'components/terraform/account-vpc/v0.1.0'

This won't use the given mirror but will (try to) use the original URL which may fail due to whatever networking issues are the reason the config was added in the first place.

Expected Behavior

I should not have to strace the atmos command to find out why my Git config which worked with Terraform does not work anymore with Atmos vendoring :-) It would be nice if it just worked as expected (like for Terraform) or the behaviour (ie. that a subdirectory-based module will be cloned to a subdirectory below $TMPDIR) was documented.

Steps to Reproduce

See above (there is probably a more minimalistic reproducer possible). Some other Git features (badly written hooks?) than the one described might be affected, too.

Screenshots

No response

Environment

  • OS: Linux
  • Version: 1.77.0

Additional Context

This is caused by an undocumented behaviour of go-getter (cf. hashicorp/go-getter#493) to pull an URL which refers to a subdirectory to $TMPDIR first and then copy over the wanted contents. So the $GIT_DIR does not match the directory from the includeIf because the code is actually checked out to a temporary location like /tmp/getter12345/temp.

It does work in Terraform because they resolve the double-slash syntax themselves as pointed out in this comment (I linked to the OpenTofu source due to the current Terraform license but the code is the same).

mss avatar Jun 03 '24 12:06 mss