atmos vendor pull fails when source repository contains symlinks

[VentByte]

Describe the Bug

When using the atmos vendor pull command, the process fails if the vendor repository contains a symlink. Specifically, if there is a symlink in the repository structure, atmos does not handle it correctly and triggers an error during the pull operation. This issue occurs consistently when the defined source repository contains a symlink.

Expected Behavior

atmos vendor pull should correctly handle symlinks in vendor repositories, ensuring that the linked paths are included without causing errors.

Steps to Reproduce

1. Create vendor.yaml
2. Add a source git repo which contains symlinks
3. Run atmos vendor pull

Screenshots

$ atmos vendor pull
Processing vendor config file 'vendor.yaml'
Pulling sources from 'github.com/ventbyte/repro-symlink-structure' into 'first_vendor'
Including 'dir-1'

Including 'dir-1/.gitkeep'

Including 'dir-2'

Including 'dir-2/dir-1'

lstat ../dir-1: no such file or directory

goroutine 1 [running]:
runtime/debug.Stack()
        /opt/hostedtoolcache/go/1.23.0/x64/src/runtime/debug/stack.go:26 +0x5e
runtime/debug.PrintStack()
        /opt/hostedtoolcache/go/1.23.0/x64/src/runtime/debug/stack.go:18 +0x13
github.com/cloudposse/atmos/pkg/utils.LogError({0x37648e0, 0xc00051e300})
        /home/runner/work/atmos/atmos/source/pkg/utils/log_utils.go:61 +0x1ca
github.com/cloudposse/atmos/pkg/utils.LogErrorAndExit({0x37648e0, 0xc00051e300})
        /home/runner/work/atmos/atmos/source/pkg/utils/log_utils.go:35 +0x38
github.com/cloudposse/atmos/cmd.init.func25(0x5084f60, {0x5253000, 0x0, 0x0})
        /home/runner/work/atmos/atmos/source/cmd/vendor_pull.go:22 +0x4f
github.com/spf13/cobra.(*Command).execute(0x5084f60, {0x5253000, 0x0, 0x0})
        /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:989 +0xa91
github.com/spf13/cobra.(*Command).ExecuteC(0x5082ce0)
        /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:1117 +0x3ff
github.com/spf13/cobra.(*Command).Execute(...)
        /home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:1041
github.com/cloudposse/atmos/cmd.Execute()
        /home/runner/work/atmos/atmos/source/cmd/root.go:88 +0x27c
main.main()
        /home/runner/work/atmos/atmos/source/main.go:9 +0x13

Environment

• Atmos Version: 1.88.1 on linux/amd64
• Git version: 2.39.5
• OS: Debian 12 (WSL)

Additional Context

Content of vendor.yaml:

apiVersion: atmos/v1
kind: AtmosVendorConfig
metadata:
  name: vendor-config
spec:
  sources:
    # Repo that contains symlinks
    - source: github.com/ventbyte/repro-symlink-structure
      targets: ["first_vendor"]
    # This vendor is not pulled because of the fatal
    # error when pulling prev. source
    - source: https://example.com/
      targets: ["second_vendor"]

Content of vendor repository:

.
├── dir-1
├── dir-2
│   └── dir-1 -> ../dir-1
├── file-1.txt
└── link-1.txt -> file-1.txt

Working directory before running atmos vendor pull:

.
└── vendor.yaml

Working directory before after atmos vendor pull:

.
├── first_vendor
│   ├── dir-1
│   │   └── .gitkeep
│   └── dir-2
└── vendor.yaml

[osterman]

Thanks for reporting! We'll take a look.

[VentByte]

I've found a workaround by setting core.symlinks to false in my gitconfig, but I'm unsure if this will work for every use case. The effectiveness of this workaround likely depends on whether and how symlinks are utilized down the line.

[osterman]

OS: Debian 12 (WSL)

Symlinks are tricky across platforms. If we detect WSL or Windows, we should probably force core.symlinks to false, which dereferences symlinks to actual files. That might not work ideally for directories, but not sure what's best.

[osterman]

@VentByte we plan to ignore all symlinks in vendoring with go-getter due the security issues associated with it. Do you have concerns with this approach?

[VentByte]

@osterman thanks for the Update. While ignoring all symlinks might not be ideal for some use cases, I agree that it's likely the best approach from a security standpoint.

That said, it would be great if there were an option to explicitly allow symlinks for trusted sources. This could provide flexibility for cases where symlinks are necessary while still maintaining security for the default behavior.