syft icon indicating copy to clipboard operation
syft copied to clipboard

publish go-mod/vendor as release artifact

Open ctr49 opened this issue 1 year ago • 2 comments

What would you like to be added: I would like to see the go-mod/cache added as release artifact next to the source and the compiled binaries. Please extend the Release action to also publish a compressed tarball of all used dependencies.

Why is this needed: To have an available and consistent dependency package for distributions that want to distribute syft.

Additional context: For source packages, some (many/all?) distributions require all artifacts to build the package (including their dependencies) to be present if not relying on distribution specific library/include dependencies (which would be rather unusual for a go package). These dependencies also need to be available for offline usage and may not be fetched at compile time through the language-specific module resolver (i.e. go mod download or npm i). Usually this results in someone creating a tarball of he deps. But this tarball could be much easier created and made available by the source package (here) rather than by each distribution package maintainer. Also, there would be higher consistency across distributions when made available from an authoritative party.

If size is an issue (> 500MB per version!), it could be limited to still supported or last x versions.

Example: Gentoo's syft package has a tarball of all dependencies at https://dev.gentoo.org/~williamh/dist/${VERSION}-deps.tar.xz which makes the whole package update process unnecessarily complicated.

ctr49 avatar Feb 11 '24 14:02 ctr49

I see how that would be useful to the downstream packagings. Instead of a full go mod download, what about an ad-hoc go mod vendor?

# in repo root...
$ go mod vendor -v
$ tar -cJf syft-vendor-cache.tar.xz  ./vendor
$ ls -alh *.tar.xz
-rw-r--r--@  1 wagoodman  staff    10M Feb 13 10:47 syft-vendor-cache.tar.xz

vs

# in repo root...
$ GOMODCACHE=$(pwd)/cache go mod download
$ tar -cJf syft-mod-cache.tar.xz  ./cache
$ ls -alh *.tar.xz
-rw-r--r--@   1 wagoodman  staff   524M Feb 13 10:46 syft-mod-cache.tar.xz

The nice thing about this is that it's much smaller, doesn't contain unnecessary content such as tests for dependencies, and would be easier to publish.

My question is that would using the vendor cache over the mod cache work for your use case?

I think the only modification to the workflow would be:

  • no need to set GOMODCACHE or GOPATH
  • unarchive the vendor tar into the root of the repo
  • run go build -mod=vendor instead

wagoodman avatar Feb 13 '24 15:02 wagoodman

Yes, you are perfectly right. Having the vendor-dir should be sufficient.

ctr49 avatar Feb 13 '24 17:02 ctr49