gozstd icon indicating copy to clipboard operation
gozstd copied to clipboard

build: enable build without cgo

Open giuseppe opened this issue 2 years ago • 6 comments

we have moved from github.com/klauspost/compress to github.com/valyala/gozstd in github.com/containers/storage for performance reasons when pulling container images, but unfortunately we need to support build also on targets where cgo is missing: https://github.com/containers/podman/pull/12794

Without this PR, I get:

$ CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build
# github.com/valyala/gozstd
./stream.go:14:48: undefined: DefaultCompressionLevel
./stream.go:31:59: undefined: CDict
./stream.go:35:64: undefined: CDict
./stream.go:47:20: undefined: Writer
./stream.go:56:22: undefined: NewWriterLevel
./stream.go:101:61: undefined: DDict
./stream.go:110:6: undefined: Reader
./stream.go:117:8: undefined: NewReader

now the build works fine

Signed-off-by: Giuseppe Scrivano [email protected]

giuseppe avatar Jan 11 '22 09:01 giuseppe

@valyala PTAL

giuseppe avatar Jan 11 '22 12:01 giuseppe

The Podman team is waiting on this to merge.

rhatdan avatar Jan 11 '22 15:01 rhatdan

@valyala PTAL

rhatdan avatar Jan 14 '22 12:01 rhatdan

@giuseppe , sorry for the delay and thanks for the pull request! I'm unsure this is a good way to go, since it allows building github.com/valyala/gozstd without cgo, but the resulting build is unusable - it is just a stub. Probably the approach taken in VictoriaMetrics is better - e.g. to create a wrapper package containing two files:

  • the first one is built when cgo is enabled. It imports github.com/valyala/gozstd
  • the second one is built when cgo is disabled. It imports github.com/klauspost/compress/zstd

See https://github.com/VictoriaMetrics/VictoriaMetrics/tree/master/lib/encoding/zstd .

valyala avatar Jan 31 '22 18:01 valyala

We are fine with that. The issue we are seeing when we vendor in your code into a package, it blows up on compile, even if we are not using gozstd in the revendored package.

rhatdan avatar Jan 31 '22 20:01 rhatdan

The issue we are seeing when we vendor in your code into a package, it blows up on compile, even if we are not using gozstd in the revendored package.

This sounds strange, since VictoriaMetrics is successfully built with both CGO_ENABLED=0 and CGO_ENABLED=1 when using the trick described above. Make sure that the github.com/valyala/gozstd is imported in the file, which compiles only for CGO_ENABLED=1. See build instructions and this particular section for details.

valyala avatar Feb 07 '22 16:02 valyala