build: use ko to build Go applications instead of the Alpine image to statically link against CGO for better security when building gaia binaries

[odeke-em]

trafficstars

Summary

A security assesment from Chainguard for supply chain security yielded a recommendation that given we use CGO in some of this code, that we should use ko directly to build gaiad images e.g when using clevel_db

Proposal

from the Chainguard report

ko is a tool for building go applications and distributing them as OCI containers. The default base image for images built with ko is a distroless base image, which will be distroless.dev/static:latest in the next ko release., which is sufficient for supporting the gaiad binary. We have noticed that there are variants of the gaiad binary which contain components built with cgo, notably when building gaiad with support for cleveldb using the cleveldb build tag. For stability and reduced binary size, Chainguard strongly recommends building these components statically linked against musl. To facilitate this, the Distroless project provides a distroless.dev/ko image which we will use to build the final gaiad image.

Conversion process

Assuming that you have already authenticated to a registry, it is possible to use the distroless.dev/ko image to build a gaiad image using docker from the root of the cosmos/gaia repository by running: $ KO_DOCKER_REPO=... ko build ./cmd/gaiad The KO_DOCKER_REPO variable should be set to the prefix of the repository you want to publish to, for example, ghcr.io/cosmos for Cosmos Network. When ko completes, it will print the sha256 reference that can be used with commands such as docker pull, etc.

/cc @kirbyquerby from Orijtech Inc to help out with addressing this

---

For Admin Use

• [x] Not duplicate issue
• [ ] Appropriate labels applied
• [ ] Appropriate contributors tagged
• [ ] Contributor assigned/self-assigned
• [ ] Is a spike necessary to map out how the issue should be approached?

[faddat]

counterproposal:

Let's drop cgo database support for gaia.

That'll leave us with the two real contenders anyhow:

• goleveldb
• pebbledb

None of the other databases provide compelling advantages -- rocksdb did provide compelling advantages, until pebbledb.

https://github.com/tendermint/tm-db/pull/284

Pebbledb is a rocks-alike from the cockroachdb team. In our early tests (narrative data only at this time) it performs better than rocks or goleveldb, and much better than cleveldb

[tac0turtle]

Let's drop cgo database support for gaia.

Please no, you have been pushing rocskdb for a while and got a lot of people to migrate and now we cant remove it since it is being used.

we can migrate to this image, but I don't think the images provided by the gaia team are built with cgo support. If it is then we can remove it on the provided docker. If others want to use cleveldb or rocks then they will need to build their own image

[faddat]

Please no, you have been pushing rocskdb for a while and got a lot of people to migrate and now we cant remove it since it is being used.

because you said please 👅

Also the reasoning is decent. We still don't understand the tradeoffs btw rocks and pebble just yet.

So we need to ensure full support for:

• goleveldb
• cleveldb
• rocksdb
• pebble

honestly I still don't understand the recommendation from the security firm.

Can anyone eli5 me or link to the report?

[mmulji-ic]

Hi @odeke-em , do we need to do more for this issue than is in #1908 ?

[odeke-em]

Hi @odeke-em , do we need to do more for this issue than is in #1908 ?

@mmulji-ic aside from also the conversion instructions that I provided in this issue (they show the appropriate environment variables) I believe that that PR covers it. Thank you

[mpoke]

@mmulji-ic What's the status of https://github.com/cosmos/gaia/pull/1908? I see it's in draft. Do you know at what stage of completeness it is?

[mpoke]

@MSalopek is this still relevant now that we use goreleaser?

[MSalopek]

For stability and reduced binary size, Chainguard strongly recommends building these components statically linked against musl. To facilitate this, the Distroless project provides a distroless.dev/ko image which we will use to build the final gaiad image.

I will need to go deeper into this, we dont distribute docker images for gaia as far as I can tell.

The issue would only affect our CI, so I don't think there's much impact whether we change it or not.