go-ethereum icon indicating copy to clipboard operation
go-ethereum copied to clipboard

downloads-page not up to date

Open holiman opened this issue 1 year ago • 19 comments

https://geth.ethereum.org/downloads showing latest as v1.14.5 (all platforms), so hasn't been updated in three releases - we're now at v1.14.8

holiman avatar Aug 16 '24 05:08 holiman

For develop-builds, the last one is 1.14.6, https://github.com/ethereum/go-ethereum/tree/7cfff30b, somewhere around June 28, 2024 at 06:54 PM was the last successfull publish.

holiman avatar Aug 16 '24 05:08 holiman

https://github.com/ethereum/go-ethereum/commits/master?since=2024-06-28&until=2024-07-03 -- the breakage happens suspiciously close to https://github.com/ethereum/go-ethereum/pull/30037

holiman avatar Aug 16 '24 05:08 holiman

Unrelated tangent, we need to fix this at some point (from travis log, where we build a docker container):

>>> docker build --build-arg COMMIT=bd57f35f8d1e920ef2b234ea6cd1aa18715b358f --build-arg VERSION=1.14.9-unstable --build-arg BUILDNUM=30159 --tag ethereum/client-go:alltools-TAG -f Dockerfile.alltools .

DEPRECATED: The legacy builder is deprecated and will be removed in a future release.

            Install the buildx component to build images with BuildKit:

            https://docs.docker.com/go/buildx/

holiman avatar Aug 16 '24 06:08 holiman

Interesting, from https://app.travis-ci.com/github/ethereum/go-ethereum/jobs/625078325 :

78dc4cc8d04a: Pushed

alltools-latest-arm64: digest: sha256:f8d72417d9a75653858be93cfb4bb77c4f70268f46824909369abd589c1a7ffd size: 951

ci.go:831: Build number mismatch on ethereum/client-go:latest-amd64: want 30159, have 30157

ci.go:831: Build number mismatch on ethereum/client-go:latest-amd64: want 30159, have 30157

ci.go:857: Relinquishing publish to other builder

The command "go run build/ci.go docker -image -manifest amd64,arm64 -upload ethereum/client-go" exited with 0.

This error just goes in the log, but the job is still "green"

EDIT: I think this is fine, it's the docker builder, and the docker builder has no problem

holiman avatar Aug 16 '24 06:08 holiman

The building and updload to azure appears to work fine: https://app.travis-ci.com/github/ethereum/go-ethereum/jobs/625078326#L5684 .

So maybe the upload is not the problem, but rather the website regeneration?

holiman avatar Aug 16 '24 06:08 holiman

Yes, the blobs are there

$ curl  "https://gethstore.blob.core.windows.net/builds?restype=container&comp=list&maxresults=10&prefix=geth-linux-amd64-1.14.8"
<?xml version="1.0" encoding="utf-8"?><EnumerationResults ServiceEndpoint="https://gethstore.blob.core.windows.net/" ContainerName="builds"><Prefix>geth-linux-amd64-1.14.8</Prefix><MaxResults>10</MaxResults><Blobs><Blob><Name>geth-linux-amd64-1.14.8-a9523b64.tar.gz</Name><Properties><Last-Modified>Mon, 12 Aug 2024 13:00:37 GMT</Last-Modified><Etag>0x8DCBACEC30E9F8F</Etag><Content-Length>29994640</Content-Length><Content-Type>application/octet-stream</Content-Type><Content-Encoding /><Content-Language /><Content-MD5>YRV3bM4ymlwo1+/82qxe/Q==</Content-MD5><Cache-Control /><Content-Disposition /><BlobType>BlockBlob</BlobType><LeaseStatus>unlocked</LeaseStatus><LeaseState>available</LeaseState></Properties></Blob><Blob><Name>geth-linux-amd64-1.14.8-a9523b64.tar.gz.asc...

holiman avatar Aug 16 '24 06:08 holiman

This seems to be the issue: https://app.netlify.com/sites/geth-website/deploys/66ad0e7d8182b00008e98725#L113

holiman avatar Aug 16 '24 06:08 holiman

cc @jmcook1186 @samajammin @wackerow PTAL

holiman avatar Aug 16 '24 06:08 holiman

Fixed by https://github.com/ethereum/go-ethereum/pull/30310

holiman avatar Aug 16 '24 07:08 holiman

Broken by https://github.com/ethereum/go-ethereum/pull/30101

I don't mind that we sometime break the build, but I find it a bit scary that we had broken production deployments for six weeks but didn't get any notification. Perhaps we can provision netlify with some email address to report to?

holiman avatar Aug 16 '24 07:08 holiman

Thanks for raising @holiman. I've added an email notification to the website team in Netlify for any failed deployments. Limited bandwidth though for ongoing management, so let me know if you'd like any alerts directed your way as well (@ me please), thanks!

wackerow avatar Aug 16 '24 12:08 wackerow

Agreed! Perhaps also don't allow PRs to be merged if the Netlify build fails?

image

General FYI - I know no longer work on the ethereum.org team so I'm not be available to help with the website. Much love ✌️😎

samajammin avatar Aug 17 '24 19:08 samajammin

Perhaps also don't allow PRs to be merged if the Netlify build fails?

IIRC, that won't work. All PRs from external branches do fail the netlify build -- only builds from branches on this repo succeeds. So we don't really want to be notified of every fail, what's important is if the deployment of the merged website branch fails. That is, the prod deployment.

General FYI - I know no longer work on the ethereum.org team so I'm not be available to help with the website

Right, sorry for the noise! Cheers!

holiman avatar Aug 19 '24 07:08 holiman

Sorry, going to have to reopen this. https://geth.ethereum.org/downloads -- the develop builds are showing the latest being Aug 15, 6 days ago. There are actually later binaries, so it appears the site hasn't been cron-updating. The deploy-page https://app.netlify.com/sites/geth-website/deploys shows latest production deploy Aug 16, nothing since.

I had supposed it should run nightly? Is that disabled?

holiman avatar Aug 21 '24 06:08 holiman

I had supposed it should run nightly? Is that disabled?

Ah, no it's not regenerated in full, rather some "incremental static regeneration" of the data on the downloads-page:

https://github.com/ethereum/go-ethereum/blob/website/src/pages/downloads.tsx#L37

holiman avatar Aug 21 '24 06:08 holiman

The latest deploy log does mention the ISR: https://app.netlify.com/sites/geth-website/deploys/66befd84c18f230008a49946#L134

holiman avatar Aug 21 '24 06:08 holiman

I'm not sure what to make of this. "ISR is great and awesome. OBS: Not supported though" ... what? Are we using edge? Strange

Screenshot 2024-08-21 at 08-35-52 Data Fetching Incremental Static Regeneration (ISR) Next js

holiman avatar Aug 21 '24 06:08 holiman

ping @wackerow

holiman avatar Aug 23 '24 05:08 holiman

Interesting, from https://app.travis-ci.com/github/ethereum/go-ethereum/jobs/625078325 :

78dc4cc8d04a: Pushed

alltools-latest-arm64: digest: sha256:f8d72417d9a75653858be93cfb4bb77c4f70268f46824909369abd589c1a7ffd size: 951

ci.go:831: Build number mismatch on ethereum/client-go:latest-amd64: want 30159, have 30157

ci.go:831: Build number mismatch on ethereum/client-go:latest-amd64: want 30159, have 30157

ci.go:857: Relinquishing publish to other builder

The command "go run build/ci.go docker -image -manifest amd64,arm64 -upload ethereum/client-go" exited with 0.

This error just goes in the log, but the job is still "green"

EDIT: I think this is fine, it's the docker builder, and the docker builder has no problem

Just to answer this bit of the ticket. To build a multi-architecture docker image, we need to build each on it's own platform, and then have a 3rd job that merges the two images into a multi-arch image.

To avoid having 3 travis jobs and a weird dependency across them, I did a dirty trick in the amd/arm64 docker builders: after they push their own architecture image, they check if the couterpart is already pushed or not. If not (i.e. mismatching build number upstream), then we early exit as the sibling builder is still running. If the counterpart image matches our build numbers, then our sibling already quit and it's up to us to combine the two images into the multi-arch one.

The error message I think is simple a log saying that "we finished first, exit and leave the other builder to do the multi-arch assembly".

karalabe avatar Aug 28 '24 03:08 karalabe

This works now, thanks @corwintines !

holiman avatar Sep 04 '24 13:09 holiman