node icon indicating copy to clipboard operation
node copied to clipboard

Published 20 hours ago • verified publisher icon nodejs

build, tools: copy release assets to staging R2 bucket once built

Open flakey5 opened this issue 1 year ago • 10 comments

cc @MoLow @targos

flakey5 avatar Jan 06 '24 21:01 flakey5

Background: https://github.com/nodejs/build/issues/3602

flakey5 avatar Jan 06 '24 21:01 flakey5

I did a test release build (https://ci-release.nodejs.org/job/iojs+release/9917/ for those who have access), but it failed to upload:

09:07:10 ssh node-www "aws s3 cp nodejs/custom/v22.0.0-test2024012187d2b3c18f/node-v22.0.0-test2024012187d2b3c18f-linux-x64.tar.gz s3://dist-staging/nodejs/custom/v22.0.0-test2024012187d2b3c18f/node-v22.0.0-test2024012187d2b3c18f-linux-x64.tar.gz --endpoint=https://07be8d2fbc940503ca1be344714cb0d1.r2.cloudflarestorage.com/" --profile=nodejs-r2-dist-bucket
09:07:12 
09:07:12 The config profile (nodejs-r2-dist-bucket) could not be found
09:07:12 make: *** [Makefile:1291: binary-upload] Error 255

targos avatar Jan 21 '24 18:01 targos

I did a test release build (https://ci-release.nodejs.org/job/iojs+release/9917/ for those who have access), but it failed to upload:

Should be the correct profile now (or at least this is the profile used here https://github.com/nodejs/build/blob/a721695e2b652bee056502e5c7da3ec03d1ca150/ansible/www-standalone/tools/promote/settings#L41)

flakey5 avatar Jan 22 '24 21:01 flakey5

New build: https://ci-release.nodejs.org/job/iojs+release/9924/

targos avatar Jan 23 '24 07:01 targos

It looks much better:

CleanShot 2024-01-23 at 09 31 55

There are a few failures, but I'll report them all when the job is finished.

targos avatar Jan 23 '24 08:01 targos

On Windows (all three builds):

upload: nodejs/custom/v22.0.0-test20240123a1b77670ae/node-v22.0.0-test20240123a1b77670ae-arm64.msi to s3://dist-staging/nodejs/custom/v22.0.0-test20240123a1b77670ae/node-v22.0.0-test20240123a1b77670ae-arm64.msi
09:47:52 touch: cannot touch 's3://dist-staging/nodejs/custom/v22.0.0-test20240123a1b77670ae/node-v22.0.0-test20240123a1b77670ae-win-arm64.zip.done': No such file or directory

upload: nodejs/custom/v22.0.0-test20240123a1b77670ae/node-v22.0.0-test20240123a1b77670ae-x64.msi to s3://dist-staging/nodejs/custom/v22.0.0-test20240123a1b77670ae/node-v22.0.0-test20240123a1b77670ae-x64.msi
09:17:15 touch: cannot touch 's3://dist-staging/nodejs/custom/v22.0.0-test20240123a1b77670ae/node-v22.0.0-test20240123a1b77670ae-win-x64.zip.done': No such file or directory

upload: nodejs/custom/v22.0.0-test20240123a1b77670ae/node-v22.0.0-test20240123a1b77670ae-x86.msi to s3://dist-staging/nodejs/custom/v22.0.0-test20240123a1b77670ae/node-v22.0.0-test20240123a1b77670ae-x86.msi
09:19:02 touch: cannot touch 's3://dist-staging/nodejs/custom/v22.0.0-test20240123a1b77670ae/node-v22.0.0-test20240123a1b77670ae-win-x86.zip.done': No such file or directory

On macOS (osx11-release-pkg)

09:33:01 scp -p node-v22.0.0-test20240123a1b77670ae.pkg node-www:nodejs/custom/v22.0.0-test20240123a1b77670ae/node-v22.0.0-test20240123a1b77670ae.pkg
09:33:06 ssh node-www "aws s3 cp nodejs/custom/v22.0.0-test20240123a1b77670ae/node-v22.0.0-test20240123a1b77670ae.pkg s3://dist-staging/nodejs/custom/v22.0.0-test20240123a1b77670ae/node-v22.0.0-test20240123a1b77670ae.pkg --endpoint=https://07be8d2fbc940503ca1be344714cb0d1.r2.cloudflarestorage.com/" --profile=worker"
09:33:06 /bin/sh: -c: line 0: unexpected EOF while looking for matching `"'
09:33:06 /bin/sh: -c: line 1: syntax error: unexpected end of file
09:33:06 make: *** [pkg-upload] Error 2

targos avatar Jan 23 '24 15:01 targos

please review each new line in the Makefile. The double quotes are not placed correctly.

targos avatar Jan 24 '24 16:01 targos

New build: https://ci-release.nodejs.org/job/iojs+release/9936/

targos avatar Jan 28 '24 11:01 targos

Are there any more outstanding issues with these changes?

flakey5 avatar Feb 16 '24 22:02 flakey5

Started a new test build: https://ci-release.nodejs.org/job/iojs+release/9980/ There were some errors on Windows in the previous one, but I'm not sure they're related.

targos avatar Feb 17 '24 07:02 targos

@nodejs/build

targos avatar Feb 18 '24 11:02 targos

I can rebase with the latest main to see if that fixes it

flakey5 avatar Feb 18 '24 17:02 flakey5

Rebased with the latest main, can we try it again to see if that fixed it?

flakey5 avatar Mar 14 '24 17:03 flakey5

I will trigger a new test build after the release of Node.js 22.

targos avatar Apr 22 '24 14:04 targos

https://ci-release.nodejs.org/job/iojs+release/10153/

targos avatar Apr 25 '24 06:04 targos

Rebuild for failed job (failure unrelated to this change) https://ci-release.nodejs.org/job/iojs+release/nodes=rhel8-ppc64le-release/10159/

targos avatar Apr 26 '24 13:04 targos

I confirm all files are in /dist-staging/nodejs/custom/v22.0.0-test202404257121813364/

targos avatar May 05 '24 08:05 targos

Commit Queue failed
- Loading data for nodejs/node/pull/51394
✔  Done loading data for nodejs/node/pull/51394
----------------------------------- PR info ------------------------------------
Title      build, tools: copy release assets to staging R2 bucket once built (#51394)
   ⚠  Could not retrieve the email or name of the PR author's from user's GitHub profile!
Branch     flakey5:flakey5/nodejs/build/3602 -> nodejs:main
Labels     windows, build, tools, needs-ci, commit-queue-squash
Commits    2
 - build, tools: copy release assets to staging R2 bucket once built
 - Update tools/release.sh
Committers 1
 - flakey5 
PR-URL: https://github.com/nodejs/node/pull/51394
Reviewed-By: Matteo Collina 
Reviewed-By: James M Snell 
Reviewed-By: Moshe Atlow 
------------------------------ Generated metadata ------------------------------
PR-URL: https://github.com/nodejs/node/pull/51394
Reviewed-By: Matteo Collina 
Reviewed-By: James M Snell 
Reviewed-By: Moshe Atlow 
--------------------------------------------------------------------------------
   ℹ  This PR was created on Sat, 06 Jan 2024 21:42:33 GMT
   ✔  Approvals: 3
   ✔  - Matteo Collina (@mcollina) (TSC): https://github.com/nodejs/node/pull/51394#pullrequestreview-2014829830
   ✔  - James M Snell (@jasnell) (TSC): https://github.com/nodejs/node/pull/51394#pullrequestreview-2014852122
   ✔  - Moshe Atlow (@MoLow) (TSC): https://github.com/nodejs/node/pull/51394#pullrequestreview-2018841776
   ✔  Last GitHub CI successful
   ✘  No Jenkins CI runs detected
--------------------------------------------------------------------------------
   ✔  Aborted `git node land` session in /home/runner/work/node/node/.ncu
https://github.com/nodejs/node/actions/runs/8956985975

nodejs-github-bot avatar May 05 '24 08:05 nodejs-github-bot

I want to land https://github.com/nodejs/node/pull/45427 first as it's building a whole new ccache.

targos avatar May 05 '24 09:05 targos

CI: https://ci.nodejs.org/job/node-test-pull-request/58959/

nodejs-github-bot avatar May 05 '24 14:05 nodejs-github-bot

@flakey5 Can you please rebase on the latest main? It seems that the CI wrongly sets the major version as 22 instead of 23.

targos avatar May 05 '24 20:05 targos

Done

flakey5 avatar May 05 '24 22:05 flakey5

CI: https://ci.nodejs.org/job/node-test-pull-request/58974/

nodejs-github-bot avatar May 06 '24 04:05 nodejs-github-bot

CI: https://ci.nodejs.org/job/node-test-pull-request/58975/

nodejs-github-bot avatar May 06 '24 05:05 nodejs-github-bot

Landed in e5a76f16e11b0681a0df7bba5990a14d95bbad52

nodejs-github-bot avatar May 06 '24 07:05 nodejs-github-bot