arrow icon indicating copy to clipboard operation
arrow copied to clipboard

GH-46336: [Release][Packaging] Add support for Reproducible Builds for source archive

Open kou opened this issue 7 months ago • 9 comments

Rationale for this change

See https://reproducible-builds.org/ for Reproducible Builds.

Automated Release Signing requires this: https://infra.apache.org/release-signing.html#automated-release-signing

What changes are included in this PR?

  • Make dev/release/utils-create-release-tarball.sh reproducible
  • Test it by reprotest: https://salsa.debian.org/reproducible-builds/reprotest
  • Verify source archive reproducibility in RC verification script

Are these changes tested?

Yes.

Are there any user-facing changes?

No.

  • GitHub Issue: #46336

kou avatar May 07 '25 07:05 kou

:warning: GitHub issue #46336 has been automatically assigned in GitHub to PR creator.

github-actions[bot] avatar May 07 '25 07:05 github-actions[bot]

Ah, https://infra.apache.org/release-signing.html#automated-release-signing includes the following:

The release procedure contains a validation step where all artifacts are reproduced on trusted hardware before publication to pages intended for end users

We need to add a reproducible check to dev/release/verify-release-candidate.sh for it.

kou avatar May 07 '25 07:05 kou

We need to add a reproducible check to dev/release/verify-release-candidate.sh for it.

Implemented.

kou avatar May 07 '25 08:05 kou

@assignUser @raulcd Do you want to review this before we request a review from INFRA?

kou avatar May 08 '25 06:05 kou

I asked INFRA to review and enable automatic release signing for source arhicve: https://issues.apache.org/jira/browse/INFRA-26808

kou avatar May 09 '25 08:05 kou

Sorry, I didn't have time to have a look so far, but this is great!

assignUser avatar May 10 '25 01:05 assignUser

https://issues.apache.org/jira/browse/INFRA-26808?focusedCommentId=17982692&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17982692

INFRA set secrets.ARROW_GPG_SECRET_KEY. I'll use it in this PR.

kou avatar Jun 19 '25 01:06 kou

I'm waiting an answer from INFRA because INFRA might set typo-ed secret variable name.

If there is no answer in this week, I'll merge this without an answer from INFRA.

kou avatar Jun 23 '25 05:06 kou

There is a typo in secret variable name. I'll merge this after the typo is fixed by INFRA.

kou avatar Jun 23 '25 05:06 kou

The typo was fixed: https://issues.apache.org/jira/browse/INFRA-26808?focusedCommentId=17986958&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17986958

I'll merge this to try this in 21.0.0 release.

kou avatar Jul 01 '25 02:07 kou

After merging your PR, Conbench analyzed the 3 benchmarking runs that have been run so far on merge-commit a4d735aa80bf138651071f110f676aaab8ebf0a2.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details.