dotnet-docker icon indicating copy to clipboard operation
dotnet-docker copied to clipboard

Automation should exist to pull in latest builds of PowerShell into nightly branch

Open MichaelSimons opened this issue 5 years ago • 15 comments

The value in providing this automation is to get the latest PS running with the latest .NET as soon as possible to catch and flush out any issues early giving us more confidence in releases.

The update-dependencies tool can be utilized to update the dockerfiles and create a PR.

dotnet run --project .\eng\update-dependencies\ -- 5.0 --product-version powershell=7.1.0-preivew.6

The update-dependencies pipeline can be updated to automatically run the tool on a schedule.

In order to complete this work the following needs to be in place for the PS daily builds:

  1. Binaries need to be published to a public location
  2. Binaries need to be signed
  3. Checksums need to be published to a public location

MichaelSimons avatar Aug 03 '20 14:08 MichaelSimons

This is currently blocked on the daily builds of PS not getting signed.

MichaelSimons avatar Aug 06 '20 16:08 MichaelSimons

@adityapatwardhan, Has there been any progress on the getting the PS daily builds signed?

MichaelSimons avatar Nov 13 '20 15:11 MichaelSimons

We have finally got a new certificate created for us. We would be working on releasing our daily packages using the certificate soon.

adityapatwardhan avatar Nov 13 '20 15:11 adityapatwardhan

[Triage] @adityapatwardhan, has there been any progress in this space? Has the cert issue been resolved?

MichaelSimons avatar Jun 23 '21 18:06 MichaelSimons

We have finally got the certificate and will have daily signed builds soon. The PR for using the new certificate is here: https://github.com/PowerShell/PowerShell/pull/15642

adityapatwardhan avatar Jun 23 '21 19:06 adityapatwardhan

We still have some work for publishing check-sums

adityapatwardhan avatar Jun 23 '21 19:06 adityapatwardhan

@adityapatwardhan - Should this be ready to implement now that 7.2.0-preview.8 has been released?

mthalman avatar Jul 29 '21 16:07 mthalman

Yes I plan to work on this tomorrow.

adityapatwardhan avatar Jul 29 '21 19:07 adityapatwardhan

@adityapatwardhan - Any update on the work to enable this?

mthalman avatar Mar 02 '22 16:03 mthalman

Example SHA file output for a release build: https://pwshtool.blob.core.windows.net/tool/7.3.0-preview.3/SHA512SUMS

mthalman avatar Mar 23 '22 18:03 mthalman

The necessary parts to implement this should be available now.

URL to get info on the latest available build: https://aka.ms/pwsh-buildinfo-daily

Example output:

{
    "ReleaseDate":  "\/Date(1648236251000)\/",
    "BlobName":  "v7-3-0-daily20220325-1",
    "ReleaseTag":  "v7.3.0-daily20220325.1"
}

Use the BlobName to construct the checksums URL (e.g. https://pwshtool.blob.core.windows.net/tool/7.3.0-daily20220323.1/SHA512SUMS)

mthalman avatar Mar 28 '22 12:03 mthalman

Here's a proposed workflow of how things would work in nightly and then main:

  1. Configure nightly to take automated PRs to reference the latest daily build from PowerShell
  2. As PRs come in they are approved and merged by maintainers.
  3. When a candidate release build of PowerShell is known, the PowerShell maintainers create a PR for nightly to target that release. This means nightly is no longer targeting a daily build URL but rather the official release URL.
  4. Any subsequent automated PRs are disable or ignored once nightly is targeting an official release.
  5. As part of the merge from nightly to main for the .NET release, the PowerShell changes are included which target the official PowerShell build.
  6. After the release, nightly can continue to accept new automated PRs that target daily builds.

@adityapatwardhan - Let us know your thoughts on this workflow and whether it seems like a reasonable approach to take.

mthalman avatar May 04 '22 18:05 mthalman

This sounds good in general. Is step # 4 manual? Do the dotnet-docker maintainers need to be notified when a PR is made?

adityapatwardhan avatar May 17 '22 23:05 adityapatwardhan

This sounds good in general. Is step # 4 manual? Do the dotnet-docker maintainers need to be notified when a PR is made?

It's not manual. These PRs would be generated by a bot. An example for the base .NET product is https://github.com/dotnet/dotnet-docker/pull/3744. The idea is that once we've settled on a release candidate, we'd keep the nightly branch stable by ignoring/not merging any further automated PRs from the bot until the official release.

mthalman avatar May 18 '22 12:05 mthalman

[Triage] I checked on the PowerShell GitHub and their public releases include binaries and sha256 hashes, e.g. https://github.com/PowerShell/PowerShell/releases/tag/v7.4.2. However they don't include NuGet packages which are what we install in our images, so this is still blocked.

lbussell avatar May 16 '24 18:05 lbussell