Transition from Digicert keylocker to Azure Trusted Signing

[ryanaslett]

Our digicert keylocker certificates have a limited number of signatures available (598 as of this issue).

We'll need to ensure continuity in signing windows releases (nightly/canary/full releases).

Azure trusted signing is a more cost effective mechanism for signing code going forward.

The OpenJS foundation has established a Trusted Signing account as per: https://learn.microsoft.com/en-us/azure/trusted-signing/quickstart?tabs=registerrp-portal%2Caccount-portal%2Corgvalidation%2Ccertificateprofile-portal%2Cdeleteresources-portal to allow OpenJS projects to use our Identity to sign windows binaries.

The rough estimate is that we have a couple of months worth of signatures before we run out, (worst case 75 days).

So the next steps to get this addressed:

• [x] Set up a nodejs signing account in azure that uses the trusted signing account, and has access to the appropriate secrets that we can inject into our release pipeline
• [x] Set up the release machines/install trusted signing
• [x] Modify the release pipelines to sign the code with the new mechanism

[felixrieseberg]

Here's how I think we'll solve this problem:

1. Ensure that build machines have the required binaries, namely:

nuget install Microsoft.Windows.SDK.BuildTools -Version 10.0.26100.1742 -x
nuget install Microsoft.Trusted.Signing.Client -Version 1.0.60 -x

2. Ensure that build machines have a trusted_code_signing.json in a well-defined path

{
	"Endpoint": "https://wus2.codesigning.azure.net/",
	"CodeSigningAccountName": "TBD",
	"CertificateProfileName": "tbd"
}

3. Ensure that build machines have AZURE_CLIENT_ID, AZURE_CLIENT_SECRET, AZURE_TENANT_ID set. We also need to know the path to Azure.CodeSigning.Dlib.dll and to bin\10.0.26100.0\x64\signtool.exe (it needs to be that version or newer).

4. Then, we replace https://github.com/nodejs/node/blob/main/tools/sign.bat with:

@echo off

@REM From March 2024, we use Azure Trusted Signing for code signing.
@REM Release CI machines are configured to have it in the PATH so this can be used safely.
path/to/signtool.exe sign /tr "http://timestamp.acs.microsoft.com" /td sha256 /fd sha256 /v /dlib %AZURE_CODE_SIGNING_DLIB% /dmdf %AZURE_METADATA_JSON% %1
if not ERRORLEVEL 1 (
    echo Successfully signed %1 using signtool
    exit /b 0
)
echo Could not sign %1 using signtool
exit /b 1

[ryanaslett]

I have set up an App registration with Azure that is tied to the OpenJS Trusted signing account, so everything is prepared on the Azure side of things.

Since it appears as though the windows release machines are not controlled by ansible, I'm not sure what the next step is to get the secrets onto the windows release machines.

@StefanStojanovic would the NodeJS onepassword work as a mechanism to share these, or shall I stash these in the secrets repo?

[ryanaslett]

(Side note: we have 432 signatures remaining as of April 16th)

[StefanStojanovic]

Hey all, ClangCL is done. I'll probably have to help with the V8 update, but I can start working on this from next Monday (OOF before that). With that in mind, if there is something I should go through first (e.g. some docs, or anything like that), let me know here and I'll start with that.

[StefanStojanovic]

@StefanStojanovic would the NodeJS onepassword work as a mechanism to share these, or shall I stash these in the secrets repo?

@ryanaslett Digicert provided a CLI to safely store secrets, so we used that. Does Azure provide something like that?

[ryanaslett]

So If I understand correctly the secrets for digicert (the ones that allow auth into their vault) are stored locally on the machines using Window Credential Manager (similar to how the secrets for the OSX machines are stored locally in apple keychain).

I think it would be more ideal if we set those secrets in jenkins and exposed them as environment variables to the build process.

Essentially AZURE_CLIENT_SECRET can be put into the release jobs for windows builds and azure will use those (https://learn.microsoft.com/en-us/dotnet/api/azure.identity.environmentcredential?view=azure-dotnet)

[ryanaslett]

The contents of trusted_code_signing.json should be as follows:

{
  "Endpoint": "https://eus.codesigning.azure.net",
  "CodeSigningAccountName": "OpenJS-CodeSigning",
  "CertificateProfileName": "NodeJS",
}

I have added the following secrets to the build environments on ci-release builds: AZURE_TENANT_ID AZURE_CLIENT_ID AZURE_CLIENT_SECRET

which should allow the signtool to understand how to authenticate to azure.

@StefanStojanovic if you'd like to test outside of the release environment, I shared them in the secrets repo.

Im pretty sure that Felix's instructions (https://github.com/nodejs/build/issues/4036#issuecomment-2722623667) are the way we do this (It roughly follows the docs: https://learn.microsoft.com/en-us/azure/trusted-signing/how-to-signing-integrations#set-up-signtool-to-use-trusted-signing) (though I didnt check the version numbers etc)

[StefanStojanovic]

@ryanaslett I was able to sign the binaries manually with the data from the secrets repo. Based on that I configured CI machines and made changes in the Node repo needed for signing. However, when I tried testing it in the release CI, it failed. The build can be found here. Based on the log, and the env vars, I think there is probably some issue with passing the AZURE_... variables. When I set them directly in the env vars on the machine, the signing passed. Could you please look into that and let me know about your findings?

[ryanaslett]

Whoops. I added the three credentials, but accidentally assigned the wrong one to the client_ID:

I've fixed that, and am re-running that job.

[ryanaslett]

Splendid. It appears to have signed this time: https://ci-release.nodejs.org/job/iojs+release/nodes=vs2022_clang-x64/10977/console

00:53:57.801 Trusted Signing
00:53:57.801 
00:53:57.801 Version: 1.0.68
00:53:57.801 
00:53:57.801 "Metadata": {
00:53:57.801   "Endpoint": "https://eus.codesigning.azure.net/",
00:53:57.801   "CodeSigningAccountName": "OpenJS-CodeSigning",
00:53:57.801   "CertificateProfileName": "NodeJS",
00:53:58.967   "ExcludeCredentials": []
00:53:58.968 }
00:54:00.040 
00:54:00.041 Submitting digest for signing...
00:54:00.041 
00:54:00.559 OperationId cf6866d7-7350-43a1-9ca7-ff9a9799cd49: InProgress
00:54:00.559 
00:54:00.561 Signing completed with status 'Succeeded' in 2.2386901s
00:54:00.561 
00:54:00.561 Successfully signed: node-v25.0.0-testf46e3ead2efedc23ab9f38ab36ada692ecd72437-x64.msi
00:54:00.577 
00:54:35.359 
00:54:35.359 Number of files successfully Signed: 1
00:54:35.362 
00:54:35.362 Number of warnings: 0
00:54:35.364 
00:54:35.364 Number of errors: 0
00:54:35.367 
00:54:35.367 Successfully signed node-v25.0.0-testf46e3ead2efedc23ab9f38ab36ada692ecd72437-x64.msi using signtool

[ryanaslett]

Okay, there appears to be one issue:

The cert appears to be valid for 3 days. I'll investigate on the azure side of things.

(Edit: I may be misunderstanding how this works, and the cert is able to autorenew)

[ryanaslett]

I was misunderstanding this. The cert/signature combination works because the signature is timestamped within the window that the cert is valid. So we should be good to go. (Though other eyes confirming that would give me more confidence)

[ryanaslett]

This looks like its working.

The console output for the most recent nightly builds have the following:

Trusted Signing

Version: 1.0.68

"Metadata": {
  "Endpoint": "https://eus.codesigning.azure.net/",
  "CodeSigningAccountName": "OpenJS-CodeSigning",
  "CertificateProfileName": "NodeJS",
  "ExcludeCredentials": []
}

Submitting digest for signing...

OperationId 7dea2e97-196c-4320-9930-3b38e00ea768: InProgress

Signing completed with status 'Succeeded' in 2.1539244s

Successfully signed: node-v25.0.0-nightly2025060625f2f27a55-arm64.msi


Number of files successfully Signed: 1

Number of warnings: 0

Number of errors: 0

Successfully signed node-v25.0.0-nightly2025060625f2f27a55-arm64.msi using signtool

Digital Signatures show:

Hooray!

[richardlau]

Does the old signing still work, e.g. for https://github.com/nodejs/node/pull/58588? Or will will need to immediately land the change for that? (We'd usually prefer the change to go out first in a current release (i.e. Node.js 24.x) and then later in the LTS release lines.

[ryanaslett]

I do not know if changes were made to the machines that prevent the old signing from working, but it should, if we need to revert the v20 backport to get the release out.