pub-dev Package signing

Note sure if this should be reported here or for the pub client, it probably affects both. After seeing the latest instance of taking over NPM packages by stealing e-mail addresses (which I believe would also be possible on pub.dev), I'm wondering if pub should optionally support signed releases for security-conscious package maintainers and users.

My idea is essentially the following:

Add an option --sign to pub lish that causes the uploaded package to be signed. E.g. pub lish --sign <keyId> for which pub could shell out to something like gpg --armor --local-user <keyId> --detach-sig package.tar.gz to get a package.tar.gz.asc which would also be uploaded to the package repository.
Verifying signatures should probably happen when pub downloads packages. But as a sanity check without security benefits, pub.dev could fetch the key from a signature (for reference, Maven central considers these keyservers) and perhaps check whether the user id of the key matches the publisher.
Then there could either be an environment variable or perhaps a --verify flag to pub get and pub upgrade to handle signatures. I'm thinking of different levels that might be possible:
1. Ignore signatures (the default).
2. Verify signatures if there are any, print warnings for failed checks / unknown keys.
3. Verify signatures if there are any, refuse to download package versions that don't have a signature or can't be verified.
Maybe there could be a small bump to a package score for signing or a tag for signed packages in the search results page to encourage signing, but it would be optional for both publishers and users.
Perhaps pub.dev should enforce new versions being signed if a previous version of a package was signed. Or there could be an option in the admin page of the package to enforce signed releases.

I think an approach like that has some advantages:

If they want to, users can be reasonably sure that neither an attacker or pub.dev added or modified a package.
pub.dev only needs to host signatures, the creation and verification of signatures happens in the pub client.
It is entirely optional and doesn't impact users who don't care about this.

There are some downsides of this though:

This would only work if dependencies of a package are signed as well, so there's a network effect and it's probably not too useful initially.
It's questionable how many end users are actually interested in this.

May 11 '22 12:05 simolus3

Not that we should not do signing - but a few comments:

After seeing the latest instance of taking over NPM packages by stealing e-mail addresses (which I believe would also be possible on pub.dev)

Users on pub.dev are bound to their google oauth id - not their email - so recreating the emails on the domain is not enough to take over the accounts.

For publishers we do not delete a publisher if someone's domain expires, so a new buyer cannot take over the already registered publisher.

For malicious users "selling out" the story is different - but here signing also won't help you.

May 12 '22 08:05 sigurdm

For the .NET ecosystem, package signing can protect against account hijacking attacks. NuGet has the concept of "author signing" - an author can sign their package using their code signing certificate. For an attacker to upload malicious Microsoft packages they would need to both hijack Microsoft's account and steal its code signing certificate.

That said, account hijacking can also be mitigated with 2FA 😄

May 13 '22 19:05 loic-sharma

Users on pub.dev are bound to their google oauth id - not their email - so recreating the emails on the domain is not enough to take over the accounts.

That's reassuring to hear!

NuGet has the concept of "author signing" - an author can sign their package using their code signing certificate

That looks pretty interesting. Using certificates from a public authority also means that users don't have to import keys themselves. Combining that with verified publishers enables pub.dev to check the legitimacy of a signature by only allowing certificates with a matching domain. However, it looks like most free CAs don't hand out certificates with a key usage for signing (at least Let's Encrypt doesn't appear to), so the feature might be less accessible than GPG keys.

May 13 '22 21:05 simolus3

However, it looks like most free CAs don't hand out certificates with a key usage for signing (at least Let's Encrypt doesn't appear to), so the feature might be less accessible than GPG keys.

NuGet also has a concept of "repository signing" where the package feed signs all packages it hosts. As a result all packages are signed by the repository, and a subset are also signed by their authors.

May 13 '22 21:05 loic-sharma

off-topic: thoughts on other ideas for mitigating account compromises (and repository compromises)

I think that we should consider low-hanging fruit like:

Adding content-hash to pubspec.yaml, PUB_CACHE and version-listing from pub.dev (@sigurdm already started on this),
Signing of version-listing responses from pub.dev using a key held by pub.dev (making mirrors less sketchy)
(possibly) Adopting elements/ideas from the-update-framework (to make mirroring/http-proxies reasonably sane).

I think mitigating account compromises is important, today we already have:

Usage of Google accounts which support 2FA -- I wish oauth APIs would let us check if users have 2FA enabled.
Activity logs (audit log of actions taken on a package), allowing collaborators to figure out what a compromised account did before it was discovered.
Publishers, which can have multiple members -- so somebody else can take over if an account is compromised.
Email other members of a publisher when a package is published, increasing the likelihood that another member will discover a malicious publishing event.

To further mitigate account compromises we could:

Support multiple roles for members of a publisher (allowing for non-admin members of a publisher),
Enable publishing from github using OIDC tokens (planned, and work slowly started), (This removes the need to store a login session on the users machine, making stealing the credentials harder)
Match contents of a package tarball against git revisions of the repository containing the package
- This would make it much easier to discover malicious code.
- We can score packages lower if they don't have a revision containing the files published.
- The matching done by pub.deb could be verified by third-parties.
Support multi-part approval for publishing on pub.dev (for publisher who enable it)

I think the space of end-to-end signing is very interesting. The idea makes me very exciting because it feels like to solution all trust issues :D But once you get into the details end-to-end signing is really hard, most end-to-end systems run into issues like:

What to do when keys are lost?
What to do when keys expire?
Key expiration is not enforced.
How to track keys being issued...
How to do multi-party sign-off... possible with some certificate schemes.
Users leave keys in plaintext on their machines
Keys are not stored in HSMs (classic gpg issue).
Cost of acquiring a signing key / certificate.
What keys have been used in the past, can an attacker hide that a keys was allowed at some point.

Often, it's easy to run into:

Limited adoption, because setup for publisher incurs a high overhead.
Disregard of invalid signatures because publisher lost/expired their key.

There is some talk about doing end-to-end signing using https://www.sigstore.dev/, which involves exchanging the developers OIDC token (google or github) for a certificate to a private key, then sign the artifact with the private key. The certificate is temporary, I'm guessing that the signature is co-signed by a server that adds a timestamp, or something like that -- I haven't understood the full details. But from what I got the idea is create a signature/certificate that proves that the publisher was in control a github / google account at the time of publishing. It's not perfect, but it alleviate many of the problems with using a GPG key.

I've also heard a crazy idea about using webauthn keys (yubikeys) for signing packages. So after publishing the package, the publisher would go to pub.dev/signing and be prompted to authenticate with a webauthn key. When the website prompts for authentication with webauthn key it can send a random challenge the physical webauthn key will sign with a private key specific to each website. So one could possibly use the hash of the published package as random challenge, and that way get a signature package using a physical key. This has the downside that it's a massive hack, hehe :rofl: (and users will still throw away the physical keys), but the upside is that the user doesn't need to install anything -- everything just works -- all they need is a $25 FIDO key.

Another crazy idea is to have publisher create a subdomain like pubkey-<hash(publicKey)>.mydomain.com, then abuse the fact that by requesting a TLS certificate for this subdomain, the subdomain will end up in certificate-transparency logs. Thus, you need to control a domain inorder to register a key for signing, and it's possible to monitor if new keys are registered by monitoring the certificate-transparency logs. I suspect that sigstore.dev might be a slightly saner way to do the same thing.

Just some random thoughts.

May 17 '22 13:05 jonasfj

Thanks for sharing these ideas! Good to hear that work is already being done in this direction.

About the low-hanging fruits you mentioned:

Signing of version-listing responses from pub.dev using a key held by pub.dev (making mirrors less sketchy)

I assume this is about servers like the pub proxy for China? If I explicitly get my packages from some hosted url (either on a per-package basis or with the environment variable), wouldn't I have to trust that host either way (or at least as much as I trust pub.dev)?

As you said, having this end-to-end chain of trust is interesting because formally I don't even have to trust pub.dev itself. Of course, in practice there's a question of whether I should be using Dart in the first place if I can't trust the Dart team enough to run a package server :D

Usage of Google accounts which support 2FA

But isn't pub's model of (long-lived) tokens side-stepping this? There's an unencrypted file on my machine that would allow anyone to upload packages in my name, and I don't see anything obvious on pub.dev that would help me manage or revoke these tokens if I ever lost access to a device. I'm not asked for a second factor when uploading a package.

When publishing packages to maven central for instance, I need to explicitly log in again and mark a staging repository as ready to be synchronized to the central repository. Having a similar step for pub.dev would already be helpful (I assume that's what you mean with multi-part approval), especially since I can easily manage and revoke my active Google sessions.

What to do when keys are lost?

PGP let's you revoke keys and inform key servers about revoked keys.

Key expiration is not enforced.

If we enforce that keys used to sign packages on pub.dev be uploaded to a set of known keyservers, pub.dev could fetch the key and enforce a not-too-far expiration date.

Users leave keys in plaintext on their machines

To be fair, recent GPG versions ask you to set a passphrase for a secret key.

But from what I got the idea is create a signature/certificate that proves that the publisher was in control a github / google account at the time of publishing. It's not perfect, but it alleviate many of the problems with using a GPG key.

sigstore with cosign looks very interesting! Much easier to use than direct GPG for sure. Having extremely short-lived certificates also sounds easier to manage than dealing with potentially lost keys.

When the website prompts for authentication with webauthn key it can send a random challenge the physical webauthn key will sign with a private key specific to each website. So one could possibly use the hash of the published package as random challenge, and that way get a signature package using a physical key.

This doesn't sound too hacky either, Yubico is describing a similar approach to obtain signatures through WebAuthn.

Thus, you need to control a domain inorder to register a key for signing, and it's possible to monitor if new keys are registered by monitoring the certificate-transparency logs. I suspect that sigstore.dev might be a slightly saner way to do the same thing.

Sigstore sounds like it's essentially the same thing but for OpenID identities instead of domain names. But since there's a push towards using verified publishers (which require control over a domain name), this is something that most package maintainers could probably do already.

May 17 '22 20:05 simolus3

Signing of version-listing responses from pub.dev using a key held by pub.dev (making mirrors less sketchy)

I assume this is about servers like the pub proxy for China? If I explicitly get my packages from some hosted url (either on a per-package basis or with the environment variable), wouldn't I have to trust that host either way (or at least as much as I trust pub.dev)?

Yes, this would do a lot of better enable the Chinese community mirror. But there are also people mirroring in other environments, think institutions with strict network policies (probably some banks and the likes). Currently, these mirrors aren't considered by the client to be mirrors of pub.dev -- they are just third-party repositories that happens to contain the same packages as pub.dev. If there was signing maybe we could do something like letting the client pretend the mirror is pub.dev, so pubspec.lock wouldn't have to change when switching between mirrors (probably other benefits).

Also most cloud environments have something like Cloud Key Management. So most pub servers would have an easy time ensuring that the key used to sign the repository can't be extracted/downloaded. This would increase the ability recover gracefully after a pub server has been compromised.

For third-party pub server, I guess the client would have to trust-on-first-use (TOFU). But for pub.dev we could probably pin the public key into the SDK. It would mostly help mirroring, but would also provide a separate layer of protection aside from just HTTPS. That an attacker could somehow temporarily get hold of a "valid" TLS certificate is not unthinkable, especially not with some users having questionable "security software" installed.

With a bit of inspiration from the-update-framework this could possibly be used mitigate replay-attacks too.

As you said, having this end-to-end chain of trust is interesting because formally I don't even have to trust pub.dev itself. Of course, in practice there's a question of whether I should be using Dart in the first place if I can't trust the Dart team enough to run a package server :D

Yes, and in an ideal world we wouldn't have trust intermediary services like pub.dev. The GPG / PGP model is attractive, but realistically, people just don't go to key-signing parties :partying_face:

Usage of Google accounts which support 2FA

But isn't pub's model of (long-lived) tokens side-stepping this? There's an unencrypted file on my machine that would allow anyone to upload packages in my name, and I don't see anything obvious on pub.dev that would help me manage or revoke these tokens if I ever lost access to a device. I'm not asked for a second factor when uploading a package.

I think you can revoke the token given to the dart pub client using: https://myaccount.google.com/permissions

Possibly, it'll also be revoked when changing password, I don't recall the specifics. It's certainly not ideal, and 2FA mainly makes it harder to break into your account, so an attacker has to steal the file or steal your 2FA -- just guessing your email/password combination isn't enough.

When publishing packages to maven central for instance, I need to explicitly log in again and mark a staging repository as ready to be synchronized to the central repository. Having a similar step for pub.dev would already be helpful (I assume that's what you mean with multi-part approval), especially since I can easily manage and revoke my active Google sessions.

Yes, something like that would be nice. For publishers with multiple members it would also be attractive to require two separate people to approve a publishing event (one staging and another one approving). That would be very strong because breaking into two accounts can get very hard.

Not saying that we're going to solve this right away.

We do right now send an email to all members of a publisher when a package is published. The goal with this is really that we want people to notice unauthorized publishing events (or at least have a chance of doing so). An unauthorized publishing event is not necessarily so bad if it's discovered quickly -- just saying that this is a very different way of thinking about security. If we can discover that something bad happened and fix the situation that can possibly involve less complexity, than trying build mechanisms for preventing unauthorized publishing.

Signing is among other things a mechanism for preventing unauthorized publishing. Doing more to find cases where something potentially bad happened might be better. Sure it's not as strong. But it doesn't necessarily involve as much upfront work on the part of the package author.

PGP let's you revoke keys and inform key servers about revoked keys.

Only if you have key or the revocation codes or something.

Honestly, yes, GPG isn't the worst. The usability is remarkably poor -- every time I have do something with gpg, I spend a non-trivial amount of time reading the manual (that might just be my experience).

And it's hard to start trusting people without a key signing party. I think keybase.io has done a lot for GPG usability, by letting users attest to identity on github, twitter, website, etc (my profile: https://keybase.io/jonasfj). But if with GPG you want to keep your key on a physical token (like yubikey) it can be a fairly complicated setup -- I know I did it: in my personal setup I sign git commits with GPG running of a yubikey, and I use gpg-agent to authenticate SSH using my GPG key sitting on a yubikey. And I have an encrypted offline master key stored on a physical flash disk, so that I register new keys.. But I disabled expiration because refreshing the setup is a hassle, and SSH using GPG doesn't support authenticating with new subkeys, so if I create new subkeys I still have to update places where they are registered. Just saying GPG is awesome, but doing it right is also a decent amount of work.

To be fair, recent GPG versions ask you to set a passphrase for a secret key.

Yes, and an encrypted GPG key is certainly better. But most users don't have their GPG key sitting on a physical key (like yubikey). I wouldn't be surprised if many users store and unlock their master key with the same password as their subkeys.

I guess, ultimately the problem with GPG is that: if we promote signing with GPG on pub.dev, then a lot of users might do it (motivated by "pub points" or a badge on pub.dev). But we have no idea how they manage their key. Using something like webauthn is attractive because we know it involves some physical token that can't be copied (we could even require vendor attestations for the webauthn keys used).

This doesn't sound too hacky either, Yubico is describing a similar approach to obtain signatures through WebAuthn.

Nice, I hadn't seen that link before -- that was exactly what I was thinking of :D

It's the worst idea, it's a bit messy because the signatures published aren't just signatures of package name, version, date-time, content-hash, it also has to include all sorts of extra data like authenticatorData and clientData (which includes the challange). And playing with it yesterday it actually seems like additional fields may be added to clientData in the future -- which should then be ignored when checking in clientData is allowed.

So the verification process certainly has to understand some of the webauthn aspects, it can't just verify some data as being signed by a key. That's why I think it's a bit hacky, but from all other perspectives it's very attractive, because irresponsible key management is hard.

sigstore with cosign looks very interesting! Much easier to use than direct GPG for sure. Having extremely short-lived certificates also sounds easier to manage than dealing with potentially lost keys.

Yes, but if you loose your google/github account, then it would sort of be the same as loosing your GPG key. So one can argue that this isn't much different from repository signing with staged publishing that requires sign-off in the browser before publishing.

May 18 '22 11:05 jonasfj

Sorry, this got a bit long :rofl:

If we want to do end-to-end signing I think we should figure out:

Threat model
- What do want to protect against?
- What properties do we want? (replay attacks? compromised server? compromised package author laptop/account?)
What have other eco-systems done?
- How did this work out?
- Are people using it? or do they use an ignore signatures option?
What options do we have? what the pro/cons?
- Leverage GPG ecosystem
  - pro: key-servers and procedures for revocation, sub key, etc.. already exists.
  - con: users can be irresponsible with their keys; usability is poor.
- Use webauthn for signing
  - pro: physical key
  - con: key rotation and management is procedures have to be defined by us.
Build on sigstore/cosign
- pro: might be used in SLSA
- con: a compromised account can still publish

I think there are way more pros/cons, and we probably have other questions that we should ask. Like do we need a merkle-tree on the server, so packages/keys once published, can only be retracted, never deleted -- or at-least the history that they existed at some point can't be denied.

May 18 '22 12:05 jonasfj

@jonasfj that's a great point about issues that end-to-end signing systems run into. I'll try to summarize how NuGet's package signing fares for these points.

Overview of .NET package signing

First, here's a quick overview of .NET package signing:

All NuGet packages are signed, providing integrity and non-repudiation benefits
All packages are "repository signed" by the package source, nuget.org. The repository signature contains the usernames of the package's owners at the time of upload.
Packages can also be "author signed" by the author before being uploaded to the package source. These packages will have 2 signatures (1 author & 1 repository).
Signatures are created using a code signing certificate. Signatures may live past the code signing certificate's validity date by adding a trusted timestamp to the signature.
End users can configure an allowlist of author/repository certificates.

.NET package signing vs Jonas's concerns

Here's how .NET's package signing fares for the issues previously mentioned by @jonasfj:

What to do when keys are lost?

Signatures are created using an X.509 certificates, these certificates support revocation.

Today, nuget.org declares which code signing certificates it uses for repository signing through an API. To use a new certificate, nuget.org must declare it before using it (otherwise clients will reject the repository signatures).

End users downloading packages can configure an allowlist of accepted code signing certificates. If the end user has such an allowlist, they will need to update this configuration whenever an author/repository moves to a new certificate. In practice, this allowlist configuration has low usage as it is only used by customers with the highest security needs.

What to do when keys expire? Key expiration is not enforced.

NuGet strictly verifies code signing certificates' validity. However, .NET package signatures are accepted past the certificate's expiration if the signature contains a trusted timestamp within the code signing certificate's validity period.

How to track keys being issued... What keys have been used in the past, can an attacker hide that a keys was allowed at some point.

I'm not sure I understand this bit. That said, packages' signatures must contain the full certificate chain (for both code signing and timestamping certificates). In other words, if you have a package you know which certificates were used to sign it. You could download all package versions and determine which certificates were used for each package.

X.509 certificate revocation is known to be... interesting. While supported, it isn't bulletproof.

How to do multi-party sign-off... possible with some certificate schemes.

A NuGet package may have multiple signatures. For example, a package can have an author signature and a repository countersignature.

Users leave keys in plaintext on their machines Keys are not stored in HSMs (classic gpg issue).

The repository signing certificates used by nuget.org are strongly protected. However, there's nothing preventing an author from doing silly things with their code signing certificates...

Cost of acquiring a signing key / certificate. Limited adoption, because setup for publisher incurs a high overhead.

This is exactly why NuGet has repository signing. All packages are signed for free.

Authors can choose to acquire a code signing certificate and sign their own packages. For example, all Microsoft packages are author signed using one of Microsoft's code signing certificates.

The .NET ecosystem also has the .NET Foundation, which provides its projects free code signing certificates. Any open-source .NET project can apply to join the .NET Foundation.

Disregard of invalid signatures because publisher lost/expired their key.

NuGet does not allow end users to opt out of signature verification.

Shortcomings of .NET package signing

In my mind, the real shortcomings are:

Different platforms have different trusted certificate roots. A surprising amount of platforms do not officially support code signing. This is particularly problematic for time stamp authority servers. In the future, .NET will ship with its own certificate trust list to make this experience consistent across platforms.
Certificate revocation checks affect performance. This affects folks in offline environments. NuGet added a feature to disable revocation checks.
The certificate allowlist configuration is inconvenient and verbose. That said, this is an optional feature for those that want to really secure their environment.

Overall, I would say these shortcomings are reasonable and that .NET package signing is an excellent solution to the signing problem. (Disclaimer: I worked on .NET package signing and am biased!)

Resources

See:

https://docs.microsoft.com/en-us/nuget/reference/signed-packages-reference
https://github.com/NuGet/Home/wiki/Package-Signatures-Technical-Details

May 18 '22 18:05 loic-sharma

Still relevant to consider. No new plans though

Sep 25 '25 08:09 sigurdm