"0x8a15005e : The server certificate did not match any of the expected values." when trying to search on msstore with winget 1.4

[sgrienen]

Brief description of your issue

I'm trying to search for a software on msstore specifically with winget 1.4.10173 and I get this error message :

Failed when opening source(s); try the 'source reset' command if the problem persists. An unexpected error occurred while executing the command: 0x8a15005e : The server certificate did not match any of the expected values.

Steps to reproduce

1. winget search 7zip -s msstore
2. winget source reset: "The following sources will be reset if the --force option is given: Name Argument

---

msstore https://storeedgefd.dsx.mp.microsoft.com/v9.0 winget https://cdn.winget.microsoft.com/cache" 3. I opened https://storeedgefd.dsx.mp.microsoft.com/v9.0 in Edge to check if URL was reachable and I got this: "This XML file does not appear to have any style information associated with it. The document tree is shown below. <Error> <Message>No HTTP resource was found that matches the request URI 'https://useast.storeedgefd-origin.xbetservices.akadns.net/v9.0'.</Message> </Error>"

Expected behavior

I would expect to see the available software on the Microsoft Store listed

Actual behavior

Failed when opening source(s); try the 'source reset' command if the problem persists. An unexpected error occurred while executing the command: 0x8a15005e : The server certificate did not match any of the expected values.

Environment

Windows Package Manager v1.4.10173
Copyright (c) Microsoft Corporation. All rights reserved.

Windows: Windows.Desktop v10.0.22621.1105
System Architecture: X64
Package: Microsoft.DesktopAppInstaller v1.19.10173.0

[denelon]

@sgrienen thanks for reporting this.

If you call one of the defined endpoints you will get a more useful/meaningful response. https://storeedgefd.dsx.mp.microsoft.com/v9.0/information

Here is the applicable code for the REST API (latest Swagger document)

https://github.com/microsoft/winget-cli-restsource/blob/da3300c19eff2f5d7378bb8bfc47800d056af23d/documentation/WinGet-1.4.0.yaml#L548-L565

If you use https://editor.swagger.io/ and upload the contents of the API Document you will see a more user-friendly view of the API.

[jadodd-CCG]

I encountered this same error this morning. In my case, my traffic to both the winget and msstore sources was being decrypted/reencrypted by our Palo Alto firewall as part of its SSL inspection feature. Our PKI is properly configured, so certificates issued by the firewall are valid and trusted on my machine itself, but it seems winget is sensitive to any tampering (legitimate or not) with the certificate from the msstore endpoint. Adding a SSL inspection bypass for the msstore endpoint URL resolved this issue in my environment.

I'm not sure when this certificate sensitivity was introduced into winget as I have not encountered this error in the past. Also of note, SSL inspection is currently running on the winget source's URL, and everything is working correctly, so this seems limited to just the msstore source.

Please let me know if any additional information would be useful. I don't have a ton of experience contributing to projects like this, but I did want to leave a comment when I realized the same problem I was having was sitting in a day old issue.

[denelon]

@jadodd-CCG,

Thanks for the information! Yes, we recently introduced certificate pinning for the Microsoft Store source in the latest release. It was also present in some earlier preview releases, but this is the first report I've seen.

[obesser]

So, what would be a solution for networks with SSL inspection that replaces the certificate? Maybe an ignore-certificate flag?

[jadodd-CCG]

Based on the pull request that added certificate pinning here: https://github.com/microsoft/winget-cli/pull/2347

An admin setting is added to disable pinning, both as an emergency measure in the event that there is a bug or rolled certificate that was not communicated, but also because there are test scenarios where the user actively wants to disable it (HTTPS redirection via something like Fiddler).

The configuration can be loaded from JSON for future dynamic configuration, but it is currently only as a test hook to enable configuration via Group Policy.

By poking around the commits from that PR, it looks like the overall goal is to have this configurable via group policy and JSON locally, but only the GPO option appears to be written. I'm not sure exactly how the group policy configuration is intended to work, but perhaps someone with some more insight into winget's workings and related group policies could shed some light.

On the more philosophical front, I believe the most user-friendly way to bypass pinning while maintaining at least some of the protection cert pinning is meant to achieve would be to add a flag such as --use-system-cas or --trusted-CA-file that would instruct winget to trust certificate chains from the system certificate store (how normal certificate validation is done) or to provide a specific CA certificate to trust.

[jadodd-CCG]

I love it when I manage to answer my on question about a minute after typing a reply. Changes to the ADMX to support disabling certificate pinning for the Microsoft Store were committed in https://github.com/microsoft/winget-cli/pull/2637 That commit was included in the 1.4.10173 release, I just missed seeing it in the release notes.

For the lazy and anyone who may end up here via Google:

1. You really should use the ADMX template and apply this setting via group policy if you need this to be done on more than one machine. Group policy can be your friend if you let it.
2. If you just need this to work on your machine and would rather just sledgehammer it with regedit, then create a DWORD named EnableBypassCertificatePinningForMicrosoftStore with a value of 1 at the path HKLM\Software\Policies\Microsoft\Windows\AppInstaller Note that this information is only current as of the PR linked above and may change at any time. Always consult current documentation if possible.

[denelon]

Some additional information:

The certificate pinning for the "msstore" source was put in place as an additional security measure to ensure your machine is actually talking to the "msstore" source. Disabling this check increases the potential risk of a MITM attack.

[faustool]

I would expect an urgent fix to allow a list of trusted CAs for Winget, or maybe just pull from the Windows Trusted CAs store.

On a side note, this SSL inspection feature is a pain. There's no standard configuration across applications and tools to configure trusted CAs. In my computer I have REQUESTS_CA_BUNDLE for Azure CLI, GIT_SSL_CAINFO for git, NODE_EXTRA_CA_CERTS for node and the multiple Java trust stores (one per JVM) for Java applications and tools like Maven and Gradle.

Now I need one for Winget. What will it be this time? Winget is a Windows-only tool, right? Please get it from Windows Trust Store.

[denelon]

I'm working on updating the documentation at Microsoft Learn to clearly explain the certificate pinning enhancement for the Microsoft Store "msstore" source.

The enhancement is designed to help reduce the risk of a site impersonating the Microsoft Store REST endpoint. WinGet was enlightened with the thumbprint for the certificate used by the Microsoft Store REST endpoint so WinGet will know it is communicating with the correct "source".

In the event described above, where networking infrastructure is modifying the connection, WinGet will return the error. The two options are to bypass these checks for the Microsoft Store REST endpoint in the networking infrastructure, or to disable this check in WinGet. This can be done by Group Policy, or by the administrative setting: winget settings --enable BypassCertificatePinningForMicrosoftStore

[denelon]

winget settings export will export the administrator settings. This work was done as a part of building DSC Resources to manage WinGet.

[faustool]

I'm working on updating the documentation at Microsoft Learn to clearly explain the certificate pinning enhancement for the Microsoft Store "msstore" source.

The enhancement is designed to help reduce the risk of a site impersonating the Microsoft Store REST endpoint. WinGet was enlightened with the thumbprint for the certificate used by the Microsoft Store REST endpoint so WinGet will know it is communicating with the correct "source".

In the event described above, where networking infrastructure is modifying the connection, WinGet will return the error. The two options are to bypass these checks for the Microsoft Store REST endpoint in the networking infrastructure, or to disable this check in WinGet. This can be done by Group Policy, or by the administrative setting: winget settings --enable BypassCertificatePinningForMicrosoftStore

So the implementation is not actually validating the certificate chain like apt, git, npm, node, openssl, Java and others do. That's why it can't simply "trust" the proxy re-encryption certificate. The only alternative is to disable the feature and fallback to the same level of security we had before.

[JohnMcPMS]

The certificate chain was previously, and continues to be, validated as trusted on the system; regardless of any configuration of settings applied. That is not something we would change.

The pinning feature adds an additional check to ensure that the chain is not just any trusted chain, but is a fairly specific one. Disabling the feature as described previously simply turns that check off and goes back to allowing any trusted chain.

The goal is to prevent supply chain attacks, securing the channel all the way up to the application level. It is properly detecting tampering on the channel via this SSL inspection (aka man-in-the-middle). If an exception for the Store DNS name is not acceptable for your organization, then disabling the feature is the correct action to take.

[faustool]

That settles it for me then, thank you very much for the clarification.

[jcrben]

The two options are to bypass these checks for the Microsoft Store REST endpoint in the networking infrastructure, or to disable this check in WinGet. This can be done by Group Policy, or by the administrative setting: winget settings --enable BypassCertificatePinningForMicrosoftStore.

You say there's two options the proposed solution here sounds like it's just disabling the check in winget? How about the bypassing of these checks? And is there a way to do this without involving an administrator?

[JohnMcPMS]

I was saying the two options are:

1. Changes to the network device policies that prevent it from inspecting the traffic to the Microsoft Store REST service.
2. Disabling the certificate pinning in winget

While I don't know what access would be required to enact option 1, I suspect it is under the control of a very few IT admins for any given operation.

And option 2 requires one to be an administrator on the machine (or to be put in place by group policy). There is no way around this requirement as it is in place to prevent an EoP chain attack on the user.

[jcrben]

@JohnMcPMS in effect strongly discouraging the usage of the winget cli in secure corporate environments. Good to know. Hard to see how either of those options is acceptable in secure corporate environments.

The first seems like the right approach but as you note likely under the control of a few IT people who are tricky to access and convince.

[aydeisen]

@JohnMcPMS in effect strongly discouraging the usage of the winget cli in secure corporate environments. Good to know. Hard to see how either of those options is acceptable in secure corporate environments.

The first seems like the right approach but as you note likely under the control of a few IT people who are tricky to access and convince. @jcrben

The IT people have probably already run into this because certificate pinning is heavily used by mobile apps. They most likely work around it by issuing corporate mobile devices that are tightly controlled by an MDM solution.

It's also important to point out that certificate pinning is only coming up with the msstore as a source, and not with winget itself as a source. The winget cache is not preventing HTTPS Inspection due to HTTPS inspection.

@denelon

I'm personally seeing two things related to this issue that are frustrating:

1. If I try to run winget upgrade against an app that the winget data shows was sourced from winget, and not msstore, I should not have to explicitly define the source switch to run the upgrade, especially when running winget upgrade --all
2. Something seems off with how certificate pinning is working with winget. I have *.mp.microsoft.com excluded from DPI-SSL on my firewall. The Microsoft Store app, which I assume is using the same endpoints, is working fine. winget is continuing to throw the error regarding the server certificate. What's different about winget that it's not working when the Microsoft Store app is?

[denelon]

@JohnMcPMS John McPherson FTE in effect strongly discouraging the usage of the winget cli in secure corporate environments. Good to know. Hard to see how either of those options is acceptable in secure corporate environments.

The first seems like the right approach but as you note likely under the control of a few IT people who are tricky to access and convince.

@jcrben it's just another layer of security. It's actually intended to improve security. If an enterprise has such a firewall, it would likely be an IT function to either disable the SSL inspection for the Microsoft Store source in their firewall, or to disable the check for the certificate pinning by the client.

[denelon]

@aydeisen if you run winget source export you can see the URL for the "msstore" REST source:

winget source export
{"Arg":"https://storeedgefd.dsx.mp.microsoft.com/v9.0","Data":"","Identifier":"StoreEdgeFD","Name":"msstore","Type":"Microsoft.Rest"}
{"Arg":"https://cdn.winget.microsoft.com/cache","Data":"Microsoft.Winget.Source_8wekyb3d8bbwe","Identifier":"Microsoft.Winget.Source_8wekyb3d8bbwe","Name":"winget","Type":"Microsoft.PreIndexed.Package"}

The URL we're pinning the certificate for is: https://storeedgefd.dsx.mp.microsoft.com/v9.0

[aydeisen]

@aydeisen if you run winget source export you can see the URL for the "msstore" REST source:

winget source export
{"Arg":"https://storeedgefd.dsx.mp.microsoft.com/v9.0","Data":"","Identifier":"StoreEdgeFD","Name":"msstore","Type":"Microsoft.Rest"}
{"Arg":"https://cdn.winget.microsoft.com/cache","Data":"Microsoft.Winget.Source_8wekyb3d8bbwe","Identifier":"Microsoft.Winget.Source_8wekyb3d8bbwe","Name":"winget","Type":"Microsoft.PreIndexed.Package"}

The URL we're pinning the certificate for is: https://storeedgefd.dsx.mp.microsoft.com/v9.0

@denelon

I am fully aware of this. Based on the URL, my exclusion for *.mp.microsoft.com from DPI-SSL inspection would encompass that URL. I know that my exclusion works because I'm not receiving errors from the Microsoft Store app.

If both the Microsoft Store App (22212.1401.8.0) and winget (v1.4.10173) are executing from the same machine with the same firewall restrictions and looking in the msstore endpoint, I would assume consistent results between the two. Instead, the Microsoft Store app is working whereas winget continues to state the server certificate doesn't match.

I can't find documentation that tells me how to identify the difference between the two or whether the issue is truly related to the app or DPI-SSL inspection

[denelon]

@adydeisen If you enable the policy for "Bypass Certificate Pinning For Microsoft Store" on the machine, do you still get the error when you try to install packages?

Do you see different behavior if you specify the source in the command?

[aydeisen]

@denelon

Do you see different behavior if you specify the source in the command?

Yes, if I explicitly specify --source winget, I do not receive an error. I object to being required to provide the parameter when winget already knows I installed a package from the winget source instead of the msstore source, but it does work.

The error does persist if I say --source msstore, so the issue is with the msstore source

If you enable the policy for "Bypass Certificate Pinning For Microsoft Store" on the machine, do you still get the error when you try to install packages?

no, enabling the setting prevents the error. Given the choice, I would prefer not to bypass certificate pinning and allow winget to confirm its certificate chain is what is expected

[denelon]

When multiple sources are configured, there may be matches returned from any of those sources.

WinGet doesn't "know" if the current version of the package was installed from a specific source or by the user manually, or if the package upgraded itself. The source column is used to indicate a match with a manifest in one or more configured sources.

I don't intend for users to "have" to specify a source. I was asking as a troubleshooting mechanism.

WinGet still does check the root of trust for the "msstore" source as @JohnMcPMS stated above. The certificate pinning is simply another layer of validation intended to ensure the connection is with the expected endpoint.

[aydeisen]

WinGet doesn't "know" if the current version of the package was installed from a specific source or by the user manually, or if the package upgraded itself. The source column is used to indicate a match with a manifest in one or more configured sources.

I must be misunderstanding something then:

If I run winget list to view installed packages, I still get a source column that's populated for packages I installed from winget, and blank for packages I did not.

When referring to packages I already have installed, am I to understand that the source column is not telling that it's a winget installed package when the source column is populated?

[Masamune3210]

List currently doesnt show msstore matches iirc due to some matching weirdness that is still being worked out

[denelon]

WinGet doesn't "know" if the current version of the package was installed from a specific source or by the user manually, or if the package upgraded itself. The source column is used to indicate a match with a manifest in one or more configured sources.

I must be misunderstanding something then:

If I run winget list to view installed packages, I still get a source column that's populated for packages I installed from winget, and blank for packages I did not.

There is a subtle distinction here. The source column is populated with "winget" when an installed package appears to match a manifest in the "winget" source. It doesn't matter how they were actually intsalled.

When referring to packages I already have installed, am I to understand that the source column is not telling that it's a winget installed package when the source column is populated?

[aydeisen]

There is a subtle distinction here. The source column is populated with "winget" when an installed package appears to match a manifest in the "winget" source. It doesn't matter how they were actually intsalled.

Got it; that was a failure on my part on how I understood what that column meant, and why my first question was nonsensical.

Thanks for the clarification

[denelon]

WinGet 1.4 originally had a certificate root for the "msstore" source that was deprecated. We did have a 1.4 servicing release with the new certificate. https://github.com/microsoft/winget-cli/releases/tag/v1.4.11071

WinGet 1.5 is now out and also has the correct certificate. https://github.com/microsoft/winget-cli/releases/tag/v1.5.1572 We've updated the https://aka.ms/getwinget link to point to the latest stable release.

[Maxime-ADAGP]

WinGet 1.5 is now out and also has the correct certificate.

This seemed to be the problem in my case and the new version solved it. I hope everyone else will have the same experience!

EDIT: I was previously running winget v1.4.10173

[mimoguz]

I still have this problem on my home desktop:

➜ winget --version
v1.5.1572

➜ winget upgrade --source msstore
Failed when opening source(s); try the 'source reset' command if the problem persists.
An unexpected error occurred while executing the command:
0x8a15005e : The server certificate did not match any of the expected values.

Enabling BypassCertificatePinningForMicrosoftStore just changes the error from 0x8a15005e : The server certificate did not match any of the expected values. to WinHttpReceiveResponse: 12152: The server returned an invalid or unrecognized response.