OpenIDConnect provider's HTTPS certificate doesn't match configured thumbprint

[jamesbrooks94]

Hey,

For the past hour or so I’ve been getting this error while using this action: Error: OpenIDConnect provider's HTTPS certificate doesn't match configured thumbprint

I’ve checked the thumbprint and it’s correctly set, would this be an issue with GitHub, are there any known issues at the moment?

[jamesbrooks94]

Additionally (this may be completely unrelated, if so I will open a new issue) intermittently we are getting this error from the action

Run aws-actions/configure-aws-credentials@ea7b857d8a33dc2fb4ef5a724500044281b49a5e
with:
    role-to-assume: arn:aws:iam::***:role/***
    role-duration-seconds: 3600
    aws-region: eu-west-1

Error: Credentials could not be loaded, please check your action inputs: Could not load credentials from any providers

[suzuki-shunsuke]

I found a tweet.

https://twitter.com/arkadiyt/status/1481418230082248714

For anyone using Github Actions -> AWS role assumption & suddenly getting "OpenIDConnect provider's HTTPS certificate doesn't match configured thumbprint" - Github updated their certificate chain and the new thumbprint to use is 6938fd4d98bab03faadb97b34396831e3780aea1

[jamesbrooks94]

Thanks @suzuki-shunsuke I've also noticed there’s a PR open already, however the thumbprints are different!

https://github.com/aws-actions/configure-aws-credentials/pull/355/files

[int128]

I could obtain the new thumbprint as the AWS official doc https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc_verify-thumbprint.html.

% curl https://token.actions.githubusercontent.com/.well-known/openid-configuration
{"issuer":"https://token.actions.githubusercontent.com","jwks_uri":"https://token.actions.githubusercontent.com/.well-known/jwks","subject_types_supported":["public","pairwise"],"response_types_supported":["id_token"],"claims_supported":["sub","aud","exp","iat","iss","jti","nbf","ref","repository","repository_owner","run_id","run_number","run_attempt","actor","workflow","head_ref","base_ref","event_name","ref_type","environment","job_workflow_ref"],"id_token_signing_alg_values_supported":["RS256"],"scopes_supported":["openid"]}%

% openssl s_client -servername token.actions.githubusercontent.com -showcerts -connect token.actions.githubusercontent.com:443
CONNECTED(00000005)
...
 1 s:/C=US/O=DigiCert Inc/CN=DigiCert TLS RSA SHA256 2020 CA1
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root CA
-----BEGIN CERTIFICATE-----
MIIE6jCCA9KgAwIBAgIQCjUI1VwpKwF9+K1lwA/35DANBgkqhkiG9w0BAQsFADBh
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3
d3cuZGlnaWNlcnQuY29tMSAwHgYDVQQDExdEaWdpQ2VydCBHbG9iYWwgUm9vdCBD
QTAeFw0yMDA5MjQwMDAwMDBaFw0zMDA5MjMyMzU5NTlaME8xCzAJBgNVBAYTAlVT
MRUwEwYDVQQKEwxEaWdpQ2VydCBJbmMxKTAnBgNVBAMTIERpZ2lDZXJ0IFRMUyBS
U0EgU0hBMjU2IDIwMjAgQ0ExMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKC
AQEAwUuzZUdwvN1PWNvsnO3DZuUfMRNUrUpmRh8sCuxkB+Uu3Ny5CiDt3+PE0J6a
qXodgojlEVbbHp9YwlHnLDQNLtKS4VbL8Xlfs7uHyiUDe5pSQWYQYE9XE0nw6Ddn
g9/n00tnTCJRpt8OmRDtV1F0JuJ9x8piLhMbfyOIJVNvwTRYAIuE//i+p1hJInuW
raKImxW8oHzf6VGo1bDtN+I2tIJLYrVJmuzHZ9bjPvXj1hJeRPG/cUJ9WIQDgLGB
Afr5yjK7tI4nhyfFK3TUqNaX3sNk+crOU6JWvHgXjkkDKa77SU+kFbnO8lwZV21r
eacroicgE7XQPUDTITAHk+qZ9QIDAQABo4IBrjCCAaowHQYDVR0OBBYEFLdrouqo
qoSMeeq02g+YssWVdrn0MB8GA1UdIwQYMBaAFAPeUDVW0Uy7ZvCj4hsbw5eyPdFV
MA4GA1UdDwEB/wQEAwIBhjAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIw
EgYDVR0TAQH/BAgwBgEB/wIBADB2BggrBgEFBQcBAQRqMGgwJAYIKwYBBQUHMAGG
GGh0dHA6Ly9vY3NwLmRpZ2ljZXJ0LmNvbTBABggrBgEFBQcwAoY0aHR0cDovL2Nh
Y2VydHMuZGlnaWNlcnQuY29tL0RpZ2lDZXJ0R2xvYmFsUm9vdENBLmNydDB7BgNV
HR8EdDByMDegNaAzhjFodHRwOi8vY3JsMy5kaWdpY2VydC5jb20vRGlnaUNlcnRH
bG9iYWxSb290Q0EuY3JsMDegNaAzhjFodHRwOi8vY3JsNC5kaWdpY2VydC5jb20v
RGlnaUNlcnRHbG9iYWxSb290Q0EuY3JsMDAGA1UdIAQpMCcwBwYFZ4EMAQEwCAYG
Z4EMAQIBMAgGBmeBDAECAjAIBgZngQwBAgMwDQYJKoZIhvcNAQELBQADggEBAHer
t3onPa679n/gWlbJhKrKW3EX3SJH/E6f7tDBpATho+vFScH90cnfjK+URSxGKqNj
OSD5nkoklEHIqdninFQFBstcHL4AGw+oWv8Zu2XHFq8hVt1hBcnpj5h232sb0HIM
ULkwKXq/YFkQZhM6LawVEWwtIwwCPgU7/uWhnOKK24fXSuhe50gG66sSmvKvhMNb
g0qZgYOrAKHKCjxMoiWJKiKnpPMzTFuMLhoClw+dj20tlQj7T9rxkTgl4ZxuYRiH
as6xuwAwapu3r9rxxZf+ingkquqTgLozZXq8oXfpf2kUCwA/d5KxTVtzhwoT0JzI
8ks5T1KESaZMkE4f97Q=
-----END CERTIFICATE-----
...

# saved the root certificate as cert.crt

% openssl x509 -in cert.crt -fingerprint -noout | sed -e 's/://g'
SHA1 Fingerprint=6938FD4D98BAB03FAADB97B34396831E3780AEA1

% openssl x509 -in Downloads/cert.crt -fingerprint -noout | sed -e 's/://g' | tr '[:upper:]' '[:lower:]'
sha1 fingerprint=6938fd4d98bab03faadb97b34396831e3780aea1

It seems the new thumbprint is 6938fd4d98bab03faadb97b34396831e3780aea1.

[jakejscott]

FYI you can also run this command:

openssl s_client -connect token.actions.githubusercontent.com:443 < /dev/null 2>/dev/null | openssl x509 -fingerprint -noout -in /dev/stdin
SHA1 Fingerprint=15:E2:91:08:71:81:11:E5:9B:3D:AD:31:95:46:47:E3:C3:44:A2:31

Just remove the : and lowercase everything

[stevie-]

How to get new thumbprint in advance? This will break once every year.

[zarnovican]

@jakejscott that command is giving a different fingerprint because you take the first cert of the chain, not the last as doc suggests.

https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc_verify-thumbprint.html

If you see more than one certificate, find the last certificate displayed (at the end of the command output). This contains the certificate of the top intermediate CA in the certificate authority chain.

[danieljamesscott]

Does anyone know if this was this replaced due to expiration of the old cert, or a security issue? I'd like to know whether to add the new cert in addition to the old one, or should I replace the old thumbprint.

[sylr]

For those using terraform to manage the OIDC provider in AWS:

data "tls_certificate" "github" {
  url = "https://token.actions.githubusercontent.com/.well-known/openid-configuration"
}

resource "aws_iam_openid_connect_provider" "github" {
  url             = "https://token.actions.githubusercontent.com"
  thumbprint_list = [data.tls_certificate.github.certificates[0].sha1_fingerprint]
  client_id_list  = ["sts.amazonaws.com"]
}

[altjx]

I understand there's a new thumbprint, but how exactly do we apply this to the github action?

      - name: Configure AWS credentials via role assumption
        uses: aws-actions/configure-aws-credentials@v1
        with:
          role-to-assume: arn:aws:iam::<my_account_id>:role/<my_role>
          role-duration-seconds: 1800 # the ttl of the session, in seconds.
          aws-region: us-east-1

Can we provide the new thumbprint here, or what exactly must be done? Does this get applied on the AWS side? A bit confused. but still researching.

[lex64]

@altjx has to be updated in AWS IAM --> Identity Providers -->(token.actions.githubusercontent.com) Find Thumbprints on right side and add new one 6938fd4d98bab03faadb97b34396831e3780aea1 or through cloudformation/terraform is you are using IaC. But, also, I am expecting some

[zarnovican]

FYI, even after 12h+ since the change, I'm still getting two different cert chains from the same hostname. Simple test:

while true; do echo | openssl s_client -servername token.actions.githubusercontent.com -showcerts -connect token.actions.githubusercontent.com:443 2>&1 | grep -c 'BEGIN CERTIFICATE'; sleep .1; done

In ~30% of cases, the chain has 3 certificates, instead of 2:

 0 s:C = US, ST = California, L = San Francisco, O = "GitHub, Inc.", CN = *.actions.githubusercontent.com
 1 s:C = US, O = DigiCert Inc, CN = DigiCert TLS RSA SHA256 2020 CA1
 2 s:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA
   i:C = IE, O = Baltimore, OU = CyberTrust, CN = Baltimore CyberTrust Root

fingerprints:
15:E2:91:08:71:81:11:E5:9B:3D:AD:31:95:46:47:E3:C3:44:A2:31
1C:58:A3:A8:51:8E:87:59:BF:07:5B:76:B7:50:D4:F2:DF:26:4F:CD
FB:20:FA:8A:6A:93:B3:75:F0:54:81:4F:9E:00:27:3E:A5:1A:61:38

vs

 0 s:C = US, ST = California, L = San Francisco, O = "GitHub, Inc.", CN = *.actions.githubusercontent.com
 1 s:C = US, O = DigiCert Inc, CN = DigiCert TLS RSA SHA256 2020 CA1
   i:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA

fingerprints:
15:E2:91:08:71:81:11:E5:9B:3D:AD:31:95:46:47:E3:C3:44:A2:31
69:38:FD:4D:98:BA:B0:3F:AA:DB:97:B3:43:96:83:1E:37:80:AE:A1

Fortunatelly, this is not a problem for OIDC, since Amazon is getting the same cert chain every time. At least from what I have seen. This problem may be ISP/country specific.

[demiurg]

Is there recommended course of action for mitigation? Spent an hour thinking I broke something until I found this thread.

[sylr]

Is there recommended course of action for mitigation? Spent an hours thinking I broke something until I found this thread.

No, I don't think there is. The trust chain you built depends on the certificate used by the provider. If the provider changes it, you must update the fingerprint you trust in the receiving end of the chain.

[buffyg]

There is a viable mitigation here: rotating certificates includes the possibility of changing trust chains in the course of that. What Github could do is publish the trust bundles for their expected CAs, users retrieve that bundle, extract the thumbprints for all valid roots. This is a list, so you can provide multiple fingerprints so that you can tolerate getting different CAs per connection while a rotation is happening, since this is a distributed consistency problem that shouldn't be taken for an atomic operation for clients.

Publishing trust bundles is in fact what Google does for the GCP IdP, but here it seems that the AWS resource definition requires not just the trust vector but enumeration of all the certificates used. Trust vector and name really should be sufficient, but Github could still give users relief by publishing the certs with their chains.

This is preferable to having to look at CT to look for all issuance against the host name, retrieving all valid roots and passing those, as CT is a detective rather than a preventive control against incorrect issuance. If you want to do pinning right, which you generally should for an IdP, you need to verify the root of trust for first use, can use that root to pull the bundle thereafter for updates.

You could also pull published trust bundles then look at what's in CT to determine, but this really points to trust qualification being fairly wonky in the OIDC/OAuth world, which generally expect you either to trust a name via whatever WebPKI universe you accept, or you trust a specific key. AWS wants the latter, the sane middle ground is that you want a name and a specific trust vector, such that you can give trust bundles as opposed to individual HTTPS certificate.

Whatever should be done, there's a bunch of complexity that really should be hidden via an IdP datasource provider.

[pruthvirajksuresh]

@buffyg aws allows upto 5 thumbprint . (just a consideration )

[buffyg]

Larger point:

@buffyg aws allows upto 5 thumbprint . (just a consideration )

But the point is that getting all possible server certificates during a rotation while pinning is not something that a client has ready visibility into because you aren't supposed to need to know that. Such are the mechanics of TLS with WebPKI: you don't look up a key or set of keys for a name, you connect to the thing with that DNS name, you retrieve a cert, validate it against your set of trust anchors. Self-selected published trust vectors so that you pin to issuance path rather than individual certificates, that's the most tractable form because it follows the design of TLS+WebPKI, and it still needs publication by a side channel because that's not what JOSE/JWKS provides for. Github as things stand just does plain WebPKI, as best as I can see.

All possible server certificates, that's still potentially tractable by adding a side channel for publishing all certificates that are meant to be in use, but it's not at all how things were designed to work in the first place, is treating WebPKI like self-signed keys which therefore all need to be published, hence the fragility in handling rotations.

[buffyg]

I'm pretty confident I've accurately described the way CA pinning and related trust bundle publication is supported by the GCP IdP because the details are my best recollection of what I got directly from @agl, but I'll tag him here in case I'm mistaken.

[GrahamCampbell]

Duplicate of https://github.com/aws-actions/configure-aws-credentials/pull/355.

[M1kep]

Duplicate of #355.

That's a "Pull Request" and this is an "Issue"

[andymckay]

👋🏾 Thanks all for spotting this and sorry about the issue. We've issued a change log here: https://github.blog/changelog/2022-01-13-github-actions-update-on-oidc-based-deployments-to-aws/.

This is something we'll be looking into to ensure this doesn't happen again.

[zarenner]

👋 To expand on Andy's comment, I'd like to answer a few questions raised.

Does anyone know if this was this replaced due to expiration of the old cert, or a security issue? I'd like to know whether to add the new cert in addition to the old one, or should I replace the old thumbprint.

@danieljamesscott This was a routine rotation due to upcoming expiration. As such, you do not need to remove the previous thumbprint, but may do so as it is no longer in use.

How to get new thumbprint in advance? This will break once every year.

@stevie- This issue was caused by the intermediate CA changing in our certificate's trust chain, which we don't expect to necessarily occur during every SSL rotation/renewal. We're investigating here to determine how to avoid this changing when possible, and will ensure adequate communication in the future if it does need to change and user action is required.

Whatever should be done, there's a bunch of complexity that really should be hidden via an IdP datasource provider.

@buffyg Thanks for your thorough analysis, we'll be looking into options here. The AWS documentation implies there may be some options here to reduce the need for customers to update the thumbprint themselves, although our engineering team will need to investigate if they are feasible.

Again, I'd like to apologize for this issue and reiterate that we are taking steps to ensure this doesn't happen again.

[mckalexee]

While it's always annoying to have something like this break deployments, I will say it's quite nice that I was able to:

1. Go to the repository for the actions
2. Look at the issues
3. See a resolution (thanks @suzuki-shunsuke!)
4. See that the maintainers are aware of the issue and finding ways to prevent it in the future

We've all had to deal with support hell and black boxes when it comes to issues and resolutions before. The community, collaboration, and transparency here is refreshing!

[stevie-]

For those using terraform to manage the OIDC provider in AWS:

data "tls_certificate" "github" {
  url = "https://token.actions.githubusercontent.com/.well-known/openid-configuration"
}

resource "aws_iam_openid_connect_provider" "github" {
  url             = "https://token.actions.githubusercontent.com"
  thumbprint_list = [data.tls_certificate.github.certificates[0].sha1_fingerprint]
  client_id_list  = ["sts.amazonaws.com"]
}

I really like that solution, but how secure is it, especially in terms of man-in-the-middle attacks?

[smailc]

For those using terraform to manage the OIDC provider in AWS:

data "tls_certificate" "github" {
  url = "https://token.actions.githubusercontent.com/.well-known/openid-configuration"
}

resource "aws_iam_openid_connect_provider" "github" {
  url             = "https://token.actions.githubusercontent.com"
  thumbprint_list = [data.tls_certificate.github.certificates[0].sha1_fingerprint]
  client_id_list  = ["sts.amazonaws.com"]
}

I really like that solution, but how secure is it, especially in terms of man-in-the-middle attacks?

What are the security implications of having the wrong fingerprint? The Idp queries will still go to token.actions.githubusercontent.com ?

Also out of interest why are we querying https://token.actions.githubusercontent.com/.well-known/openid-configuration rather than just https://token.actions.githubusercontent.com to get the cert information?

It works well for me by the way (using both urls), thanks!

[buffyg]

The AWS documentation implies there may be some options here to reduce the need for customers to update the thumbprint themselves, although our engineering team will need to investigate if they are feasible.

@zarenner I've also talked to AWS about making the underling trust ceremony corporate as opposed to per-customer. They already trust a number of IdPs, suggested they do the same with you, the mechanics can be between your engineering orgs.

I suggested using WebPKI issuance pins, that's what AWS says, which is then garbled by Terraform (looking at the AWS docs, it starts out talking about using CA pins, but the Terraform resource docs instead suggest these should be server thumbprints, not CA, "A list of server certificate thumbprints for the OpenID Connect (OIDC) identity provider's server certificate(s)"). The AWS doc, which gives the procedure for relying on root CAs, is https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc_verify-thumbprint.html and the Terraform resource doc is https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_openid_connect_provider.

I've since been pointed to a write-up from @sleevi on considerations for OIDC FastFed, suggesting that subsetting WebPKI issuance has viability issues and that Private PKI should be used instead. Also should be pointed out that DigiCert has reduced the lifetime of their ICAs precisely to discourage pinning via ICA: https://knowledge.digicert.com/alerts/DigiCert-ICA-Update.html

Sleevi's notes are here: https://docs.google.com/document/d/e/2PACX-1vQQ-FhNjW0ZhZVTnK1VG_87IBNZKBaJmweYZb1VBRdQCMAWXekqKfxdNl8wL6FWFDJ9pxXxpr66-GZp/pub

[sleevi]

@zarenner mentioned wanting to explore ways to avoid having the issuing CA change. To echo and emphasize @buffyg’s comment, the WebPKI doesn't provide that guarantee, somewhat intentionally.

DigiCert has been explicit about this, and it’s a huge credit to them for implementing organizational changes to help surface these risks during routine situations (such as rotations), where folks are aware and can respond to consequences, because the alternative is these sorts of things get surfaced during non-routine situations, like responses to CA compromise or distrust.

A big part of this philosophy was seeing how long it took organizations to successfully transition off certificates from DigiNotar, WoSign/StartCom, and Symantec, three CAs that demonstrated significant security control failures. For an unfortunately large number of users, they had trouble rotating certificates precisely because they had end users who had pinned.

Beyond such cases, though, the WebPKI ecosystem has intentionally been moving to shorter (effective) lifetimes to root certificates. When you use a 10-year old root, for example, you also inherit 10 years of any mistakes it has made potentially causing harm to your organization. The old Verizon PKI demonstrated this best: Verizon did not even know everyone they had issued sub-CAs to, any of which could have maliciously issued certificates that anyone who trusted the Verizon roots would accept. DigiCert, when they acquired Verizon, spent an incredible amount of time investigating and revoking those certificates, because that’s a huge risk.

As one of the authors of RFC 7469, and as one who helped drive removing support from Chrome, I’ve definitely been trying to help educate folks about the dangers of pinning. Since we’re engineers here, the way I would describe Certificate pinning is a bit like reaching into an opaque C struct: you’re reaching into implementation details that aren’t ABI or API guaranteed.

That is, if you had a foo.h that had

typedef struct FOO_internal FOO;

then it would be inherently unsafe in your libraries bar.c to do

void baz(FOO* foo) {
  *((int*)((char*)(foo) + 12)) = 42;
}

because you’re reaching in to the internal implementation details and “violating” the API contract.

Pinning to the WebPKI is basically that: reaching into implementation details of the trust graph that, by design, aren’t guaranteed to be stable. If you need a stable API, then private PKI let’s you define those contracts as you need.

Hopefully this doesn’t come off as a lecture 😅 It’s just very much “here be dragons”, and any pinning to the Web PKI is going to (eventually) explode, because just like software itself needs to get regularly updated to respond to security risks, so does the WebPKI.

Note: while Google does publish a root bundle at https://pki.goog, that’s not really for encouraging pinning, and you’ll note it’s a broad selection of CAs. However, even with that broad selection, there have still been operational issues where Google ends up getting a certificate outside of that set, and interoperability issues can come up. So publishing a set of trust anchors isn’t much of a solution: it’s primarily there to ensure at least a minimum set or “everyone Google may decide to do business with” is included.

[lorengordon]

Small adjustment to the terraform snippet provided earlier... This will use all the certificates instead of only the 0-index...

data "tls_certificate" "github" {
  url = "https://token.actions.githubusercontent.com/.well-known/openid-configuration"
}

resource "aws_iam_openid_connect_provider" "github" {
  url             = "https://token.actions.githubusercontent.com"
  thumbprint_list = data.tls_certificate.github.certificates[*].sha1_fingerprint
  client_id_list  = ["sts.amazonaws.com"]
}

[sylr]

Small adjustment to the terraform snippet provided earlier... This will use all the certificates instead of only the 0-index...

data "tls_certificate" "github" {
  url = "https://token.actions.githubusercontent.com/.well-known/openid-configuration"
}

resource "aws_iam_openid_connect_provider" "github" {
  url             = "https://token.actions.githubusercontent.com"
  thumbprint_list = data.tls_certificate.github.certificates[*].sha1_fingerprint
  client_id_list  = ["sts.amazonaws.com"]
}

I'm not sure this is advisable, I might be wrong but I think that, at best, it's useless if the target only checks the end certificate of the provider, at worst, it makes the target trust any certificates signed by the same authority GitHub is using.

[lorengordon]

I'm not sure this is advisable, I might be wrong but I think that, at best, it's useless if the target only checks the end certificate of the provider, at worst, it makes the target trust any certificates signed by the same authority GitHub is using.

Oh that may be. I'm not sure. I suppose I saw multiple fingerprints and figured that must be a mechanism to support rotation, such that both current and <next> would work for some period of time.

[lorengordon]

Here's the output of the data source... I'm certainly not an expert on TLS or OIDC. Happy to defer to others that are more knowledgeable.

    "certificates" = tolist([
      {
        "is_ca" = true
        "issuer" = "CN=DigiCert Global Root CA,OU=www.digicert.com,O=DigiCert Inc,C=US"
        "not_after" = "2030-09-23T23:59:59Z"
        "not_before" = "2020-09-24T00:00:00Z"
        "public_key_algorithm" = "RSA"
        "serial_number" = "13567650854749339296468135199911180260"
        "sha1_fingerprint" = "6938fd4d98bab03faadb97b34396831e3780aea1"
        "signature_algorithm" = "SHA256-RSA"
        "subject" = "CN=DigiCert TLS RSA SHA256 2020 CA1,O=DigiCert Inc,C=US"
        "version" = 3
      },
      {
        "is_ca" = false
        "issuer" = "CN=DigiCert TLS RSA SHA256 2020 CA1,O=DigiCert Inc,C=US"
        "not_after" = "2023-01-11T23:59:59Z"
        "not_before" = "2022-01-11T00:00:00Z"
        "public_key_algorithm" = "RSA"
        "serial_number" = "11052166568250529137901871408801069995"
        "sha1_fingerprint" = "15e29108718111e59b3dad31954647e3c344a231"
        "signature_algorithm" = "SHA256-RSA"
        "subject" = "CN=*.actions.githubusercontent.com,O=GitHub\\, Inc.,L=San Francisco,ST=California,C=US"
        "version" = 3
      },
    ])