SCEP from Cisco router failing (HTTP GET not implemented (correctly))

[lindenaar]

trafficstars

Subject of the issue

I am trying to use step-ca as SCEP server for my Cisco homelab. I am confident that Cisco implementation is working but step-ca complains about invalid base64 data in the request.

There seems to be a problem in the decoding of the HTTP GET request (it uses base64.URLEncoding which in case of Cisco should be base64.StdEncoding and I noticed that step-a dit not implement SCEP using HTTP GET requests for the PKIOperation in the Get handler.

I got it working with minor changes (merely reading the TODO and copying existing code), see below.

Please include my changes in the next version, or, at least, provide a proper error message when an SCEP client uses the PKIOperation method over HTTP GET. At this moment the client receives an status 200 with an empty response, which is not correct and not in line with the spec.

Your environment

• OS - Debian (Raspbian) 11.3 armv7
• Client: Cisco 1100 running IOS-XE 16.09
• step-ca version: 0.19.0

Steps to reproduce

working configuration (just run step ca init and add this provisioner to the configuration as per the documentation on configuration of a SCEP provisioner:

{
    "type": "SCEP",
    "name": "scepca",
    "forceCN": true,
    "challenge": "secret1234"
}

On the Cisco side I am using this configuration:

! generate an RSA key pair for the router
crypto key generate rsa label STEP-CA-KEY modulus 2048

! define a trustpoint for SCEP with step-ca
crypto pki trustpoint STEP-CA
 enrollment retry count 100
 enrollment retry period 60
 enrollment url http://192.168.1.1:80/scep/stepca
 serial-number none
 fqdn router.mydomain.tld
 subject-name CN=router.mydomain.tld,dc=mydomain,dc=tld
 revocation-check none
 rsakeypair STEP-CA-KEY
 auto-enroll 10 regenerate

! Import the root CA fro the step-ca responder, this command requires confirmation
crypto pki authenticate STEP-CA

! and start the enrollment with step-ca
crypto pki enroll STEP-CA

Expected behaviour

The Cisco router requesst a certificate through a CSR via SCEP with step-ca and get a signed certificate which gets installed

Actual behaviour

Cisco performs a SCEP request with step-ca, which results in an error, according to the step-ca logfiles:

error="invalid scep get request: illegal base64 data at input byte 274" 

Additional context

Diving into the issue, I can see the SCEP request from the Cisco (I am reluctant to include the whole so it is truncated) from the step-ca logs looks like this:

Apr 29 12:26:41 pi step-ca[24052]: 
time="2022-04-29T12:26:41+02:00" 
level=error 
duration=1.095262ms 
duration-ns=1095262 
error="invalid scep get request: illegal base64 data at input byte 274" 
fields.time="2022-04-29T12:26:41+02:00" 
method=GET 
name=ca 
path="/scep/stepca/pkiclient.exe?operation=PKIOperation&message=MIIK1gYJKoZIhvcNAQcCoIIKxzCCCsMCAQExCzAJBgUrDgMCGgUAMIIF2wYJKoZI%0AhvcNAQcBoIIFzASCBcgwggXEBgkqhkiG9w0BBwOgggW1MIIFsQIBADGCAoQwggKA%0AAgEAMGgwUDELMAkGA1UEBhMCTkwxEjAQBgNVBAoTCUxpbmRlbmFhcjEtMCsGA1UE%0AAxMkTGluZGVuYWFyIFJvb3QgQ2VydGlmaWNhdGUgQXV0aG9yaXR5AhQN4pYduaQM%0ApXqtsP3jCkQIsX%2F%2BJjANBgkqhkiG9w0BAQEFAASCAgBmD2uSKjx%2BZPNCnyhHxFNx%0A9Yy7W6L8mHY3tVODI1jLkS5%2BwRwEiQNxrCyYq0zkrkp%2FB8ssgNQaWMpW8zYZT0s3%0Ax28UxVJOfkiACN82O8XyI%2FyV%2B05uiPGwEkt202NNEKaZP7ze1IASxCJ2K4MBsJFC%0AvnfL9enAQl4dSwcXazeqf6SdlBFAPacYBcsHxetr4BRb%2BsF5XLzgCiGkaYeypvJ0%0Adif3qzovro09Url2KKbCzczw%2Fj3ONTlUFLgBNiHm8d80pLp%2FwEPRj34XmfKCwRBC%0AaTkLZDQWvBO%2B8AxcFoExO%2BGn%2BV7XdWxbWmG1Ey%2B6TC2BCX2Gd%2F1mIJJlk%2FuIRn0w%0ADeYQA%2FB3dyS95DFF1pTjAmPEeJ
[cut]
WtG94%0AjY0P7k9QVIxMFNGVrN4Shh8Y%2BegUC7lo%2BvbsJqhGMK5BQ9JP5ybr2oCTfv3wtoY1%0AtsZOXhvzgmPz3eEYvN9yrP7LKza9CtKkk89EfFVhkaKnqJISV8N%2BMeobCffT1He4%0Ayhb4JrhvU8988IowZ%2BeLza4kxsZnibD7rXlymTQDF8q04yqbtcamkrqj%0A" 
protocol=HTTP/1.0 
referer= 
remote-address=192.168.0.1
request-id=c9lrqoccnn9psu3ug7rg 
size=64 
status=500 
user-agent= 
user-id=

the URLEncoded requests contains "%0A" (newline - this one seems to be ignored so problem) and also "%2F" and "%2B" (the '/' and '+' signs). The error turns out to be about these though they are perfectly valid in Base64 encoding but not in the currently used Base64 URL encoding variant.

Changing in function decodeRequest (scep/api/api.go line 154):

	// TODO: verify this; it seems like it should be StdEncoding instead of URLEncoding
	decodedMessage, err := base64.URLEncoding.DecodeString(message)

into

	// TODO: verify this; it seems like it should be StdEncoding instead of URLEncoding
	decodedMessage, err := base64. StdEncoding.DecodeString(message)

resolves this (though I am not sure whether other implementations may still need URLEncoding, potentially trying both would be better?).

This solves the problem with the invalid Base64 data but still does not make SCEP work from a Cisco.

Looking at tcpdump output for the requests it turns out that step-ca does not send any response back for the request from the Cisco. When performing the logged request from the command line step-ca simply closes the connection without a response. Looking in the code I see that in the Get handler in scep/api/api.go on line 88 step-ca does not do anything:

	case opnPKIOperation:
		// TODO: implement the GET for PKI operation? Default CACAPS doesn't specify this is in use, though

SCEP using HTTP GET is currently not supported at all, unfortunately this is not documented and the client is not informed and no error is visible about this anywhere. Fortunately, copying over the implementation from the Post handler below:

	case opnPKIOperation:
		res, err = h.PKIOperation(ctx, req)

seems to make it work (at least in my case) and allows a Cisco router to request a certificate using SCEP from step-ca

Could you please apply these changes to the next version as well? Thanks!

regards,

Frederik

[hslatman]

Hi @lindenaar,

Thank you for reporting the issue. I've opened a PR here: https://github.com/smallstep/certificates/pull/917.

As you probably already guessed from the code, I initially had my doubts if these are the right things to do. They do make sense technically, but the hard thing with SCEP is that clients may react differently to it. Initially I based my implementation on https://datatracker.ietf.org/doc/html/rfc8894, which tries to set some stricter guidelines for how SCEP should be used. The CA should already have sent a POSTPKIOperation to your client, indicating that the SCEP client should use POST requests instead of GETs. I know of at least one other person who also tried step-ca with Cisco gear and did so successfully, so it may also be possible to configure your Cisco SCEP client to work with the current step-ca implementation.

That said, I think it makes sense to also support your use case, so that's why I created the PR. I'll need to verify that other SCEP clients don't break because of the encoding change or figure out another solution for that. I doubt that it'll be an issue, since the changes are only for GET requests, and I don't know of other people that experienced your issue, so far. Re-reading some relevant parts of the RFC also seem to indicate that standard base64 encoding should be used, so the change makes sense. It's just hard to test many different SCEP clients, because their code is not always readily available.

[lindenaar]

Hi Herman,

Thanks for picking this up immediately. I am definitely not a SCEP expert but looking at the SCEP spec you refer to (especially 4.1. HTTP POST and GET Message Formats) it plain Base46 encoding indeed is the right thing to do.

As for GET vs. POST, the same section states that:

If the remote CA supports POST, the CMS-encoded SCEP messages MUST be sent via HTTP POST instead of HTTP GET. This applies to any SCEP message except GetCACert, GetNextCACert, and GetCACaps and avoids the need for base64 and URL encoding that's required for GET messaging.

However, Cisco routers do not seem to check this at all and simply perform a GET SCEP request. It seems I can have it perform a POST request instead though that seems to requires quite some additional configuration so I am very happy you include changes.

In the mean time, I have seen that with these changes also renewals works nicely in my environment. I will do some more testing with other devices when I can. Let me know in case you need me to test anything specific.

I am still struggling with getting it pass on SANs but at this stage I am unsure what is causing this (whether the Cisco is sending these at all or whether my step-ca setup is not correct). Is there a way to get the decoded request in the debug log (without any sensitive values) ?

[hslatman]

Hey @lindenaar, all these clients doing their own thing is the joy of SCEP 😅

At this moment we don't log the CSR embedded in the SCEP request when the request is successful; it should be logged when it failed, though. That's probably not going to help you further, but it's possible to add some additional output to step-ca: since you already customized step-ca before, you can do so as follows:

In scep/api/api.go, add the following (at about line 292)

fmt.Println(certinfo.CertificateRequestText(csr))

It should come right after

csr := msg.CSRReqMessage.CSR

You'll need to add an appropriate import at the start of the file:

"github.com/smallstep/certinfo"

And run the following before starting the CA:

go get github.com/smallstep/certinfo
go mod vendor

That should result in the full CSR being printed in a readable format.

[lindenaar]

Hi Herman,

Many thanks! this really helped to understand what the Cisco was actually requesting... it turns out they are not included in the SCEP request :-(

Apart from that SCEP now works, looking forward to the next version with your PR included.

I noticed that Cisco routers can pass their fqdn, IP address and serial number as PKCS#9 OIDs as subject. Currently these are not available in templates unfortunately. I will look further into this and raise a separate ticket for that if needed.

Frederik

[hslatman]

Hi @lindenaar,

I've just merged https://github.com/smallstep/certificates/pull/917, so in the next release the GET requests will be supported.

Do you happen to know what SCEP client your Cisco gear uses? I suspect it's something closed source, but it may be based on some existing libraries and we may be able to use that instead to test more easily. Do you have some references to documentations where usage of FQDN, IP and the serial number are explained? I did find some mentions of the serial number here: https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/sec_conn_pki/configuration/15-mt/sec-pki-15-mt-book/sec-cert-enroll-pki.html.

[lindenaar]

Hi Herman,

Thanks again for getting this implemented.

Cisco is pretty closed source and has SCEP implemented for ages in all kind of devices so I doubt whether there is any open-source code or library that they use. Documentation seems to be mostly about how to configure or how to use the protocol in general, not how they have implemented it (till now I have not encountered that in my searches).

I don't mind reverse engineering this and have gotten quite used to that working with these routers for a while). BTW: it looks like Cisco routers implement both a SCEP client and server so if necessary I could test how that implementation responds if that helps. The documentation I use is the one that you found for my IOS routers and in addition this one for IOS-XE.

As I was playing with the other configuration options available in the Cisco router I ended up with the following configuration:

! generate an RSA key pair for the router
crypto key generate rsa label STEP-CA-KEY modulus 2048

! define a trustpoint for SCEP with step-ca
crypto pki trustpoint STEP-CA
 enrollment retry count 100
 enrollment retry period 60
 enrollment url http://192.168.1.1:80/scep/stepca
 usage ike
 usage ssl-server
 serial-number
 fqdn router.mydomain.tld
 ip-address 192.168.0.1
 password 7 XXXXXXXXXXXXXXXXXXXXXX
 subject-name CN=router.mydomain.tld,dc=mydomain,dc=tld
 subject-alt-name san.mydomain.tld
 revocation-check none
 rsakeypair STEP-CA-KEY
 auto-enroll 10 regenerate

! Import the root CA fro the step-ca responder, this command requires confirmation
crypto pki authenticate STEP-CA

! and start the enrollment with step-ca
crypto pki enroll STEP-CA

With a modified version of step-ca as per your instructions that prints the request to the log I see that a request now looks like:

Certificate Request:
     Data:
         Version: 0 (0x0)
         Subject: DC=tld,DC=mydomain,CN=router.mydomain.tld,SERIALNUMBER=FCZXXXXXXXX,UnknownOID=1.2.840.113549.1.9.8,UnknownOID=1.2.840.113549.1.9.2
         Subject Public Key Info:
             Public Key Algorithm: RSA
                 Public-Key: (2048 bit)
                 Modulus:
[CUT]
                 Exponent: 65537 (0x10001)
     Signature Algorithm: SHA1-RSA
[CUT]
  <nil>

Unfortunately the Cisco does not seem to send the subject-alt-name, the Serial Number of the router becomes available in the .Subject.SerialNumber field in templates. This is however far less useful than the two UnknownOIDs, which seem to be PKCS#9 attributes:

UnknownOID	PKCS#9 attribute
1.2.840.113549.1.9.2	unstructuredName
1.2.840.113549.1.9.8	unstructuredAddress

Would be great if these would be available at least in the. Subject or .Insecure.CR.Subject attribute ExtraNames, which is where I would already expect them but at this moment that is not the case. It looks like the ExtraNames are copied over when preparing the template data but still ends up as an empty array for the template. Looks like this is not specific to SCEP but a lot deeper in the CSR handling code?

Does it make sense to keep this issue open now that the initial problem is resolved? I can raise a separate request for this one as well?

[maraino]

Hi @lindenaar, @hslatman,

There seems to be a bug in go.step.sm/crypto in how those extra subject fields are parsed, we should get them from pkix.Name Names instead of ExtraNames as it is right now. I'll create a PR that will allow extracting those from .Insecure.CR.Subject.

[maraino]

@hslatman with https://github.com/smallstep/crypto/pull/42 @lindenaar should be able to extract non-default subject attributes from ExtraNames, and populate them directly to the subject using templates.

[hslatman]

Thank you, @maraino! 😄

So the template to use these should be something like the below after the patch is merged, or am I completely off?

         {
		"subject": {
		        "commonName":  {{ toJson .Insecure.CR.Subject.CommonName }},
                        "extraNames": {{ toJson .Insecure.CR.Subject.ExtraNames }}
		},
	{{- if typeIs "*rsa.PublicKey" .Insecure.CR.PublicKey }}
		"keyUsage": ["keyEncipherment", "digitalSignature"],
	{{- else }}
		"keyUsage": ["digitalSignature"],
	{{- end }}
		"extKeyUsage": ["serverAuth", "clientAuth"]
	}

Or, the most simple case, use this?

         {
		"subject": {{ toJson .Insecure.CR.Subject }},
	{{- if typeIs "*rsa.PublicKey" .Insecure.CR.PublicKey }}
		"keyUsage": ["keyEncipherment", "digitalSignature"],
	{{- else }}
		"keyUsage": ["digitalSignature"],
	{{- end }}
		"extKeyUsage": ["serverAuth", "clientAuth"]
	}

We may want to add some more (somewhat standard) OIDs to the certinfo utility too to be able to show their contents.

[maraino]

So the template to use these should be something like the below after the patch is merged, or am I completely off?

Well, you probably want a DNS SAN too, but yes that's the idea.

[hslatman]

@lindenaar: I've just merged #938, which contains @maraino's patch. You can try it to see if you can use the ExtraNames in a template now. The certinfo utility hasn't been updated yet, so the OIDs will still show using their numeric identifier instead of a textual representation.

I'm OK with continuing in this issue if there are more things, since it's still about requests from Cisco gear failing. If they become more like a detail, then it might be good to create a new one instead.

[lindenaar]

@hslatman: Sorry for the delay in response, I will try to build a new version tomorrow and test it against an IOS-XE router.

[hslatman]

@lindenaar: we're aiming for a release today/tomorrow.

[lindenaar]

@hslatman looking forward to this release, thanks!

I did some testing today with the master branch today and can confirm the new fields indeed are included in the certificate with:

	"subject": {
		        "commonName":  {{ toJson .Insecure.CR.Subject.CommonName }},
                        "extraNames": {{ toJson .Insecure.CR.Subject.ExtraNames }}
		},

I will try to play a bit more with the later today.

[lindenaar]

@hslatman I was a bit to optimistic to close this already, things seem to work fine with IOS-XE 16.09 but when using a similar setup on a Cisco router running IOS 15.7 I get the following error message:

May 26 21:48:42 syamozero step-ca[14513]: time="2022-05-26T21:48:42+02:00" level=error duration=410.143889ms duration-ns=410143889 error="scep get request failed: parse CSR from pkiEnvelope: asn1: structure error: tags don't match (16 vs {class:0 tag:20 length:15 isCompound:false}) {optional:false explicit:false application:false private:false defaultValue:<nil> tag:<nil> stringType:0 timeType:0 set:false omitEmpty:false}  @2" fields.time="2022-05-26T21:48:42+02:00" method=GET name=ca path="/scep/routers/pkiclient.exe?operation=PKIOperation&message=

(I removed the request itself as it's encrypted anyway)

This problem also occurs in the version with only my changes so this is not related to the changes of @maraino .

Any clues where the problem might be or what I am missing?

[hslatman]

@lindenaar: it sounds like the request from this IOS 15.7 client contains something in the CSR that cannot be properly decoded using ASN1. At least not according to the standard Go encoding/asn1 library. In this case it seems to expect a sequence (asn tag 16) but got teletexstring/t61string (asn tag 20).

In the error I seem to be missing the fieldType.Name() as this seems to be empty in the error you got, which originates from https://cs.opensource.google/go/go/+/master:src/encoding/asn1/asn1.go;l=842 (just before the @2 I would expect the name). Without knowing anything about the CSR, could it be that some configuration property is not set?

Can you share the bytes that are getting parsed? The error happens at line 210 in scep/authority.go:

csr, err := x509.ParseCertificateRequest(msg.pkiEnvelope)

Can you print and share the bytes (if no sensitive data is in there) by adding this on the next line:

fmt.Println(base64.StdEncoding.EncodeToString(msg.pkiEnvelope))

[lindenaar]

@hslatman I have minimised / anonymised the request, and added the print of the CRS (just had to add an import for "encoding/base64" to make it work). The only real difference with the one I had yesterday is that I use different names now and removed the serial number of the router. I trust that is not the culprit as I still get the same error (and yes, I know I am using the default example password / secret now)

The raw CSR is:

MIIDJzCCAg8CAQAwgaYxEzARBgoJkiaJk/IsZAE
ZFgN0bGQxGDAWBgoJkiaJk/IsZAEZFghteWRvbWFpbjEQMA4GA1UECxMHcm91dGVyczEgMB4GA1UEAxM
XY2lzY28tMTkyMS5teWRvbWFpbi50bGQxQTAZBgkqhkiG9w0BCQgTDDE5Mi4xNjguMTAuMTAkBgkqhki
G9w0BCQIWF2Npc2NvLTE5MjEubXlkb21haW4udGxkMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgK
CAQEAqWnc/gEQkXGrJmkNDUjUwEL/foD+feftVLjYtT+B5ZjA3B3xIuts72Iz0k67J6U3UYA9Q5yGtT1
rdB4AkdJDpFAZqAhJ6hY0SUHc+6Nr6mBKDbTGK+Q1NEVYWbWuLTJrwaxz8Q68PPZh/ugYi09SDTnFTlg
qALFuc34SsJ0KXbQqbUs5zOeCmfdFsZofoqXcov7HGVT6QuINWondGQdGPPONRizt7qkld8aYLlhZQTj
LJxpJFMgq6AQDWaDerU6LtPwrPJoQ3w0Rctpxpr+AaoXS5RmPxFwyAYVEEXP7RmPUaYdv+FOatzdguS4
dKU4o0lmgDazQq29oCQOOEFaVQwIDAQABoDswGQYJKoZIhvcNAQkHMQwTCnNlY3JldDEyMzQwHgYJKoZ
IhvcNAQkOMREUDzANMAsGA1UdDwQEAwIFoDANBgkqhkiG9w0BAQQFAAOCAQEAhipl54JhTTfttr5U/Mn
ypMP9U6WKSR2vhs9ubez9IISSsN47jy6vlxS4lMIjgU9hc6brlTPzojMM8seYHGO61jx8uAR7habXdR+
Dm7kGADPLMUg41UWTs5xUD5ut8PmNCY+Zfv1mTTdLlIh0MhhjjHWOx6a4sDR6L6pRJmA1+CTrQ31w5t9
6hZUN01yahHaOnYIwZqcwGN4GEwOBDlBHa1Mf3HsuaeViFm2d3GybsFUq2VNzGOQag9ZkY9QNGw5aQ6Z
tM/FzXCa7nIljPUN+464KhCcLX+Eq/SAp3G8gd2NfKZF0ObfDZUs9Q/NfH0P1OPHvnP0M55MRJj5j5h5
LFw==

the error for this specific request is (again):

step-ca[4314]: time="2022-05-27T19:45:25+02:00" level=error duration=434.385466ms duration-ns=434385466 error="scep get request failed: parse CSR from pkiEnvelope: asn1: structure error: tags don't match (16 vs {class:0 tag:20 length:15 isCompound:false}) {optional:false explicit:false application:false private:false defaultValue:<nil> tag:<nil> stringType:0 timeType:0 set:false omitEmpty:false}  @2" fields.time="2022-05-27T19:45:24+02:00" method=GET name=ca path="/scep/routers/pkiclient.exe?operation=PKIOperation&message=

OpenSSL does not see any thing odd (after adding -----BEGIN CERTIFICATE REQUEST----- and -----END CERTIFICATE REQUEST----- before/after the raw request.

I found this online CSR decoder, which dumps the ASN.1 information and does not seem to have an issue with the CRS data. I cannot influence the CSR other then changing settings but and the configuration is similar to the IOS-XE router so it seems there is a difference in implementation.

I hope this helps to further understand what is causing this. Please let me know what you think could be tested again or what should be debugged, I may have some time for that over the weekend as well.

[hslatman]

@lindenaar: I did a bit of digging, but I don't have a solution yet. Here's some notes:

Decoded your CSR and then put it in here, which shows no error (like the decoder you used).

Then traced through the Go ASN.1 decoding and found that it breaks on the TeletexString/T61String, because it expects a Sequence with tag 16 (based on some logic that determines it needs to find tag 16). On the web page it's the last TeletexString before the end of the object. Its byte contents are [20 15 48 13 48 11 6 3 85 29 15 4 4 3 2 5 160]. 20 is the tag; 15 is its length. The remaining 15 bytes is the value. It encodes an ExtensionReq, indicated by its OID 1.2.840.113549.1.9.14. The hex representation of the value is 300b0603551d0f0404030205a0 (a broken string representation is shown on the web page). I don't know what this value represents.

The type that is being parsed is []pkix.Extension and because it's a slice, it is currently seen as a Sequence (tag 16). This could mean that there's a bug in the ASN.1 parser, but I'd need to look more into this to verify this.

Do you know what extension the device might be trying to encode? Is there also an extension in the IOS 16.x CSR?

[maraino]

@hslatman the oid 1.2.840.113549.1.9.14 hods a set of extensions that are requested to the CA, but that CSR is not encoding a set of sequences like any other CSR with requested extensions:

SEQUENCE (2 elem)
    OBJECT IDENTIFIER 1.2.840.113549.1.9.14 extensionRequest (PKCS #9 via CRMF)
    SET (1 elem)
        SEQUENCE (2 elem)
            SEQUENCE (2 elem)
                OBJECT IDENTIFIER 1.2.3.4
                OCTET STRING (3 byte) foo
            SEQUENCE (2 elem)
                OBJECT IDENTIFIER 1.2.3.5
                OCTET STRING (3 byte) bar

In fact after looking a little bit more close, 300b0603551d0f0404030205a0 looks like a keyUsage extension that is not embedded in a sequence:

• 300b: length 11
• 0603551D0F: 2.5.29.15 (key usage)
• 0404: octet string (04) - 4 bytes
• 030205a0: Digital Signature, Key Encipherment (appropriate for an RSA key)

But a proper key usage extension request would look like this:

SEQUENCE (2 elem)
    OBJECT IDENTIFIER 1.2.840.113549.1.9.14 extensionRequest (PKCS #9 via CRMF)
    SET (1 elem)
      SEQUENCE (1 elem)
        SEQUENCE (2 elem)
          OBJECT IDENTIFIER 2.5.29.15 keyUsage (X.509 extension)
          OCTET STRING (4 byte) 030205A0
            BIT STRING (3 bit) 101

See this example.

[hslatman]

@maraino: thank you! Makes sense. I hadn't spotted this 🙂

So, the IOS 15.x client encodes it wrongly. When looking into this I tried to find changelogs for this software that mentioned fixing something like this, but didn't find anything related. Can have a look again.

@lindenaar: if this is the case, I expect an update should fix this.

[lindenaar]

@maraino and @hslatman thanks for diving into this, I looked into the Cisco Bug tracker but did not really see something related to this. Release notes for Cisco IOS releases are not always very clear indeed, I have given up on those a long time ago already.

Updating is not really an updating as these are different devices running different OSes (IOS-XE is the next generation OS that only runs on newer models)

Let me raise a ticket for this with Cisco Support (luckily I still have a maintenance contract for this device) based on the above analysis and see how they respond. I will keep you updated.

[lindenaar]

Hi @maraino and @hslatman my apologies for the delay in response, I got side-tracked by some other project so was not yet able to follow-up on this much further.

I tried to raise a request with Cisco TAC but I am not quite sure what to ask them as the problem is not quite clear to me. Besides, when I understand correctly, the ASN.1 encoding of the messages is correct, though not as the Go libraries expect. The Cisco documentation is also not clear about whether this is as expected.

I have disabled all extKeyUsage in the requests (... I think, by not specifying it in the config) but even that did not help to make it work. The requests from the older IOS 15.7 routers are still not being accepted. Now this firmware runs on thousands of routers in corporate networks and for sure some of them use SCEP so if there indeed is an issue in the way they encode their PKCS#7/PCS#10 content of the SCEP requests that would be noticed somewhere. Requesting them to fix something is fine (I had them resolve bugs in the past though in that case they were causing crashes) though it is still not clear based on what this is an issue on their side.

I did some further searching and used the commands described here to analyse the requests with OpenSSL, which also did not complain about the messages sent.

I do understand from @maraino 's analysis here that the encoding is not as expected, but isn't that something OpenSSL then also should complain about? Do you have any reference that I can share with Cisco to make clear this is not right (Or... since OpenSSL does not complain, is this something that the Go libraries also should accept) ?

Looking forward to your response, please let me know in case I can do anything to bring this closer to a closure. Thanks!

Regards,

Frederik