MacOS RSA DER / OpenSSL 3.4.0 RSA256 EncodingKey::from_rsa_der broken

[ben-kaye]

Seen also #55

Description: Using OpenSSL 3.4.0 on MacOS, EncodingKey::from_rsa_der fails to parse standard RSA DER keys.

Steps to Reproduce: Minimal repro here: der_broken jsonwebtoken = "9.3.1"

Shell script to generate the key:

#!/bin/zsh

if [ -z "$1" ]; then
    num_bits=3072
else
    num_bits=$1
fi

# Generate 3072-bit RSA private key directly in PEM
openssl genrsa -out private.pem $num_bits

openssl rsa -outform DER -in private.pem -out private.der
openssl rsa -RSAPublicKey_out -outform DER -in private.pem -out public.der

# Base64 encode the DER files (without newlines)
base64 < private.der | tr -d '\n' > private.der.b64
base64 < public.der | tr -d '\n' > public.der.b64

# Write to .env cleanly without extra newlines
{
  printf "JWT_PRIVATE=%s\n" "$(cat private.der.b64)"
  printf "JWT_PUBLIC=%s\n" "$(cat public.der.b64)"
} > .env

Notes:

• The DER produced is valid (can be parsed by OpenSSL and other libraries).
• Likely regression related to stricter ASN.1 parsing.
• Happens on both 2048 and 3072-bit keys.
• PEM-based loading (from_rsa_pem) still works fine.

[d-s-d]

The exact same issue exists when trying to parse ed25519 keys. This is a structural bug whereby the der encoding is used as is, whereas the pem-codepath extracts the first bitstring it finds in the der-encoding.

[Keats]

Does someone has a patch?

[dsykes16]

@ben-kaye, I was unable to reproduce the issue after tweaking your openssl command to generate proper PKCS1 DER RSA keys.

Add -traditional to your openssl command to output keys in PKCS1 format.

-openssl rsa -outform DER -in private.pem -out private.der
+openssl rsa -outform DER -in private.pem -out private.der -traditional

With proper RSA key inputs, this was validated as working on v9.3.1 and on v10 w/ rust_crypto and aws_lc_rs. This is not a bug in jsonwebtoken.

@Keats, this can probably be closed?

[Keats]

Let's wait to see if @d-s-d has a reproduction on v10

[dsykes16]

I just tried Ed25519 keys (loading with from_ed_der) on v10 and it's able to encode and results in a valid JWT with both aws_lc_rs and rust_crypto, but decoding/validation is failing with rust_crypto. I'm working on adding some tests and will see if I can track down the source of the issue.