How to generate self-signed certificates by using openssl?

[heibor]

I use the example code server.rs. And when I use the sample.pem and sample.rsa in the example fold, the server will crash and give me these messages:

Starting to serve on https://127.0.0.1:1337.
[!] Voluntary server halt due to client-connection error...
FAILED: error accepting connection: TLS Error: Custom { kind: InvalidData, error: AlertReceived(BadCertificate) }

And when I generated the self-signed certificates follow these instructions How to enable SSL and TLS 1.3 on NGINX.

After I load the ca.crt and ca.key file, I still got these message:

Starting to serve on https://127.0.0.1:1337.
[!] Voluntary server halt due to client-connection error...
FAILED: error accepting connection: TLS Error: Custom { kind: InvalidData, error: AlertReceived(BadCertificate) }

I want to use TLS 1.3, but I don't know how to generate the self-signed certificates. BTW, my environment is MacosX + Rust 1.45.2.

Please give me some advice, thank you.

[lucaswxp]

@heibor Did you find a workaround? I'm having the same issue.

First that I could not get the private keys to load without using this python script https://github.com/ctz/rustls/issues/74#issuecomment-306055454

And now I'm facing the same issue than you:

FAILED: error accepting connection: TLS Error: Custom { kind: InvalidData, error: AlertReceived(BadCertificate) }

Is there any workaround? I'm almost giving up on rustls, it seems too restrictive. Should I raise a issue in their repo? I'm not sure what is happening yet.

[heibor]

I use the command:

openssl req -x509 -nodes -days 1825 -newkey rsa:4096 -keyout server.key -out server.crt

The generated self-signed certificates can be used in rustls. Pls check if you forget the x509 option.

It seems the mkcert is a better tool to generate the certificates, but I haven't tried it yet.

[lucaswxp]

Thanks, mkcert worked, I still had to use the python script tho, or else I would get:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Custom { kind: Other, error: "expected a single private key" }',

[heibor]

@lucaswxp The private key contains rsa key and pkcs8 key, and I added some codes to the load_private_key function, and rename it to load_keys:

fn load_keys(path: &Path) -> io::Result<PrivateKey> {
    let rsa_keys = {
        let keyfile = File::open(path)
            .expect("cannot open private key file");
        let mut reader = io::BufReader::new(keyfile);
        pemfile::rsa_private_keys(&mut reader)
            .expect("file contains invalid rsa private key")
    };

    let pkcs8_keys = {
        let keyfile = File::open(path)
            .expect("cannot open private key file");
        let mut reader = io::BufReader::new(keyfile);
        pemfile::pkcs8_private_keys(&mut reader)
            .expect("file contains invalid pkcs8 private key (encrypted keys not supported)")
    };

    // prefer to load pkcs8 keys
    if !pkcs8_keys.is_empty() {
        Ok(pkcs8_keys[0].clone())
    } else {
        assert!(!rsa_keys.is_empty());
        Ok(rsa_keys[0].clone())
    }
}

I'm not sure if this will help you.

[nwtgck]

Hi. I use the following openssl commands and Rust code.

mkdir ssl_certs && cd ssl_certs && openssl req -x509 -newkey rsa:4096 -keyout server.key -out server.crt -days 365 -sha256 -nodes --subj '/CN=localhost/' && cd -

The command above creates ssl_certs/server.key and ssl_certs/server.crt.

The rust code below shows how to load the files.

let tls_cfg = load_tls_config("./ssl_certs/server.crt", "./ssl_certs/server.key")?;
let tls_acceptor = TlsAcceptor::from(std::sync::Arc::new(tls_cfg));

// use tls_acceptor...

// (base: https://github.com/ctz/hyper-rustls/blob/5f073724f7b5eee3a2d72f0a86094fc2718b51cd/examples/server.rs)
pub fn load_tls_config(
    cert_path: impl AsRef<std::path::Path>,
    key_path: impl AsRef<std::path::Path> + std::fmt::Display,
) -> std::io::Result<rustls::ServerConfig> {
    // Load public certificate.
    let mut cert_reader = std::io::BufReader::new(std::fs::File::open(cert_path)?);
    let certs = rustls::internal::pemfile::certs(&mut cert_reader)
        .map_err(|_| error("unable to load certificate".to_owned()))?;
    // Load private key.
    let mut key_reader = std::io::BufReader::new(std::fs::File::open(key_path)?);
    // Load and return a single private key.
    let key = rustls::internal::pemfile::pkcs8_private_keys(&mut key_reader)
        .map_err(|_| error("unable to load private key".to_owned()))?
        .remove(0);
    // Do not use client certificate authentication.
    let mut cfg = rustls::ServerConfig::new(rustls::NoClientAuth::new());
    // Select a certificate to use.
    cfg.set_single_cert(certs, key).unwrap();
    // Configure ALPN to accept HTTP/2, HTTP/1.1 in that order.
    cfg.set_protocols(&[b"h2".to_vec(), b"http/1.1".to_vec()]);
    Ok(cfg)
}

Here is an actual example using the command in CI: https://github.com/nwtgck/piping-server-rust/blob/1dc4563fa90511579d145271f4131fdf24fcd610/.github/workflows/ci.yml#L20.

[cpu]

Thanks to @nwtgck it looks like there's an example for the original poster to follow. I would echo the suggestion of others to try and avoid openssl if you're looking for easy to use tooling. mkcert or rcgen are likely going to give you a better user experience. The openssl command line tools are notoriously difficult to work with.