Loading native certs takes 300ms on OS X

[rcoh]

Loading native certs on OS X takes 300ms:

use std::time::SystemTime;
fn main() {
    let now = SystemTime::now();
    let https = rustls_native_certs::load_native_certs();
    println!("{:?}", now.elapsed());
}

    Finished release [optimized + debuginfo] target(s) in 0.69s
     Running `target/release/scratch`
Ok(334.809ms)
native (includes connection to google): Ok(90.305ms)

flamegraph.pdf

Apologies for the PDF flamegraph, GitHub is blocking SVGs.

For comparison, the crate native-tls only takes 100ms to load TLS certs & connect to google over HTTPS, so it seems like it should definitely be possible to improve:

    let native_now = SystemTime::now();
    let connector = TlsConnector::new().unwrap();
    let stream = TcpStream::connect("google.com:443").unwrap();
    let mut stream = connector.connect("google.com", stream).unwrap();
    println!("native: {:?}", native_now.elapsed());

Cargo tree

scratch v0.1.0 (/Users/rcoh/code/scratch)
├── native-tls v0.2.8
│   ├── lazy_static v1.4.0
│   ├── libc v0.2.103
│   ├── security-framework v2.4.2
│   │   ├── bitflags v1.3.2
│   │   ├── core-foundation v0.9.1
│   │   │   ├── core-foundation-sys v0.8.2
│   │   │   └── libc v0.2.103
│   │   ├── core-foundation-sys v0.8.2
│   │   ├── libc v0.2.103
│   │   └── security-framework-sys v2.4.2
│   │       ├── core-foundation-sys v0.8.2
│   │       └── libc v0.2.103
│   ├── security-framework-sys v2.4.2 (*)
│   └── tempfile v3.2.0
│       ├── cfg-if v1.0.0
│       ├── libc v0.2.103
│       ├── rand v0.8.4
│       │   ├── libc v0.2.103
│       │   ├── rand_chacha v0.3.1
│       │   │   ├── ppv-lite86 v0.2.10
│       │   │   └── rand_core v0.6.3
│       │   │       └── getrandom v0.2.3
│       │   │           ├── cfg-if v1.0.0
│       │   │           └── libc v0.2.103
│       │   └── rand_core v0.6.3 (*)
│       └── remove_dir_all v0.5.3
└── rustls-native-certs v0.5.0
    ├── rustls v0.19.1
    │   ├── base64 v0.13.0
    │   ├── log v0.4.14
    │   │   └── cfg-if v1.0.0
    │   ├── ring v0.16.20
    │   │   ├── spin v0.5.2
    │   │   └── untrusted v0.7.1
    │   │   [build-dependencies]
    │   │   └── cc v1.0.70
    │   ├── sct v0.6.1
    │   │   ├── ring v0.16.20 (*)
    │   │   └── untrusted v0.7.1
    │   └── webpki v0.21.4
    │       ├── ring v0.16.20 (*)
    │       └── untrusted v0.7.1
    └── security-framework v2.4.2 (*)

[ctz]

Thanks for the detailed report!

I don't presently have a mac to make progress on this, do you have any suggestions of improvements we can make here? It looks from the flamegraph that ~all the time is in required security framework calls, which makes me think this is another good argument for taking a different approach to the goals of this crate, as mentioned in #25

[djc]

It might make more sense to file an issue against the security-framework crate, since ~all the work is being done in that crate anyway.

[cpu]

Based on the discussion above and the progress made on https://github.com/rustls/rustls-platform-verifier I think we should close this issue. If performance on MacOS is still a problem for this crate I would encourage you to file an issue against https://github.com/kornelski/rust-security-framework, we can pull in any performance improvements that are made downstream.

[mdecimus]

Hi @rcoh , I am having the same problem on a MacOS dev environment. Have you reported this issue on https://github.com/kornelski/rust-security-framework or were you able to find a workaround? Thanks

[rcoh]

I believe we "fixed" it by loading certs in lazy static so it only happened once

[mdecimus]

Thanks! I am not sure the devs at rust-security-framework can do much either as all that time is spent in the MacOS libraries:

[djc]

@mdecimus have you considered trying rustls-platform-verifier as suggested above?

[mdecimus]

@djc I don't have that option as in my case the slowdown was caused by reqwest which by default uses rustls with native certs. What I did instead is to configure reqwest to use webpki-roots and now everything works as expected. Thanks!

[djc]

For what it's worth, you can use ClientBuilder::use_preconfigured_tls() to pass a custom ClientConfig (for example, one that uses rustls-platform-verifier).

[rcoh]

I'm looking into rust-platform-verifier—seems like it's not actually on crates.io though? https://crates.io/crates/rustls-platform-verifier

[cpu]

seems like it's not actually on crates.io though?

That's correct. We're tracking tasks for an initial release here.