Memory leak on macOS Catalina

[inejge]

See sfackler/rust-native-tls#171, which also contains the test program for the leak. Cargo.lock says:

[[package]]
name = "security-framework"
version = "0.4.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "64808902d7d99f78eaddd2b4e2509713babc3dc3c85ad6f4c447680f3c01e535"
dependencies = [
 "bitflags",
 "core-foundation",
 "core-foundation-sys",
 "libc",
 "security-framework-sys",
]

[[package]]
name = "security-framework-sys"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "17bf11d99252f512695eb468de5516e5cf75455521e69dfe343f3b74e4748405"
dependencies = [
 "core-foundation-sys",
 "libc",
]

Note that I have limited access to macOS machines, and can't run GUI tools such as Instruments.

[inejge]

I managed to do some work on a Catalina machine and run a test program with MallocStackLogging=1. The program is basically that in the original bug report, but instead of opening connections indefinitely, it opens 300 connections and goes to sleep. I used heap and malloc_history to capture the logs.

heap_16209.log malloc_history_16209.log

Perhaps someone who knows macOS better than me might find them informative.

[kornelski]

I think it's Apple's code that is leaking.

The leaked memory is allocated in the call to SSLHandshake, but for every handshake the corresponding SslStream's drop is called. The handshake belongs to SslContext, which has drop implemented by core foundation crate, so this should be correct, or everything would leak everywhere.

I've checked that the stream context has retain count == 1 in fn drop, so it should definitely get released by us.