OpenSC
engine_pkcs11 leaks memory
Hey, I've implemented a custom memory tracking in BIND 9 for the external libraries, including OpenSSL, and it found that engine_pkcs11 module is leaking following memory (lines should match OpenSSL 3.0.5):
ptr 0x56056e775b50 size 264 file ../crypto/evp/pmeth_lib.c line 130
ptr 0x56056e708570 size 120 file ../crypto/ec/ec_kmeth.c line 184
ptr 0x56056e708310 size 120 file ../crypto/rsa/rsa_meth.c line 48
ptr 0x56056e708280 size 24 file ../crypto/rsa/rsa_meth.c line 71
ptr 0x56056e774c10 size 264 file ../crypto/evp/pmeth_lib.c line 130
ptr 0x56056e6f6020 size 56 file ../crypto/threads_pthread.c line 50
It's following methods:
ptr 0x55dd947bb8f0 size 24 file ../crypto/rsa/rsa_meth.c line 71-->RSA_meth_set1_name()ptr 0x55dd947bbbe0 size 120 file ../crypto/ec/ec_kmeth.c line 184-->EC_KEY_METHOD_new()ptr 0x55dd947bb980 size 120 file ../crypto/rsa/rsa_meth.c line 48-->RSA_meth_dup()ptr 0x55dd948287b0 size 264 file ../crypto/evp/pmeth_lib.c line 130-->EVP_PKEY_meth_new()
The call tree can be found in the attached PDF: jeprof.pdf
This simple code ENGINE_free(ENGINE_by_id("pkcs11")); is enough to reproduce the issue.
Looking at the code, it seems like the code that would free the method allocated in PKCS11_get_rsa_method() (and others) was never actually implemented:
/* This function is *not* currently exported */
void PKCS11_rsa_method_free(void)
{
free_rsa_ex_index();
}
Ssee https://github.com/OpenSC/libp11/pull/470#issuecomment-1219913973 and make sure the free callback is also added. Further comments discuses OpenSSL 1.1 and OpenSSL 3.0 while engines are still available
#471 tries to address the dup problem.
