cryptography
cryptography copied to clipboard
Published
20 hours ago •
pyca
pyca
Memory leak tests fail on windows with OpenSSL 3.0.4
After finally working out the kinks of the DLL loading in conda-forge, there are now some failing tests with CPython 3.7 / 3.9 / 3.10 on windows (also after a restart; interestingly, they pass on PyPy and CPython 3.8, and reading the source does not look like they're being skipped there).
=========================== short test summary info ===========================
FAILED tests/hazmat/backends/test_openssl_memleak.py::TestOpenSSLMemoryLeaks::test_x509_csr_extensions
FAILED tests/hazmat/backends/test_openssl_memleak.py::TestOpenSSLMemoryLeaks::test_x25519_pubkey_from_private_key
FAILED tests/hazmat/backends/test_openssl_memleak.py::TestOpenSSLMemoryLeaks::test_load_pkcs12_key_and_certificates[pkcs12/cert-aes256cbc-no-key.p12]
FAILED tests/hazmat/backends/test_openssl_memleak.py::TestOpenSSLMemoryLeaks::test_load_pkcs12_key_and_certificates[pkcs12/cert-key-aes256cbc.p12]
FAILED tests/hazmat/backends/test_openssl_memleak.py::TestOpenSSLMemoryLeaks::test_write_pkcs12_key_and_certificates
= 5 failed, 2816 passed, 56 skipped, 296 warnings, 85319 subtests passed in 340.54s (0:05:40) =
This might be an OpenSSL issue, but the test suite for OpenSSL 3.0.4 ran through fine in the conda-forge builds, and since it shows up in the cryptography test suite, I thought I'd open the issue here first.
PS. One fringe benefit of this issue actually is to - very belatedly - thank @Tarnum-tst for their input in #6392 (which is locked for further comments) - it was spot-on and ultimately what got things unblocked (my initial mistake was wanting to mirror the build OpenSSL instructions between unix & windows; for some reason though, the legacy provider gets built by default on unix, but not windows).
Interesting, we do run our memleak tests on windows with openssl 3.0.4 right now and aren’t seeing this. (As an aside, conda should not ship 3.0.4 because of https://github.com/openssl/openssl/issues/18625)
Maybe there is some difference in the way you’re building and linking openssl…
Thanks for the quick response and the heads-up. Opened https://github.com/conda-forge/openssl-feedstock/pull/100 to fix 3.0.4 in conda-forge.
Maybe there is some difference in the way you’re building and linking openssl…
Our build instructions on windows are pretty minimal: https://github.com/conda-forge/openssl-feedstock/blob/main/recipe/bld.bat
Considering that 3/6 builds failed, and in particular, intermittently across CPython versions, it's not impossible IMO that this had something to do with the OpenSSL issue you cited (i.e. getting an AVX512 runner or not). I'll retry once the patched 3.0.4 makes its way through the CDN.
Unfortunately, the errors persist also with the backported fix for https://github.com/openssl/openssl/issues/18625
Which version of cryptography is being used here? As @reaperhulk notes on both main and the 37.0.x branch we test against OpenSSL 3, so both of those should be green here.
It's using cryptography 37.0.2 (as we're currently running into some cargo problems in cross-compilation, so 37.0.3 is stuck).
Can you share the full output for these failing tests. Ideally with line 31 of test_openssl_memleak.py changed to be BACKTRACE_ENABLED = True.
So switching BACKTRACE_ENABLED=True also makes the OpenSSL 1.1.1 tests fail. The full logs are here (ignore the aarch/ppc failures).
Could it be that the backtrace is incompatible in some way with the DLL loading? What particularly caught my eye was: OSError: dlopen(None) cannot work on Windows for Python 3, also for the OpenSSL 1.1.1 builds. As soon as the CI is through (so azure doesn't delete the logs), I'll give it another shot without the backtrace
Can you share the full output for these failing tests.
Here's the full output:
Without backtrace (failing only on windows + openssl 3.0.4 + cpython 3.7/3.9/3.10)
================================== FAILURES ===================================
_______________ TestOpenSSLMemoryLeaks.test_x509_csr_extensions _______________
self = <tests.hazmat.backends.test_openssl_memleak.TestOpenSSLMemoryLeaks object at 0x000001A28653DA00>
def test_x509_csr_extensions(self):
> assert_no_memory_leaks(
textwrap.dedent(
"""
def func():
from cryptography import x509
from cryptography.hazmat.backends.openssl import backend
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import rsa
private_key = rsa.generate_private_key(
key_size=2048, public_exponent=65537, backend=backend
)
cert = x509.CertificateSigningRequestBuilder().subject_name(
x509.Name([])
).add_extension(
x509.OCSPNoCheck(), critical=False
).sign(private_key, hashes.SHA256(), backend)
cert.extensions
"""
)
)
tests\hazmat\backends\test_openssl_memleak.py:299:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '\ndef func():\n from cryptography import x509\n from cryptography.hazmat.backends.openssl import backend\n f...n x509.OCSPNoCheck(), critical=False\n ).sign(private_key, hashes.SHA256(), backend)\n\n cert.extensions\n'
argv = ['D:\\bld\\cryptography_1656482881112\\_test_env\\python.exe', '-c', '\ndef func():\n from cryptography import x509...n for ptr in remaining\n )))\n sys.stdout.flush()\n sys.exit(255)\n\nmain(sys.argv)\n']
def assert_no_memory_leaks(s, argv=[]):
env = os.environ.copy()
env["PYTHONPATH"] = os.pathsep.join(sys.path)
# When using pytest-cov it attempts to instrument subprocesses. This
# causes the memleak tests to raise exceptions.
# we don't need coverage so we remove the env vars.
env.pop("COV_CORE_CONFIG", None)
env.pop("COV_CORE_DATAFILE", None)
env.pop("COV_CORE_SOURCE", None)
argv = [
sys.executable,
"-c",
"{}\n\n{}".format(s, MEMORY_LEAK_SCRIPT),
] + argv
# Shell out to a fresh Python process because OpenSSL does not allow you to
# install new memory hooks after the first malloc/free occurs.
proc = subprocess.Popen(
argv,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
proc.wait()
if proc.returncode == 255:
# 255 means there was a leak, load the info about what mallocs
# weren't freed.
out = json.loads(proc.stdout.read().decode())
> raise AssertionError(out)
E AssertionError: {'2209339288832': {'size': 2, 'path': 'crypto\\provider_core.c', 'line': 1616, 'backtrace': None}}
tests\hazmat\backends\test_openssl_memleak.py:167: AssertionError
_________ TestOpenSSLMemoryLeaks.test_x25519_pubkey_from_private_key __________
self = <tests.hazmat.backends.test_openssl_memleak.TestOpenSSLMemoryLeaks object at 0x000001A28653DB50>
def test_x25519_pubkey_from_private_key(self):
> assert_no_memory_leaks(
textwrap.dedent(
"""
def func():
from cryptography.hazmat.primitives.asymmetric import x25519
private_key = x25519.X25519PrivateKey.generate()
private_key.public_key()
"""
)
)
tests\hazmat\backends\test_openssl_memleak.py:364:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '\ndef func():\n from cryptography.hazmat.primitives.asymmetric import x25519\n private_key = x25519.X25519PrivateKey.generate()\n private_key.public_key()\n'
argv = ['D:\\bld\\cryptography_1656482881112\\_test_env\\python.exe', '-c', '\ndef func():\n from cryptography.hazmat.prim...n for ptr in remaining\n )))\n sys.stdout.flush()\n sys.exit(255)\n\nmain(sys.argv)\n']
def assert_no_memory_leaks(s, argv=[]):
env = os.environ.copy()
env["PYTHONPATH"] = os.pathsep.join(sys.path)
# When using pytest-cov it attempts to instrument subprocesses. This
# causes the memleak tests to raise exceptions.
# we don't need coverage so we remove the env vars.
env.pop("COV_CORE_CONFIG", None)
env.pop("COV_CORE_DATAFILE", None)
env.pop("COV_CORE_SOURCE", None)
argv = [
sys.executable,
"-c",
"{}\n\n{}".format(s, MEMORY_LEAK_SCRIPT),
] + argv
# Shell out to a fresh Python process because OpenSSL does not allow you to
# install new memory hooks after the first malloc/free occurs.
proc = subprocess.Popen(
argv,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
proc.wait()
if proc.returncode == 255:
# 255 means there was a leak, load the info about what mallocs
# weren't freed.
out = json.loads(proc.stdout.read().decode())
> raise AssertionError(out)
E AssertionError: {'2050124924512': {'size': 3, 'path': 'crypto\\provider_core.c', 'line': 1616, 'backtrace': None}}
tests\hazmat\backends\test_openssl_memleak.py:167: AssertionError
_ TestOpenSSLMemoryLeaks.test_load_pkcs12_key_and_certificates[pkcs12/cert-aes256cbc-no-key.p12] _
self = <tests.hazmat.backends.test_openssl_memleak.TestOpenSSLMemoryLeaks object at 0x000001A28653DBE0>
path = 'pkcs12/cert-aes256cbc-no-key.p12'
@pytest.mark.parametrize(
"path",
["pkcs12/cert-aes256cbc-no-key.p12", "pkcs12/cert-key-aes256cbc.p12"],
)
def test_load_pkcs12_key_and_certificates(self, path):
> assert_no_memory_leaks(
textwrap.dedent(
"""
def func(path):
from cryptography import x509
from cryptography.hazmat.backends.openssl import backend
from cryptography.hazmat.primitives.serialization import pkcs12
import cryptography_vectors
with cryptography_vectors.open_vector_file(path, "rb") as f:
pkcs12.load_key_and_certificates(
f.read(), b"cryptography", backend
)
"""
),
[path],
)
tests\hazmat\backends\test_openssl_memleak.py:405:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '\ndef func(path):\n from cryptography import x509\n from cryptography.hazmat.backends.openssl import backend\n ...h, "rb") as f:\n pkcs12.load_key_and_certificates(\n f.read(), b"cryptography", backend\n )\n'
argv = ['D:\\bld\\cryptography_1656482881112\\_test_env\\python.exe', '-c', '\ndef func(path):\n from cryptography import ... )))\n sys.stdout.flush()\n sys.exit(255)\n\nmain(sys.argv)\n', 'pkcs12/cert-aes256cbc-no-key.p12']
def assert_no_memory_leaks(s, argv=[]):
env = os.environ.copy()
env["PYTHONPATH"] = os.pathsep.join(sys.path)
# When using pytest-cov it attempts to instrument subprocesses. This
# causes the memleak tests to raise exceptions.
# we don't need coverage so we remove the env vars.
env.pop("COV_CORE_CONFIG", None)
env.pop("COV_CORE_DATAFILE", None)
env.pop("COV_CORE_SOURCE", None)
argv = [
sys.executable,
"-c",
"{}\n\n{}".format(s, MEMORY_LEAK_SCRIPT),
] + argv
# Shell out to a fresh Python process because OpenSSL does not allow you to
# install new memory hooks after the first malloc/free occurs.
proc = subprocess.Popen(
argv,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
proc.wait()
if proc.returncode == 255:
# 255 means there was a leak, load the info about what mallocs
# weren't freed.
out = json.loads(proc.stdout.read().decode())
> raise AssertionError(out)
E AssertionError: {'2441160132704': {'size': 3, 'path': 'crypto\\provider_core.c', 'line': 1616, 'backtrace': None}}
tests\hazmat\backends\test_openssl_memleak.py:167: AssertionError
_ TestOpenSSLMemoryLeaks.test_load_pkcs12_key_and_certificates[pkcs12/cert-key-aes256cbc.p12] _
self = <tests.hazmat.backends.test_openssl_memleak.TestOpenSSLMemoryLeaks object at 0x000001A28653DFD0>
path = 'pkcs12/cert-key-aes256cbc.p12'
@pytest.mark.parametrize(
"path",
["pkcs12/cert-aes256cbc-no-key.p12", "pkcs12/cert-key-aes256cbc.p12"],
)
def test_load_pkcs12_key_and_certificates(self, path):
> assert_no_memory_leaks(
textwrap.dedent(
"""
def func(path):
from cryptography import x509
from cryptography.hazmat.backends.openssl import backend
from cryptography.hazmat.primitives.serialization import pkcs12
import cryptography_vectors
with cryptography_vectors.open_vector_file(path, "rb") as f:
pkcs12.load_key_and_certificates(
f.read(), b"cryptography", backend
)
"""
),
[path],
)
tests\hazmat\backends\test_openssl_memleak.py:405:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '\ndef func(path):\n from cryptography import x509\n from cryptography.hazmat.backends.openssl import backend\n ...h, "rb") as f:\n pkcs12.load_key_and_certificates(\n f.read(), b"cryptography", backend\n )\n'
argv = ['D:\\bld\\cryptography_1656482881112\\_test_env\\python.exe', '-c', '\ndef func(path):\n from cryptography import ...\n )))\n sys.stdout.flush()\n sys.exit(255)\n\nmain(sys.argv)\n', 'pkcs12/cert-key-aes256cbc.p12']
def assert_no_memory_leaks(s, argv=[]):
env = os.environ.copy()
env["PYTHONPATH"] = os.pathsep.join(sys.path)
# When using pytest-cov it attempts to instrument subprocesses. This
# causes the memleak tests to raise exceptions.
# we don't need coverage so we remove the env vars.
env.pop("COV_CORE_CONFIG", None)
env.pop("COV_CORE_DATAFILE", None)
env.pop("COV_CORE_SOURCE", None)
argv = [
sys.executable,
"-c",
"{}\n\n{}".format(s, MEMORY_LEAK_SCRIPT),
] + argv
# Shell out to a fresh Python process because OpenSSL does not allow you to
# install new memory hooks after the first malloc/free occurs.
proc = subprocess.Popen(
argv,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
proc.wait()
if proc.returncode == 255:
# 255 means there was a leak, load the info about what mallocs
# weren't freed.
out = json.loads(proc.stdout.read().decode())
> raise AssertionError(out)
E AssertionError: {'1386551076512': {'size': 3, 'path': 'crypto\\provider_core.c', 'line': 1616, 'backtrace': None}}
tests\hazmat\backends\test_openssl_memleak.py:167: AssertionError
________ TestOpenSSLMemoryLeaks.test_write_pkcs12_key_and_certificates ________
self = <tests.hazmat.backends.test_openssl_memleak.TestOpenSSLMemoryLeaks object at 0x000001A28653D0D0>
def test_write_pkcs12_key_and_certificates(self):
> assert_no_memory_leaks(
textwrap.dedent(
"""
def func():
import os
from cryptography import x509
from cryptography.hazmat.backends.openssl import backend
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.serialization import pkcs12
import cryptography_vectors
path = os.path.join('x509', 'custom', 'ca', 'ca.pem')
with cryptography_vectors.open_vector_file(path, "rb") as f:
cert = x509.load_pem_x509_certificate(
f.read(), backend
)
path2 = os.path.join('x509', 'custom', 'dsa_selfsigned_ca.pem')
with cryptography_vectors.open_vector_file(path2, "rb") as f:
cert2 = x509.load_pem_x509_certificate(
f.read(), backend
)
path3 = os.path.join('x509', 'letsencryptx3.pem')
with cryptography_vectors.open_vector_file(path3, "rb") as f:
cert3 = x509.load_pem_x509_certificate(
f.read(), backend
)
key_path = os.path.join("x509", "custom", "ca", "ca_key.pem")
with cryptography_vectors.open_vector_file(key_path, "rb") as f:
key = serialization.load_pem_private_key(
f.read(), None, backend
)
encryption = serialization.NoEncryption()
pkcs12.serialize_key_and_certificates(
b"name", key, cert, [cert2, cert3], encryption)
"""
)
)
tests\hazmat\backends\test_openssl_memleak.py:537:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '\ndef func():\n import os\n from cryptography import x509\n from cryptography.hazmat.backends.openssl import....NoEncryption()\n pkcs12.serialize_key_and_certificates(\n b"name", key, cert, [cert2, cert3], encryption)\n'
argv = ['D:\\bld\\cryptography_1656482881112\\_test_env\\python.exe', '-c', '\ndef func():\n import os\n from cryptogra...n for ptr in remaining\n )))\n sys.stdout.flush()\n sys.exit(255)\n\nmain(sys.argv)\n']
def assert_no_memory_leaks(s, argv=[]):
env = os.environ.copy()
env["PYTHONPATH"] = os.pathsep.join(sys.path)
# When using pytest-cov it attempts to instrument subprocesses. This
# causes the memleak tests to raise exceptions.
# we don't need coverage so we remove the env vars.
env.pop("COV_CORE_CONFIG", None)
env.pop("COV_CORE_DATAFILE", None)
env.pop("COV_CORE_SOURCE", None)
argv = [
sys.executable,
"-c",
"{}\n\n{}".format(s, MEMORY_LEAK_SCRIPT),
] + argv
# Shell out to a fresh Python process because OpenSSL does not allow you to
# install new memory hooks after the first malloc/free occurs.
proc = subprocess.Popen(
argv,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
proc.wait()
if proc.returncode == 255:
# 255 means there was a leak, load the info about what mallocs
# weren't freed.
out = json.loads(proc.stdout.read().decode())
> raise AssertionError(out)
E AssertionError: {'1984856455008': {'size': 3, 'path': 'crypto\\provider_core.c', 'line': 1616, 'backtrace': None}}
With backtrace (failing on all windows builds for openssl 1.1.1 & 3.0.4 + cpython & pypy)
================================== FAILURES ===================================
_______________ TestAssertNoMemoryLeaks.test_no_leak_no_malloc ________________
self = <tests.hazmat.backends.test_openssl_memleak.TestAssertNoMemoryLeaks object at 0x0000017392220400>
def test_no_leak_no_malloc(self):
> assert_no_memory_leaks(
textwrap.dedent(
"""
def func():
pass
"""
)
)
tests\hazmat\backends\test_openssl_memleak.py:187:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '\ndef func():\n pass\n'
argv = ['D:\\bld\\cryptography_1656441144990\\_test_env\\python.exe', '-c', '\ndef func():\n pass\n\n\n\nimport sys\n\n\nd...n for ptr in remaining\n )))\n sys.stdout.flush()\n sys.exit(255)\n\nmain(sys.argv)\n']
def assert_no_memory_leaks(s, argv=[]):
env = os.environ.copy()
env["PYTHONPATH"] = os.pathsep.join(sys.path)
# When using pytest-cov it attempts to instrument subprocesses. This
# causes the memleak tests to raise exceptions.
# we don't need coverage so we remove the env vars.
env.pop("COV_CORE_CONFIG", None)
env.pop("COV_CORE_DATAFILE", None)
env.pop("COV_CORE_SOURCE", None)
argv = [
sys.executable,
"-c",
"{}\n\n{}".format(s, MEMORY_LEAK_SCRIPT),
] + argv
# Shell out to a fresh Python process because OpenSSL does not allow you to
# install new memory hooks after the first malloc/free occurs.
proc = subprocess.Popen(
argv,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
proc.wait()
if proc.returncode == 255:
# 255 means there was a leak, load the info about what mallocs
# weren't freed.
out = json.loads(proc.stdout.read().decode())
raise AssertionError(out)
elif proc.returncode != 0:
# Any exception type will do to be honest
> raise ValueError(proc.stdout.read(), proc.stderr.read())
E ValueError: (b'', b'Traceback (most recent call last):\r\n File "<string>", line 120, in <module>\r\n File "<string>", line 27, in main\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 150, in dlopen\r\n lib, function_cache = _make_ffi_library(self, name, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 832, in _make_ffi_library\r\n backendlib = _load_backend_lib(backend, libname, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 821, in _load_backend_lib\r\n raise OSError("dlopen(None) cannot work on Windows for Python 3 "\r\nOSError: dlopen(None) cannot work on Windows for Python 3 (see http://bugs.python.org/issue23606)\r\n')
tests\hazmat\backends\test_openssl_memleak.py:170: ValueError
__________________ TestAssertNoMemoryLeaks.test_no_leak_free __________________
self = <tests.hazmat.backends.test_openssl_memleak.TestAssertNoMemoryLeaks object at 0x0000017392222DA0>
def test_no_leak_free(self):
> assert_no_memory_leaks(
textwrap.dedent(
"""
def func():
from cryptography.hazmat.bindings.openssl.binding import Binding
b = Binding()
name = b.lib.X509_NAME_new()
b.lib.X509_NAME_free(name)
"""
)
)
tests\hazmat\backends\test_openssl_memleak.py:197:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '\ndef func():\n from cryptography.hazmat.bindings.openssl.binding import Binding\n b = Binding()\n name = b.lib.X509_NAME_new()\n b.lib.X509_NAME_free(name)\n'
argv = ['D:\\bld\\cryptography_1656441144990\\_test_env\\python.exe', '-c', '\ndef func():\n from cryptography.hazmat.bind...n for ptr in remaining\n )))\n sys.stdout.flush()\n sys.exit(255)\n\nmain(sys.argv)\n']
def assert_no_memory_leaks(s, argv=[]):
env = os.environ.copy()
env["PYTHONPATH"] = os.pathsep.join(sys.path)
# When using pytest-cov it attempts to instrument subprocesses. This
# causes the memleak tests to raise exceptions.
# we don't need coverage so we remove the env vars.
env.pop("COV_CORE_CONFIG", None)
env.pop("COV_CORE_DATAFILE", None)
env.pop("COV_CORE_SOURCE", None)
argv = [
sys.executable,
"-c",
"{}\n\n{}".format(s, MEMORY_LEAK_SCRIPT),
] + argv
# Shell out to a fresh Python process because OpenSSL does not allow you to
# install new memory hooks after the first malloc/free occurs.
proc = subprocess.Popen(
argv,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
proc.wait()
if proc.returncode == 255:
# 255 means there was a leak, load the info about what mallocs
# weren't freed.
out = json.loads(proc.stdout.read().decode())
raise AssertionError(out)
elif proc.returncode != 0:
# Any exception type will do to be honest
> raise ValueError(proc.stdout.read(), proc.stderr.read())
E ValueError: (b'', b'Traceback (most recent call last):\r\n File "<string>", line 123, in <module>\r\n File "<string>", line 30, in main\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 150, in dlopen\r\n lib, function_cache = _make_ffi_library(self, name, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 832, in _make_ffi_library\r\n backendlib = _load_backend_lib(backend, libname, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 821, in _load_backend_lib\r\n raise OSError("dlopen(None) cannot work on Windows for Python 3 "\r\nOSError: dlopen(None) cannot work on Windows for Python 3 (see http://bugs.python.org/issue23606)\r\n')
tests\hazmat\backends\test_openssl_memleak.py:170: ValueError
___________________ TestAssertNoMemoryLeaks.test_no_leak_gc ___________________
self = <tests.hazmat.backends.test_openssl_memleak.TestAssertNoMemoryLeaks object at 0x0000017392221270>
def test_no_leak_gc(self):
> assert_no_memory_leaks(
textwrap.dedent(
"""
def func():
from cryptography.hazmat.bindings.openssl.binding import Binding
b = Binding()
name = b.lib.X509_NAME_new()
b.ffi.gc(name, b.lib.X509_NAME_free)
"""
)
)
tests\hazmat\backends\test_openssl_memleak.py:210:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '\ndef func():\n from cryptography.hazmat.bindings.openssl.binding import Binding\n b = Binding()\n name = b.lib.X509_NAME_new()\n b.ffi.gc(name, b.lib.X509_NAME_free)\n'
argv = ['D:\\bld\\cryptography_1656441144990\\_test_env\\python.exe', '-c', '\ndef func():\n from cryptography.hazmat.bind...n for ptr in remaining\n )))\n sys.stdout.flush()\n sys.exit(255)\n\nmain(sys.argv)\n']
def assert_no_memory_leaks(s, argv=[]):
env = os.environ.copy()
env["PYTHONPATH"] = os.pathsep.join(sys.path)
# When using pytest-cov it attempts to instrument subprocesses. This
# causes the memleak tests to raise exceptions.
# we don't need coverage so we remove the env vars.
env.pop("COV_CORE_CONFIG", None)
env.pop("COV_CORE_DATAFILE", None)
env.pop("COV_CORE_SOURCE", None)
argv = [
sys.executable,
"-c",
"{}\n\n{}".format(s, MEMORY_LEAK_SCRIPT),
] + argv
# Shell out to a fresh Python process because OpenSSL does not allow you to
# install new memory hooks after the first malloc/free occurs.
proc = subprocess.Popen(
argv,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
proc.wait()
if proc.returncode == 255:
# 255 means there was a leak, load the info about what mallocs
# weren't freed.
out = json.loads(proc.stdout.read().decode())
raise AssertionError(out)
elif proc.returncode != 0:
# Any exception type will do to be honest
> raise ValueError(proc.stdout.read(), proc.stderr.read())
E ValueError: (b'', b'Traceback (most recent call last):\r\n File "<string>", line 123, in <module>\r\n File "<string>", line 30, in main\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 150, in dlopen\r\n lib, function_cache = _make_ffi_library(self, name, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 832, in _make_ffi_library\r\n backendlib = _load_backend_lib(backend, libname, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 821, in _load_backend_lib\r\n raise OSError("dlopen(None) cannot work on Windows for Python 3 "\r\nOSError: dlopen(None) cannot work on Windows for Python 3 (see http://bugs.python.org/issue23606)\r\n')
tests\hazmat\backends\test_openssl_memleak.py:170: ValueError
______________________ TestAssertNoMemoryLeaks.test_leak ______________________
self = <tests.hazmat.backends.test_openssl_memleak.TestAssertNoMemoryLeaks object at 0x00000173922229E0>
def test_leak(self):
with pytest.raises(AssertionError):
> assert_no_memory_leaks(
textwrap.dedent(
"""
def func():
from cryptography.hazmat.bindings.openssl.binding import (
Binding
)
b = Binding()
b.lib.X509_NAME_new()
"""
)
)
tests\hazmat\backends\test_openssl_memleak.py:224:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '\ndef func():\n from cryptography.hazmat.bindings.openssl.binding import (\n Binding\n )\n b = Binding()\n b.lib.X509_NAME_new()\n'
argv = ['D:\\bld\\cryptography_1656441144990\\_test_env\\python.exe', '-c', '\ndef func():\n from cryptography.hazmat.bind...n for ptr in remaining\n )))\n sys.stdout.flush()\n sys.exit(255)\n\nmain(sys.argv)\n']
def assert_no_memory_leaks(s, argv=[]):
env = os.environ.copy()
env["PYTHONPATH"] = os.pathsep.join(sys.path)
# When using pytest-cov it attempts to instrument subprocesses. This
# causes the memleak tests to raise exceptions.
# we don't need coverage so we remove the env vars.
env.pop("COV_CORE_CONFIG", None)
env.pop("COV_CORE_DATAFILE", None)
env.pop("COV_CORE_SOURCE", None)
argv = [
sys.executable,
"-c",
"{}\n\n{}".format(s, MEMORY_LEAK_SCRIPT),
] + argv
# Shell out to a fresh Python process because OpenSSL does not allow you to
# install new memory hooks after the first malloc/free occurs.
proc = subprocess.Popen(
argv,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
proc.wait()
if proc.returncode == 255:
# 255 means there was a leak, load the info about what mallocs
# weren't freed.
out = json.loads(proc.stdout.read().decode())
raise AssertionError(out)
elif proc.returncode != 0:
# Any exception type will do to be honest
> raise ValueError(proc.stdout.read(), proc.stderr.read())
E ValueError: (b'', b'Traceback (most recent call last):\r\n File "<string>", line 124, in <module>\r\n File "<string>", line 31, in main\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 150, in dlopen\r\n lib, function_cache = _make_ffi_library(self, name, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 832, in _make_ffi_library\r\n backendlib = _load_backend_lib(backend, libname, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 821, in _load_backend_lib\r\n raise OSError("dlopen(None) cannot work on Windows for Python 3 "\r\nOSError: dlopen(None) cannot work on Windows for Python 3 (see http://bugs.python.org/issue23606)\r\n')
tests\hazmat\backends\test_openssl_memleak.py:170: ValueError
_____________________ TestAssertNoMemoryLeaks.test_errors _____________________
self = <tests.hazmat.backends.test_openssl_memleak.TestAssertNoMemoryLeaks object at 0x0000017392223760>
def test_errors(self):
with pytest.raises(ValueError, match="ZeroDivisionError"):
> assert_no_memory_leaks(
textwrap.dedent(
"""
def func():
raise ZeroDivisionError
"""
)
)
tests\hazmat\backends\test_openssl_memleak.py:239:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '\ndef func():\n raise ZeroDivisionError\n'
argv = ['D:\\bld\\cryptography_1656441144990\\_test_env\\python.exe', '-c', '\ndef func():\n raise ZeroDivisionError\n\n\n...n for ptr in remaining\n )))\n sys.stdout.flush()\n sys.exit(255)\n\nmain(sys.argv)\n']
def assert_no_memory_leaks(s, argv=[]):
env = os.environ.copy()
env["PYTHONPATH"] = os.pathsep.join(sys.path)
# When using pytest-cov it attempts to instrument subprocesses. This
# causes the memleak tests to raise exceptions.
# we don't need coverage so we remove the env vars.
env.pop("COV_CORE_CONFIG", None)
env.pop("COV_CORE_DATAFILE", None)
env.pop("COV_CORE_SOURCE", None)
argv = [
sys.executable,
"-c",
"{}\n\n{}".format(s, MEMORY_LEAK_SCRIPT),
] + argv
# Shell out to a fresh Python process because OpenSSL does not allow you to
# install new memory hooks after the first malloc/free occurs.
proc = subprocess.Popen(
argv,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
proc.wait()
if proc.returncode == 255:
# 255 means there was a leak, load the info about what mallocs
# weren't freed.
out = json.loads(proc.stdout.read().decode())
raise AssertionError(out)
elif proc.returncode != 0:
# Any exception type will do to be honest
> raise ValueError(proc.stdout.read(), proc.stderr.read())
E ValueError: (b'', b'Traceback (most recent call last):\r\n File "<string>", line 120, in <module>\r\n File "<string>", line 27, in main\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 150, in dlopen\r\n lib, function_cache = _make_ffi_library(self, name, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 832, in _make_ffi_library\r\n backendlib = _load_backend_lib(backend, libname, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 821, in _load_backend_lib\r\n raise OSError("dlopen(None) cannot work on Windows for Python 3 "\r\nOSError: dlopen(None) cannot work on Windows for Python 3 (see http://bugs.python.org/issue23606)\r\n')
tests\hazmat\backends\test_openssl_memleak.py:170: ValueError
During handling of the above exception, another exception occurred:
self = <tests.hazmat.backends.test_openssl_memleak.TestAssertNoMemoryLeaks object at 0x0000017392223760>
def test_errors(self):
> with pytest.raises(ValueError, match="ZeroDivisionError"):
E AssertionError: Regex pattern 'ZeroDivisionError' does not match '(b\'\', b\'Traceback (most recent call last):\\r\\n File "<string>", line 120, in <module>\\r\\n File "<string>", line 27, in main\\r\\n File "D:\\\\bld\\\\cryptography_1656441144990\\\\_test_env\\\\lib\\\\site-packages\\\\cffi\\\\api.py", line 150, in dlopen\\r\\n lib, function_cache = _make_ffi_library(self, name, flags)\\r\\n File "D:\\\\bld\\\\cryptography_1656441144990\\\\_test_env\\\\lib\\\\site-packages\\\\cffi\\\\api.py", line 832, in _make_ffi_library\\r\\n backendlib = _load_backend_lib(backend, libname, flags)\\r\\n File "D:\\\\bld\\\\cryptography_1656441144990\\\\_test_env\\\\lib\\\\site-packages\\\\cffi\\\\api.py", line 821, in _load_backend_lib\\r\\n raise OSError("dlopen(None) cannot work on Windows for Python 3 "\\r\\nOSError: dlopen(None) cannot work on Windows for Python 3 (see http://bugs.python.org/issue23606)\\r\\n\')'.
tests\hazmat\backends\test_openssl_memleak.py:238: AssertionError
_ TestOpenSSLMemoryLeaks.test_der_x509_certificate_extensions[x509/PKITS_data/certs/ValidcRLIssuerTest28EE.crt] _
self = <tests.hazmat.backends.test_openssl_memleak.TestOpenSSLMemoryLeaks object at 0x0000017392222200>
path = 'x509/PKITS_data/certs/ValidcRLIssuerTest28EE.crt'
@pytest.mark.parametrize(
"path", ["x509/PKITS_data/certs/ValidcRLIssuerTest28EE.crt"]
)
def test_der_x509_certificate_extensions(self, path):
> assert_no_memory_leaks(
textwrap.dedent(
"""
def func(path):
from cryptography import x509
from cryptography.hazmat.backends.openssl import backend
import cryptography_vectors
with cryptography_vectors.open_vector_file(path, "rb") as f:
cert = x509.load_der_x509_certificate(
f.read(), backend
)
cert.extensions
"""
),
[path],
)
tests\hazmat\backends\test_openssl_memleak.py:256:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '\ndef func(path):\n from cryptography import x509\n from cryptography.hazmat.backends.openssl import backend\n\... f:\n cert = x509.load_der_x509_certificate(\n f.read(), backend\n )\n\n cert.extensions\n'
argv = ['D:\\bld\\cryptography_1656441144990\\_test_env\\python.exe', '-c', '\ndef func(path):\n from cryptography import ... sys.stdout.flush()\n sys.exit(255)\n\nmain(sys.argv)\n', 'x509/PKITS_data/certs/ValidcRLIssuerTest28EE.crt']
def assert_no_memory_leaks(s, argv=[]):
env = os.environ.copy()
env["PYTHONPATH"] = os.pathsep.join(sys.path)
# When using pytest-cov it attempts to instrument subprocesses. This
# causes the memleak tests to raise exceptions.
# we don't need coverage so we remove the env vars.
env.pop("COV_CORE_CONFIG", None)
env.pop("COV_CORE_DATAFILE", None)
env.pop("COV_CORE_SOURCE", None)
argv = [
sys.executable,
"-c",
"{}\n\n{}".format(s, MEMORY_LEAK_SCRIPT),
] + argv
# Shell out to a fresh Python process because OpenSSL does not allow you to
# install new memory hooks after the first malloc/free occurs.
proc = subprocess.Popen(
argv,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
proc.wait()
if proc.returncode == 255:
# 255 means there was a leak, load the info about what mallocs
# weren't freed.
out = json.loads(proc.stdout.read().decode())
raise AssertionError(out)
elif proc.returncode != 0:
# Any exception type will do to be honest
> raise ValueError(proc.stdout.read(), proc.stderr.read())
E ValueError: (b'', b'Traceback (most recent call last):\r\n File "<string>", line 130, in <module>\r\n File "<string>", line 37, in main\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 150, in dlopen\r\n lib, function_cache = _make_ffi_library(self, name, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 832, in _make_ffi_library\r\n backendlib = _load_backend_lib(backend, libname, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 821, in _load_backend_lib\r\n raise OSError("dlopen(None) cannot work on Windows for Python 3 "\r\nOSError: dlopen(None) cannot work on Windows for Python 3 (see http://bugs.python.org/issue23606)\r\n')
tests\hazmat\backends\test_openssl_memleak.py:170: ValueError
_ TestOpenSSLMemoryLeaks.test_pem_x509_certificate_extensions[x509/cryptography.io.pem] _
self = <tests.hazmat.backends.test_openssl_memleak.TestOpenSSLMemoryLeaks object at 0x0000017392222080>
path = 'x509/cryptography.io.pem'
@pytest.mark.parametrize("path", ["x509/cryptography.io.pem"])
def test_pem_x509_certificate_extensions(self, path):
> assert_no_memory_leaks(
textwrap.dedent(
"""
def func(path):
from cryptography import x509
from cryptography.hazmat.backends.openssl import backend
import cryptography_vectors
with cryptography_vectors.open_vector_file(path, "rb") as f:
cert = x509.load_pem_x509_certificate(
f.read(), backend
)
cert.extensions
"""
),
[path],
)
tests\hazmat\backends\test_openssl_memleak.py:278:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '\ndef func(path):\n from cryptography import x509\n from cryptography.hazmat.backends.openssl import backend\n\... f:\n cert = x509.load_pem_x509_certificate(\n f.read(), backend\n )\n\n cert.extensions\n'
argv = ['D:\\bld\\cryptography_1656441144990\\_test_env\\python.exe', '-c', '\ndef func(path):\n from cryptography import ...ining\n )))\n sys.stdout.flush()\n sys.exit(255)\n\nmain(sys.argv)\n', 'x509/cryptography.io.pem']
def assert_no_memory_leaks(s, argv=[]):
env = os.environ.copy()
env["PYTHONPATH"] = os.pathsep.join(sys.path)
# When using pytest-cov it attempts to instrument subprocesses. This
# causes the memleak tests to raise exceptions.
# we don't need coverage so we remove the env vars.
env.pop("COV_CORE_CONFIG", None)
env.pop("COV_CORE_DATAFILE", None)
env.pop("COV_CORE_SOURCE", None)
argv = [
sys.executable,
"-c",
"{}\n\n{}".format(s, MEMORY_LEAK_SCRIPT),
] + argv
# Shell out to a fresh Python process because OpenSSL does not allow you to
# install new memory hooks after the first malloc/free occurs.
proc = subprocess.Popen(
argv,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
proc.wait()
if proc.returncode == 255:
# 255 means there was a leak, load the info about what mallocs
# weren't freed.
out = json.loads(proc.stdout.read().decode())
raise AssertionError(out)
elif proc.returncode != 0:
# Any exception type will do to be honest
> raise ValueError(proc.stdout.read(), proc.stderr.read())
E ValueError: (b'', b'Traceback (most recent call last):\r\n File "<string>", line 130, in <module>\r\n File "<string>", line 37, in main\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 150, in dlopen\r\n lib, function_cache = _make_ffi_library(self, name, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 832, in _make_ffi_library\r\n backendlib = _load_backend_lib(backend, libname, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 821, in _load_backend_lib\r\n raise OSError("dlopen(None) cannot work on Windows for Python 3 "\r\nOSError: dlopen(None) cannot work on Windows for Python 3 (see http://bugs.python.org/issue23606)\r\n')
tests\hazmat\backends\test_openssl_memleak.py:170: ValueError
_______________ TestOpenSSLMemoryLeaks.test_x509_csr_extensions _______________
self = <tests.hazmat.backends.test_openssl_memleak.TestOpenSSLMemoryLeaks object at 0x0000017392222680>
def test_x509_csr_extensions(self):
> assert_no_memory_leaks(
textwrap.dedent(
"""
def func():
from cryptography import x509
from cryptography.hazmat.backends.openssl import backend
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import rsa
private_key = rsa.generate_private_key(
key_size=2048, public_exponent=65537, backend=backend
)
cert = x509.CertificateSigningRequestBuilder().subject_name(
x509.Name([])
).add_extension(
x509.OCSPNoCheck(), critical=False
).sign(private_key, hashes.SHA256(), backend)
cert.extensions
"""
)
)
tests\hazmat\backends\test_openssl_memleak.py:299:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '\ndef func():\n from cryptography import x509\n from cryptography.hazmat.backends.openssl import backend\n f...n x509.OCSPNoCheck(), critical=False\n ).sign(private_key, hashes.SHA256(), backend)\n\n cert.extensions\n'
argv = ['D:\\bld\\cryptography_1656441144990\\_test_env\\python.exe', '-c', '\ndef func():\n from cryptography import x509...n for ptr in remaining\n )))\n sys.stdout.flush()\n sys.exit(255)\n\nmain(sys.argv)\n']
def assert_no_memory_leaks(s, argv=[]):
env = os.environ.copy()
env["PYTHONPATH"] = os.pathsep.join(sys.path)
# When using pytest-cov it attempts to instrument subprocesses. This
# causes the memleak tests to raise exceptions.
# we don't need coverage so we remove the env vars.
env.pop("COV_CORE_CONFIG", None)
env.pop("COV_CORE_DATAFILE", None)
env.pop("COV_CORE_SOURCE", None)
argv = [
sys.executable,
"-c",
"{}\n\n{}".format(s, MEMORY_LEAK_SCRIPT),
] + argv
# Shell out to a fresh Python process because OpenSSL does not allow you to
# install new memory hooks after the first malloc/free occurs.
proc = subprocess.Popen(
argv,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
proc.wait()
if proc.returncode == 255:
# 255 means there was a leak, load the info about what mallocs
# weren't freed.
out = json.loads(proc.stdout.read().decode())
raise AssertionError(out)
elif proc.returncode != 0:
# Any exception type will do to be honest
> raise ValueError(proc.stdout.read(), proc.stderr.read())
E ValueError: (b'', b'Traceback (most recent call last):\r\n File "<string>", line 134, in <module>\r\n File "<string>", line 41, in main\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 150, in dlopen\r\n lib, function_cache = _make_ffi_library(self, name, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 832, in _make_ffi_library\r\n backendlib = _load_backend_lib(backend, libname, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 821, in _load_backend_lib\r\n raise OSError("dlopen(None) cannot work on Windows for Python 3 "\r\nOSError: dlopen(None) cannot work on Windows for Python 3 (see http://bugs.python.org/issue23606)\r\n')
tests\hazmat\backends\test_openssl_memleak.py:170: ValueError
_________ TestOpenSSLMemoryLeaks.test_ec_private_numbers_private_key __________
self = <tests.hazmat.backends.test_openssl_memleak.TestOpenSSLMemoryLeaks object at 0x0000017392221330>
def test_ec_private_numbers_private_key(self):
> assert_no_memory_leaks(
textwrap.dedent(
"""
def func():
from cryptography.hazmat.backends.openssl import backend
from cryptography.hazmat.primitives.asymmetric import ec
ec.EllipticCurvePrivateNumbers(
private_value=int(
'280814107134858470598753916394807521398239633534281633982576099083'
'35787109896602102090002196616273211495718603965098'
),
public_numbers=ec.EllipticCurvePublicNumbers(
curve=ec.SECP384R1(),
x=int(
'10036914308591746758780165503819213553101287571902957054148542'
'504671046744460374996612408381962208627004841444205030'
),
y=int(
'17337335659928075994560513699823544906448896792102247714689323'
'575406618073069185107088229463828921069465902299522926'
)
)
).private_key(backend)
"""
)
)
tests\hazmat\backends\test_openssl_memleak.py:323:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = "\ndef func():\n from cryptography.hazmat.backends.openssl import backend\n from cryptography.hazmat.primitives.... '575406618073069185107088229463828921069465902299522926'\n )\n )\n ).private_key(backend)\n"
argv = ['D:\\bld\\cryptography_1656441144990\\_test_env\\python.exe', '-c', '\ndef func():\n from cryptography.hazmat.back...n for ptr in remaining\n )))\n sys.stdout.flush()\n sys.exit(255)\n\nmain(sys.argv)\n']
def assert_no_memory_leaks(s, argv=[]):
env = os.environ.copy()
env["PYTHONPATH"] = os.pathsep.join(sys.path)
# When using pytest-cov it attempts to instrument subprocesses. This
# causes the memleak tests to raise exceptions.
# we don't need coverage so we remove the env vars.
env.pop("COV_CORE_CONFIG", None)
env.pop("COV_CORE_DATAFILE", None)
env.pop("COV_CORE_SOURCE", None)
argv = [
sys.executable,
"-c",
"{}\n\n{}".format(s, MEMORY_LEAK_SCRIPT),
] + argv
# Shell out to a fresh Python process because OpenSSL does not allow you to
# install new memory hooks after the first malloc/free occurs.
proc = subprocess.Popen(
argv,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
proc.wait()
if proc.returncode == 255:
# 255 means there was a leak, load the info about what mallocs
# weren't freed.
out = json.loads(proc.stdout.read().decode())
raise AssertionError(out)
elif proc.returncode != 0:
# Any exception type will do to be honest
> raise ValueError(proc.stdout.read(), proc.stderr.read())
E ValueError: (b'', b'Traceback (most recent call last):\r\n File "<string>", line 139, in <module>\r\n File "<string>", line 46, in main\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 150, in dlopen\r\n lib, function_cache = _make_ffi_library(self, name, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 832, in _make_ffi_library\r\n backendlib = _load_backend_lib(backend, libname, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 821, in _load_backend_lib\r\n raise OSError("dlopen(None) cannot work on Windows for Python 3 "\r\nOSError: dlopen(None) cannot work on Windows for Python 3 (see http://bugs.python.org/issue23606)\r\n')
tests\hazmat\backends\test_openssl_memleak.py:170: ValueError
______________ TestOpenSSLMemoryLeaks.test_ec_derive_private_key ______________
self = <tests.hazmat.backends.test_openssl_memleak.TestOpenSSLMemoryLeaks object at 0x0000017392221930>
def test_ec_derive_private_key(self):
> assert_no_memory_leaks(
textwrap.dedent(
"""
def func():
from cryptography.hazmat.backends.openssl import backend
from cryptography.hazmat.primitives.asymmetric import ec
ec.derive_private_key(1, ec.SECP256R1(), backend)
"""
)
)
tests\hazmat\backends\test_openssl_memleak.py:352:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '\ndef func():\n from cryptography.hazmat.backends.openssl import backend\n from cryptography.hazmat.primitives.asymmetric import ec\n ec.derive_private_key(1, ec.SECP256R1(), backend)\n'
argv = ['D:\\bld\\cryptography_1656441144990\\_test_env\\python.exe', '-c', '\ndef func():\n from cryptography.hazmat.back...n for ptr in remaining\n )))\n sys.stdout.flush()\n sys.exit(255)\n\nmain(sys.argv)\n']
def assert_no_memory_leaks(s, argv=[]):
env = os.environ.copy()
env["PYTHONPATH"] = os.pathsep.join(sys.path)
# When using pytest-cov it attempts to instrument subprocesses. This
# causes the memleak tests to raise exceptions.
# we don't need coverage so we remove the env vars.
env.pop("COV_CORE_CONFIG", None)
env.pop("COV_CORE_DATAFILE", None)
env.pop("COV_CORE_SOURCE", None)
argv = [
sys.executable,
"-c",
"{}\n\n{}".format(s, MEMORY_LEAK_SCRIPT),
] + argv
# Shell out to a fresh Python process because OpenSSL does not allow you to
# install new memory hooks after the first malloc/free occurs.
proc = subprocess.Popen(
argv,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
proc.wait()
if proc.returncode == 255:
# 255 means there was a leak, load the info about what mallocs
# weren't freed.
out = json.loads(proc.stdout.read().decode())
raise AssertionError(out)
elif proc.returncode != 0:
# Any exception type will do to be honest
> raise ValueError(proc.stdout.read(), proc.stderr.read())
E ValueError: (b'', b'Traceback (most recent call last):\r\n File "<string>", line 122, in <module>\r\n File "<string>", line 29, in main\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 150, in dlopen\r\n lib, function_cache = _make_ffi_library(self, name, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 832, in _make_ffi_library\r\n backendlib = _load_backend_lib(backend, libname, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 821, in _load_backend_lib\r\n raise OSError("dlopen(None) cannot work on Windows for Python 3 "\r\nOSError: dlopen(None) cannot work on Windows for Python 3 (see http://bugs.python.org/issue23606)\r\n')
tests\hazmat\backends\test_openssl_memleak.py:170: ValueError
_________ TestOpenSSLMemoryLeaks.test_x25519_pubkey_from_private_key __________
self = <tests.hazmat.backends.test_openssl_memleak.TestOpenSSLMemoryLeaks object at 0x0000017392222770>
def test_x25519_pubkey_from_private_key(self):
> assert_no_memory_leaks(
textwrap.dedent(
"""
def func():
from cryptography.hazmat.primitives.asymmetric import x25519
private_key = x25519.X25519PrivateKey.generate()
private_key.public_key()
"""
)
)
tests\hazmat\backends\test_openssl_memleak.py:364:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '\ndef func():\n from cryptography.hazmat.primitives.asymmetric import x25519\n private_key = x25519.X25519PrivateKey.generate()\n private_key.public_key()\n'
argv = ['D:\\bld\\cryptography_1656441144990\\_test_env\\python.exe', '-c', '\ndef func():\n from cryptography.hazmat.prim...n for ptr in remaining\n )))\n sys.stdout.flush()\n sys.exit(255)\n\nmain(sys.argv)\n']
def assert_no_memory_leaks(s, argv=[]):
env = os.environ.copy()
env["PYTHONPATH"] = os.pathsep.join(sys.path)
# When using pytest-cov it attempts to instrument subprocesses. This
# causes the memleak tests to raise exceptions.
# we don't need coverage so we remove the env vars.
env.pop("COV_CORE_CONFIG", None)
env.pop("COV_CORE_DATAFILE", None)
env.pop("COV_CORE_SOURCE", None)
argv = [
sys.executable,
"-c",
"{}\n\n{}".format(s, MEMORY_LEAK_SCRIPT),
] + argv
# Shell out to a fresh Python process because OpenSSL does not allow you to
# install new memory hooks after the first malloc/free occurs.
proc = subprocess.Popen(
argv,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
proc.wait()
if proc.returncode == 255:
# 255 means there was a leak, load the info about what mallocs
# weren't freed.
out = json.loads(proc.stdout.read().decode())
raise AssertionError(out)
elif proc.returncode != 0:
# Any exception type will do to be honest
> raise ValueError(proc.stdout.read(), proc.stderr.read())
E ValueError: (b'', b'Traceback (most recent call last):\r\n File "<string>", line 122, in <module>\r\n File "<string>", line 29, in main\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 150, in dlopen\r\n lib, function_cache = _make_ffi_library(self, name, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 832, in _make_ffi_library\r\n backendlib = _load_backend_lib(backend, libname, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 821, in _load_backend_lib\r\n raise OSError("dlopen(None) cannot work on Windows for Python 3 "\r\nOSError: dlopen(None) cannot work on Windows for Python 3 (see http://bugs.python.org/issue23606)\r\n')
tests\hazmat\backends\test_openssl_memleak.py:170: ValueError
_______________ TestOpenSSLMemoryLeaks.test_create_ocsp_request _______________
self = <tests.hazmat.backends.test_openssl_memleak.TestOpenSSLMemoryLeaks object at 0x0000017392222890>
def test_create_ocsp_request(self):
> assert_no_memory_leaks(
textwrap.dedent(
"""
def func():
from cryptography import x509
from cryptography.hazmat.backends.openssl import backend
from cryptography.hazmat.primitives import hashes
from cryptography.x509 import ocsp
import cryptography_vectors
path = "x509/PKITS_data/certs/ValidcRLIssuerTest28EE.crt"
with cryptography_vectors.open_vector_file(path, "rb") as f:
cert = x509.load_der_x509_certificate(
f.read(), backend
)
builder = ocsp.OCSPRequestBuilder()
builder = builder.add_certificate(
cert, cert, hashes.SHA1()
).add_extension(x509.OCSPNonce(b"0000"), False)
req = builder.build()
"""
)
)
tests\hazmat\backends\test_openssl_memleak.py:376:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '\ndef func():\n from cryptography import x509\n from cryptography.hazmat.backends.openssl import backend\n f...(\n cert, cert, hashes.SHA1()\n ).add_extension(x509.OCSPNonce(b"0000"), False)\n req = builder.build()\n'
argv = ['D:\\bld\\cryptography_1656441144990\\_test_env\\python.exe', '-c', '\ndef func():\n from cryptography import x509...n for ptr in remaining\n )))\n sys.stdout.flush()\n sys.exit(255)\n\nmain(sys.argv)\n']
def assert_no_memory_leaks(s, argv=[]):
env = os.environ.copy()
env["PYTHONPATH"] = os.pathsep.join(sys.path)
# When using pytest-cov it attempts to instrument subprocesses. This
# causes the memleak tests to raise exceptions.
# we don't need coverage so we remove the env vars.
env.pop("COV_CORE_CONFIG", None)
env.pop("COV_CORE_DATAFILE", None)
env.pop("COV_CORE_SOURCE", None)
argv = [
sys.executable,
"-c",
"{}\n\n{}".format(s, MEMORY_LEAK_SCRIPT),
] + argv
# Shell out to a fresh Python process because OpenSSL does not allow you to
# install new memory hooks after the first malloc/free occurs.
proc = subprocess.Popen(
argv,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
proc.wait()
if proc.returncode == 255:
# 255 means there was a leak, load the info about what mallocs
# weren't freed.
out = json.loads(proc.stdout.read().decode())
raise AssertionError(out)
elif proc.returncode != 0:
# Any exception type will do to be honest
> raise ValueError(proc.stdout.read(), proc.stderr.read())
E ValueError: (b'', b'Traceback (most recent call last):\r\n File "<string>", line 135, in <module>\r\n File "<string>", line 42, in main\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 150, in dlopen\r\n lib, function_cache = _make_ffi_library(self, name, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 832, in _make_ffi_library\r\n backendlib = _load_backend_lib(backend, libname, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 821, in _load_backend_lib\r\n raise OSError("dlopen(None) cannot work on Windows for Python 3 "\r\nOSError: dlopen(None) cannot work on Windows for Python 3 (see http://bugs.python.org/issue23606)\r\n')
tests\hazmat\backends\test_openssl_memleak.py:170: ValueError
_ TestOpenSSLMemoryLeaks.test_load_pkcs12_key_and_certificates[pkcs12/cert-aes256cbc-no-key.p12] _
self = <tests.hazmat.backends.test_openssl_memleak.TestOpenSSLMemoryLeaks object at 0x0000017392222BC0>
path = 'pkcs12/cert-aes256cbc-no-key.p12'
@pytest.mark.parametrize(
"path",
["pkcs12/cert-aes256cbc-no-key.p12", "pkcs12/cert-key-aes256cbc.p12"],
)
def test_load_pkcs12_key_and_certificates(self, path):
> assert_no_memory_leaks(
textwrap.dedent(
"""
def func(path):
from cryptography import x509
from cryptography.hazmat.backends.openssl import backend
from cryptography.hazmat.primitives.serialization import pkcs12
import cryptography_vectors
with cryptography_vectors.open_vector_file(path, "rb") as f:
pkcs12.load_key_and_certificates(
f.read(), b"cryptography", backend
)
"""
),
[path],
)
tests\hazmat\backends\test_openssl_memleak.py:405:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '\ndef func(path):\n from cryptography import x509\n from cryptography.hazmat.backends.openssl import backend\n ...h, "rb") as f:\n pkcs12.load_key_and_certificates(\n f.read(), b"cryptography", backend\n )\n'
argv = ['D:\\bld\\cryptography_1656441144990\\_test_env\\python.exe', '-c', '\ndef func(path):\n from cryptography import ... )))\n sys.stdout.flush()\n sys.exit(255)\n\nmain(sys.argv)\n', 'pkcs12/cert-aes256cbc-no-key.p12']
def assert_no_memory_leaks(s, argv=[]):
env = os.environ.copy()
env["PYTHONPATH"] = os.pathsep.join(sys.path)
# When using pytest-cov it attempts to instrument subprocesses. This
# causes the memleak tests to raise exceptions.
# we don't need coverage so we remove the env vars.
env.pop("COV_CORE_CONFIG", None)
env.pop("COV_CORE_DATAFILE", None)
env.pop("COV_CORE_SOURCE", None)
argv = [
sys.executable,
"-c",
"{}\n\n{}".format(s, MEMORY_LEAK_SCRIPT),
] + argv
# Shell out to a fresh Python process because OpenSSL does not allow you to
# install new memory hooks after the first malloc/free occurs.
proc = subprocess.Popen(
argv,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
proc.wait()
if proc.returncode == 255:
# 255 means there was a leak, load the info about what mallocs
# weren't freed.
out = json.loads(proc.stdout.read().decode())
raise AssertionError(out)
elif proc.returncode != 0:
# Any exception type will do to be honest
> raise ValueError(proc.stdout.read(), proc.stderr.read())
E ValueError: (b'', b'Traceback (most recent call last):\r\n File "<string>", line 128, in <module>\r\n File "<string>", line 35, in main\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 150, in dlopen\r\n lib, function_cache = _make_ffi_library(self, name, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 832, in _make_ffi_library\r\n backendlib = _load_backend_lib(backend, libname, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 821, in _load_backend_lib\r\n raise OSError("dlopen(None) cannot work on Windows for Python 3 "\r\nOSError: dlopen(None) cannot work on Windows for Python 3 (see http://bugs.python.org/issue23606)\r\n')
tests\hazmat\backends\test_openssl_memleak.py:170: ValueError
_ TestOpenSSLMemoryLeaks.test_load_pkcs12_key_and_certificates[pkcs12/cert-key-aes256cbc.p12] _
self = <tests.hazmat.backends.test_openssl_memleak.TestOpenSSLMemoryLeaks object at 0x0000017392222D10>
path = 'pkcs12/cert-key-aes256cbc.p12'
@pytest.mark.parametrize(
"path",
["pkcs12/cert-aes256cbc-no-key.p12", "pkcs12/cert-key-aes256cbc.p12"],
)
def test_load_pkcs12_key_and_certificates(self, path):
> assert_no_memory_leaks(
textwrap.dedent(
"""
def func(path):
from cryptography import x509
from cryptography.hazmat.backends.openssl import backend
from cryptography.hazmat.primitives.serialization import pkcs12
import cryptography_vectors
with cryptography_vectors.open_vector_file(path, "rb") as f:
pkcs12.load_key_and_certificates(
f.read(), b"cryptography", backend
)
"""
),
[path],
)
tests\hazmat\backends\test_openssl_memleak.py:405:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '\ndef func(path):\n from cryptography import x509\n from cryptography.hazmat.backends.openssl import backend\n ...h, "rb") as f:\n pkcs12.load_key_and_certificates(\n f.read(), b"cryptography", backend\n )\n'
argv = ['D:\\bld\\cryptography_1656441144990\\_test_env\\python.exe', '-c', '\ndef func(path):\n from cryptography import ...\n )))\n sys.stdout.flush()\n sys.exit(255)\n\nmain(sys.argv)\n', 'pkcs12/cert-key-aes256cbc.p12']
def assert_no_memory_leaks(s, argv=[]):
env = os.environ.copy()
env["PYTHONPATH"] = os.pathsep.join(sys.path)
# When using pytest-cov it attempts to instrument subprocesses. This
# causes the memleak tests to raise exceptions.
# we don't need coverage so we remove the env vars.
env.pop("COV_CORE_CONFIG", None)
env.pop("COV_CORE_DATAFILE", None)
env.pop("COV_CORE_SOURCE", None)
argv = [
sys.executable,
"-c",
"{}\n\n{}".format(s, MEMORY_LEAK_SCRIPT),
] + argv
# Shell out to a fresh Python process because OpenSSL does not allow you to
# install new memory hooks after the first malloc/free occurs.
proc = subprocess.Popen(
argv,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
proc.wait()
if proc.returncode == 255:
# 255 means there was a leak, load the info about what mallocs
# weren't freed.
out = json.loads(proc.stdout.read().decode())
raise AssertionError(out)
elif proc.returncode != 0:
# Any exception type will do to be honest
> raise ValueError(proc.stdout.read(), proc.stderr.read())
E ValueError: (b'', b'Traceback (most recent call last):\r\n File "<string>", line 128, in <module>\r\n File "<string>", line 35, in main\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 150, in dlopen\r\n lib, function_cache = _make_ffi_library(self, name, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 832, in _make_ffi_library\r\n backendlib = _load_backend_lib(backend, libname, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 821, in _load_backend_lib\r\n raise OSError("dlopen(None) cannot work on Windows for Python 3 "\r\nOSError: dlopen(None) cannot work on Windows for Python 3 (see http://bugs.python.org/issue23606)\r\n')
tests\hazmat\backends\test_openssl_memleak.py:170: ValueError
_______________ TestOpenSSLMemoryLeaks.test_create_crl_with_idp _______________
self = <tests.hazmat.backends.test_openssl_memleak.TestOpenSSLMemoryLeaks object at 0x0000017392222B30>
def test_create_crl_with_idp(self):
> assert_no_memory_leaks(
textwrap.dedent(
"""
def func():
import datetime
from cryptography import x509
from cryptography.hazmat.backends.openssl import backend
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import ec
from cryptography.x509.oid import NameOID
key = ec.generate_private_key(ec.SECP256R1(), backend)
last_update = datetime.datetime(2002, 1, 1, 12, 1)
next_update = datetime.datetime(2030, 1, 1, 12, 1)
idp = x509.IssuingDistributionPoint(
full_name=None,
relative_name=x509.RelativeDistinguishedName([
x509.NameAttribute(
oid=x509.NameOID.ORGANIZATION_NAME, value=u"PyCA")
]),
only_contains_user_certs=False,
only_contains_ca_certs=True,
only_some_reasons=None,
indirect_crl=False,
only_contains_attribute_certs=False,
)
builder = x509.CertificateRevocationListBuilder().issuer_name(
x509.Name([
x509.NameAttribute(
NameOID.COMMON_NAME, u"cryptography.io CA"
)
])
).last_update(
last_update
).next_update(
next_update
).add_extension(
idp, True
)
crl = builder.sign(key, hashes.SHA256(), backend)
crl.extensions.get_extension_for_class(
x509.IssuingDistributionPoint
)
"""
)
)
tests\hazmat\backends\test_openssl_memleak.py:424:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '\ndef func():\n import datetime\n from cryptography import x509\n from cryptography.hazmat.backends.openssl ...hashes.SHA256(), backend)\n crl.extensions.get_extension_for_class(\n x509.IssuingDistributionPoint\n )\n'
argv = ['D:\\bld\\cryptography_1656441144990\\_test_env\\python.exe', '-c', '\ndef func():\n import datetime\n from cry...n for ptr in remaining\n )))\n sys.stdout.flush()\n sys.exit(255)\n\nmain(sys.argv)\n']
def assert_no_memory_leaks(s, argv=[]):
env = os.environ.copy()
env["PYTHONPATH"] = os.pathsep.join(sys.path)
# When using pytest-cov it attempts to instrument subprocesses. This
# causes the memleak tests to raise exceptions.
# we don't need coverage so we remove the env vars.
env.pop("COV_CORE_CONFIG", None)
env.pop("COV_CORE_DATAFILE", None)
env.pop("COV_CORE_SOURCE", None)
argv = [
sys.executable,
"-c",
"{}\n\n{}".format(s, MEMORY_LEAK_SCRIPT),
] + argv
# Shell out to a fresh Python process because OpenSSL does not allow you to
# install new memory hooks after the first malloc/free occurs.
proc = subprocess.Popen(
argv,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
proc.wait()
if proc.returncode == 255:
# 255 means there was a leak, load the info about what mallocs
# weren't freed.
out = json.loads(proc.stdout.read().decode())
raise AssertionError(out)
elif proc.returncode != 0:
# Any exception type will do to be honest
> raise ValueError(proc.stdout.read(), proc.stderr.read())
E ValueError: (b'', b'Traceback (most recent call last):\r\n File "<string>", line 159, in <module>\r\n File "<string>", line 66, in main\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 150, in dlopen\r\n lib, function_cache = _make_ffi_library(self, name, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 832, in _make_ffi_library\r\n backendlib = _load_backend_lib(backend, libname, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 821, in _load_backend_lib\r\n raise OSError("dlopen(None) cannot work on Windows for Python 3 "\r\nOSError: dlopen(None) cannot work on Windows for Python 3 (see http://bugs.python.org/issue23606)\r\n')
tests\hazmat\backends\test_openssl_memleak.py:170: ValueError
_______ TestOpenSSLMemoryLeaks.test_create_certificate_with_extensions ________
self = <tests.hazmat.backends.test_openssl_memleak.TestOpenSSLMemoryLeaks object at 0x0000017392222590>
def test_create_certificate_with_extensions(self):
> assert_no_memory_leaks(
textwrap.dedent(
"""
def func():
import datetime
from cryptography import x509
from cryptography.hazmat.backends.openssl import backend
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import ec
from cryptography.x509.oid import (
AuthorityInformationAccessOID, ExtendedKeyUsageOID, NameOID
)
private_key = ec.generate_private_key(ec.SECP256R1(), backend)
not_valid_before = datetime.datetime.now()
not_valid_after = not_valid_before + datetime.timedelta(days=365)
aia = x509.AuthorityInformationAccess([
x509.AccessDescription(
AuthorityInformationAccessOID.OCSP,
x509.UniformResourceIdentifier(u"http://ocsp.domain.com")
),
x509.AccessDescription(
AuthorityInformationAccessOID.CA_ISSUERS,
x509.UniformResourceIdentifier(u"http://domain.com/ca.crt")
)
])
sans = [u'*.example.org', u'foobar.example.net']
san = x509.SubjectAlternativeName(list(map(x509.DNSName, sans)))
ski = x509.SubjectKeyIdentifier.from_public_key(
private_key.public_key()
)
eku = x509.ExtendedKeyUsage([
ExtendedKeyUsageOID.CLIENT_AUTH,
ExtendedKeyUsageOID.SERVER_AUTH,
ExtendedKeyUsageOID.CODE_SIGNING,
])
builder = x509.CertificateBuilder().serial_number(
777
).issuer_name(x509.Name([
x509.NameAttribute(NameOID.COUNTRY_NAME, u'US'),
])).subject_name(x509.Name([
x509.NameAttribute(NameOID.COUNTRY_NAME, u'US'),
])).public_key(
private_key.public_key()
).add_extension(
aia, critical=False
).not_valid_before(
not_valid_before
).not_valid_after(
not_valid_after
)
cert = builder.sign(private_key, hashes.SHA256(), backend)
cert.extensions
"""
)
)
tests\hazmat\backends\test_openssl_memleak.py:473:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '\ndef func():\n import datetime\n\n from cryptography import x509\n from cryptography.hazmat.backends.openss... not_valid_after\n )\n\n cert = builder.sign(private_key, hashes.SHA256(), backend)\n cert.extensions\n'
argv = ['D:\\bld\\cryptography_1656441144990\\_test_env\\python.exe', '-c', '\ndef func():\n import datetime\n\n from c...n for ptr in remaining\n )))\n sys.stdout.flush()\n sys.exit(255)\n\nmain(sys.argv)\n']
def assert_no_memory_leaks(s, argv=[]):
env = os.environ.copy()
env["PYTHONPATH"] = os.pathsep.join(sys.path)
# When using pytest-cov it attempts to instrument subprocesses. This
# causes the memleak tests to raise exceptions.
# we don't need coverage so we remove the env vars.
env.pop("COV_CORE_CONFIG", None)
env.pop("COV_CORE_DATAFILE", None)
env.pop("COV_CORE_SOURCE", None)
argv = [
sys.executable,
"-c",
"{}\n\n{}".format(s, MEMORY_LEAK_SCRIPT),
] + argv
# Shell out to a fresh Python process because OpenSSL does not allow you to
# install new memory hooks after the first malloc/free occurs.
proc = subprocess.Popen(
argv,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
proc.wait()
if proc.returncode == 255:
# 255 means there was a leak, load the info about what mallocs
# weren't freed.
out = json.loads(proc.stdout.read().decode())
raise AssertionError(out)
elif proc.returncode != 0:
# Any exception type will do to be honest
> raise ValueError(proc.stdout.read(), proc.stderr.read())
E ValueError: (b'', b'Traceback (most recent call last):\r\n File "<string>", line 174, in <module>\r\n File "<string>", line 81, in main\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 150, in dlopen\r\n lib, function_cache = _make_ffi_library(self, name, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 832, in _make_ffi_library\r\n backendlib = _load_backend_lib(backend, libname, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 821, in _load_backend_lib\r\n raise OSError("dlopen(None) cannot work on Windows for Python 3 "\r\nOSError: dlopen(None) cannot work on Windows for Python 3 (see http://bugs.python.org/issue23606)\r\n')
tests\hazmat\backends\test_openssl_memleak.py:170: ValueError
________ TestOpenSSLMemoryLeaks.test_write_pkcs12_key_and_certificates ________
self = <tests.hazmat.backends.test_openssl_memleak.TestOpenSSLMemoryLeaks object at 0x0000017392221630>
def test_write_pkcs12_key_and_certificates(self):
> assert_no_memory_leaks(
textwrap.dedent(
"""
def func():
import os
from cryptography import x509
from cryptography.hazmat.backends.openssl import backend
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.serialization import pkcs12
import cryptography_vectors
path = os.path.join('x509', 'custom', 'ca', 'ca.pem')
with cryptography_vectors.open_vector_file(path, "rb") as f:
cert = x509.load_pem_x509_certificate(
f.read(), backend
)
path2 = os.path.join('x509', 'custom', 'dsa_selfsigned_ca.pem')
with cryptography_vectors.open_vector_file(path2, "rb") as f:
cert2 = x509.load_pem_x509_certificate(
f.read(), backend
)
path3 = os.path.join('x509', 'letsencryptx3.pem')
with cryptography_vectors.open_vector_file(path3, "rb") as f:
cert3 = x509.load_pem_x509_certificate(
f.read(), backend
)
key_path = os.path.join("x509", "custom", "ca", "ca_key.pem")
with cryptography_vectors.open_vector_file(key_path, "rb") as f:
key = serialization.load_pem_private_key(
f.read(), None, backend
)
encryption = serialization.NoEncryption()
pkcs12.serialize_key_and_certificates(
b"name", key, cert, [cert2, cert3], encryption)
"""
)
)
tests\hazmat\backends\test_openssl_memleak.py:537:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
s = '\ndef func():\n import os\n from cryptography import x509\n from cryptography.hazmat.backends.openssl import....NoEncryption()\n pkcs12.serialize_key_and_certificates(\n b"name", key, cert, [cert2, cert3], encryption)\n'
argv = ['D:\\bld\\cryptography_1656441144990\\_test_env\\python.exe', '-c', '\ndef func():\n import os\n from cryptogra...n for ptr in remaining\n )))\n sys.stdout.flush()\n sys.exit(255)\n\nmain(sys.argv)\n']
def assert_no_memory_leaks(s, argv=[]):
env = os.environ.copy()
env["PYTHONPATH"] = os.pathsep.join(sys.path)
# When using pytest-cov it attempts to instrument subprocesses. This
# causes the memleak tests to raise exceptions.
# we don't need coverage so we remove the env vars.
env.pop("COV_CORE_CONFIG", None)
env.pop("COV_CORE_DATAFILE", None)
env.pop("COV_CORE_SOURCE", None)
argv = [
sys.executable,
"-c",
"{}\n\n{}".format(s, MEMORY_LEAK_SCRIPT),
] + argv
# Shell out to a fresh Python process because OpenSSL does not allow you to
# install new memory hooks after the first malloc/free occurs.
proc = subprocess.Popen(
argv,
env=env,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
assert proc.stdout is not None
assert proc.stderr is not None
try:
proc.wait()
if proc.returncode == 255:
# 255 means there was a leak, load the info about what mallocs
# weren't freed.
out = json.loads(proc.stdout.read().decode())
raise AssertionError(out)
elif proc.returncode != 0:
# Any exception type will do to be honest
> raise ValueError(proc.stdout.read(), proc.stderr.read())
E ValueError: (b'', b'Traceback (most recent call last):\r\n File "<string>", line 149, in <module>\r\n File "<string>", line 56, in main\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 150, in dlopen\r\n lib, function_cache = _make_ffi_library(self, name, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 832, in _make_ffi_library\r\n backendlib = _load_backend_lib(backend, libname, flags)\r\n File "D:\\bld\\cryptography_1656441144990\\_test_env\\lib\\site-packages\\cffi\\api.py", line 821, in _load_backend_lib\r\n raise OSError("dlopen(None) cannot work on Windows for Python 3 "\r\nOSError: dlopen(None) cannot work on Windows for Python 3 (see http://bugs.python.org/issue23606)\r\n')
tests\hazmat\backends\test_openssl_memleak.py:170: ValueError
Were you able to have a look, and if yes, did you see anything interesting/suspicious in the logs/traces?
I'm still working through the cargo- & setuptool-rust related issues we have on the cf-side with aarch/ppc, but if it helps, I can also run the test suite with 37.0.4 instead of .0.2.
I've reviewed -- unfortunately the backtrace didn't end up being useful. All of the leaked memory is at https://github.com/openssl/openssl/blob/openssl-3.0/crypto/provider_core.c#L1615-L1616 and I don't have enough context to say why that is. Or more importantly, why this happens in your build environment but not ours, since as we've said we run all of these tests in our own CI.
Just saw that your OpenSSL build flags differ a bit from ours. Anything that stands out to you? no-dynamic-engine maybe?
And thanks a lot for the quick response & analysis. Worst case we can raise an issue upstream...
So the failures persist with cryptography 37.0.4. Would you be able to comment on what build flags you think might be worth adapting?
I also noted that despite building with enable-fips (and passing the tests) on all platforms, cryptography thinks there's no fips provider, e.g. it shows up in the skips like:
<OpenSSLBackend(version: OpenSSL 3.0.5 5 Jul 2022, FIPS: False)>
Tests still fail for 37.0.4....
Would you be able to comment on which build flags you think might be relevant for us to change?
I also noticed (in the test skips) that despite building OpenSSL with enable-fips on all platforms (and passing the test suite), cryptography thinks that there is no FIPS provider, i.e. the recognized backend looks like: <OpenSSLBackend(version: OpenSSL 3.0.5 5 Jul 2022, FIPS: False)>
enable-fips builds the fips provider, but enabling it requires additional configuration. You can see how we do that here https://github.com/pyca/cryptography/blob/main/.github/workflows/build_openssl.sh#L32-L34
And even if you do that config the FIPS provider still isn't loaded by default by cryptography. To make that happen you need to activate it, which we do via a pytest flag in our own tests (https://github.com/pyca/cryptography/blob/main/.github/workflows/ci.yml#L40)
But you really don't want FIPS 😄. No one should unless they have a specific audit requirement for it (e.g. selling to the US govt)
OK, thanks a lot. That covers the fips part...
Regarding the memory leaks, any idea which flags might be causing that? Yours:
export OPENSSL_BUILD_FLAGS_WINDOWS="no-ssl3 no-ssl3-method no-zlib no-shared no-module no-comp no-dynamic-engine"
Ours (currently):
no-module enable-legacy shared enable-fips
Could it be because we're building a shared lib?
Anything is possible in this world sadly. We strictly statically link. Isolating this more is going to require some actual debugging on Windows unfortunately.
FWIW we do run these tests under dynamic linking on Linux. I suppose a minimal test would be to run our CI modified to dynamically link on windows.
I suppose a minimal test would be to run our CI modified to dynamically link on windows.
Is that possible with the split into the infra-repo? I.e. merging something in infra that does not get picked up on main in this repo?
Ooof, not easily, I forgot that for Windows we built static only.
On Wed, Jul 13, 2022 at 3:55 AM h-vetinari @.***> wrote:
I suppose a minimal test would be to run our CI modified to dynamically link on windows.
Is that possible with the split into the infra-repo? I.e. merging something in infra that does not get picked up on main in this repo?
— Reply to this email directly, view it on GitHub https://github.com/pyca/cryptography/issues/7379#issuecomment-1182890301, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAAGBGMLLUBIJSV7JK3VV3VTZY55ANCNFSM5Z56GR4A . You are receiving this because you commented.Message ID: @.***>
-- All that is necessary for evil to succeed is for good people to do nothing.
Perhaps unsurprisingly, the openssl build without fips still shows the same error pattern - I find it very curious though that the leaks do not appear on pypy, and even more so, neither on one particular CPython version (3.8), while the others (3.7; 3.9; 3.10) fail.
IMO that makes it sound more likely that the issue is on the side of (C)Python rather than the openssl?
CC @tiran, if you have any thoughts on this, it would be much appreciated.
PS. For context - the windows builds for cryptography in conda-forge (based on a shared openssl build) run into memory leaks, see PR & CI
On conda the whole stack is built with the same compiler so this couldn't be some kind of mixture of runtimes getting malloc/free mixed up by passing objects across different dlls, right?
Shouldn't be -- we don't instrument malloc/free, we instrument OpenSSL's allocators, via CRYPTO_set_mem_functions, and there's only a single OpenSSL dll.
It looks like the windows builders are failing because BACKTRACE_ENABLED = True doesn't actually work on windows (not surprising, I've never tested
it there). Maybe we can enable it only for Linux builders?
On Tue, Jun 28, 2022 at 3:00 PM h-vetinari @.***> wrote:
So switching BACKTRACE_ENABLED=True also makes the OpenSSL 1.1.1 tests fail. The full logs are here https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=528401&view=results (ignore the aarch/ppc failures).
Could it be that the backtrace is incompatible in some way with the DLL loading? What particularly caught my eye was: OSError: dlopen(None) cannot work on Windows for Python 3, also for the OpenSSL 1.1.1 builds. As soon as the CI is through (so azure doesn't delete the logs), I'll give it another shot without the backtrace
— Reply to this email directly, view it on GitHub https://github.com/pyca/cryptography/issues/7379#issuecomment-1169109400, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAAGBA7WQK3Z2OMABSJ3RLVRNDUFANCNFSM5Z56GR4A . You are receiving this because you commented.Message ID: @.***>
-- All that is necessary for evil to succeed is for good people to do nothing.
no-dynamic-engine does seem plausible.
On Wed, Jul 6, 2022 at 7:23 AM h-vetinari @.***> wrote:
Just saw that your OpenSSL build flags https://github.com/pyca/infra/blob/c32125d7b5a1b43d895e9f8f5b3919351eba3487/cryptography-linux/openssl-version.sh#L5 differ a bit from ours https://github.com/conda-forge/openssl-feedstock/blob/main/recipe/bld.bat#L8-L14. Anything that stands out to you? no-dynamic-engine maybe?
— Reply to this email directly, view it on GitHub https://github.com/pyca/cryptography/issues/7379#issuecomment-1176103630, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAAGBHFS3AVJ5IZBRIHYFLVSVUDPANCNFSM5Z56GR4A . You are receiving this because you commented.Message ID: @.***>
-- All that is necessary for evil to succeed is for good people to do nothing.
We also run these tests on Linux distro where we're dynamically linking.
בתאריך יום ה׳, 7 ביולי 2022, 11:44, מאת Paul Kehrer < @.***>:
Anything is possible in this world sadly. We strictly statically link. Isolating this more is going to require some actual debugging on Windows unfortunately.
— Reply to this email directly, view it on GitHub https://github.com/pyca/cryptography/issues/7379#issuecomment-1177821184, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAAGBGFDIG7IBUT7NRMFO3VS33PNANCNFSM5Z56GR4A . You are receiving this because you commented.Message ID: @.***>
Sorry, github appears to have lost its mind and posted months old emails I sent.
@alex and I have discussed this and we don't believe we can effectively offer any help unless someone comes up with a way to reproduce this in our CI system. Accordingly, we're going to close this issue out, but if you find a way to reproduce the correct conditions in our own CI we'd be happy to take a look again.
Total shot in the dark here, but any chance this has something to do with the tp_traverse change mentioned in the Python 3.9 C API release notes? I don't see any direct use of this functionality in the Cryptography Rust code, but maybe it happens implicitly somehow? If I'm understanding the description of the change correctly, it sounds exactly like something that could introduce a subtle memory leak in Python 3.9 that would be hidden in 3.8.
