pyopenssl icon indicating copy to clipboard operation
pyopenssl copied to clipboard

self test problem on NetBSD

Open 0-wiz-0 opened this issue 8 years ago • 12 comments

There is one self test failure in 17.1.0 on NetBSD (down from 21 in 17.0.0):

____________________________________________________________________________ TestContext.test_set_verify_callback_exception ____________________________________________________________________________

self = <tests.test_ssl.TestContext object at 0x7ba1466627d8>

    def test_set_verify_callback_exception(self):
        """
            If the verify callback passed to `Context.set_verify` raises an
            exception, verification fails and the exception is propagated to the
            caller of `Connection.do_handshake`.
            """
        serverContext = Context(TLSv1_METHOD)
        serverContext.use_privatekey(
            load_privatekey(FILETYPE_PEM, cleartextPrivateKeyPEM))
        serverContext.use_certificate(
            load_certificate(FILETYPE_PEM, cleartextCertificatePEM))
    
        clientContext = Context(TLSv1_METHOD)
    
        def verify_callback(*args):
            raise Exception("silly verify failure")
        clientContext.set_verify(VERIFY_PEER, verify_callback)
    
        with pytest.raises(Exception) as exc:
>           self._handshake_test(serverContext, clientContext)
E           Failed: DID NOT RAISE <class 'Exception'>

tests/test_ssl.py:1301: Failed
========================================================================== 1 failed, 490 passed, 14 skipped in 15.51 seconds ===========================================================================

0-wiz-0 avatar Jul 03 '17 20:07 0-wiz-0

What version of OpenSSL does NetBSD have these days?

alex avatar Jul 03 '17 20:07 alex

OpenSSL 1.0.2k 26 Jan 2017

0-wiz-0 avatar Jul 03 '17 20:07 0-wiz-0

Any notable custom patches, or just direct from upstream?

alex avatar Jul 03 '17 20:07 alex

Many minor changes. openssl.zip

0-wiz-0 avatar Jul 03 '17 21:07 0-wiz-0

This isn't substantive feedback, but it is MUCH harder to review this diff because it is filled with pointless s/free_func/freefunc/g. It'd be significantly nicer to downstream if NetBSD restricted itself to substantive changes (or upstreamed their changes).

alex avatar Jul 03 '17 21:07 alex

Ok, I don't think I see anything relevant in the diff, though I only skimmed it.

alex avatar Jul 03 '17 21:07 alex

I agree that a smaller diff would be nice, but I don't maintain OpenSSL in NetBSD. I asked around, the particular diff is probably to fix -Wshadow builds.

0-wiz-0 avatar Jul 03 '17 22:07 0-wiz-0

I have the same failure on Gentoo/FreeBSD. This is using vanilla pyopenssl-17.5.0 and openssl-1.0.2n (with some Gentoo patches; the same patchset on Linux doesn't cause problems). Tried with CPython 2.7 and 3.5.

==================================================================== FAILURES =====================================================================
_________________________________________________ TestContext.test_set_verify_callback_exception __________________________________________________

self = <tests.test_ssl.TestContext object at 0x808bc8860>

    def test_set_verify_callback_exception(self):
        """
            If the verify callback passed to `Context.set_verify` raises an
            exception, verification fails and the exception is propagated to the
            caller of `Connection.do_handshake`.
            """
        serverContext = Context(TLSv1_METHOD)
        serverContext.use_privatekey(
            load_privatekey(FILETYPE_PEM, cleartextPrivateKeyPEM))
        serverContext.use_certificate(
            load_certificate(FILETYPE_PEM, cleartextCertificatePEM))
    
        clientContext = Context(TLSv1_METHOD)
    
        def verify_callback(*args):
            raise Exception("silly verify failure")
        clientContext.set_verify(VERIFY_PEER, verify_callback)
    
        with pytest.raises(Exception) as exc:
>           self._handshake_test(serverContext, clientContext)
E           Failed: DID NOT RAISE <class 'Exception'>

tests/test_ssl.py:1326: Failed
================================================ 1 failed, 477 passed, 14 skipped in 10.79 seconds ================================================

mgorny avatar Feb 22 '18 06:02 mgorny

Related Gentoo bug: https://bugs.gentoo.org/648446

mgorny avatar Feb 22 '18 06:02 mgorny

NetBSD and Darwin (macOS) with pyOpenSSL 19.0.0:

__________________________________________ TestX509StoreContext.test_verify_with_time __________________________________________

self = <tests.test_crypto.TestX509StoreContext object at 0x74e26cffc9d8>

    def test_verify_with_time(self):
        """
        `verify_certificate` raises error when the verification time is
        set at notAfter.
        """
        store = X509Store()
        store.add_cert(self.root_cert)
        store.add_cert(self.intermediate_cert)
    
        expire_time = self.intermediate_server_cert.get_notAfter()
        expire_datetime = datetime.strptime(
            expire_time.decode('utf-8'), '%Y%m%d%H%M%SZ'
        )
        store.set_time(expire_datetime)
    
        store_ctx = X509StoreContext(store, self.intermediate_server_cert)
        with pytest.raises(X509StoreContextError) as exc:
>           store_ctx.verify_certificate()
E           Failed: DID NOT RAISE <class 'OpenSSL.crypto.X509StoreContextError'>

tests/test_crypto.py:3604: Failed
________________________________________ TestContext.test_set_verify_callback_exception ________________________________________

self = <tests.test_ssl.TestContext object at 0x74e26853dbd8>

    def test_set_verify_callback_exception(self):
        """
        If the verify callback passed to `Context.set_verify` raises an
        exception, verification fails and the exception is propagated to the
        caller of `Connection.do_handshake`.
        """
        serverContext = Context(TLSv1_2_METHOD)
        serverContext.use_privatekey(
            load_privatekey(FILETYPE_PEM, cleartextPrivateKeyPEM))
        serverContext.use_certificate(
            load_certificate(FILETYPE_PEM, cleartextCertificatePEM))
    
        clientContext = Context(TLSv1_2_METHOD)
    
        def verify_callback(*args):
            raise Exception("silly verify failure")
        clientContext.set_verify(VERIFY_PEER, verify_callback)
    
        with pytest.raises(Exception) as exc:
>           self._handshake_test(serverContext, clientContext)
E           Failed: DID NOT RAISE <class 'Exception'>

tests/test_ssl.py:1323: Failed

moubctez avatar Jan 22 '19 09:01 moubctez

If you're able to replicate the pyOpenSSL test failures in a clean virtualenv on macOS with an up-to-date cryptography please open a new issue. There's something odd going on there since it's passing in CI as well as on my laptop.

reaperhulk avatar Jan 22 '19 15:01 reaperhulk

I confirm that on FreeBSD the test is also failing. Tested with last release 20.0.1. Result below:

_____________________________________________________________ TestContext.test_set_verify_default_callback[1] ______________________________________________________________

self = <tests.test_ssl.TestContext object at 0x803a03050>, mode = 1

    @pytest.mark.parametrize("mode", [SSL.VERIFY_PEER, SSL.VERIFY_NONE])
    def test_set_verify_default_callback(self, mode):
        """
        If the verify callback is omitted, the preverify value is used.
        """
        serverContext = Context(TLSv1_2_METHOD)
        serverContext.use_privatekey(
            load_privatekey(FILETYPE_PEM, root_key_pem)
        )
        serverContext.use_certificate(
            load_certificate(FILETYPE_PEM, root_cert_pem)
        )
    
        clientContext = Context(TLSv1_2_METHOD)
        clientContext.set_verify(mode, None)
    
        if mode == SSL.VERIFY_PEER:
            with pytest.raises(Exception) as exc:
>               self._handshake_test(serverContext, clientContext)
E               Failed: DID NOT RAISE <class 'Exception'>

tests/test_ssl.py:1456: Failed

Environment of the tests:

$ uname -a && openssl version
FreeBSD freebsd 12.2-RELEASE-p4 FreeBSD 12.2-RELEASE-p4 GENERIC  amd64
OpenSSL 1.1.1h-freebsd  22 Sep 2020

sbz avatar Mar 29 '21 21:03 sbz