Unit tests are failing with m2crypto 0.38.0

[chrisburr]

SSLTransport.py::test_simpleMessage[M2-M2] FAILED [100%]
=================================== FAILURES ===================================________ test_simpleMessage[M2-M2] ___________________________
create_serverAndClient = (<DIRAC.Core.DISET.private.Transports.test.Test_SSLTransport.DummyServiceReactor object at 0x7fd43ed8cac0>, <DIRAC.Core.DISET.private.Transports.M2SSLTransport.SSLTransport object at 0x7fd415eccca0>)answer """
      serv, client = create_serverAndClient
>     serverAnswer = ping_server(client)
client     = <DIRAC.Core.DISET.private.Transports.M2SSLTransport.SSLTransport object at 0x7fd415eccca0>ET.private.Transports.test.Test_SSLTransport.DummyServiceReactor object at 0x7fd43ed8cac0>, <DIRAC.Core.DISET.private.Transports.M2SSLTransport.SSLTransport object at 0x7fd415eccca0>)s/test/Test_SSLTransport.py:204:  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ clientTransport = <DIRAC.Core.DISET.private.Transports.M2SSLTransport.SSLTransport object at 0x7fd415eccca0>
    def ping_server(clientTransport):
      """ This sends a message to the server and expects an answer      This basically does the same as BaseClient.py
              :param clientTransport: the Transport object to be used as client
    
      clientTransport.setSocketTimeout(5)
      result = clientTransport.sendData(MAGIC_QUESTION)
      assert result["OK"]
      serverReturn = clientTransport.receiveData()
      if isinstance(serverReturn, dict):
>       assert serverReturn["OK"]
E       assert False

clientTransport = <DIRAC.Core.DISET.private.Transports.M2SSLTransport.SSLTransport object at 0x7fd415eccca0>
result     = {'OK': True, 'Value': None}
serverReturn = {'CallStack': ['  File "/home/cburr/miniconda3/envs/dirac-development/bin/pytest", line 11, in <module>\n    sys.exit(...= lambda hook, methods, kwargs: hook.multicall(\n', ...], 'Errno': 0, 'Message': 'Peer closed connection', 'OK': False}

[chrisburr]

Running the server process separately with DIRAC_DEBUG_M2CRYPTO=yes:

LOOP: SSL accept: before SSL initialization
LOOP: SSL accept: before SSL initialization
LOOP: SSL accept: SSLv3/TLS read client hello
LOOP: SSL accept: SSLv3/TLS write server hello
LOOP: SSL accept: SSLv3/TLS write change cipher spec
LOOP: SSL accept: TLSv1.3 write encrypted extensions
LOOP: SSL accept: SSLv3/TLS write certificate request
LOOP: SSL accept: SSLv3/TLS write certificate
LOOP: SSL accept: TLSv1.3 write server certificate verify
LOOP: SSL accept: SSLv3/TLS write finished
LOOP: SSL accept: TLSv1.3 early data
INFO: SSL accept: TLSv1.3 early data
LOOP: SSL accept: TLSv1.3 early data
SSL DEBUG ERRNUM 40 ERRMSG proxy certificates not allowed, please set the appropriate flag
ALERT: write: fatal: certificate unknown
INFO: SSL accept: error

{
    'OK': False,
    'Errno': 0,
    'Message': "Error in handhsake: certificate verify failed SSLError('certificate verify failed')",
    'CallStack': [
        '  File "/home/cburr/Development/DIRAC-dev/DIRAC/server.py", line 10, in <module>\n    sr.serve()\n',
        '  File "DIRAC/Core/DISET/private/Transports/test/Test_SSLTransport.py", line 96, in serve\n    self.__acceptIncomingConnection()\n',
        '  File "DIRAC/Core/DISET/private/Transports/test/Test_SSLTransport.py", line 116, in __acceptIncomingConnection\n    self.handleConnection(clientTransport)\n',
        '  File "DIRAC/Core/DISET/private/Transports/test/Test_SSLTransport.py", line 80, in handleConnection\n    res = clientTransport.handshake()\n',
        '  File "DIRAC/Core/DISET/private/Transports/M2SSLTransport.py", line 353, in handshake_multipleSteps\n    return S_ERROR("Error in handhsake: %s %s" % (e, repr(e)))\n'
    ]
}

[fstagni]

Basic questions:

• why for py2 version 0.36 is used?
• 0.38 was on pypi since January, why we discover now? https://pypi.org/project/M2Crypto/#history
• https://github.com/conda-forge/m2crypto-feedstock/pull/58 was created by the conda forge bot based on what logic?

[chrisburr]

why for py2 version 0.36 is used?

• For DIRACOS: Recent versions of m2crypto require SWIG 2 which isn't available for CentOS 6 so we can't update DIRACOS and it's pinned to an older version
• For the other CI: We install using conda and conda-forge hasn't supported Python 2 for a long time now. It's possible to still build for Python 2 but it's more involved. I did this for a few versions but I have no motivation to waste more time on it.

0.38 was on pypi since January, why we discover now?

No, it was released three days ago. (I guess Jun != Jan?)

conda-forge/m2crypto-feedstock#58 was created by the conda forge bot based on what logic?

Based on the source URL appearing to have a newer version. In this case by monitoring PyPI directly though the bot has other strategies (GitHub releases, directory listing or just guessing what the next version will be by incrementing each digit).

[chrisburr]

It looks like this issue is this commit which is preventing the constants from being set: https://gitlab.com/m2crypto/m2crypto/-/commit/fd81cf8a5ce81647287eb77790448e3edc9b9dbb

I don't quite understand why the macros don't work correctly but I also don't know SWIG very well...

[fstagni]

Indeed I misread Jun != Jan, sorry about that.

[chrisburr]

I've now understood the issue though I don't see a nice solution so I've opened an issue at https://gitlab.com/m2crypto/m2crypto/-/issues/298

For now I've worked around it by banning that version 42e47d1.