check for libcrypto using EVP_CIPHER_CTX_new

[ryancdotorg]

I build static versions of tcpdump for my own convenience, and I disable DES in OpenSSL when doing so. This causes tcpdump to fail to detect libcrypto.

Currently, I have a script that applies this patch - looking for AES_cbc_encrypt rather than DES_cbc_encrypt - before building.

Given that AES is now over a quarter of a century old, and is a required algorithm in TLS, I think we can assume that if libcrypto exsists, it will support AES.

DES, on the other hand, seems very reasonable to disable when building OpenSSL, and I expect that some Linux distributions will begin doing this by default in the next few years.

[infrastation]

This is exactly where #1111 started.

[ryancdotorg]

From having a quick look at #1111 I have changed this to use EVP_CIPHER_CTX_new.

Could the maintainers please let me know if they're willing to accept a patch to bundle an MD5 implementation to fall back to if libcrypto isn't available to address concerns raised in #1111?

I would be happy to provide HMAC-MD5 that can reuse key material across multiple calls (somewhat faster) as well.

[ryancdotorg]

Actually, there's a public domain implementation of MD5 that's compatible with the OpenSSL function prototypes here

https://openwall.info/wiki/people/solar/software/public-domain-source-code/md5

which could be used.

[ryancdotorg]

This needs more changes, please do not merge.

[infrastation]

With long-term maintenance in mind, a useful way to reason about it seems to be "if a user wants to use a particular old/new cipher/mode, it is their responsibility to provide a library version that supports it, and it is tcpdump code responsibility to detect and to use the required library functions, but not to reimplement any functionality that the library does not have".

[ryancdotorg]

That seems sensible. Separate checks for MD5 and disabling that code when not available is straightforward.

So, then:

• I will switch this back to doing a gate on AES_cbc_encrypt.

In order to future-proof, in anticipation of not being able to rely on the long term availability if MD5_Init/MD5_Update/MD5_Final in libcrypto:

• I will add checks for MD5 availability to the configure/cmake scripts.
• I will change the ifdefs for code depending on MD5 availability to use HAVE_MD5_INIT.

[guyharris]

Given that we use neither DES_cbc_encrypt() nor AES_cbc_encrypt(), we shouldn't check for either one.

We currently use EVP_CIPHER_CTX_new() if it's present, and use calloc() to allocate a EVP_CIPHER_CTX if it's not.

We do use EVP_CIPHER_CTX_block_size() unconditionally, so, in order to check whether we can build with libcrypto, we can just check for that. See pull request #1197.