test_symbol_presence broken on Linux "devtoolset" compilers, maybe doesn't need to be disabled on macOS

[nickwilliams-zaxiom]

This is reported as fixed in #16837, but either isn't fixed or is broken again.

The test test_symbol_presence is broken on Linux "devtoolset" compilers. "devtoolset" is a way to use a newer compiler on older LTS versions of RHEL and CentOS. It's failing for us on GCC10 on CentOS 7, RHEL 7, and RHEL 8, reporting that essentially all symbols are missing. For now I'm going to skip this test while we're in development phase on this project, but I'm concerned compliance issues will prevent us from moving beyond development with this test skipped.

CentOS 7 with Devtoolset GCC 10

$ nm --version
GNU nm version 2.35-5.el7.3
...
# OPT
$ nm -DPg libcrypto.so.3 | grep AES_options
AES_options @@OPENSSL_3.0.0T b6e10 8
# DEBUG
$ nm -DPg libcrypto.so.3 | grep AES_options
AES_options @@OPENSSL_3.0.0T b6c6c d

Result: Test fails, reports hundreds of missing symbols

RHEL 7 with Devtoolset GCC 10

$ nm --version
GNU nm version 2.35-5.el7.4
...
# OPT
$ nm -DPg libcrypto.so.3 | grep AES_options
AES_options @@OPENSSL_3.0.0T b6e10 8
# DEBUG
$ nm -DPg libcrypto.so.3 | grep AES_options
AES_options @@OPENSSL_3.0.0T b6c6c d

Result: Test fails, reports hundreds of missing symbols

RHEL 8 with Devtoolset GCC 10

$ nm --version
GNU nm version 2.35-8.el8_5.6
...
# OPT
$ nm -DPg libcrypto.so.3 | grep AES_options
AES_options @@OPENSSL_3.0.0T b6e10 8
# DEBUG
$ nm -DPg libcrypto.so.3 | grep AES_options
AES_options @@OPENSSL_3.0.0T b6c6c d

Result: Test fails, reports hundreds of missing symbols

RHEL 8 with stock GCC 8

$ nm --version
GNU nm version 2.30-108.el8_5.1
...
# OPT
$ nm -DPg libcrypto.so.3 | grep AES_options
AES_options T 00000000000b6cc0 0000000000000008
# DEBUG
$ nm -DPg libcrypto.so.3 | grep AES_options
AES_options T 00000000000b6c78 000000000000000d

Result: Test passes

macOS 11.6.2 with Clang 13

$ nm --version
Apple LLVM version 13.0.0 (clang-1300.0.29.30)
...
# OPT
$ nm -Pg libcrypto.3.dylib | grep AES_options
_AES_options T 7c40 0
# DEBUG
$ nm -Pg libcrypto.3.dylib | grep AES_options
_AES_options T 4e40 0

Result: Test disabled on macOS, but does it need to be?

[xtkoba]

This is caused by a bug in GNU Binutils 2.35 which is fixed in 2.35.2. Upstream issue: https://sourceware.org/bugzilla/show_bug.cgi?id=27128.

We could make the test skipped for the buggy versions of nm, although I doubt if it should be handled on the OpenSSL side.

[paulidale]

Agreed, its seem like a stretch for OpenSSL to address this one.

[mattcaswell]

I wonder if we could somehow test for sane output from nm and skip the test if its not sane?

[nickwilliams-zaxiom]

I like Matt's idea. This approach could also get the test defacto working on macOS (currently always disabled on macOS): In newer versions of macOS, the nm output is sane, and so the test would run on those platforms.

[t8m]

The problem is how to check the nm output is sane and it is not the case that the symbol export is somehow broken.

[mattcaswell]

Check a well known symbol from libcrypto. Grep the output from nm to see if it exists at all. If it exists we assume symbol export is working in some capacity. If no symbol export at all then die. Then check the format of the output for that symbol and see if it is a format we recognise. If an unrecognised format then skip the test. Otherwise run the test normally.