FIPS mode self check error

[MattTheCuber]

We use a Red Hat Enterprise Linux 8.5 machine with FIPS mode enabled. After installing any program (folder or one-file mode) and running it, we get the following error:

Error in GnuTLS initialization: Error while performing self checks.

Note, this is a non-critical bug as it does not seem to affect the usage of the program except the undesired terminal output. Googling the error will return a few posts, but they don't seem to have any solutions.

gnutls-cli version: 3.6.16

[rokm]

Hmm, can you give me a minimal reproducer that uses GnuTLS from python? Are you building under RHEL 8.5 as well, or under some other distribution? Also, what PyInstaller version are you using - gnutls seems to be using .hmac files, and we added collection of those in 6.4.0.

[MattTheCuber]

Great point, I kept seeing the error with everything I tried to compile, but I never tried to create a minimal example. I am using the latest pyinstaller version (6.9.0) and building and running on the same RHEL (8.5) system. Here were my findings in this process:

Issue 1

test.py:

import IPython

import comm

import ipykernel

import jupyter_client

import matplotlib

import matplotlib_inline

Compiling and running one of these returns the GnuTLS error originally stated.

$ pyinstaller test.py
...
$ ./dist/test/test
Error in GnuTLS initialization: Error while performing self checks.

Issue 2

test.py:

import hashlib

import OpenSSL

import parso

Compiling and running one of these returns:

$ pyinstaller test.py
...
$ ./dist/test/test
crypto/fips/fips.c:154: OpenSSL internal error: FATAL FIPS SELFTEST FAILURE
Aborted (core dumped)

What is strange is that it doesn't happen when thrid-party libraries import hashlib, openssl, or parso.

[rokm]

Can you check if .libname.hmac files are collected along with the shared libraries? I.e., assuming that libgnutls.so.XY is collected into frozen application, there should be an accompanying .libgnutls.so.XY.hmac file (note the starting dot) in the same directory. Same for the openssl libs.

[MattTheCuber]

For the import matplotlib program, the following files are collected:

dist/test/_internal/.libcrypto.so.1.1.hmac
dist/test/_internal/.libgcrypt.so.20.hmac
dist/test/_internal/.libgnutls.so.30.hmac
dist/test/_internal/.libhogweed.so.4.hmac
dist/test/_internal/.libnettle.so.6.hmac
dist/test/_internal/.libssl.so.1.1.hmac

For the import hashlib program, only the .libcrypto.so.1.1.hmac file is collected.

[rokm]

Hmm, last time time I was looking into FIPS mode (https://github.com/pyinstaller/pyinstaller/issues/8273) collecting these seemed to suffice.

I'll try to set up a test RHEL8 system again.

[MattTheCuber]

If it is easier for you to explain debugging steps, I would be happy to help you test.

[rokm]

If it is easier for you to explain debugging steps, I would be happy to help you test.

I think it is, because I have no idea what to look for at the moment.

[rokm]

Can you tell me what python version you used, and how did you install it? (If I recall correctly, default is 3.6 which does not work with PyInstaller 6.9). And tested packages are installed from PyPI via pip?

[MattTheCuber]

$ python --version
Python 3.11.6

installed using the yum package manager

Packages are installed from PyPi using pip, yes.

[rokm]

Can you give me pip freeze list for the environment? The comm and matplotlib examples are not pulling in GnuTLS on my test system, so it might be pulled in via some optional dependency.

That said, hashlib example does pull in openssl libs and their hmac files, but it runs on my test system (admittedly, it is 8.10 with python 3.11.9, with FIPS enabled post-hoc via "sudo fips-mode-setup --enable && sudo reboot"). In my case, both libssl.so.1.1 and libcrypto.so.1.1 and their corresponding hmac files are collected. And if I remove either hmac file, I get

crypto/fips/fips.c:154: OpenSSL internal error: FATAL FIPS SELFTEST FAILURE
Aborted (core dumped)

[rokm]

Hmm, let's focus on hashlib and OpenSSL first. If your hashlib build did not collect libssl.so.1.1 and its hmac file (I see that _hashlib extension is, indeed, linked only against libcrypto.so.1.1), can you try to either:

• use --add-binary /usr/lib64/libssl.so.1.1:. to ensure that it is collected
• remove the bundled libcrypto.so.1.1 and try running the application

Could be that issue is that one lib is bundled and the other is not.

[rokm]

Could be that issue is that one lib is bundled and the other is not.

Yep, that seems to be the case on my system; the libssl.so.1.1.so is pulled in by python's _blake2 extension, so if I add --exclude _blake2 to my PyInstaller command for hashlib example, only libcrypto.so.1.1 is collected, and I get the selftest failure.

[MattTheCuber]

Can you give me pip freeze list for the environment?

I am in a giant catch-all environment (probably should have made a fresh one). Let me know if you want me to make a new one or send you a paste bin link of my current environment.

admittedly, it is 8.10 with python 3.11.9, with FIPS enabled post-hoc via "sudo fips-mode-setup --enable && sudo reboot"

Not an expert, but enabling FIPS post-boot shouldn't be problem, I doubt the Python version change makes a difference, but we have noticed big differences between libraries on different RHEL 8.x versions.

Trying you suggestions for hashlib now...

[MattTheCuber]

Adding --add-binary /usr/lib64/libssl.so.1.1:. worked for import hashlib (no errors)

[MattTheCuber]

remove the bundled libcrypto.so.1.1 and try running the application

This also worked

[MattTheCuber]

For reference, when I package import hashlib with no extra arguments it outputs:

Edit: the same structure for import openssl

[MattTheCuber]

Running pyinstaller test.py --exclude _blake2 also did not work on my system (without any modifications to the script or files generated).

[rokm]

Is there a /usr/lib64/python3.11/lib-dynload/_blake2.cpython-311-x86_64-linux-gnu.so file on your system? Is it collected into _internal/lib-dynload? If it is, what is it linked against (ldd <filename>)?

If I try importing it in python REPL, I get the following, though:

>>> import _blake2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: blake2 is not available in FIPS mode

So it could be that it is completely unavailable for you... (and that's why hashlib does not work by default on your system).

[MattTheCuber]

$ ll /usr/lib64/python3.11/lib-dynload/_blake2.cpython-311-x86_64-linux-gnu.so
-rwxr-xr-x. 1 root root 86696 Sep 22  2023 /usr/lib64/python3.11/lib-dynload/_blake2.cpython-311-x86_64-linux-gnu.so

It is collected at _internal/lib-dynload/_blake2.cpython-311-x86_64-linux-gnu.so

$ ldd dist/test/_internal/lib-dynload/_blake2.cpython-311-x86_64-linux-gnu.so
        linux-vdso.so.1 (0x00007ffe345d9000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f481356c000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f48131a7000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f48139ae000)

I get no errors when importing _blake2:

$ python
Python 3.11.6 (main, Nov  2 2023, 14:08:07) [GCC 8.5.0 20210514 (Red Hat 8.5.0-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import _blake2
>>> 

[MattTheCuber]

In an interpreter, import hashlib, and hashlib.blake2b() work fine.

[rokm]

Aha, then it's different in your python version vs. mine. In my case, it is linked against ssl and crypto:

$ ldd /usr/lib64/python3.11/lib-dynload/_blake2.cpython-311-x86_64-linux-gnu.so 
	linux-vdso.so.1 (0x00007ffeec929000)
	libssl.so.1.1 => /lib64/libssl.so.1.1 (0x00007fcd8e911000)
	libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x00007fcd8e426000)
	libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fcd8e206000)
	libc.so.6 => /lib64/libc.so.6 (0x00007fcd8de30000)
	libz.so.1 => /lib64/libz.so.1 (0x00007fcd8dc18000)
	libdl.so.2 => /lib64/libdl.so.2 (0x00007fcd8da14000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fcd8edbb000)

[MattTheCuber]

Wrong file, here's what I get:

$ ldd /usr/lib64/python3.11/lib-dynload/_blake2.cpython-311-x86_64-linux-gnu.so 
        linux-vdso.so.1 (0x00007ffe173eb000)
        libssl.so.1.1 => /lib64/libssl.so.1.1 (0x00007ff65cc24000)
        libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x00007ff65c73b000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ff65c51b000)
        libc.so.6 => /lib64/libc.so.6 (0x00007ff65c156000)
        libz.so.1 => /lib64/libz.so.1 (0x00007ff65bf3e000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007ff65bd3a000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ff65d0cd000)

[rokm]

Hmmm, that confuses situation somewhat. Can you check the build/<name>/Analysis-00.toc and find out where _blake2 was collected from?

Or if you run

import _blake2
print(_blake2.__file__)

in interpreter?

[MattTheCuber]

The lines from build/test/Analysis-00.toc are that mention _blake2 are:

  ('lib-dynload/_blake2.cpython-311-x86_64-linux-gnu.so',
   '/usr/local/lib/python311/lib/python3.11/lib-dynload/_blake2.cpython-311-x86_64-linux-gnu.so',
   'EXTENSION'),

$ python
Python 3.11.6 (main, Nov  2 2023, 14:08:07) [GCC 8.5.0 20210514 (Red Hat 8.5.0-4)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import _blake2
>>> print(_blake2.__file__)
/usr/local/lib/python311/lib/python3.11/lib-dynload/_blake2.cpython-311-x86_64-linux-gnu.so

[rokm]

Do you have two python 3.11 installations on that system? RPM-installed (python3.11, python3.11-libs) in /usr and something manually installed in /usr/local?

[MattTheCuber]

it appears so...

[rokm]

So I take it if you try to import _blake2 using /usr/bin/python3.11, it will raise an error?

[MattTheCuber]

Yep XD

I guess we need to uninstall all instances of Python 3.11.5 from our system?

[rokm]

Yep XD

I guess we need to uninstall all instances of Python 3.11.5 from our system?

Nah, don't, because then we might lose the chance to debug the real issue at hand.

[rokm]

So aside this detour with different python versions, the OpenSSL case is clear: if we collect ssl or crypto shared lib and there are accompanying hmac files, we need to ensure that the other lib is also collected (if that is not already the case).