ImportError in multithreaded context with cx_Logging [works with Python 3.10]

[deymundson]

Describe the bug "Just in time" imports using the form from foo import bar intermittently fail with an ImportError. When running the Python scripts directly the imports always succeed.

The behavior shows up when two or more threads are attempting to import a given module at the same time. In order for the bug to manifest itself, the following must be true:

• The program is run from a frozen executable.
• The module is being imported from more than one thread simultaneously.
• The module is being imported "just in time" and all threads are importing the module for the first time.
• The thread must make a call to cx_Logging before attempting to import the module.
• The module being imported must import cProfile or pstats. It likely isn't constrained to just these modules, but in the scenario in which I encountered the bug, these imports were involved.

To Reproduce main.py

import cx_Logging
import threading


def import_test():
    # Thread must write with cx_Logging before importing module
    cx_Logging.Trace("import_test(): start")
    try:
        # Import needs to be "from x import y", not "import x"
        from module import Module
    except ImportError:
        cx_Logging.Trace("import_test(): import failed")
    cx_Logging.Trace("import_test(): end")


if __name__ == "__main__":
    cx_Logging.StartLoggingStdout(20, prefix="%t [%i]")

    cx_Logging.Trace("run_test(): Round 1")
    threads1 = [threading.Thread(target=import_test) for i in range(3)]
    for thread in threads1:
        thread.start()
    for thread in threads1:
        thread.join()

    cx_Logging.Trace("run_test(): Round 2")
    threads2 = [threading.Thread(target=import_test) for i in range(3)]
    for thread in threads2:
        thread.start()
    for thread in threads2:
        thread.join()

module.py

# Module needs to import cProfile or pstats
import cProfile


class Module(object):
    pass

Freeze Command

cxfreeze.exe --includes cx_Logging -c main.py

Sample Output

15:58:45.986 [27604] run_test(): Round 1
15:58:45.988 [27532] import_test(): start
15:58:45.990 [21256] import_test(): start
15:58:45.991 [23600] import_test(): start
15:58:45.992 [23600] import_test(): import failed
15:58:45.993 [27532] import_test(): end
15:58:45.994 [21256] import_test(): end
15:58:45.995 [23600] import_test(): end
15:58:45.997 [27604] run_test(): Round 2
15:58:45.998 [04608] import_test(): start
15:58:45.998 [23008] import_test(): start
15:58:45.999 [04608] import_test(): end
15:58:46.000 [12200] import_test(): start
15:58:46.001 [23008] import_test(): end
15:58:46.001 [12200] import_test(): end

Round 1 shows that the initial attempt to import the module fails for one of the threads. Round 2 shows that the import succeeds for all threads after the module was initially loaded.

Expected behavior As is the case when running the Python scripts directly, the imports should succeed.

Desktop (please complete the following information):

• Platform information: Windows 10.0.19044
• OS architecture: amd64
• cx_Freeze version: 6.11.1
• Python version: 3.8

[marcelotduarte]

I tested w/ py 3.8.10 and confirm. I tested also installing cx_Freeze and cx_Logging from my packages*: pip install --pre --extra-index-url https://marcelotduarte.github.io/packages/ cx_Logging pip install --pre --extra-index-url https://marcelotduarte.github.io/packages/ cx_Freeze

and its works. With py 3.10 it worked for me w/ cx_Freeze 6.11.1 and the cx_Logging from my packages.

• cx_Logging from my packages are built with your PR https://github.com/anthony-tuininga/cx_Logging/pull/6

[marcelotduarte]

Can you test with the latest version of cx_Freeze and cx_Logging 3.1?

[marcelotduarte]

Can I close this issue?

[deymundson]

Sorry for the delay on getting back to this. I tested with Python 3.8.10, cx_Freeze 6.15.3, and cx_Logging 3.1.0 and I am still getting the issue.

[marcelotduarte]

Can you test with Python 3.10?

[deymundson]

Tested with Python 3.10.11 and I no longer get the issue.

[marcelotduarte]

I discovered that if you put a time.sleep(0.1) before the `thread.start() in round 1, it runs without problems with Python 3.8.10/64.