Python tests hanging when compiling Zendoo in debug mode

[ptagl]

Some Python tests are getting stuck during execution if Zendoo is compiled in debug mode.

Steps to reproduce

CONFIGURE_FLAGS="--enable-debug"
make distclean
./zcutil/build.sh
cd qa/rpc-tests
python prioritisetransaction.py --nocleanup --tracerpc

Analysis

The Python test hangs for a deadlock in Zendoo, the responsible seems to be the call to ShrinkDebugFile() inside the function LogPrintStr() (util.cpp).

The issue shows only in debug mode because the macro DEBUG_LOCKORDER is activated. Specifically, the deadlock is caused by the fact that some thread performs a LOCK that is detected as a potential deadlock (see the function potential_deadlock_detected()) and tries to print some debug strings. If this print is the first one requested after the initialization of the debug file, a call to ShrinkDebugFile() is performed, causing a call to LOCK(csPathCached) to hang forever since the mutex dd_mutex is already locked inside the function push_lock (sync.cpp).

Possible solutions

1. Move the call to ShrinkDebugFile() somewhere outside LogPrintStr() so that the function push_lock() is never called recursively.
2. Edit the push_lock() function so that dd_mutex is reentrant/recursive and does not cause a deadlock anymore (in this case we should carefully analyse the implications of such change).

Additional notes

The issue can be reproduced for any Python test that does not start the Zendoo nodes with the parameter -debug, since this flag automatically disables the shrink of the debug file.