gil_load icon indicating copy to clipboard operation
gil_load copied to clipboard

gil_load inside docker has another threading scheme

Open DerLegatLabienus opened this issue 1 year ago • 4 comments

Setup: WSL2 ubuntu22 over windows 11

In our deployment, we have two options to run gil_load module:

  1. django application running on our laptop
  2. django application running inside containers

The gil module is loaded as part of the application init on a different spawned thread.

The first one works ok. Before we invoke "./manage.py runserver" the application detects only one thread with "threading.active_count()) and continues to load the module and gather gil stats.

The second one detects bunch of other python threads such as: WriterThread, ReaderThread, PyDBCommandThread, TimeoutThread, CheckAliveThread. Those threads seems to pop up only when runserver django command is executed

The following code demonstrated this. This is a entrypoint script that is executed as part of the django boot up. The code that prints the number of active threads is also executed from within the django app. In the comments I am writing down the output, number of detected threads, for each command:

export LD_PRELOAD=$(python3 -c "import gil_load; print(gil_load.preload_path)"):$LD_PRELOAD # LD_PRELOAD is set OK! echo $(python3 -c "import threading; print(threading.active_count())") # shows one thread only, OK python manage.py runserver 0.0.0.0:8000 # shows 6 threads when one of the inner django application initalizes the gil module, Not good

PS: This issue also happens when gunicorn is used. instead of django runserver. Therefore, the gil library probably needs to be written for docker containers.

DerLegatLabienus avatar Jan 18 '23 16:01 DerLegatLabienus