django-q2 icon indicating copy to clipboard operation
django-q2 copied to clipboard

Using the multiprocessing module to start, unable to stop qcluster properly

Open afroginawell opened this issue 1 year ago • 0 comments

I hope to automatically execute the qcluster command before running the runserver command. I used the multiprocessing and subprocess modules to execute the command python manage.py qcluster, avoiding blocking the main process. However, after starting it, I cannot stop qcluster normally; using ctrl + C causes the program to crash and then automatically restart. Is there a more elegant way to stop qcluster?

Test code:

import sys
from multiprocessing import Process
from subprocess import run
from sys import stdout

from djangoproject.settings import BASE_DIR


def run_command(command: str, cwd: str, shell=False, stdout_=stdout, ):
    if not stdout_:
        stdout_ = stdout
    print(f"execute command: {command}")
    run(command, cwd=cwd, stdout=stdout_, shell=shell)


def run_command_multiprocess(command: str, cwd: str, shell=False) -> Process:
    p = Process(target=run_command, args=(command, cwd, shell, None))
    p.start()
    return p


def djangoproject_qcluster(prod=False) -> Process:
    if prod:
        manage_py = BASE_DIR / 'manage_prod.py'
    else:
        manage_py = BASE_DIR / 'manage.py'
    return run_command_multiprocess(command=f'{sys.executable} {manage_py} qcluster', cwd=BASE_DIR)


if __name__ == '__main__':
    djangoproject_qcluster()

Output after Ctrl + C.

02:23:14 [Q] CRITICAL reincarnated worker Process-eee46fb113404184abc93a0f7936965a after death
02:23:14 [Q] CRITICAL reincarnated worker Process-74ff89c2bc38453197eedee72d991b98 after death
02:23:14 [Q] CRITICAL reincarnated monitor Process-255929e38745465dbe83fcd2b396e098 after sudden death
02:23:14 [Q] CRITICAL reincarnated pusher Process-42b52b2f7c7e40f49225dcb466f6fb16 after sudden death
02:23:15 [Q] CRITICAL reincarnated worker Process-c6a976bc513747768f029b64c6f5b648 after death
02:23:16 [Q] INFO Process-75e97c6995aa47239e027b98b2476778 ready for work at 12928
02:23:16 [Q] CRITICAL reincarnated worker Process-24d7df774d88446cae30919a895dd5c2 after death
02:23:16 [Q] INFO Process-0f8a641b23a846e5b8c3bde3d4c59275 ready for work at 8120
02:23:16 [Q] INFO Process-5ee275eae9b54749b029919d5c5c688c monitoring at 14560
02:23:16 [Q] INFO Process-21522f61a8304d51aafb6cfcbaab4954 pushing tasks at 13448
02:23:17 [Q] INFO Process-c87c724f2fe5499496cc14295d1c3e03 ready for work at 1324
02:23:17 [Q] INFO Process-714b49d3def54bfd83267bd3296f1a25 ready for work at 5660

django-q2==1.7.3 Django==4.2.16 Python: 3.9.10

afroginawell avatar Oct 30 '24 18:10 afroginawell