Threadpool executor breaks child process exit codes on Python3.11

[ivanvenosdel]

Things to check first

• [X] I have checked that my issue does not already have a solution in the FAQ

• [X] I have searched the existing issues and didn't find my bug already reported there

• [X] I have checked that my bug is still present in the latest release

Version

3.10.4

What happened?

Say you have a function that starts a process blocks till it completes and then returns. Running that function as a job with the ThreadPoolExecutor works fine with all APScheduler versions on Python3.8. However on Python3.11 the ThreadPoolExecutor (but not the ProcessPoolExecutor) will cause the exit code of the child process to always be 1. Regardless of the exit code it actually exited with.

How can we reproduce the bug?

Here is some minimal Python code that reproduces and explains the bug.

import multiprocessing
from datetime import datetime, timezone
import sys

from apscheduler.executors.pool import ThreadPoolExecutor
from apscheduler.schedulers.blocking import BlockingScheduler


def print_something():
    print('something')


def print_something_exit9():
    print('something, exit 9')
    sys.exit(9)


def print_something_in_process():
    process = multiprocessing.Process(target=print_something)
    process.start()
    process.join()

    print(f'Expected exit code: 0, Actual exit code: {process.exitcode}')
    process.close()


def print_something_in_process_exit9():
    process = multiprocessing.Process(target=print_something_exit9)
    process.start()
    process.join()

    print(f'Expected exit code: 9, Actual exit code: {process.exitcode}')
    process.close()
    

scheduler = BlockingScheduler()
scheduler.add_job(print_something_in_process, next_run_time=datetime.now(timezone.utc))
scheduler.add_job(print_something_in_process_exit9, next_run_time=datetime.now(timezone.utc))
scheduler.start()

Python 3.8 output

something Expected exit code: 0, Actual exit code: 0 something, exit 9 Expected exit code: 9, Actual exit code: 9

Python 3.11 output

something something, exit 9 Expected exit code: 0, Actual exit code: 1 Expected exit code: 9, Actual exit code: 1

[agronholm]

If you mix multiprocessing with threads, make sure that you don't use forking when creating subprocesses. Forking is easily abused and causes a lot of difficult problems when not used correctly.

[agronholm]

multiprocessing.set_start_method('spawn') should fix your issue. I don't think this was ever an APScheduler issue.