codechecker icon indicating copy to clipboard operation
codechecker copied to clipboard

[server] Use the `multiprocess` library on every platform

Open vodorok opened this issue 2 years ago • 5 comments

Multiprocessing is broken on osx, (probably because of the fork -> spawn change). This patch intends to correct it by using the newly added multiprocess library.

vodorok avatar Nov 22 '23 13:11 vodorok

Did some tests on windows, and I could use the multiprocess lib without issues. I will remove the branching completely.

vodorok avatar Nov 27 '23 13:11 vodorok

Did some tests on windows, and I could use the multiprocess lib without issues. I will remove the branching completely.

@vodorok Let's just make sure these claims hold true for sufficiently old Python versions such as 3.9. Other than that, I think this is good to go.

whisperity avatar Dec 05 '23 13:12 whisperity

Do not merge yet Edit: more explanation: The web tests got really flaky after non functIon change modification (changed import order). I am not completely sure why is this happens, but there can be cases where the zip file disappears during the store process.

vodorok avatar Dec 07 '23 15:12 vodorok

  1. Create a new file codechecker_common/multiprocessing.py which deals with the platform-specific branching.
if sys.platform in ["darwin", "win32"]:
  from multiprocessing import Pool as MultiProcessPool
elif sys.platform == ":
  from concurrent.futures import ProcessPoolExecutor as MultiProcessPool
  1. In the usage places, import this new module and use the aliased name
from codechecker_common.multiprocessing import MultiProcessPool

# ...

with MultiProcessPool(...) as pool:
  ...(..., pool.map)

whisperity avatar Dec 07 '23 16:12 whisperity

Made a different implementation where the linux platform still uses the ProcessPoolExecutor class. See #4118 Will debug this after the release.

vodorok avatar Dec 07 '23 22:12 vodorok