[BUG][stdlib] Limitation of `parallelize` with Mojo code that interacts with Python

[johnsoez4]

Bug description

Actual behavior

When a parallelized function call interacts with Python, it crashes at runtime under certain conditions. Referring to the code example below, struct Bar.start() uses parallelize to execute Foo.start() on instances stored in a variable of type List[Foo]. The wheels fall off if num_work_items and num_workers each is greater than 1, and the function interacts with Python. Otherwise, num_workers greater than 1 works as expected with Mojo-only code in Foo.start().

Expected behavior

Parallelize works with num_workers greater than 1 in Mojo code that interacts with Python.

Discord

Original post https://discord.com/channels/1087530497313357884/1151418092052815884/1329614332744564787 Feedback from Owen Hilyard and Joe Loser https://discord.com/channels/1087530497313357884/1151418092052815884/1330181446345953302 https://discord.com/channels/1087530497313357884/1151418092052815884/1330190214035017768

Steps to reproduce

from algorithm.functional import parallelize
from python import Python, PythonObject


@value
struct Foo(Copyable & Movable):
    var name: String
    var py_uuid: PythonObject

    fn __init__(out self, name: String) raises:
        self.name = name
        self.py_uuid = Python.import_module("uuid")

    fn start(self, out result: PythonObject):
        try:
            _uuid = self.py_uuid.uuid4()
        except e:
            print("start:", e)
            _uuid = PythonObject("<no uuid>")

        result = _uuid


alias Foos = List[Foo]


@value
struct Bar(Copyable & Movable):
    var name: String
    var _foos: Foos

    fn __init__(out self, name: String):
        self.name = name
        self._foos = Foos()

    fn append(mut self, foo: Foo):
        self._foos.append(foo)

    fn start(self):
        # Serial: This works - OK
        # for index in range(len(self._foos)):
        #     print(self._foos[index].start())

        # Parallel: If num_workers > 1 - KO
        @parameter
        fn parallelize_start(index: Int):
            print("parallelize_start:", index)
            print(self._foos[index].start())

        num_work_items = len(self._foos)
        # num_workers = 1  # OK - Essentially Serial
        num_workers = 2  # KO
        # num_workers = num_work_items  # KO
        parallelize[parallelize_start](num_work_items, num_workers)
        print("Parallel start - Done!")


fn main() raises:
    bars = Bar("My Foos")
    bars.append(Foo("One"))
    bars.append(Foo("Two"))
    bars.start()

System information

Package Version Build Size Kind Source max 25.1.0.dev2025021205 release 9.7 KiB conda max max-core 25.1.0.dev2025021205 release 231 MiB conda max-core max-python 25.1.0.dev2025021205 release 113.9 MiB conda max-python

 Magic version: 0.6.4

System

   Pixi version: 0.40.3
       Platform: linux-64

Virtual packages: __unix=0=0 : __linux=6.11.0=0 : __glibc=2.40=0 : __cuda=12.6=0 : __archspec=1=skylake Cache dir: /home/johnsoe1/.cache/rattler/cache Auth storage: /home/johnsoe1/.rattler/credentials.json Config locations: No config files found

Global

        Bin dir: /home/johnsoe1/.modular/bin
Environment dir: /home/johnsoe1/.modular/envs
   Manifest dir: /home/johnsoe1/.modular/manifests/pixi-global.toml

Project

           Name: EzAsanaDL
        Version: 0.1.0
  Manifest file: /home/johnsoe1/dev/EzProjects/EzAsanaDL/mojoproject.toml
   Last updated: 12-02-2025 09:19:49

Environments

    Environment: default
       Features: default
       Channels: conda-forge, https://conda.modular.com/max-nightly

Dependency count: 7 Dependencies: max, python, pycryptodome, screeninfo, pytest, pip, jupyter PyPI Dependencies: asana, timeloop-ng, dearpygui Target platforms: linux-64 Tasks: tests