BrokenProcessPool with "__ (Double Underscore)" methods in Python 3.8

[c-thiel]

Remark I am not sure if this is a loky Issue. It might be a pickle issue too. If you could point me in the right direction that would be very helpful.

Problem Name mangling with double underscore methods is not performed correctly at some point of the execution in Python 3.8, so that the following code snippet fails:

import joblib
import functools

print(joblib.__version__)


class MyClass:
    b = 1

    def process_parallel(self, list_of_a):
        with joblib.parallel_backend('loky'):
            f = functools.partial(self.__process_wrapper)
            r = joblib.Parallel(n_jobs=2)(
                joblib.delayed(f)(i) for i in list_of_a)
        return r

    def __process_wrapper(self, a):
        r = self.process(a)
        return r


    def process(self, a):
        return self.b + a

c = MyClass()
print(c.process_parallel(range(3)))

It fails with the following error: AttributeError: 'MyClass' object has no attribute '__process_wrapper' and thus joblib.externals.loky.process_executor.BrokenProcessPool: A task has failed to un-serialize. Please ensure that the arguments of the function are all picklable. The exact same snippet works in Python 3.7.

Simply renaming __process_wrapper to _process_wrapper omits name wrangling and works as expected.

Version I am using joblib 0.16.0. It also fails with older versions of joblib with Python 3.8. If you need any further details of my environment please let me know.