python
bpo-34837: Multiprocessing.Pool API Extension - Pass Data to Workers w/o Globals
Proposing a new kwarg in the __init__() method of multiprocessing.Pool named expect_initret.
This kwarg defaults to False. When set to True, the return value of the initializer function is passed to the function we are applying (i.e. func in Pool.map(func, ls)), as a kwarg named initret.
This PR includes thorough test coverage, and provides backwards compatibility (at least in Python3.x).
See this blog post for example use cases, as well as an initial defense for why this makes the multiprocessing.Pool API more approachable, to more Python end-users, and augments the library.
https://bugs.python.org/issue34837
Hello, and thanks for your contribution!
I'm a bot set up to make sure that the project can legally accept your contribution by verifying you have signed the PSF contributor agreement (CLA).
Unfortunately our records indicate you have not signed the CLA. For legal reasons we need you to sign this before we can look at your contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.
You can check yourself to see if the CLA has been received.
Thanks again for your contribution, we look forward to reviewing it!
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.
Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.
Thanks for making the requested changes!
@terryjreedy: please review the changes made to this pull request.
@pitrou see my recent email response to @njsmith 's suggestion. I think we are on to something there...
@seanharr11 I came across your blog about multiprocessing. Seems this PR is still in progress. What blockers do we have? I'm curious if the issue in your blog still exists in the latest version of python?
To my knowledge, this issue still exists in the latest version of Python3. Per comments above, we were looking for consensus from core developers to merge this in.