loky icon indicating copy to clipboard operation
loky copied to clipboard

Worker ID functionality

Open pmla opened this issue 4 years ago • 6 comments

Hello loky team.

I would like to implement a "worker ID" function for loky. I originally wrote an issue and a pull request on the joblib github page. @tomMoral has kindly pointed me to the correct place for this PR.

@tomMoral: I have stripped this PR down so that it is relevant to loky and added a unit test. The worker ID assignment now accounts for available IDs with the process_worker_id variable. I'm not sure if I have accounted for all mechanisms by which processes can be ended though.

pmla avatar Jul 03 '20 14:07 pmla

Any progress? Very useful feature!

danielyan86129 avatar Jul 15 '22 22:07 danielyan86129

Would love to see this merged!

scottgigante-immunai avatar May 04 '23 22:05 scottgigante-immunai

any progress of getting merged??

qianyizhang avatar Apr 09 '24 01:04 qianyizhang

Thanks for the ping on this feature. To make progress, the best would be to have people testing this feature extensively to see that it does not break. One would need to rebase this one and see how it goes.

tomMoral avatar Apr 09 '24 05:04 tomMoral

Thanks for the ping on this feature. To make progress, the best would be to have people testing this feature extensively to see that it does not break. I can try to rebase this one and see how it goes.

Bote that another implementation of this feature is in #285 (IIRC, I forgot this PR existed and give a try to make this feature, sorry for the duplication).

tomMoral avatar Apr 09 '24 05:04 tomMoral

Thanks for the ping on this feature. To make progress, the best would be to have people testing this feature extensively to see that it does not break. I can try to rebase this one and see how it goes.

Bote that another implementation of this feature is in #285 (IIRC, I forgot this PR existed and give a try to make this feature, sorry for the duplication).

Either implementation appears to be satisfactory, but I prefer this one as it introduces an environment variable rather than an attribute, which seems less complex. This feature does not alter any existing functionality, so please proceed with a release if feasible.

qianyizhang avatar Apr 09 '24 23:04 qianyizhang