common-workflow-service icon indicating copy to clipboard operation
common-workflow-service copied to clipboard

Published 20 hours ago • verified publisher icon NASA-AMMOS

Multiple processes running when limit is set to 1

Open stephenykp opened this issue 4 years ago • 3 comments

We have a singleton process (only enabled on 1 worker, and limit is 1), but we are occasionally seeing 2 concurrent runs of that process.

It seems like this problem is more likely to occur when the process is failing.

stephenykp avatar Jul 15 '21 16:07 stephenykp

One theory is that there are various increment and decrement counter points in the code, and it's probably some race condition where the counter is somehow 0 when it really should be 1 for a very short time, and the worker thinks it's okay to grab and launch another process.

galenatjpl avatar Jul 15 '21 16:07 galenatjpl

This could be complicated and time-consuming to fix. Doesn't happen all that often.. Rough time estimate: 1 week

galenatjpl avatar Oct 05 '21 18:10 galenatjpl

Related to #61

kgrimes2 avatar Nov 30 '21 00:11 kgrimes2