webmagic
webmagic copied to clipboard
Atomicity Violation
Hello! I just wanted to let you know there's a possible bug here that allows more threadAlive.get()
to become greater than threadNum
.
For example, if threadAlive.get() == threadNum - 1
and two threads call execute, they may interleave like this:
t1:
threadAlive.get() >= threadNum
returns false
t2
: threadAlive.get() >= threadNum
returns false
t1
: threadAlive.getAndIncrement()
t2
: threadAlive.getAndIncrement()
https://github.com/code4craft/webmagic/blob/be892b80bf6682cd063d30ac25a79be0c079a901/webmagic-core/src/main/java/us/codecraft/webmagic/thread/CountableThreadPool.java#L53-L69
One possible fix is
if(threadAlive.getAndUpdate(x -> x < threadNum ? x + 1 : x) >= threadNum)
{
reentrantLock.lock();
try
{
while (threadAlive.getAndUpdate(x -> x < threadNum ? x + 1 : x) >= threadNum)
{
try
{
condition.await();
} catch (InterruptedException e)
{
}
}
} finally
{
reentrantLock.unlock();
}
}