webmagic icon indicating copy to clipboard operation
webmagic copied to clipboard

Atomicity Violation

Open benSepanski opened this issue 3 years ago • 0 comments

Hello! I just wanted to let you know there's a possible bug here that allows more threadAlive.get() to become greater than threadNum.

For example, if threadAlive.get() == threadNum - 1 and two threads call execute, they may interleave like this: t1: threadAlive.get() >= threadNum returns false t2: threadAlive.get() >= threadNum returns false t1: threadAlive.getAndIncrement() t2: threadAlive.getAndIncrement()

https://github.com/code4craft/webmagic/blob/be892b80bf6682cd063d30ac25a79be0c079a901/webmagic-core/src/main/java/us/codecraft/webmagic/thread/CountableThreadPool.java#L53-L69

One possible fix is

if(threadAlive.getAndUpdate(x -> x < threadNum ? x + 1 : x) >= threadNum)
    {
        reentrantLock.lock();
        try
        {
            while (threadAlive.getAndUpdate(x -> x < threadNum ? x + 1 : x) >= threadNum)
            {
                try
                {
                    condition.await();
                } catch (InterruptedException e)
                {
                }
            }
        } finally
        {
            reentrantLock.unlock();
        }
    }

benSepanski avatar Oct 07 '21 23:10 benSepanski