Thredsafe cachedmethod

[vit-zikmund]

Hello, this is rather a notice about a decorator I wrote for cachetools for a different OSS project dealing with concurrent access. My goal was to prevent the race condition where multiple concurrent threads populate the same cache entry, which I solved by letting just one thread call a method at a time, while all other concurrent threads in the same call are blocked till the result/exception is available, at which moment the result is passed on (or exception re-raised).

Providing all this as a standalone method decorator, I though it might be an interesting contribution to the current pack of decorators. Feel free to have a look if that would be a fitting contribution :wink:

[Dobatymo]

Could you go into more detail what's the difference between this decorator and simply passing a threading.Lock to cachedmethod?

[vit-zikmund]

Thanks for the question! Passing a Lock to the cachedmethod makes sure a result of the method call is not stored twice. But that's it. First it's used to check whether the value is not already cached and then it's released, after which the wrapped method is being called. After it returns, the lock is used again to make sure the result is properly stored. The catch is that between the locks multiple threads can run free, so the (potentially expensive) method can be called multiple times. This is what is being tackled by the decorator, which makes sure the wrapped method gets called only once for any threads that would otherwise call it until one returns.

[Dobatymo]

Thanks for the reply! Methods called on different arguments are still executed in parallel?

[vit-zikmund]

Methods called on different arguments are still executed in parallel?

Sure thing ;)

[tkem]

@vit-zikmund: Finally found the time to give that a closer look, so AFAICS this is rather similar to https://github.com/tkem/cachetools/pull/224, i.e. there might be a rather large number of (R)Locks (or SingleCallContext in this case), which therefore become another ressource that needs to be managed. I rejected https://github.com/tkem/cachetools/pull/224 because of the added complexity, and - unless I did not fully understand your proposal, for which you did not provide much background information, I might say - would probably do so here. However, if you plan to provide your decorator as a seperate module or whatever, I'd be glad to link to that in the Related Projects section!