incubator-uniffle icon indicating copy to clipboard operation
incubator-uniffle copied to clipboard

Published 20 hours ago • verified publisher icon apache

[Improvement] There should be a bounded and blocked queue when sendShuffleData in client side

Open xianjingfeng opened this issue 2 years ago • 5 comments

In RssShuffleManager，the workQueue of threadPoolExecutoris unbounded now. If sendShuffleData not fast enough, it will cost a lot of memory

xianjingfeng avatar Jul 29 '22 13:07 xianjingfeng

Will it bring any other problems?

jerqi avatar Jul 29 '22 15:07 jerqi

Will it bring any other problems?

It may cause OOM

xianjingfeng avatar Jul 30 '22 04:07 xianjingfeng

We have a class WriteBufferManager. The class will control the memory that we can use. It is strange to cause OOM. I doubt whether it is caused by the issue that pr #67 want to fix.

jerqi avatar Jul 30 '22 04:07 jerqi

May be it is, i am not sure. I once found out an application failed due to requestExecutorMemory fail. Should we limit memory usage when sending data.

xianjingfeng avatar Jul 30 '22 08:07 xianjingfeng

May be it is, i am not sure. I once found out an application failed due to requestExecutorMemory fail. Should we limit memory usage when sending data.

I think we don't need, because we have WriteBufferManager. If our WriteBufferManager works well, OOM shouldn't occur. If our WriteBufferManager has bug, we should fix it.

jerqi avatar Jul 30 '22 08:07 jerqi