incubator-uniffle icon indicating copy to clipboard operation
incubator-uniffle copied to clipboard

Published 20 hours ago • verified publisher icon apache

Negative value of preAllocatedSize

Open zuston opened this issue 3 years ago • 4 comments

I found sometimes the value of preAllocatedSize is negative in the shuffle server log, which makes me confused. Maybe this problem is caused by race condition.

reliao_img_1663580756793

zuston avatar Sep 19 '22 09:09 zuston

It's abnormal. It should be caused by some bugs.

jerqi avatar Sep 20 '22 06:09 jerqi

We have found this problem before, and it has not occurred since #159

xianjingfeng avatar Sep 21 '22 09:09 xianjingfeng

We have found this problem before, and it has not occurred since #159

#159 has been applied in our internal uniffle cluster, but it still occurred.

zuston avatar Sep 21 '22 09:09 zuston

If the requireBufferId just expires when writing data, it may occurred. Maybe we need to introduce locks when memory be released.

xianjingfeng avatar Sep 21 '22 10:09 xianjingfeng