xianjingfeng
xianjingfeng
> If the range of [low-watermark, high-watermark] is large, this problem with this patch should still exist, right? You are right, but if the range of [low-watermark, high-watermark] is not...
> > My goal is to make full use of the memory, so i hope that it only writes 40g to the disk when the memory reaches 340g. But actually...
> @xianjingfeng , when this is ready for review, you can ping me or @zuston. It's ready for review. @advancedxy @jerqi @zuston
> Do you test this PR and check it works as expected. I have only tested it with UT.
> LGTM. And it would be great if we could test this PR has been worked as expected. @advancedxy
Merged. Thanks for your review. @zuston @advancedxy
> BTW, I'm curious how much can the performance be improved? It is difficult to estimate. In theory, this pr will reduce the amount of data written to disk. >...
> > This pr is for reducing the amount of data written to disk. > > When the server's memory is insufficient, it always has to flush to the disk....
> I got your point, this PR will benefit in your cases. We don't know if it will still benefit in other cases, e.g. the shuffle size way larger than...
@summaryzb @zuston @jerqi