matrixone icon indicating copy to clipboard operation
matrixone copied to clipboard

[Enhancement]: S3 read ordering

Open fengttt opened this issue 1 year ago • 1 comments

Is there an existing issue for enhancement?

  • [x] I have checked the existing issues.

What would you like to be added ?

Good read ordering may improve performance significantly once we have runtime filter.

Why is this needed ?

Let's use an example, select * from T order by x limit 10;

If we look at zonemap, read S3 objects according to order min of x -- only a few S3 blocks with probably raise the threshold of x and we can filter most of later S3 objects at run time.

Additional information

No response

fengttt avatar Aug 27 '23 09:08 fengttt