datafusion-ballista icon indicating copy to clipboard operation
datafusion-ballista copied to clipboard

Provide way to control collect_stat for the table provider

Open Dandandan opened this issue 3 years ago • 0 comments

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

Now that statistics are extended by @isidentical and others, we should be able to use statistics in Ballista as well. This has some nice benefits for performance: https://github.com/apache/arrow-datafusion/pull/3889#issuecomment-1283276230

It looks like currently the statistics are disabled in Ballista (as DataFusion was hardcoded to use collect_stat: false.

Describe the solution you'd like

https://github.com/apache/arrow-datafusion/pull/3846

Use the new setting in DataFusion.

Describe alternatives you've considered

Additional context

Dandandan avatar Oct 19 '22 16:10 Dandandan