datafusion-ballista
datafusion-ballista copied to clipboard
Provide way to control collect_stat for the table provider
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Now that statistics are extended by @isidentical and others, we should be able to use statistics in Ballista as well. This has some nice benefits for performance: https://github.com/apache/arrow-datafusion/pull/3889#issuecomment-1283276230
It looks like currently the statistics are disabled in Ballista (as DataFusion was hardcoded to use collect_stat: false.
Describe the solution you'd like
https://github.com/apache/arrow-datafusion/pull/3846
Use the new setting in DataFusion.
Describe alternatives you've considered
Additional context