datafusion icon indicating copy to clipboard operation
datafusion copied to clipboard

Don't fetch and decode parquet metadata multiple times

Open tustvold opened this issue 2 years ago • 1 comments

Draft as I'm not sure this is a good idea

Which issue does this PR close?

Closes #.

Rationale for this change

https://github.com/apache/arrow-datafusion/discussions/7737#discussioncomment-7186343 shows parquet metadata decoding showing up as a bottleneck in query execution, this is a quick and dirty hack to potentially mitigate this somewhat

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

tustvold avatar Oct 04 '23 13:10 tustvold

Thank you for your contribution. Unfortunately, this pull request is stale because it has been open 60 days with no activity. Please remove the stale label or comment or this will be closed in 7 days.

github-actions[bot] avatar Apr 26 '24 01:04 github-actions[bot]