[WIP] parallel processing for column-based TRowSet generation

[bowenliang123]

:mag: Description

Issue References 🔗

This pull request fixes #

Describe Your Solution 🔧

• Improvement comparison:

• Support parallel processing for column-based TRowSet generation, within a fork-join pool on the engine side

• The order of columns in TRowSet is still guaranteed by sorting the column index, which is a very light cost operation

• Add a config to enable/disable this feature

Types of changes :bookmark:

• [ ] Bugfix (non-breaking change which fixes an issue)
• [ ] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to change)

Test Plan 🧪

Behavior Without This Pull Request :coffin:

Behavior With This Pull Request :tada:

Related Unit Tests

---

Checklist 📝

• [ ] This patch was not authored or co-authored using Generative Tooling

Be nice. Be informative.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 0% with 33 lines in your changes missing coverage. Please review.

Project coverage is 0.00%. Comparing base (96c5632) to head (74106c7). Report is 92 commits behind head on master.

Files with missing lines	Patch %	Lines
...apache/kyuubi/engine/result/TRowSetGenerator.scala	0.00%	14 Missing :warning:
...ain/scala/org/apache/kyuubi/util/ThreadUtils.scala	0.00%	9 Missing :warning:
...in/scala/org/apache/kyuubi/config/KyuubiConf.scala	0.00%	5 Missing :warning:
...kyuubi/engine/spark/operation/SparkOperation.scala	0.00%	4 Missing :warning:
...gine/spark/schema/SparkArrowTRowSetGenerator.scala	0.00%	1 Missing :warning:

Additional details and impacted files

@@          Coverage Diff           @@
##           master   #6667   +/-   ##
======================================
  Coverage    0.00%   0.00%           
======================================
  Files         683     683           
  Lines       42213   42236   +23     
  Branches     5755    5756    +1     
======================================
- Misses      42213   42236   +23     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[github-actions[bot]]

Thanks for the PR! This PR is being closed due to inactivity. This isn't a judgement on the merit of the PR in any way. If this is still an issue with the latest version of Kyuubi, please reopen it and ask a committer to remove the Stale tag!

Thank you for using Kyuubi!