apache
[GLUTEN-7532][CORE] Rewrite TakeOrderedAndProject as actually executed operators when do offload
What changes were proposed in this pull request?
Rewrite the TakeOrderedAndProjectExecTransformer to the operators which are actually executed when do offload. Currently, the logic is wrapped in the RDD of the operator. Which means that in the RDD computation we play with the sql operators to generate new RDDs which is not a common practice processing sql queries. And the final executed plan and metrics information won't be shown correctly in the Spark UI page.
In this PR, rewrite the node TakeOrderedAndProjectExecTransformer when do offload so that the operators can be shown correctly.
How was this patch tested?
UT
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
+1 to the idea. I remember there was vanilla Spark limitation as blocker against a similar effort. But let's see if it's doable at this time.
This PR is stale because it has been open 45 days with no activity. Remove stale label or comment or this will be closed in 10 days.
This PR was auto-closed because it has been stalled for 10 days with no activity. Please feel free to reopen if it is still valid. Thanks.