spark icon indicating copy to clipboard operation
spark copied to clipboard

[SPARK-48660][SQL] Fix explain result for CreateTableAsSelect

Open yuexing opened this issue 7 months ago • 3 comments

What changes were proposed in this pull request?

To fix the explain result of 'CreateTableAsSelect', according to how ExplainCommand processes, I find out the following work is needed

  • CreateDataSourceTableAsSelectCommand should implement stats
  • ExecutedCommandExec should also implement innerChildren for CreateDataSourceTableAsSelectCommand

Here's why:

  • As to 'CREATE TABLE ... AS SELECT'
  • CreateDataSourceTableAsSelectCommand will finally be the LogicPlan
  • ExecutedCommandExec(cmd=CreateDataSourceTableAsSelectCommand) will finally be the PhysicalPlan

Thus , to have the expected output

  • CreateDataSourceTableAsSelectCommand adds stats implementation
  • ExecutedCommandExec adds innerChildren for QueryExecution.stringWithStats

Why are the changes needed?

as reported in SPARK-48660, explain result of 'CreateTableAsSelect' is incorrect. It's missing stats for logic plan and optimization for physical plan.

Does this PR introduce any user-facing change?

Yes, it affects the explain result.

How was this patch tested?

UT.

Was this patch authored or co-authored using generative AI tooling?

No

yuexing avatar May 25 '25 15:05 yuexing

cc @wangyum FYI

LuciferYang avatar May 26 '25 02:05 LuciferYang

Could you fix the failed tests first? @yuexing

LuciferYang avatar May 26 '25 02:05 LuciferYang

The result seems correct. Do you have time to review this one? @wangyum

LuciferYang avatar May 28 '25 11:05 LuciferYang

friendly ping @wangyum

LuciferYang avatar Jun 17 '25 03:06 LuciferYang

We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!

github-actions[bot] avatar Sep 26 '25 00:09 github-actions[bot]