hive HIVE-26986: Fix OperatorGraph when a query plan contains UnionOperator

What changes were proposed in this pull request?

I rewrite OperatorGraph to make sure that OperatorGraph represents actual Tez DAG. In new OperatorGraph, each cluster has a unique root operator and one Operator can belong to more than one cluster.

Why are the changes needed?

Tez DAG and current OperatorGraph group operators differently when a query plan contains union operators. More precisely, OperatorGraph groups all root operators of a union operator into the same cluster, but Tez DAG creates a BaseWork for each of the root operators and clones descendants of the union operator if needed. This difference makes false-positive errors when detecting parallel edges, which leads to insertion of unnecessary concentrator RS by ParallelEdgeFixer.

Does this PR introduce any user-facing change?

No

How was this patch tested?

I run the below maven tests and add a unit test for OperatorGraph.

mvn test -Dtest=TestSharedWorkOptimizer
cd itests; mvn test -Dtest=TestCliDriver

Jan 31 '23 02:01 ngsg

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
14 Code Smells

No Coverage information
No Duplication information

Feb 01 '23 10:02 sonarqubecloud[bot]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the [email protected] list if the patch is in need of reviews.

Apr 03 '23 00:04 github-actions[bot]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the [email protected] list if the patch is in need of reviews.

Jul 25 '23 00:07 github-actions[bot]

@ngsg, could you please reopen this PR, can't do it from UI

Dec 04 '23 17:12 deniskuzZ

@deniskuzZ , it seems that I don't have a permission to reopen this PR. I rebased and force-pushed my branch recently so there was a difference between the PR's head and that of my branch. I restore my branch and now they are pointing the same commit. Could you please check whether you can reopen this PR? If you can't, I'll create a new PR based on the latest commit on master branch.

Dec 05 '23 04:12 ngsg

@ngsg, could you please rebase

Dec 08 '23 09:12 deniskuzZ

@deniskuzZ , I rebased the branch.

Dec 08 '23 10:12 ngsg

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
22 Code Smells

No Coverage information
No Duplication information

warning The version of Java (11.0.8) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17. Read more here