[KYLIN-4442 HiveProducer is not able to create partition in system cube

[isonusingh]

Proposed changes

HiveProducer class is not able to create a partition for system cubes in Hive tables. I found this bug while using Vanilla Hadoop 3.1 and below is the Jira ticket link : https://issues.apache.org/jira/browse/KYLIN-4442

Instead of using the below code, we can use my proposed change as its more effective for creating partitions in the Hive table.

            driver = new Driver(hiveConf);
            session = new CliSessionState(hiveConf);
            SessionState.start(session);
            CommandProcessorResponse res = driver.run(hql.toString());

I have tested my solution with Vanilla Hadoop3.1 on AWS & Azure.

Types of changes

What types of changes does your code introduce to Kylin? Put an x in the boxes that apply

• [x] Bugfix (non-breaking change which fixes an issue)
• [ ] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
• [ ] Documentation Update (if none of the other choices appl

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

• [x] I have create an issue on Kylin's jira, and have described the bug/feature there in detail
• [x] Commit messages in my PR start with the related jira ID, like "KYLIN-0000 Make Kylin project open-source"
• [ ] Compiling and unit tests pass locally with my changes
• [ ] I have added tests that prove my fix is effective or that my feature works
• [x] If this change need a document change, I will prepare another pr against the document branch
• [ ] Any dependent changes have been merged

Further comments

If this is a relatively large or complex change, kick off the discussion at user@kylin or dev@kylin by explaining why you chose the solution you did and what alternatives you considered, etc...

[codecov-io]

Codecov Report

Merging #1194 into master will decrease coverage by 0.02%. The diff coverage is 0.00%.

@@             Coverage Diff              @@
##             master    #1194      +/-   ##
============================================
- Coverage     24.87%   24.85%   -0.03%     
+ Complexity     6258     6256       -2     
============================================
  Files          1452     1452              
  Lines         89036    89030       -6     
  Branches      12437    12434       -3     
============================================
- Hits          22151    22126      -25     
- Misses        64709    64724      +15     
- Partials       2176     2180       +4     

Impacted Files	Coverage Δ	Complexity Δ
...ache/kylin/metrics/lib/impl/hive/HiveProducer.java	0.00% <0.00%> (ø)	0.00 <0.00> (ø)
...lin/stream/core/storage/columnar/FragmentData.java	75.67% <0.00%> (-13.52%)	11.00% <0.00%> (-1.00%)
...he/kylin/dict/lookup/cache/RocksDBLookupTable.java	72.97% <0.00%> (-5.41%)	6.00% <0.00%> (-1.00%)
...rg/apache/kylin/cube/inmemcubing/MemDiskStore.java	66.56% <0.00%> (-4.56%)	7.00% <0.00%> (ø%)
...ream/core/storage/columnar/ColumnarStoreCache.java	48.86% <0.00%> (-3.41%)	8.00% <0.00%> (ø%)

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update b53ba2d...8462f93. Read the comment docs.

[coveralls]

Pull Request Test Coverage Report for Build 5875

• 0 of 9 (0.0%) changed or added relevant lines in 1 file are covered.
• 24 unchanged lines in 5 files lost coverage.
• Overall coverage decreased (-0.02%) to 27.304%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
metrics-reporter-hive/src/main/java/org/apache/kylin/metrics/lib/impl/hive/HiveProducer.java	0	9	0.0%
<!--	Total:	0	9

Files with Coverage Reduction	New Missed Lines	%
core-dictionary/src/main/java/org/apache/kylin/dict/lookup/cache/RocksDBLookupTable.java	1	81.08%
metrics-reporter-hive/src/main/java/org/apache/kylin/metrics/lib/impl/hive/HiveProducer.java	2	0%
stream-core/src/main/java/org/apache/kylin/stream/core/storage/columnar/ColumnarStoreCache.java	5	54.55%
stream-core/src/main/java/org/apache/kylin/stream/core/storage/columnar/FragmentData.java	7	75.68%
core-cube/src/main/java/org/apache/kylin/cube/inmemcubing/MemDiskStore.java	9	76.29%
<!--	Total:	24

Totals
Change from base Build 5865:	-0.02%
Covered Lines:	24309
Relevant Lines:	89030

---

💛 - Coveralls

[isonusingh]

I was facing issue with below code on Vanilla Hadoop -

Driver driver = new Driver(hiveConf);
CliSessionState session = new CliSessionState(hiveConf);
SessionState.start(session); // control is coming to this line 
CommandProcessorResponse res = driver.run(hql.toString());// after this line of code control is not going to the next line.
if (res.getResponseCode() != 0) {
logger.warn("Fail to add partition. HQL: {}; Cause by: {}",
hql.toString(),
res.toString());
}
session.close();
driver.close();

The problem was above code not able to create the partition in hive tables for system cube. There was no error in logs but I was not able to see writing partition success in logs.

After Adding my code changes I am able to see the partition in Hive table for system cubes also able to see metrics write success.