carbondata icon indicating copy to clipboard operation
carbondata copied to clipboard

Published 20 hours ago • verified publisher icon apache

[WIP][CARBONDATA-3935]Support partition table transactional write in presto

Open akashrn5 opened this issue 4 years ago • 14 comments

Why is this PR needed?

Currently, we support only reading the tables created in spark in presto. Its a bottleneck and writing the trasactional is required in presto for easy write and read via presto.

What changes were proposed in this PR?

This PR iis on top of #3875 This PR supports writing the partition transactional data in presto, it supports multiple partition columns too.

Does this PR introduce any user interface change?

  • No

Is any new testcase added?

  • Yes

akashrn5 avatar Sep 08 '20 16:09 akashrn5

Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2270/

CarbonDataQA1 avatar Sep 08 '20 18:09 CarbonDataQA1

Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4010/

CarbonDataQA1 avatar Sep 08 '20 18:09 CarbonDataQA1

Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2404/

CarbonDataQA1 avatar Sep 21 '20 11:09 CarbonDataQA1

Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4145/

CarbonDataQA1 avatar Sep 21 '20 11:09 CarbonDataQA1

Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2454/

CarbonDataQA1 avatar Sep 23 '20 19:09 CarbonDataQA1

Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4197/

CarbonDataQA1 avatar Sep 23 '20 19:09 CarbonDataQA1

Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2564/

CarbonDataQA1 avatar Oct 06 '20 09:10 CarbonDataQA1

Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4314/

CarbonDataQA1 avatar Oct 06 '20 10:10 CarbonDataQA1

please do rebase

QiangCai avatar Oct 23 '20 02:10 QiangCai

@akashrn5 : please rebase as presto write PR is merged

ajantha-bhat avatar Oct 27 '20 05:10 ajantha-bhat

@ajantha-bhat @QiangCai i have rebased, please have a look.

akashrn5 avatar Oct 27 '20 08:10 akashrn5

@akashrn5 a) Please update the PR description about what all problems were there and what changes done to support it. Now it is not clear. b) Please confirm here whether the cluster test is passed.

ajantha-bhat avatar Oct 27 '20 09:10 ajantha-bhat

Build Failed with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/4700/

CarbonDataQA1 avatar Oct 27 '20 10:10 CarbonDataQA1

Build Failed with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2943/

CarbonDataQA1 avatar Oct 27 '20 10:10 CarbonDataQA1