carbondata icon indicating copy to clipboard operation
carbondata copied to clipboard

Published 20 hours ago • verified publisher icon apache

[CARBONDATA-3232] Add test framework and test case for alluxio UT

Open xubo245 opened this issue 6 years ago • 102 comments

Add test case for alluxio UT

  1. install alluxio UT environment, start alluxio minicluster before running test case. This PR only add webapp into spark-common-test, it won't affect other module and non-test class
  2. add CarbonSession test case for UT => done
  3. add external table test case for UT=> done
  4. add spark carbon file format test case for UT=>done
  5. add SDK test case for UT=> done

Test case:

  • CarbonSession and external test:
  1. all data type and Create/Load/Select/Insert/Update/Delete.
  2. Create table and segment manage.
  3. alter operation.
  • SDK and spark carbon file format:
  1. write and read.
  2. select

TODO after this PR:

TODO: remove the webapp, but need Alluxio community support it in the future.

Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily:

  • [ ] Any interfaces changed? No
  • [ ] Any backward compatibility impacted? No
  • [ ] Document update required? No
  • [ ] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. yes, add many test case for alluxio
  • [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. No

xubo245 avatar Jan 11 '19 10:01 xubo245

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2270/

CarbonDataQA avatar Jan 11 '19 10:01 CarbonDataQA

Build Failed with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10528/

CarbonDataQA avatar Jan 11 '19 11:01 CarbonDataQA

Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2489/

CarbonDataQA avatar Jan 11 '19 11:01 CarbonDataQA

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2275/

CarbonDataQA avatar Jan 11 '19 12:01 CarbonDataQA

Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2495/

CarbonDataQA avatar Jan 11 '19 13:01 CarbonDataQA

Build Failed with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10534/

CarbonDataQA avatar Jan 11 '19 13:01 CarbonDataQA

@xubo245 What are the jsp files for?

xuchuanyin avatar Jan 13 '19 16:01 xuchuanyin

@xuchuanyin JSP is for alluxio mini cluster. Now Alluxio don't support mini cluster without webapp. Alluxio PMC said maybe support it in the future.

xubo245 avatar Jan 14 '19 09:01 xubo245

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2286/

CarbonDataQA avatar Jan 14 '19 09:01 CarbonDataQA

Build Failed with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10545/

CarbonDataQA avatar Jan 14 '19 11:01 CarbonDataQA

Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2506/

CarbonDataQA avatar Jan 14 '19 11:01 CarbonDataQA

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2289/

CarbonDataQA avatar Jan 14 '19 12:01 CarbonDataQA

Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2509/

CarbonDataQA avatar Jan 14 '19 13:01 CarbonDataQA

retest this please

xubo245 avatar Jan 14 '19 13:01 xubo245

Build Failed with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10548/

CarbonDataQA avatar Jan 14 '19 13:01 CarbonDataQA

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2292/

CarbonDataQA avatar Jan 14 '19 14:01 CarbonDataQA

Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2512/

CarbonDataQA avatar Jan 14 '19 14:01 CarbonDataQA

Build Failed with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10551/

CarbonDataQA avatar Jan 14 '19 15:01 CarbonDataQA

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2294/

CarbonDataQA avatar Jan 14 '19 16:01 CarbonDataQA

Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2514/

CarbonDataQA avatar Jan 14 '19 17:01 CarbonDataQA

Build Failed with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10553/

CarbonDataQA avatar Jan 14 '19 17:01 CarbonDataQA

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2295/

CarbonDataQA avatar Jan 15 '19 01:01 CarbonDataQA

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2296/

CarbonDataQA avatar Jan 15 '19 01:01 CarbonDataQA

Build Failed with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10555/

CarbonDataQA avatar Jan 15 '19 03:01 CarbonDataQA

Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2516/

CarbonDataQA avatar Jan 15 '19 03:01 CarbonDataQA

retest this please

xubo245 avatar Jan 15 '19 04:01 xubo245

Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2300/

CarbonDataQA avatar Jan 15 '19 05:01 CarbonDataQA

Build Failed with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10559/

CarbonDataQA avatar Jan 15 '19 06:01 CarbonDataQA

Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2520/

CarbonDataQA avatar Jan 15 '19 07:01 CarbonDataQA

retest this please

xubo245 avatar Jan 15 '19 09:01 xubo245