dolphinscheduler icon indicating copy to clipboard operation
dolphinscheduler copied to clipboard

[Feature-10273][Datasource] Add dataSource for AWS Athena

Open guodongym opened this issue 2 years ago • 6 comments

Purpose of the pull request

add dataSource for AWS Athena

close #10273

Brief change log

  • Add the ATHENA backend
  • Add the ATHENA test case
  • Add the ATHENA frontend

Verify this pull request

This pull request is code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

guodongym avatar Jul 26 '22 15:07 guodongym

Codecov Report

Merging #11157 (661209f) into dev (764a243) will decrease coverage by 1.81%. The diff coverage is 59.09%.

@@             Coverage Diff              @@
##                dev   #11157      +/-   ##
============================================
- Coverage     40.39%   38.58%   -1.82%     
+ Complexity     4877     4058     -819     
============================================
  Files           950     1001      +51     
  Lines         37186    36770     -416     
  Branches       4079     4273     +194     
============================================
- Hits          15021    14186     -835     
- Misses        20642    20960     +318     
- Partials       1523     1624     +101     
Impacted Files Coverage Δ
...ce/api/datasource/AbstractDataSourceProcessor.java 17.39% <0.00%> (ø)
...gin/datasource/athena/AthenaDataSourceChannel.java 0.00% <0.00%> (ø)
...asource/athena/AthenaDataSourceChannelFactory.java 0.00% <0.00%> (ø)
...ugin/datasource/athena/AthenaDataSourceClient.java 0.00% <0.00%> (ø)
...asource/athena/param/AthenaDataSourceParamDTO.java 0.00% <0.00%> (ø)
.../org/apache/dolphinscheduler/spi/enums/DbType.java 0.00% <0.00%> (ø)
...g/apache/dolphinscheduler/spi/utils/Constants.java 0.00% <ø> (ø)
...source/athena/param/AthenaDataSourceProcessor.java 70.00% <70.00%> (ø)
...datasource/athena/param/AthenaConnectionParam.java 80.00% <80.00%> (ø)
...ler/server/worker/metrics/WorkerServerMetrics.java 0.00% <0.00%> (-72.92%) :arrow_down:
... and 439 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

codecov-commenter avatar Jul 26 '22 17:07 codecov-commenter

Hi @guodongym , Thx for contributing this great feature! It seems there is no related docs for this PR, could you please add the docs? Thx again.

Yes I can

guodongym avatar Aug 03 '22 03:08 guodongym

Hi @guodongym , Thx for contributing this great feature! It seems there is no related docs for this PR, could you please add the docs? Thx again.

The document has been added

guodongym avatar Aug 07 '22 07:08 guodongym

SonarCloud Quality Gate failed.    Quality Gate failed

Bug C 21 Bugs Vulnerability E 9 Vulnerabilities Security Hotspot E 37 Security Hotspots Code Smell A 2101 Code Smells

37.9% 37.9% Coverage 2.4% 2.4% Duplication

I found the sonar result is wrong, seems checked all the code, how can we retry this check again?

caishunfeng avatar Aug 11 '22 03:08 caishunfeng

SonarCloud Quality Gate failed.    Quality Gate failed Bug C 21 Bugs Vulnerability E 9 Vulnerabilities Security Hotspot E 37 Security Hotspots Code Smell A 2101 Code Smells 37.9% 37.9% Coverage 2.4% 2.4% Duplication

I found the sonar result is wrong, seems checked all the code, how can we retry this check again?

Done

guodongym avatar Aug 12 '22 07:08 guodongym

@songjianet @Amy0104 @devosend do you have time to take a look the FE part?

zhongjiajie avatar Aug 28 '22 09:08 zhongjiajie

I add missing doc side bar in 8def3b2

zhongjiajie avatar Aug 29 '22 07:08 zhongjiajie

And will merge PR when it pass

zhongjiajie avatar Aug 29 '22 07:08 zhongjiajie

Restart the failed test

zhongjiajie avatar Aug 29 '22 15:08 zhongjiajie