fhir-data-pipes icon indicating copy to clipboard operation
fhir-data-pipes copied to clipboard

add ability to switch off/on creation of parquet dwh

Open mozzy11 opened this issue 1 year ago • 4 comments
trafficstars

Fixes https://github.com/google/fhir-data-pipes/issues/1073

Add ability to switch of creation of a parquet DWH in case of syncying betwen FHIR servers

Added a flag to the createParquetDwh to the controller to switch off/on creation of parquet DWH

E2E test

Adedd e2e tests for synching from a hapi fhir sever to another using the pipeline controller for both FULL and INCREMENTAL modes while swtching on/off creation of parquet DWH

TESTED: Testes Locally syncying between FHIR server while the parquet DWH is switched off/on

Checklist: I completed these to help reviewers :)

  • [x] I have read and will follow the review process.

  • [x] I am familiar with Google Style Guides for the language I have coded in.

    No? Please take some time and review Java and Python style guides.

  • [x] My IDE is configured to follow the Google code styles.

    No? Unsure? -> configure your IDE.

  • [x] I have added tests to cover my changes. (If you refactored existing code that was well tested you do not have to add tests)

  • [x] I ran mvn clean package right before creating this pull request and added all formatting changes to my commit.

  • [x] All new and existing tests passed.

  • [x] My pull request is based on the latest changes of the master branch.

    No? Unsure? -> execute command git pull --rebase upstream master

mozzy11 avatar May 29 '24 10:05 mozzy11

Codecov Report

Attention: Patch coverage is 33.33333% with 6 lines in your changes missing coverage. Please review.

Project coverage is 51.41%. Comparing base (eb96271) to head (e9933e4).

Files with missing lines Patch % Lines
...java/com/google/fhir/analytics/DataProperties.java 33.33% 1 Missing and 3 partials :warning:
...a/com/google/fhir/analytics/ConvertResourceFn.java 0.00% 0 Missing and 1 partial :warning:
...a/com/google/fhir/analytics/FetchSearchPageFn.java 50.00% 0 Missing and 1 partial :warning:
Additional details and impacted files
@@             Coverage Diff              @@
##             master    #1074      +/-   ##
============================================
- Coverage     51.48%   51.41%   -0.07%     
+ Complexity      716      715       -1     
============================================
  Files            95       95              
  Lines          5909     5916       +7     
  Branches        787      790       +3     
============================================
  Hits           3042     3042              
- Misses         2578     2582       +4     
- Partials        289      292       +3     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov-commenter avatar May 29 '24 10:05 codecov-commenter

cc @bashir2

mozzy11 avatar May 30 '24 00:05 mozzy11

cc @bashir2

mozzy11 avatar Jul 08 '24 18:07 mozzy11

cc @bashir2

Sorry @mozzy11 this fell off my radar (perhaps due to DevDays); thanks for the reminder. I'll provide some feedback by tomorrow.

bashir2 avatar Jul 09 '24 16:07 bashir2

cc @bashir2 . i have worked on the review comments

mozzy11 avatar Jan 06 '25 10:01 mozzy11

Hello @bashir2 . I addresed the comments above

mozzy11 avatar Jan 07 '25 08:01 mozzy11

Thanks @bashir2 , ill attend the dev call this week

mozzy11 avatar Jan 13 '25 19:01 mozzy11

Hello @bashir2 . sorry i missed the dev call. Let me know if you have any other time to have a quick call

mozzy11 avatar Jan 16 '25 14:01 mozzy11

Thanks @bashir2

mozzy11 avatar Jan 17 '25 06:01 mozzy11