datahub icon indicating copy to clipboard operation
datahub copied to clipboard

feat(mssql): add multi database ingest support

Open MugdhaHardikar-GSLab opened this issue 2 years ago • 3 comments

This PR enables user to ingest multiple databases from mssql using one recipe. User can filter databases to ingest using new config paramter 'database_pattern' Setup: set databse, database_alias paramters to None. Optionally add database_pattern to filter databases.

Checklist

  • [ ] The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • [ ] Links to related issues (if applicable)
  • [ ] Tests for the changes have been added/updated (if applicable)
  • [ ] Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • [ ] For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

MugdhaHardikar-GSLab avatar Jul 28 '22 10:07 MugdhaHardikar-GSLab

Unit Test Results (build & test)

502 tests  +3   502 :heavy_check_mark: +3   8m 22s :stopwatch: -20s 116 suites +1       0 :zzz: ±0  116 files   +1       0 :x: ±0 

Results for commit 6419d4d5. ± Comparison against base commit e9494f50.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Jul 28 '22 10:07 github-actions[bot]

Unit Test Results (metadata ingestion)

       8 files         8 suites   1h 14m 22s :stopwatch:    645 tests    643 :heavy_check_mark: 2 :zzz: 0 :x: 1 290 runs  1 286 :heavy_check_mark: 4 :zzz: 0 :x:

Results for commit 6419d4d5.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Jul 28 '22 17:07 github-actions[bot]

@MugdhaHardikar-GSLab - looks like the testIntegration failed to complete. Is that related to these changes?

hsheth2 avatar Aug 10 '22 06:08 hsheth2

@hsheth2 Fixed the ingetion action issue.

MugdhaHardikar-GSLab avatar Aug 12 '22 06:08 MugdhaHardikar-GSLab