google-cloud icon indicating copy to clipboard operation
google-cloud copied to clipboard

e2e BigQueryMultiTable Additional Test Scenarios

Open itsmekumari opened this issue 2 years ago • 7 comments

@itsankit-google @rmstar Please review. cc @sawantpritam

itsmekumari avatar Apr 05 '22 15:04 itsmekumari

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

For more information, open the CLA check for this pull request.

google-cla[bot] avatar Apr 05 '22 15:04 google-cla[bot]

Please fix conflicts

rmstar avatar Apr 08 '22 17:04 rmstar

Please fix conflicts

Conflicts resolved.

itsmekumari avatar Apr 12 '22 14:04 itsmekumari

Please resolve conflicts

itsankit-google avatar Apr 13 '22 12:04 itsankit-google

Please resolve conflicts

Conflicts resolved.

itsmekumari avatar Apr 14 '22 11:04 itsmekumari

We should also test Multi Database Table Source -> BQMT sink. That's the primary usecase for multi table plugins

Will add the case once automation of Multi Database table source plugin is done. Created a task for reference https://cdap.atlassian.net/browse/CDAP-19189

itsmekumari avatar May 05 '22 17:05 itsmekumari

Can we also add a test using multisource to multisink? And also test the functionality using runtime arguments - multisink.<table_name>=<schema> with Allow Flexible Schemas set to false

As previously commented, will add the case once automation of Multi Database table source plugin is done. Created a task for reference https://cdap.atlassian.net/browse/CDAP-19189. Also, for 2nd point is it multitablesource to BQMT sink with macros, Please elaborate point 2.

itsmekumari avatar Jul 28 '22 10:07 itsmekumari