shc
shc copied to clipboard
Added enhancement in the API to support for Google BigTable
Background -
Basically Google BigTable doesn't have namespaces & name descriptors Check this
Hence, during createRelation
task we have to skip calling getter/setter methods of namespaces & name descriptors viz. getNamespaceDescriptor()
and createNamespace()
There were 2 issues -
- I had created an issue
- Another similar one was created back in 2017.
What changes were proposed in this pull request?
- Create new class variable
tableType
inHBaseTableCatalog
tableType
variable by default is initialized to value "hbase" - Add getter and setter methods to overwrite
tableType
variable - Create if else branch in
createTableIfNotExist()
inHBaseRelation
class based ontableType
variable set in catalog, so as to skip calling namespace getter methods if API is used to perform write into Google BigTable - Illustrate the usage for writing into Google BigTable iin README.md
How was this patch tested?
- Unit test is added,
HBaseTableCatalogSuite.scala
- Manual testing is performed thoroughly and I'm using this in one of my project & running since 7 months in production, therefore I think now this is stable & right time to create pull request to merge into master branch
Regards, Vitthal