amoro
amoro copied to clipboard
[spark2.3] support arctic table scan and mor
add spark conf
conf.set("arctic.catalog.url", "thrift://xxxx/catalog_name")
conf.set("arctic.resolve-all-identifier", "true" )
conf.set("spark.sql.extensions" , "com.netease.arctic.spark.ArcticSparkSessionExtensions")
note: new spark conf
- arctic.catalog.url : default arctic catalog url for lookup table meta.
- arctic.resolve-all-identifier : set to false, only resolve UnresolvedRelation to arctic Table. or resolve ALL LogicalRelation with tableIdentifier to arctic table
support merge on read for keyed table