Spark-MongoDB icon indicating copy to clipboard operation
Spark-MongoDB copied to clipboard

Cannot parse splitSize properly

Open wuciawe opened this issue 9 years ago • 0 comments

If I use the syntax provided by Spark-SQL as:

val parameters = Map(
  "host" -> "example.com",
  "database" -> "example",
  "collection" -> "test",
  "schema_samplingRatio" -> "0.0001",
  "splitSize" -> "20"
)

then, the splitSize can not be parsed properly in the file com.qiyi.mongo.datasource.mongodb.partitioner.MongodbPartitioner.

It is caused by partially parsing parameter when initializing DataSouce.

I work out a patch for this issue, will pull it right now.

wuciawe avatar Apr 28 '16 12:04 wuciawe