spark-acid icon indicating copy to clipboard operation
spark-acid copied to clipboard

Published 20 hours ago • verified publisher icon qubole

Add support for Datasource V2

Open maheshk114 opened this issue 4 years ago • 1 comments

Add support for Datasource V2

  1. This will enable vectorized execution
  2. The feature is supported only for ORC as of now.
  3. Can be enabled using config spark.acid.use.datasource.v2=true. The default value is false.

pull request https://github.com/qubole/spark-acid/pull/85/files

maheshk114 avatar Jul 25 '20 18:07 maheshk114

@maheshk114 : Thanks for submitting the PR.

Did you do any performance comparison that you can share?

sourabh912 avatar Sep 17 '20 05:09 sourabh912