[FEATURE] upgrade Spark DataSource V2 APIs for unify streaming and batch
Thanks to @hanguyen6's proposal. in PR #42
Spark DataSource V2 APIs has been refactored to unify between streaming and batch. Should we upgrade our code to follow the updated abstraction:
https://issues.apache.org/jira/browse/SPARK-25390
write API refactor: https://docs.google.com/document/d/1vI26UEuDpVuOjWw4WPoH2T6y8WAekwtI7qoowhOFnI4/edit#
read API refactoring: https://docs.google.com/document/d/1uUmKCpWLdh9vHxP7AWJ9EgbwB_U6T3EJYNjhISGmiQg/edit#
I have updated code to follow that abstraction and can help if needed.
@hanguyen6 would you like to work on this issue?
@jiazhai Yes, I am working on it right now. Will update when it completes
@jiazhai, @jianyun8023 Could you pls review PR #50 ?