Cleanup support for Spark SQL interface.

[anuragkh]

The current Spark SQL interface still uses constructs from Spark 1.4.0. The implementation can benefit from the optimizations if the Data Source is brought up to date with Spark 2.0 constructs including Datasets.

Additionally, the current Spark SQL interface supports a subset of operators. We should add support for all Spark SQL operators.