spark-csv icon indicating copy to clipboard operation
spark-csv copied to clipboard

Published 20 hours ago • verified publisher icon databricks

Upgraded to spark 2.0

Open davidcrossland opened this issue 8 years ago • 3 comments

davidcrossland avatar Oct 25 '16 10:10 davidcrossland

@davidcrossland this package is not supposed to work with Spark 2.0. spark-csv has been inlined in Apache Spark 2.0

falaki avatar Oct 25 '16 16:10 falaki

This related to a fork of spark-csv which was used by https://github.com/quartethealth/spark-fixedwidth, i needed to make use of this project for a customer. Unluckily it didnt work with spark 2.0, so i was forced to upgrade it. This was only intended to be a pull request for that specific repo, certainly not for the main codebase. Hopefully once Spark 2.0 is released i will be able to update that project, you could take a look at integrating fixed width files in spark-csv and save me a job ;)

davidcrossland avatar Oct 26 '16 08:10 davidcrossland

I see. Spark 2.0 was released a few months ago. Feel free to close this PR.

falaki avatar Oct 26 '16 18:10 falaki