spark-csv
spark-csv copied to clipboard
databricks
Fixed width support.
Adds Relation, LineReader and BulkReader traits to avoid duplicated code. Largely derived from https://github.com/quartethealth/spark-csv and https://github.com/quartethealth/spark-fixedwidth.
This is in response to the following PR (created by @blrnw3) being closed without a merge:
https://github.com/databricks/spark-csv/pull/259
A fixed-width parser is a very common use case that I think several users would enjoy using. We plan to use this in our current production environment.
If I understood correctly, this library now avoids to add new features but only accepts critical bug fixes. How about proposing this into Spark?
I hadn't realized that. This is disappointing.
I'll see if I can get this proposed to the spark readers though.
Hi, Is fixed width csv still not supported in spark / databricks for csv parsing? thanks
@kurtmaile this repository is on maintenance mode and does not accept new features. All new feature development should be directed to Apache Spark CSV data source implementation.
Sure thanks.
So with regards to my general question on csv parsing in spark, do you know of a spark fixed width csv parser feature in the spark data soruce implementation?
Cheers
On 14 June 2018 at 02:27, Hossein Falaki [email protected] wrote:
@kurtmaile https://github.com/kurtmaile this repository is on maintenance mode and does not accept new features. All new feature development should be directed to Apache Spark CSV data source implementation.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/databricks/spark-csv/pull/391#issuecomment-397137855, or mute the thread https://github.com/notifications/unsubscribe-auth/ATIvhXJPPFY-BqfoNiqezUBMmzB9sZWdks5t8bv-gaJpZM4Ki7M4 .