spark-csv
spark-csv copied to clipboard
Use the same library when Reading and Writing CSV
- Added Escape Character
- Use CSVPrinter when writing CSV Files
@abendada Would you add tests for this and make it pass?
I added a test for escape The test consist of:
- reading a csv file with "" as escape character
- check the parsed content
- save the dataFrame as a temp file without providing an escape character the default (") is used if necessary
- check that temp file is well formed
- read the temp file and check that the content well parsed
Note: quote and escape should have Character as type so nullable.
@abendada left a few comments. Would you please also update the README for the new escape character option, and rebase this? Thanks!