empirical-lang icon indicating copy to clipboard operation
empirical-lang copied to clipboard

Empirical cannot handle CSV files with separators other than `,`

Open d53dave opened this issue 5 years ago • 1 comments

Hi,

empirical looks very cool!

Here's an issue I ran into, as I just wanted to process some of my bank statements which are generated using a German locale which means the CSV separator is a semicolon. Empirical doesn't seem to be able to handle other separators.

--- ~ » ./empirical
Empirical version 0.1.0
Copyright (C) 2019 Empirical Software Solutions, LLC

>>> let t = load$("transactions.csv")
[1]    13494 segmentation fault  ./empirical

It works as soon as I sed 's/;/,/g' transactions.csv > transactions_clean.csv it. I had a look at the source and it seems there is no way to tell load to use alternative separators.

EDIT: It can also not parse German formatted numbers as such, i.e.

>>> Float64("123.4")
123.4

>>> Float64("123,4")
nan

d53dave avatar May 22 '19 14:05 d53dave

Thanks for trying it out! You are correct that I've been neglectful of locale. And I don't have any parameters for load, like separators or headers.

I will have to add those things once I get default arguments in place.

chrisaycock avatar May 22 '19 14:05 chrisaycock