MonetDBLite-R Support for compressed CSV files

Most CSV readers (notably, base R readers) support compressed (gz, bzip2...) CSV files transparently. It would be a nice addition to MonetDBLite::monetdb.read.csv, because big CSVs are commonly gzip'ed.

Jun 06 '18 13:06 Enchufa2

Yes we had removed this feature because it adds a dependency, which is a pain especially on Windows.

Jun 06 '18 14:06 hannes

And what about allowing an external command to be specified? May this be possible? Example: data.table::fread. You can read a compressed CSV as follows:

data.table::fread("zcat somefile.csv.gz")

zcat is invoked and its output feeds the reader.

Jun 06 '18 14:06 Enchufa2

That sounds pretty good. I am unlikely to implement this at the moment. Happy to review a PR though.

Jun 07 '18 06:06 hannes

And another quick and portable option would be to rely on the R.utils package, which implements gunzip and the like based on base R (efficiently copying from a gzfile, bzfile... connection to a file connection). But this would mean adding R.utils as a dependency (or porting to MonetDBLite just the relevant code). What do you think?

Jun 07 '18 11:06 Enchufa2