methylKit icon indicating copy to clipboard operation
methylKit copied to clipboard

switch to iotools for chunkwise apply

Open alexg9010 opened this issue 5 years ago • 3 comments

There is this nice package iotools (https://cran.r-project.org/web/packages/iotools/iotools.pdf) which allows fast chunkwise reading and writing from files. It supports connections, i.e. can read and write from /to gzipped files directly. It has some apply functionality build in which could become handy for our applyByTabix functions.

alexg9010 avatar Feb 05 '19 09:02 alexg9010

Iotools or fread then? How well is this maintained? Can we depend on them for the long-run?

from mobile

On Tue, Feb 5, 2019, 10:06 AM Alexander Gosdschan <[email protected] wrote:

There is this nice package iotools ( https://cran.r-project.org/web/packages/iotools/iotools.pdf) which allows fast chunkwise reading and writing from files. It supports connections, i.e. can read and write from /to gzipped files directly. It has some apply functionality build in which could become handy for our applyByTabix functions.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/al2na/methylKit/issues/143, or mute the thread https://github.com/notifications/unsubscribe-auth/AAm9EZKZh5cOSTLb2Az14roPG1z4BjLWks5vKUmJgaJpZM4aiyDD .

al2na avatar Feb 05 '19 09:02 al2na

Iotools or fread then?

I would use both, because fread determines column types automatically and still can read content very fast into memory.

How well is this maintained? Can we depend on them for the long-run?

It is there since R2.9 and is serving only the purpose of streaming files, so probably well tested for this. I does actually only have two depending packages, but itself has no other dependencies than R.

alexg9010 avatar Feb 05 '19 09:02 alexg9010

ok, sounds promising.

from mobile

On Tue, Feb 5, 2019, 10:30 AM Alexander Gosdschan <[email protected] wrote:

Iotools or fread then?

I would use both, because fread determines column types automatically and still can read content very fast into memory.

How well is this maintained? Can we depend on them for the long-run?

It is there since R2.9 and is serving only the purpose of streaming files, so probably well tested for this. I does actually only have two depending packages, but itself has no other dependencies than R.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/al2na/methylKit/issues/143#issuecomment-460569062, or mute the thread https://github.com/notifications/unsubscribe-auth/AAm9EeSZX94t5TAVbxH_li-LUMY7WW__ks5vKU8tgaJpZM4aiyDD .

al2na avatar Feb 05 '19 09:02 al2na