fst
fst copied to clipboard
fstpackage
Lightning Fast Serialization of Data Frames for R
When a sorted data set is stored as a `fst` binary file, sorting metadata is stored alongside the data. Using this metadata, a binary search can be performed on the...
That would involve creating a `fst` file-connection object (similar to base-R `file` method). With that object data can be streamed row-by-row until the file is depleted (or the connection is...
Just an idea, probably not really very important... It would be nice if there were an option to output lz4-compressed data in a form compatibile with the lz4 command line...
The scenario is as follows: A number of large fst files are written on disk. They have identical structures but reading all of them will occupy a major proportion of...
`R` 3.5.0 brings some features from the `ALTREP` framework. One of those features is that the actual vector data can be stored in an alternative structure or location. Such a...
Hello, I have a large number of rds files (each file>100MB) that I'm trying to work with. I just got familiar with your fst package which looks promising for my...
With this feature you can populate say row 1001:2000 in a 1e6 row `data.table` with a 1000 row read from `fst.read`. All this is done in memory. This feature is...
And provide fast compression with random access to the matix. Check if there is a use-case for such a feature.
See [here](http://www.boost.org/doc/libs/1_55_0/doc/html/interprocess/sharedmemorybetweenprocesses.html), boost allows for the creation of memory shared between processes. For `fst` that could mean that a single in-memory `fst` table can be shared between different processes. First...
Using code from the `microbenchmark` package directly from C++. See for example [this code](https://github.com/cran/microbenchmark/blob/master/src/nanotimer.c) for cross-platform timers.
Metadata
Owner
Metadata
Lightning Fast Serialization of Data Frames for R