fst icon indicating copy to clipboard operation
fst copied to clipboard

Write to fst binary file with an apply-like method

Open MarcusKlik opened this issue 8 years ago • 0 comments
trafficstars

This would reduce the memory footprint of writing to a fst file. It is also possible to use a parLapply approach, where data is generated in a parallel, but serialized to a fst file sequentially. In the latter scenario, we have to test the speed of data transfer from the nodes to the master, because it might be too slow for practical purposes (for example on a MPI cluster on Windows).

MarcusKlik avatar Feb 01 '17 20:02 MarcusKlik