fread memory_limit arg could rbind into single temp jay

[jangorecki]

trafficstars

memory_limit int ... In the end the returned Frame will be partially composed from data located on disk, and partially from the data in memory. It is advised to either store this data as a Jay file or filter and materialize the frame (if not the performance may be slow).

This FR asks for option for automatic storing data (AFAIU rbind) into single temporary jay file. Could be even the default IMO

[st-pasha]

But this is already possible: you can take the returned frame and save it into a jay file. The reason we are not doing it for you is that you may want to do some operations with the data before saving it.

[jangorecki]

I understand that, but it requires more branching in code. Depending if data is larger than memory on not. If having a feature requested here (to do that for user behind the scene) makes the code more agnostic of out-of-memory configuration. User just sets memory limit, and if it exceeds then all is handled behind the scene.