Julian Samaroo
Julian Samaroo
Doing just `TextParse.csvread("ok.csv", ','; header_exists=true, colparsers=fmtypes)` loads the file successfully in ~5 seconds (including inference and compilation time, which is quite good). So clearly this is a JuliaDB issue. Thanks...
@mahiki JuliaDB is in maintenance mode. Unless you plan to implement this work yourself, it is likely to never get done. If you need a more actively-developed distributed table implementation,...
> Was it lack of popularity/usage, or was is the design strategy that let to a dead end? A lack of new clients and funding to develop it, is my...
LGTM! I'd like another maintainer to confirm that the Julia 1.6 compat bound looks good before merging.
Per the hackathon discussion, step 1 is already handled; step 2 would probably be a good idea too. 3 and 4 to follow. Additionally, for step 4, we should probably...
I almost forgot, I still need to actually implement incremental saving of read blocks to the output file when specified, otherwise we'll still read the whole CSV's data into memory...
Quick update for onlookers: the latest commit attempts to split individual files into blocks before calling `_loadtable_serial` so that each block can be saved to disk (and thus removed from...
@tanmaykm @shashi done and ready for review!
Bump, anyone up for reviewing this?
Looks like some change in TextParse 1.0 is breaking the ability to pass `nrows=1` during header parsing (since this passes locally with a pre-1.0 TextParse). EDIT: `nrows` was renamed, to...