atom-tablr
atom-tablr copied to clipboard
abe33
Crash on preview of large files
Unfortunately I can't provide a sample CSV (the ones I'm working with are under lock and key and I don't have time to prepare new ones), but I have some CSVs that are inordinately large (2.5mb) that have a pretty hefty number of columns as well as just sheer length.
tablr crashes atom while generating a preview. I don't even have the chance to open it as a regular text file =/
After a short wait, a csv of about half size (1.25mb) generates the preview alright. Obviously changing options causes another wait of about 30seconds.
Otherwise I love this package. Looking forward to updates.
I have to second that. I have a 32MB csv file that has a delimiter |, and I can't even go past the selection of my custom delimiter. Once select the field, one CPU is at 100%, and after a while it just crashes.
Alas, I cannot provide the csv file.
This might be a current limitation of Atom.io as atom is slow as heck on large files. It would be great if they could fix this issue. It should be possible as vscode (MSFT) does not seem to have large file problems.
I started working on it, I plan to using stream for both the preview and the opening of the file (so that I can add a progress indicator on opening) and limiting the preview size to a value specified through a setting. So far it looks promising, I could open a 8MB file without having the preview making trouble, so I think the biggest problem was the generation of the preview table in @sylistine case, I'm still looking for a source for huge csv of several dozen MB to assert whether there's a memory issue with huge table once in the editor, but for the 8MB file even the editor was still smooth enough.
FYI version 1.2.0 was 🚢 with the new stream-based CSV open, there's still probably some bottleneck here and there (like the save routine that is still synchronous) but I think that for the most part it's now usable with big files. Let me know if you still get crashes or big hangup when manipulating such files.
I'm getting hangups working with files about 5 MB. tablr preview is fine, but after the two progress bars Atom freezes. In addition it seems that tablr skips the "Ask before opening large files" prompt if I chose "Open Table Editor".
Could the preview show the size of the file (MB and rows?) Given Atom's difficulties with large files I just wouldn't open large CSVs.
