snaketail-net icon indicating copy to clipboard operation
snaketail-net copied to clipboard

Not usable with very large logfiles (sized several GB)

Open GoogleCodeExporter opened this issue 10 years ago • 4 comments

SnakeTail is (practically) unable to process very large log files. I cancelled 
trying to open a 5GB log file after 5 minutes (SnakeTail indicated "loading"; 
high CPU + Kernel times load).

Looking at the memory consumption, it does not look like the full file is 
loaded into memory, so I guess it is just read through to find the tail of the 
file (maybe also to count the number of lines in order to display a proper 
scrollbar?).

It would be much faster if the file position would be set to end of file and 
then the last n characters read. This would really enable using this tool with 
the large log files I keep encountering.

Original issue reported on code.google.com by [email protected] on 20 Dec 2013 at 12:20

GoogleCodeExporter avatar Mar 17 '15 21:03 GoogleCodeExporter

An Enhancement, really? This app is meant (judging by the name) to give the kind of tailing performance found in typical unix systems. Given that the default behaviour of SnakeTail is to display the bottom (the "tail") for the file (exactly as I'd expect), what is the point of buffering/scanning the entire file (or whatever it is you are doing) on opening it? Might as well use a text editor. IMHO This is a bug.

brendangawn avatar Dec 08 '16 17:12 brendangawn

@brendangawn You are always welcome to come with a Github PullRequest (PR) that fixes the "bug".

snakefoot avatar Dec 08 '16 18:12 snakefoot

fair comment :) some day in my spare time perhaps

brendangawn avatar Dec 08 '16 20:12 brendangawn

@brendangawn When I created this tool, then it was both for tailing the log file, but also to search/scroll the entire log file (not just the end of the file). Tools like Notepad, Notepad++ or LogExpert performed very poorly when the size grew beyond 50 MByte. SnakeTail can handle around 200 MByte before getting tired, which covered my needs. Applying a virtual scroll-window of the raw-text-file, that used file seek would be much faster, but then you have funny issues with current selection outside the scroll-window, and handling scrollbars to show current position as the file grows. Maybe checkout the tool Tail Blazer it seems to have cracked the nut.

snakefoot avatar Dec 08 '16 20:12 snakefoot