klogg icon indicating copy to clipboard operation
klogg copied to clipboard

Partial open of log files

Open variar opened this issue 6 years ago • 3 comments

Indexing log file of 280Gb took about 4Gb memory and then line counter overflowed.

We can use 64-bit counter for lines but searching in such big file would be very slow. For very large log files it makes sense to open only part of the file.

variar avatar Aug 08 '19 14:08 variar

Can I add to this?

This would also work for a "soft" file clear - i.e. without deleting the file content, just hide away the file content so far, to get a clear view pane

This is very useful when hunting bugs during debug sessions, to have a "clean" log to work from, but without actually deleting the log file so far (thus losing some information).

dia-susca avatar Aug 09 '19 07:08 dia-susca

@dia-susca your use case to some extent can be done using existing search limit functionality

variar avatar Aug 13 '19 22:08 variar

Thank you for looking into this. I don't think my use-case is unique, or special: basically, when there are errors during an application deployment, there can be ~2500 lines of logging during startup, some of which are exceptions. When there are up to 10-20 deployments in the same log, it's difficult to localize which errors belong to the last deployment. So, to be able to easily find the problems, one needs to clear the log after each deployment. I see two possibilities:

  1. deleting the entire file content after each deployment - can be done by navigating menus and clicking on the confirmation dialog - at the cost of losing information
  2. having the possibility to "clean"/"clear" the file view - sort of an extension of opening only part of the file - just that the part that remains open is from the moment it was marked for "clean", onwards.

Anyway, if this is possible, it would improve the lives of many archaeo-log-ists :). Well, at least one person's life.

dia-susca avatar Aug 14 '19 12:08 dia-susca