lnav
lnav copied to clipboard
tstack
Custom Format not recognised consistently
lnav version v0.11.0
Describe the bug opening *.log file formatted to match a custom log format don't get recognised if regex group "body" contains some chars
To Reproduce
- Create a matching regex in regex101 https://regex101.com/r/qbAPgv/1
- generate a json format with command
lnav -m regex101 import <regex101-url> <format-name> [<regex-name>] - edit format based on references here: https://docs.lnav.org/en/latest/formats.html
- final *.json format file https://github.com/tstack/lnav/files/9582954/monolog.txt
- I attached 2 log files examples to test monolog-unrecognised.log
Basically, something in the regex group "body" cause monolog-unrecognised.log to fallback to generic_log
Both files contains the same log line, in the recognised one i cut the output after about 500 words
[UDAPTE]
if I try to expand the log line with CTRL+W on monolog-unrecognised.log, lnav crashes and attaching GNU gdb (GDB) this is the output:
Attaching to process 101998 Reading symbols from /usr/local/bin/lnav...Dwarf Error: wrong version in compilation unit header (is 0, should be 2, 3, or 4) [in module /usr/local/bin/lnav] (no debugging symbols found)...done. 0x0000000000e2da5c in sccp ()
crash-2022-09-16-12-25-53.126345.log
Would you be able to change this part of the regex:
>> (?<message>.+) >> (?<body>.*)$
Having more than one repetitive match of . is not going to work very well. If message cannot have a >, you can do something like [^>]+ to match everything but the greater-than symbol.
For unrecognized lines, you can add them as unit tests in regex101 and they will be added as samples in the lnav format file. lnav will check the samples against the regexes in the format and should give you an informative error message detailing where there's a mismatch.
As for the word-wrap crash, I can replicate it and will get it fixed, thanks for the report.
Thanks for the quick reply, I can confirm updating the regex solved the issue and right now lnav recognise the custom format correctly.
I also followed your suggestion about adding more unit tests and updating the regex accordingly making it more robust in matching lines.
About the word-wrap crash, looking forward for the next release (even a statically linked beta will do) as I need to use this on many different distros and I don't want to compile from source every time (sadly I have no idea how to create a statically linked binaries otherwise I'll do it myself)
by the way, keep up with your great work! I really enjoy this tool!
I can confirm updating the regex solved the issue
Out of curiosity what did you change? Did you find out what part of log message caused the original regexp to not match?
looking forward for the next release (even a statically linked beta will do)
Can you give this a try:
https://github.com/tstack/lnav/releases/tag/v0.11.1-beta1
I can confirm updating the regex solved the issue
Out of curiosity what did you change? Did you find out what part of log message caused the original regexp to not match?
I changed the regex from:
^\[(?<timestamp>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\] (?<level>\w+) >> (?<component>[\w\\]+) >> (?<message>.+) >> (?<body>.*)$
to
^\[(?<timestamp>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\] (?<level>\w+) >> (?<component>[\w\\]+) >> (?<message>[^>]+) >> (?<body>.*)$
basically the "message" group has a different regex than "body"
I was not able to pinpoint the exact char sequence that caused the original regex to not match
looking forward for the next release (even a statically linked beta will do)
Can you give this a try:
https://github.com/tstack/lnav/releases/tag/v0.11.1-beta1
I'll try ASAP and give you feedback, thanks
I tried the beta release and I encountered the bug just once but I'm not able to replicate it. It seems to work way better now. Thanks