support for multiple JSON line-formats and switching between them

[tstack]

From a comment on this post -- https://news.ycombinator.com/item?id=40718862

I do wish there was support for switching formats so I could switch between different "views" over the same data, maybe it will be possible someday :)

[vagran]

My case is that I am unable to make my format to be selected. It is actually pino format. I tried to add my custom format, like this:

{
    "$schema": "https://lnav.org/schemas/format-v1.schema.json",
    "pino_log": {
        "title": "Pino log",
        "json": true,
        "line-format": [
            {
                "field": "time"
            },
            " ",
            {
                "field": "__level__",
                "text-transform": "uppercase",
                "auto-width": true
            },
            " ",
            {
                "field": "msg"
            }
        ],
        "level-field": "level",
        "level": {
            "fatal": 60,
            "error": 50,
            "warning": 40,
            "info": 30,
            "debug": 20,
            "trace": 10
        },
        "value": {
            "hostname": {
                "kind": "string",
                "identifier": true,
                "hidden": true
            },
            "time": {
                "kind": "integer",
                "identifier": false
            },
            "pid": {
                "kind": "integer",
                "identifier": false
            },
            "level": {
                "kind": "integer",
                "identifier": true,
                "foreign-key": true
            },
            "msg": {
                "kind": "string"
            }
        },
        "timestamp-field": "time",
        "body-field": "msg"
    }
}

but lnav keeps selecting bunyan_log instead. I do not need all fields from bunyan_log line format. But the worst thing is that time is interpreted incorrectly. Bunyay has string kind specified, I have specified integer, but is still selects bunyan. This, by the way, sounds like a bug. Also if it would select my format, how can I specify that this is UTC milliseconds? And I want to display them in UTC.

Here is log example:

{"level":30,"time":1719324213672,"pid":3191448,"hostname":"RaspberryNode","msg":"message here"}

[aisamu]

but lnav keeps selecting bunyan_log instead.

IIUC lnav picks the one with the largest amount of fields on the line-format (e.g. "more specific"?)

[vagran]

but lnav keeps selecting bunyan_log instead.

IIUC lnav picks the one with the largest amount of fields on the line-format (e.g. "more specific"?)

In my case, I need just a subset of bunyan format fields, so my format cannot be more specific. Having just a simple command line argument (or option in UI) to select a specific format would solve everything. It also looks very natural to have such option.

If someone is interested, I currently worked-around my problem by overriding bunyan_log format in local config. Initially, I was under the impression that it is impossible without recompiling lnav itself, since all built-in formats are compiled into the binary. But later I found this feature in the docs.

[hvrauhal]

Regardless of the comment above, it took me a while to figure out that the quality of a JSON format is determined based on the line-format rather than the value matches, but this is how it indeed is:

https://github.com/tstack/lnav/blob/3d8be486d4baf744ecee32a49868f2be5379a1e8/src/log_format.cc#L4109-L4116

https://github.com/tstack/lnav/blob/3d8be486d4baf744ecee32a49868f2be5379a1e8/src/log_format.cc#L853

https://github.com/tstack/lnav/blob/3d8be486d4baf744ecee32a49868f2be5379a1e8/src/log_format.cc#L1378

https://github.com/tstack/lnav/blob/3d8be486d4baf744ecee32a49868f2be5379a1e8/src/log_format.hh#L423-L425

https://github.com/tstack/lnav/blob/3d8be486d4baf744ecee32a49868f2be5379a1e8/src/logfile.cc#L447-L448