add support for `format JSONEachRow` with Logs Panel

[mxlxm]

Is your feature request related to a problem? Please describe. display the whole logs within logs panel without complex query

Describe the solution you'd like add support for format JSONEachRow, display the whole log and allow user to specify the time column. query would by more simple select * from xxx format JSONEachRow

Describe alternatives you've considered complex query like select time,toJSONString(tuple(x,y,z)) from xxx, while in this way, column name is absent.

[gingerwizard]

We will support JSON type properly and avoid JSONEachRow - the driver uses native format. See https://github.com/grafana/clickhouse-datasource/issues/129

[mxlxm]

No, it's not JSON type, consider that we've created a table for nginx access_log, each field with a separate column. e.g.

create table nginx_access (
    `time` DateTime,
    `host` LowCardinality(String),
    `request_uri` String,
    ...
)

with FOTMAT JSONEachRow, we can see a complete log(with field name).

[gingerwizard]

Format controls the response format from ClickHouse. The driver uses the native format for performance reasons - Grafana uses this for communication. Introducing support in the driver for JSONEachRow just for a logs panel is going to be hard to justify - not to mention we'd need to detect the format usage and switch the encoder.

If you used the JSON type for all the fields except time, then it will just be rendered as a JSON line shortly (next release probably). Tuples will also be rendered as JSON in the next release btw.

[mxlxm]

JSON type is pretty good for semi-structured data, but with nginx access_log we definitely know how the log looks like.

On the other hand, it should give a significant increase in performance (especially in the right indexing) with separate column, moreover we can use LowCardinality and codec to reduce disk space.

I didn't try with JSON type, but format limited rows vs store whole log with JSON type, I think I would prefer the former

[gingerwizard]

Yes id imagine configuring some columns explicitly, to exploit codes etc, with a column for any remaining dynamic JSON. Either way in the next release of the plugin, tuples, maps and nested (and thus JSON) will be displayed as JSON.

I don't think this will be done in the driver however - I'll raise a ticket to support JSONEachRow in the driver but i don't see this happening soon given the likely minimal use outside of this Grafana usecase. We could also consider a mode in the plugin which does this transformation if it gets enough +1.

[TimSimmons]

Something fairly similar here with respect to the log view, some way to view just any arbitrary selected row in that format would be wonderful. It's a very confusing experience for folks if you have "logs" in a table.

You might have a log (like an nginx log) in a table with an actual schema, but you can only view columns that are string types at all, and even then only one shows up until you click a log and hit the 👁️ next to the field. Some way to be able to show all the fields in that view besides the Grafana table view would be wonderful. It's possible of course to toString each column, but if you wanted to dynamically select those columns somehow (like with a Grafana variable), you're out of luck.