logdyhq
Better view for JSON fields
Hello! Some of my columns are JSON. Example from the demo:
How do I show that as a "inline" table or something instead of a raw string?
If your log lines are already JSON, you can use "autogenerate" from the settings to split the data into separate columns. Otherwise you have to assemble the columns yourself as noted in the docs: https://logdy.dev/docs/explanation/columns
https://github.com/user-attachments/assets/08492ec5-3f89-42d7-8bd9-6df6f1316407
I've been using that, but I have some "polymophic" fields with different info depending on the message. To give some background, I'm writing little CLI tools with structured logging - https://github.com/bbkane/grabbit and https://github.com/bbkane/toddlerevents.
The grabbit logs currently look something like:
{"msg": "can't download image", "url": "https://image.com/image", "filepath": "/where/to/download/image", "err": "the wires are cut"}
I'd like to make this format more "viewable" by putting all fields unique to the message in an "extra" column.
However, I have multiple log statements and they each have unique key/value pairs like "url"/"filepath"/"err" above. So if I turn each key into a column, I get a really sparse wide view of the logs which isn't super helpful.
Instead, I was thinking of something like making this format more "viewable" by putting all fields unique to the message in an "extra" column:
{"msg": "can't download image", "extra": { "url": "https://image.com/image", "filepath": "/where/to/download/image", "err": "the wires are cut"}}
That's a bit better, but now all the fields are in this column, and this one colume is hard to read. Hence my ask if there's a way to more easily view JSON inside a single column.
Do you have any recommendations for me?
P.S. I'm having a lot of fun using logdy - it's much "slimmer" than competing tools. Thanks for writing/open-sourcing it! I don't have any budget for expensive hosted solutions for my side-project OSS CLIs.
So currently you can control the style of row or cell with code (issue: https://github.com/logdyhq/logdy-core/issues/76). I assume drawer is not helpful because you want to see a value for each row at a glance, right? Currently I can think of some kind of special format like trying to write a handler for a special column which would go like this:
(line: Message): CellHandler => {
return { text: `${line.json_content.foo} ${line.json_content.bar}` /*more fields can go here*/ }
}
Bascially flattens multiple values into a string.
Next thing I can think of is allowing to provide an HTML code within text's value. This way you would gain full access to format the value inside however you like and pair it with custom styles. Let me know what you think
I like the custom HTML approach (especially if I can render an inline table). I need to be able to switch on the message value to decide how to render the extras value.
I imagine most of what I want would be a key-value table, but I can also see that getting large enough that it would greatly reduce the amount of logs viewable. I could handle that just by manually limiting the number keys/values I suppose.
I've been tinkering with the problem, this is where I landed. Basically by default all html tags are sanitized, but once you add allowHtmlInText: true to a cell handler then the sanitization will be off and raw HTML will be displayed.
This is what I've got, with the code below. I think the results are nice and the the table even inherited the styles nicely
(line: Message): CellHandler => {
return {
text: `<table>
<tr>
<th>Company</th>
<th>Contact</th>
<th>Country</th>
</tr>
<tr>
<td>Alfreds Futterkiste</td>
<td>Maria Anders</td>
<td>Germany</td>
</tr>
</table>`,
allowHtmlInText: true
}
}
Wdyt?
It will render raw HTML bypassing any kind of sanitization so links, images and even custom CSS will work (although at your own risk)
@bbkane I'm happy to share that this feature is live with the latest release: https://logdy.dev/blog/post/logdy-new-version-announcement-v016
If you have any feedback for Logdy overall or for this feature feel free to deliver it!
I think also it'd be useful to generate columns automatically with jq-style headers; e.g. "extra.url", "extra.filepath", "extra.err". The application I am collecting logs from will conditionally output this extra information, and ideally I'd want to be able to filter and sort on those columns same as any other. It'd be nice to be able to click on the column for the nested object in JSON and "explode" those to normal columns, and then maybe I can hide some I don't care about and sort and filter on the others.
I think also it'd be useful to generate columns automatically with jq-style headers; e.g. "extra.url", "extra.filepath", "extra.err". The application I am collecting logs from will conditionally output this extra information, and ideally I'd want to be able to filter and sort on those columns same as any other. It'd be nice to be able to click on the column for the nested object in JSON and "explode" those to normal columns, and then maybe I can hide some I don't care about and sort and filter on the others.
@stellarpower so Logdy would have to index all fields and provide you a "schema" you can choose from to display the columns currently?
I guess so; the extended information is what is most important in these logs, so without being able to sort or filter by it, it means I'm back to using grep in the shell or rolling my own. As an example, if a firewall logs the IP address of denied connections, but also outputs warnings, information when rules have been reloaded, etc. Then the common fields (timestamp, severity, message) aren't that helpful if say, you'd want to look for repeated attempts to break in (sort by extra.sourceIP and look for repetitions). I see the value of the table-style formatting above for some usecases, but in a scenario like this, it wouldn't help much.
Besides this, it would be nice if there was a tree view option in the drawer, as an alternative to raw JSON.
