Option to turn off stdin capture to file

[ms-xy]

Is your feature request related to a problem? Please describe. Potentially running out of disk space.

Describe the solution you'd like A command line switch, lets say --no-stdin-capture which disables saving of stdin. Or possibly --stdin-capture=<file|dump|off>.

Describe alternatives you've considered

• Writing an shell function that removes the created file after running lnav
• Writing a systemd timer that periodically clears the capture folder
• Mounting ~/.config/lnav/stdin-capture as a tmp file system so all files created get cleared after reboot

All of these are viable options. However, it would be way more elegant to just add the option to turn it off.

Additional context This wonderful feature should have an option to be turned off. It's great for capturing logs of any program not producing persistent logs. Doubly logging something already persisted in e.g. journald is unnecessary redundancy, in my opinion.

[tstack]

• Writing an shell function that removes the created file after running lnav
• Writing a systemd timer that periodically clears the capture folder
• Mounting ~/.config/lnav/stdin-capture as a tmp file system so all files created get cleared after reboot

Files older than a day should be removed the next time lnav starts up. See: https://github.com/tstack/lnav/blob/cc1e79d1cc1b286cd3f4ca96096f4e20eeb1934a/src/lnav_config.cc#L176-L199

add the option to turn it off.

It's not possible to turn off completely because lnav needs to save the contents in order to support scrollback. So, the space is going to be needed for some period of time.

Doubly logging something already persisted in e.g. journald is unnecessary redundancy, in my opinion.

For something like journalctl, I would think the ideal solution would be to add native support for the systemd journal APIs to lnav. That way, queries could be done within lnav's TUI instead of having to drop out.

[ms-xy]

Files older than a day should be removed the next time lnav starts up. See:

Ahh my bad, I hadn't noticed that it does that. I implemented counter measures before it ever had the chance :sweat:

It's not possible to turn off completely because lnav needs to save the contents in order to support scrollback. So, the space is going to be needed for some period of time.

Thank you for the explanation, makes sense to have such a cache file.

For something like journalctl, I would think the ideal solution would be to add native support for the systemd journal APIs to lnav. That way, queries could be done within lnav's TUI instead of having to drop out.

That would be absolutely awesome!

For convenience and sake of aliases and shell functions, it would be good if control over the queries would not only be possible via the TUI. For example one alias/function that I use frequently:

function sshdlogs() {
    journalctl -f --since=-1weeks -a -o json --output-fields=MESSAGE,PRIORITY,_PID,SYSLOG_IDENTIFIER,_SYSTEMD_UNIT -u sshd "$@" | lnav
}

I'd assume that specifying the output fields and -f would not be necessary, but e.g. something like this here to specify the initial query (I have no idea if this would make any sense at all, I don't know if the journald API offers the ability to parse parameters or not, but it illustrates what I mean):

lnav --journald -- --since=-1weeks -u sshd

I'm also not familiar with the lnav code, but if you think me helping might help getting such a feature in more quickly, I'd try my hand.

[kylemanna]

One comment on this is that I have a number of backup scripts that assume everything under ~/.config are things I care a lot about and want to keep. Often this isn't the case for the stdin-captures.

Would be nice to move volatile (can be-recreated, isn't explicitly configuration data) data under ~/.cache/lnav/....

To be honest though, 99% of the time I'd just turn the feature off and avoid having to back up files that change everyday anyways.

[piotr-dobrogost]

According to https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html#variables *.history and view-info-* lnav's files should be kept in $XDG_STATE_HOME and lnav's stdin captures should be kept in $XDG_CACHE_HOME. ~/.config is not the right place for them.

[kylemanna]

One-liner workaround that I'm using on my systems to move to the cache directory:

mkdir -pv  ~/.cache/lnav/ && mv -v  ~/.config/lnav/stdin-captures ~/.cache/lnav/ && ln -srv ~/.cache/lnav/stdin-captures ~/.config/lnav/

[zamu-flowerpot]

Would it be possible to have an option to just use memory? I have plenty of memory and looking at the disk usage of journald (journalctl --disk-usage and journalctl --user --disk-usage), I have more than plenty to house everything in memory.

Or even just a delete on exit option would be nice. I don't think I would ever go back and re-use these files so leaving them around seems kinda wasteful.

[elandorr]

The gist of my own message from last year (https://github.com/tstack/lnav/issues/1097) so I can close the duplicate:

Can we have a toggle for turning this off? Maybe in general for all unwanted writes? Ideally off by default. Nothing should ever write without asking, in my book. (I know many applications do, but that's a bad habit. Makes it hard to keep a clean system.)

The docs yield no results for either 'stdin*' or 'clean'. How does it handle real-life usage, when you keep a pipe open, and millions of lines pour in?

It does not have to write a single byte. There can be a cutoff for lines kept in memory so it doesn't leak during live processing, and a configurable fallback to disk, if desired.

I would like to keep this running 24/7 so I can't have uncontrollably exploding files everywhere.

Thanks for your consideration!

After running for a few mins with a couple million lines for testing, lnav is just eating more and more memory, with no signs of stopping. So running it 24/7 is out of question. This needs a line-limit.