Peter Bourgon
Peter Bourgon
Looks like you've got a bad record saved to disk somehow. Can you go to your store's data dir and `tail -n1` each of the files there? I bet one...
I don’t maintain the Docker releases for OK Log, that’s done by a third party. Maybe they can chime in here?
Another first-class property of OK Log is that the final segment files on disk are human-readable and greppable. I wouldn't give up this property lightly. I would be much more...
The advantage has always been conceptual: you can still back up, grep, and analyze your logs as files-on-disk with your favorite UNIX tools, independent of the OK Log apparatus on...
~~OK Log defines log entries (log records) as single-line, as an invariant of the system.~~ edit: In the context of this issue, the above comment is counterproductive.
No reason to bring in a dependency for this. My intuition is that a custom little ringbuffer should work fine. Happy to consider anything you come up with, though.
That's fine, but then you'll want to use another log forwarding tool like svlogd — `oklog forward` will never be in the business of managing files on disk.
I think it's almost certainly not worth doing this one. Filesystems have advanced to the point where you can turn compression on at the FS layer. I struggle to imagine...
We tried pushing segments thru LZ4 before and after disk writes and reads respectively, it cost too much CPU and didn't reduce size very much.
Looks reasonable. Needs tests.