Implement proposed `k8s-stream-file` format

[portante]

Instead of parsing the contents of the data read from the stdout and stderr pipes, this commit adds support for a "stream" format, named k8s-stream-file, which just records what is read from a pipe to disk.

It significantly saves on CPU spend processing the buffer read, uses only 2 I/O vectors, and never touches the memory read from the pipe.

This is a conceptual PR, meant to illustrate a proposed stream log file format that removes the byte level interpretation of stdout/stderr in favor of simply recording what data was read on each system call. It defers the interpretation of the byte stream to the consumer, allowing this writer to operate with as little overhead as possible (avoiding bad containers that write only newlines, or small numbers of bytes between newlines). The reader of the byte stream is then tasked with reassembling the stream according to whatever interpretation it sees fit to use.

The goal of this work is to provide a simple format that will stream well into an object store, such that, given enough metadata stored with the stream, the consumer can reconstruct the I/O stream at the time it is read.

This is an updated implementation of https://github.com/cri-o/cri-o/pull/1605.

[portante]

See also #262.

[lgtm-com[bot]]

This pull request introduces 1 alert when merging c0bc805cc3409baa99e50831529a7b1c1aba9e59 into 31614525ebc5fd9668a6e084b5638d71b903bf6d - view on LGTM.com

new alerts:

• 1 for Use of potentially dangerous function

[portante]

you'll need to run make fmt and commit the changes to have the majority of CI checks run.

Fixed.

Thanks for the review!

[rhatdan]

@giuseppe PTAL