Add integration tests

[spbnick]

To test tlog end-to-end add integration tests. Possible tests follow.

General Recording

• For several payload limit setting values (minimum, default, large) check that tlog-rec produces payloads within the limit, for both input and output, for the following crossing the limit:
  • single-byte UTF-8 characters,
  • multi-byte UTF-8 characters,
  • input byte sequences producing escape sequences on the output,
  • invalid input byte sequences diverted into binary fields on the output,
  • window resize records.
• For several latency settings (minimum, default, long) check that tlog-rec writes a record on time when there was any I/O or window resizes, and doesn't when there wasn't any.
• For each --log-* option check that turning it on or off reflects in tlog-rec output. That is input/output/window resizes are (not) recorded as configured.
• Check that various rate-limiting settings affect tlog-rec log as documented. That is:
  • average rate is within 5% of configured over 10 seconds of recording,
  • configured amount of burst I/O is allowed through,
  • configured action (pass/delay/drop) is observed.
• Check that tlog-rec can record to a file correctly, and responds appropriately for various conditions preventing it from writing one. Including:
  • Filesystem permissions
  • SElinux violation (assuming the session is somehow confined)
  • Insufficient space on device
• Check that tlog-rec can record to syslog correctly, and observes facility and priority settings
• Check that tlog-rec can record to journal correctly, observes the priority setting, and adds the documented extra Journal fields correctly
• Check that both terminal input and output recorded by tlog-rec and played back by tlog-play are preserved, including:
  • Valid 1-, 2-, 3-, and 4-byte UTF-8 byte sequences
  • Invalid UTF-8 byte sequences
  • Characters that must be escaped in JSON
  • Window resizes
• Check that terminal input, output, and window resize timing is preserved when recorded by tlog-rec and played back by tlog-play.
• Check that tlog-rec records and tlog-play faithfully reproduces, without crashing/memory corruption, any input and output produced by a fuzzer.
• Check that tlog-play can process any input produced by a fuzzer without crashing or memory corruption.

Tlog-rec

• Check that tlog-rec observes settings stored in its system-wide configuration file.
• Check that tlog-rec overrides settings from its system-wide configuration file by ones found in file specified with TLOG_REC_CONF_FILE environment variable.
• Check that tlog-rec overrides settings from the file specified with TLOG_REC_CONF_FILE environment variable by settings specified in TLOG_REC_CONF_TEXT environment variable value.
• Check that tlog-rec observes SHELL environment variable value, when no command to record was specified.
• Check that tlog-rec doesn't interpret options passed to the recorded program as its own.

Tlog-rec-session

• Check that tlog-rec-session doesn't interpret arguments passed to the shell as its own, when started with -c option.
• Check that tlog-rec-session doesn't interpret arguments passed to the shell script to execute as its own, when started with arguments, but without the -c option.
• Check that tlog-rec-session starts a login shell when invoked with argv[0] starting with dash -.
• Check that tlog-rec-session starts a login shell when invoked with -l option.
• Check that tlog-rec-session observes settings stored in its system-wide configuration file.
• Check that tlog-rec-session observes the shell specified in its system-wide configuration file.
• Check that tlog-rec-session observes the notice text setting specified in its system-wide configuration file.
• Check that tlog-rec-session overrides settings from its system-wide configuration file by ones found in file specified with TLOG_REC_SESSION_CONF_FILE environment variable.
• Check that tlog-rec-session overrides settings from the file specified with TLOG_REC_SESSION_CONF_FILE environment variable by settings specified in TLOG_REC_SESSION_CONF_TEXT environment variable value.
• Check that tlog-rec-session observes TLOG_REC_SESSION_SHELL environment variable value, overriding the shell specified in TLOG_REC_SESSION_CONF_TEXT environment variable.

Tlog-play

• Check that tlog-play can playback recordings stored in files.
• Check that tlog-play can playback recordings stored in journal.
• Check that tlog-play can playback recordings stored in ElasticSearch.
• Check that tlog-play would tolerate missing log messages with lax option enabled, and would not with it disabled.
• Check that tlog-play observes -s/--speed option correctly.
• Check that tlog-play observes -p/--paused option correctly.
• Check that tlog-play observes all possible value types of -g/--goto option correctly. Including negative and ones beyond the end of the recording.
• Check that tlog-play observes -f/--follow option correctly with any log reader.
• Check that tlog-play observes -S/--journal-since, -U/--journal-until, and -M/--journal-match options correctly.
• Check that tlog-play reacts to control keys correctly during playback.
• Check that tlog-play doesn't react to escape sequences in recording, which would normally invoke a response from the terminal including characters that are used to control tlog-play. See #147.

Most, if not all, integration tests should run within make check.

[nkinder]

This looks like a good list of tests. It would be nice to call out the conditions cases we want to test for the following test:

"Check that tlog-rec can record to a file correctly, and responds appropriately for various conditions preventing it from writing one."

I think this would include (at a minimum):

• Filesystem permissions
• SElinux violation (assuming the session is somehow confined)
• Insufficient space on device

[spbnick]

Thank you, Nathan! Yes, I agree, this is a good list of conditions to handle. This brings me into thinking that we should test how failures to log to Journal are handled. There is no error detection with syslog, though (unless we're prepared to talk to the socket directly), so we don't really know if anything is logged anywhere when using it.