Improve ducktape log collection spec

[ewencp]

Currently we collect logs based on a default specified in the Service class or if the test explicitly requests it (via mark_collect either for a single log file or defaulting to all of them). In practice, this has meant that logs are either always collected or not because it's not convenient to make assertions that throw exceptions if you have to also catch that exception and mark logs for collection.

Instead, a developer should be able to specify a different set of options for log collection such that it is fully automated by the ducktape framework and our existing Service classes should be upgraded to use this new approach. In particular the set of options that would probably make sense are:

• Always – always collect the log file. Useful for basic INFO level logs from pretty much any service and test
• On failure – the log is possibly large and only useful if the test fails. Example: verifiable producer/consumer generate a lot of output, but on success that output isn't useful. On error, it's basically required to debug the issue.
• If explicitly requested – the mode currently required if you only sometimes want to collect them. Probably rarely used, but potentially useful if you only want to collect certain logs if specific failures occur (i.e. a particular validation fails)

If there's a good way to express alternatives, we may also want to support that. For example, we may have some services generate the same logs at two levels (e.g. INFO and DEBUG), and on failure you'll want the full DEBUG, but otherwise the INFO would be sufficient.