add flag to disable runner partial event json write

[chrismeyersfsu]

trafficstars

https://gist.github.com/chrismeyersfsu/e388d7c81f7d039ad617c1ac13ff0196

Above is the results of an strace of ansible-runner worker run. The majority of the IO system calls are to create and write to a very short lived job event file. There should be a flag to turn off the writing to this file.

[matburt]

Disabling these partials would result in missing event data... the partial files are how we join the stdout information to the task information, so this has the effect of removing events entirely.

I'm not specifically against doing this, but it would be part of a larger "disable event handling and just dump stdout" sort of thing.

[chrismeyersfsu]

Buffer the partials in memory instead of writing them to disk. We can balance the memory usage by overflowing to disk.

[matburt]

The problem with doing that is that these two places are on the either side of a process boundary so in order to keep them in memory we would need some sort of persistent connection between the worker process and Runner's parent collector process. That's doable but increases the complexity by a lot... especially with a high fork count.

The original goal was to leverage disk i/o which would likely be a better overflow mechanism when the worker process was dealing primarily with net i/o in the case of a remote connection