Pipe task stdout/stderr to file

[thejmazz]

Todos:

• [ ] keep stdout/stderr logs in main logs?
• [ ] set absolute path to stdout.txt and stderr.txt in task state
• [ ] how to handle this for streaming stdout/stderr to web app in future?

---

Currently, when you return a string from an operationCreator, it will be called with our utility function shell. If you have a task like

const myTask = task({
  name: 'Boring echos',
  output: '*.txt',
}, () => 'echo one && echo two && echo three > output.txt')

Then in the output you will see something like

8a93345 :   Creating operation
8a93345 :     operationProps input: null
8a93345 :     operationProps params: {}
8a93345 :     Starting: echo one && echo two && echo three > output.txt
8a93345 :         stdout: one
8a93345 :         stdout: two
8a93345 :         stdout:
8a93345 :   Starting resolve output for 8a9334528a0fe207a682d675ed5f0dfb2abcb47b8128a72e875c53053c89a212

The shell command is passed a logger instance from the createOperation lifecycle step (basically so that it can have a preset tab level). shell basically creates a child process with { shell: true } (this is why the && and | work). In terms of interacting with the store, it calls logger, which is then storing those logs in the state (along with items like "starting task", "resolving input", etc, for that specific task.

The relevant pieces of shell code are

  myProcess.stdout.on('data', onStdout)
  myProcess.stderr.on('data', onStderr)

  function onStdout(chunk) {
    for (let line of chunk.toString().split('\n')) {
      logger.emit('log', 'stdout: ' + line, 2)
    }
  }

  function onStderr(chunk) {
    for (let line of chunk.toString().split('\n')) {
      logger.emit('log', 'stderr: ' + line, 2)
    }
  }

The most immediate solution is to pipe stdout and stderr into files within the task directory. This would mean shell needs to be made known of the task directory: that is, it is a piece of the global state it is interesting in receiving - in react-redux you would do something like

const { connect } = require('react-redux')
// expects props.pertinentData, e.g.
// render() { return (<div>{this.props.pertinentData}</div>) }
const myComponent = require('./my-component.jsx')

const mapStateToProps = (state) => ({ pertinentData: state.something })
// connect constructs a shouldComponentUpdate method handling whether or not
// properties of object returned by mapStateToProps have had their reference
// changed in the global state
const connectedComponent = connect(mapStateToProps)(myComponent)

In our case, state is mostly selected from using yield(select() within the sagas lifestyle. The lifecycle methods, e.g. createOperation, accept a parameter taskState as their first parameter by convention, whose value is passed into them by the "lifecycle brain" which is the saga: for example, within operationSaga.

So, in order to let shell be aware of the taskState.dir, we need to pass it into a "lifecycle method" within the main "lifecycle saga" and that lifecycle method can then pass it into functions it calls (however, we would probably want to keep that one level deep?). In this case, there is a quick solution, notice this

const shellOpts = {
  cwd: taskState.dir
}

So we can actually just pluck the taskState.dir out of shellOpts.cwd. (But keep in mind the importance of being explicit with regards to picking out relevant data from global state for a given side effect).