Add workflow specific engine logging to WES GetRunlog response

[wleepang]

Description

Include any workflow specific engine log output in top level run_log element of the WES GetRunLog response

Use Case

If a workflow fails because a task fails, the corresponding logs are retrievable via the task_logs element of the WES GetRunLog response. However, if a workflow fails because of a workflow syntax or localization error, these are only available via the CloudWatch logs associated with an engine, retrieved by agc logs engine and mixed with engine logs from other unrelated workflow executions. This makes it difficult for tooling that only talks to the WES endpoint to retrieve engine related errors for a specific workflow execution.

Proposed Solution

Place either the output or a retrievable URI for a workflow specific engine log in the WES GetRunLog response. For example:

{
  "run_id": "string",
  "request": {...},
  "state": "UNKNOWN",
  "run_log": {
    "name": "string",
    "cmd": [
      "string"
    ],
    "start_time": "string",
    "end_time": "string",
    "stdout": "string", // <-- log output lines or s3 uri or cloudwatch logstream arn
    "stderr": "string",  // <-- log error lines or s3 uri or cloudwatch logstream arn
    "exit_code": 0
  },
  "task_logs": [...],
  "outputs": {...}
}

Other information

[adamnovak]

This probably relates a bit to #319, which teaches the AGC client to download and display those URL fields. Although that only implements fetching web URIs so far.

Toil as set up in #317 already sends nearly all the workflow-specific engine logs this way; some of the ZIP parsing logs still only show up in the main engine log stream.

[patmagee]

@wleepang will this be scheduled soon? It would be great to have the information available in the WES api