Feature request: print/download FAILED logs only

[droazen]

cromshell really needs a command that can print and/or download logs for JUST the failed tasks. Currently we have to either download all logs or grep through the metadata for failures and log file links.

[droazen]

Here is a trivial shell script that downloads just the (stderr) logs for failed shards given the metadata. Perhaps it could be adapted into a patch for cromshell:

#!/bin/bash

if [ $# -ne 1 ]
then
    echo "Usage: $0 path_to_metadata"
    exit 1
fi

grep -A 1 Failed "$1" | grep stdout | awk '{ print $2; }' | sed 's/"//g' | sed 's/,//g' | sed 's/stdout/stderr/g' | while read log
do
    SHARD_ID=`echo "$log" | awk -F/ '{ print $7; }'`
    gsutil cp "$log" "${SHARD_ID}_stderr"
done

exit 0