Pipe third-party tool output or write to temp location

[widdowquinn]

Summary:

Currently, third-party tool output (MUMmer, BLAST, etc.) is written to the output directory, and failed jobs can block re-runs.

It would be better to either pipe the output (where possible), or write it to a temporary location, so failures could be handled more gracefully.

[widdowquinn]

This is a filesystem issue also. Writing thousands to millions of small files to the filesystem degrades performance. They are also painfully slow to delete. A process of:

• call 3rd party tool and write to temp location
• check output and pull into database
• delete output

would be cleaner, at the cost of some extra time to delete output for each job.