Python's tarfile package is S-L-O-W

[jlinford]

tau trial export for a trace creates a compressed tarball, but it's about 10x slower than using the system's tar cvzf. We should invoke tar when available and fallback to Python's tarfile package otherwise. We do something similar for curl and wget in util.download()

[jlinford]

Some numbers:

1.9GB trace:

[TAU] Writing '/work1/people/jlinford/NEPTUNE/NEP-117/knl01-neptune_fcst-trace.trial0.tgz'...
[TAU] Completed in 990.592 seconds

vs.

time tar czf ~/0.tgz 0

real	1m10.381s
user	1m9.313s
sys	0m1.575s

So really Python's tarfile module is about 14x slower than the system tar command. Yikes!

[jlinford]

Could the -u flag added to resolve Issue #136 be causing this?

[zbeekman]

Very very possible. The question is did you run tests before or after I added it? Maybe there is a better way to control IO to stdout/stderr than setting -upon the command line. I’ll investigate.