gzip in gargammel.pl causing filesystem issues

[cmeesters]

Hi,

gargammel.pl contains a few | gzip commands. This can be problematic (and has been on our cluster: it caused very low overall throughput):

• filesystems do not cope well with repeated small requests. This might not be noted on a "small" filesystem and a "small" computer (e.g. a "big" server), but on huge parallel file system (e.g. of a cluster) this can be a performance killer (for gargammel-users and other users). The impact might only be noted, when many such processes run concurrently.
• in addition to this it is to be noted that compression usually is slow and piping to gzip within the same cgroup will limit the scalability of the parent.

Generally, it is a good idea to compress final results, but not intermediate files (or to do so independently of an actual calculation).

Would you consider to change the wrapper script accordingly?

Best regards, Chris