ni
ni copied to clipboard
Automate `sort-compress` program using `ni`
Compression trades speed for space, but we should be able to make that decision on the fly. There are two, perhaps three, action items here:
- Trade space and speed more intelligently
- Automatically prefer
pigztogzip, etc, if available - Rate-limit data throughput if we start running out of disk space, pause entirely if we're near critical so we don't crash the job