ni icon indicating copy to clipboard operation
ni copied to clipboard

Automate `sort-compress` program using `ni`

Open spencertipping opened this issue 4 years ago • 0 comments

Compression trades speed for space, but we should be able to make that decision on the fly. There are two, perhaps three, action items here:

  1. Trade space and speed more intelligently
  2. Automatically prefer pigz to gzip, etc, if available
  3. Rate-limit data throughput if we start running out of disk space, pause entirely if we're near critical so we don't crash the job

spencertipping avatar Sep 06 '21 13:09 spencertipping