sqlite-utils icon indicating copy to clipboard operation
sqlite-utils copied to clipboard

Research: handle gzipped CSV directly

Open simonw opened this issue 4 years ago • 2 comments

Would it be worthwhile for the sqlite-utils command-line tool to grow features to efficiently directly interact with gzipped CSV data?

Maybe add --gz options to both insert and to the various commands that output query results.

simonw avatar Aug 18 '21 21:08 simonw

Something like this then:

sqlite-utils file.db "select * from t" --csv --gz > t.csv.gz

Maybe add a -o t.csv.gz option too so you don't have to use a >.

simonw avatar Aug 18 '21 21:08 simonw

Some questions:

  • Should this support compression formats other than gzip?
  • Should memory learn to auto-detect gzipped data?

simonw avatar Aug 18 '21 21:08 simonw