ncov-ingest icon indicating copy to clipboard operation
ncov-ingest copied to clipboard

ENH: Produce "small" metadata.tsv with only essential columns

Open corneliusroemer opened this issue 1 year ago • 0 comments

Context

Our metadata.tsv is now so big that it becomes impossible to load into memory in full.

We should either create an "essential" metadata.tsv or in general partition metadata.tsv and make users merge later.

Right now we sort of require preprocessing with tsv-select to pick relevant columns for interactive work (though augur filter seems to work fine with the metadata file as is so this isn't urgent).

corneliusroemer avatar Dec 13 '22 14:12 corneliusroemer