decompress error

[ruizgo]

command:

unnaf --fastq fastq.gz.naf |gzip - >fastq.gz.naf.gz
unnaf error: can't allocate 35115481985 bytes

file size: fastq.gz.naf 43G

[ruizgo]

My PC memory:96G

[KirillKryukov]

One possibly related issue is that currently unnaf uses memory quite inefficiently for fastq decompression. Namely, the decompressor stores the entire compressed sequence in memory, while streaming qualities. This was done for speed and simplicity, but may result in problems when decompressing huge fastq datasets. We will address this in the future, hopefully soon. In the meantime, large fastq files can be divided into smaller parts before compressing. In your case, I think your RAM should be more than enough, so there may be something else.