Nalini Ganapati

@bbimber, this is another manifestation of running out of memory - `Error: (gzip_handle_error) Cannot decompress with GZIP: inflateInit error: Z_MEM_ERROR`. We have been looking into these types of issues with...

@bbimber, thanks to @mlathara and @kgururaj, here is a suggestion. With `~36 attributes+offsets` and `~80 fragments` in GenomicsDB parlance getting stored, with `--genomicsdb-segment-size=1048576(1M default)`, we are looking at memory requirements...

Yes, we do expect a `done importing batch 5/5`. The reason it is taking long is because it is doing the consolidation in this phase too. Should probably disassociate the...

@bbimber, sorry that the import with consolidate did not complete. If you are amenable to using a native tool, please download the tool from [here](https://github.com/GenomicsDB/GenomicsDB/releases/download/v1.4.3/consolidate_genomicsdb_array) for consolidation. This executable will...

Yes, the consolidation will be in-place and you can clone the workspace before trying the consolidate.

@bbimber, we are investigating some scalable solutions for you. Meanwhile, can you provide the following information? 1. What is the total and free amount of memory available to say consolidate_genomicsdb_array...

@bbimber, your approach should mostly work, this is exactly what I am going to allow with the standalone tool, a new arg for `batch-wise consolidation`. The tool is also better...

@bbimber, I have placed another version of consolidate_genomicsdb_array [here](https://github.com/GenomicsDB/GenomicsDB/releases/download/v1.4.3/consolidate_genomicsdb_array). This allows for batch-wise consolidation with the `--batch-size` or `-b` option. The tool also does better reuse of the consolidation buffers...

Are you able to run `gatk SelectVariants` on the workspace?

How large is your machine, memory-wise? Can you reduce the `-Xmx100g` to something smaller, so the native GenomicsDB process does not get starved out?