hail
hail copied to clipboard
MethodTooLargeException when exporting very wide Table
See https://github.com/broadinstitute/gnomad-browser/issues/914. In the line in question, we attempt to export a Table
with ~13,000 columns, and get the following error: is.hail.relocated.org.objectweb.asm.MethodTooLargeException: Method too large: __C19580collect_distributed_array.__m19633split_InsertFields ()V
(see above-referenced issue for full stacktrace).
Hail version was 0.2.96-39909e0a396f
This is a known limitation -- it's hard to fix without significant investment in a backend we don't want to maintain forever. This issue generally pops up when people use tables instead of matrix tables to process wide files. Can you rewrite the GTEx processing script to use import_matrix_table instead of import_table, then process as a matrix?
@tpoterba yes I think that's what we'll do when we come back to this, thanks