gnomad-browser icon indicating copy to clipboard operation
gnomad-browser copied to clipboard

feat(docs): add information on how to speed up large pipelines

Open rileyhgrant opened this issue 1 year ago • 0 comments

Adds one small paragraph about adding more workers to a dataproc cluster to run the computational portion of the variants pipeline in a shorter timeframe (~2 hours).

I had meant to add this last time I ran the browser pipeline, and found this information in my Slack messages. I figure its preferable to have it in the docs, as they currently only document how to speed up the hail table -> elasticsearch portion, not the generation of the hail table portion.


Edited for clarity

rileyhgrant avatar Jul 23 '24 17:07 rileyhgrant