varianttools icon indicating copy to clipboard operation
varianttools copied to clipboard

Faster access to HDF5 using vaex

Open BoPeng opened this issue 6 years ago • 1 comments

Interesting talk at SciPy 2019, vaex acts on HDF5 files (that we use) but does the filtering and computation extremely fast with minimal memory usage (use memory map heavily).

The library is at https://github.com/vaexio/vaex

BoPeng avatar Jul 10 '19 19:07 BoPeng

This could significantly improve the performance of our vtools update operation on statistics, and filtering operations for vtools associate.

BoPeng avatar Jul 10 '19 20:07 BoPeng