msigdbr icon indicating copy to clipboard operation
msigdbr copied to clipboard
verified publisher icon igordot

Slow load of the database

Open assaron opened this issue 2 months ago • 1 comments

Hi Igor,

I wanted to file an issue that load_msigdb_df takes rather significant amount of time, even if the file is local. It would be nice to improve it.

At the moment I have two suggestions:

  1. Very simple and practical: switch to qs2::qd_save/qs2::qd_read. For me it changes the time from ~15 seconds for RDS to ~3 seconds for qd.

  2. A bit more involved: to split the file by collections (it also will help to keep the memory footprint down). I could even look at it myself, but not right now.

assaron avatar Oct 09 '25 19:10 assaron