nilearn icon indicating copy to clipboard operation
nilearn copied to clipboard

Memory profiling of the masker objects

Open htwangtw opened this issue 3 years ago • 1 comments

This is related to #3398.

We have started to profile the masker object to find out the where's most of the memory consumption happening, aiming to see if there's possibility to trim the code / process.

Through some rough and bare bone methods, we found resampling and apply_mask seems to be taking up most of the consumption. However, with the existing profiling tools we tried (memory_profiler (line by line / time tracking), guppy3 (memory by type of objects)) won't give us detailed traced back of the masker methods.

A few directions:

  • find other profiling tools
  • identify low level functions (resample_img etc) and profile them specifically.

WIP repo:

https://github.com/htwangtw/masker-memory-profile

Profile of memory usage by time of NiftiMasker niftimasker_memory_profile

htwangtw avatar Oct 26 '22 12:10 htwangtw

This profiler might be what we want, as it look into the modules from a glance: https://github.com/bloomberg/memray

htwangtw avatar Oct 26 '22 13:10 htwangtw