mesmerize-core icon indicating copy to clipboard operation
mesmerize-core copied to clipboard

Cache size calculation does not take size of CNMF object into account

Open ethanbb opened this issue 3 months ago • 0 comments

I just rewrote this function (to recurse for sequences and estimates fields). This changed the results in test_cache, and I realized the old implementation had a bug:

def _get_cache_size_bytes(self):
        """Returns in bytes"""
        cache_size = 0
        for i in range(len(self.cache.index)):
            if isinstance(self.cache.iloc[i, 4], np.ndarray):
                cache_size += self.cache.iloc[i, 4].data.nbytes
            elif isinstance(self.cache.iloc[i, 4], (tuple, list)):
                for lists in self.cache.iloc[i, 4]:
                    for array in lists:
                        cache_size += array.data.nbytes
            elif isinstance(self.cache.iloc[i, 4], CNMF):
                sizes = list()
              # ^^^^^^^^^^^^^^ this list is never used
                for attr in self.cache.iloc[i, 4].estimates.__dict__.values():
                    if isinstance(attr, np.ndarray):
                        sizes.append(attr.data.nbytes)
                    else:
                        sizes.append(getsizeof(attr))
            else:
                cache_size += sys.getsizeof(self.cache.iloc[i, 4])

        return cache_size

So it's easy to fix but just noting that the cache size limit has not been working correctly for a while (and it seems like the test was written to preserve existing behavior rather than the correct behavior).

ethanbb avatar Sep 08 '25 19:09 ethanbb