devito icon indicating copy to clipboard operation
devito copied to clipboard

More effective symbol caching

Open FabioLuporini opened this issue 2 years ago • 0 comments

Consider:

from devito import *
from devito.types import _SymbolCache

grid = Grid(shape=(4, 4))

f = Function(name='f', grid=grid)
f0 = Function(name='f', grid=grid)

both f and f0 have the same name, 'f'. In the _SymbolCache, four entries are created:

 (f, (x, y)): <weakref at 0x14229d880; to 'f' at 0x142284b30>,
 f: <weakref at 0x14229d880; to 'f' at 0x142284b30>,
 (f, (x, y)): <weakref at 0x14229db20; to 'f' at 0x1420ea450>,
 f: <weakref at 0x14229db20; to 'f' at 0x1420ea450>

the f entries represent dynamically allocated classes. This is created by the Python interpreter and will never be freed. It's a tiny amount of bytes per symbol, but if called in a large loop it may cause a non-negligible leak. This appears extremely related to https://github.com/devitocodes/devito/issues/845.

However, this could be fixable by creating a class f only once, instead of one per new Function. This will require somewhat extending (changing?) the cache_key of the instance Function, which currently is (f, (x, y)), but this is clearly not enough if we were to use a unique class f. Perhaps we could add the id of the object. Maybe the id (or the hash) alone would be enough? if we use the id (hash), it's plausible it's as simple as that -- we don't need f, x, y, etc.

FabioLuporini avatar Mar 08 '22 09:03 FabioLuporini