dolphin
dolphin copied to clipboard
`Maximum memory usage` is wrong for parallel bursts
with 3 parallel bursts on a small test set (using the GPU):
dolphin config --slc ../gslcs/t078_16557*/*/t*h5 -s 5 3 --threads 16 --n-work 2 --n-parallel 3
The final print out was
INFO Maximum memory usage: 4.03 GB
Moving down to 1 parallel burst:
dolphin config --slc ../gslcs/t078_16557*/*/t*h5 -s 5 3 --threads 16 --n-work 2 --n-parallel 1
Result:
INFO Maximum memory usage: 7.56 GB
There's a very small chance this isn't wrong... but it doesn't make sense, as I would expect 3x the RAM used for 3 burst processes running concurrently. This might be easier to debug if we had a smaller dedicated machine to watch the actual RAM usage.