EGSnrc icon indicating copy to clipboard operation
EGSnrc copied to clipboard

I/O bottleneck for EGS_ScoringSingle and EGS_ScoringArray

Open ftessier opened this issue 2 years ago • 0 comments

Reopening this as an issue: originally posted as issue #605 and then transferred to Discussion #688.

There are 2 pull requests related to this issue (#606, #707) and a suggested improvement on branch fix-egspp-scoring-speed (extracted from #707). Note also a somewhat related pull request for sparse matrix storage in DOSXYZnrc (#405).

I ran into this issue myself a couple weeks ago with a 250x250x250 voxels XYZ geometry: simulation for a few histories took 0.03 seconds, but data I/O took 30 seconds! This makes simulation with large phantoms (in terms of number of voxels) completely impractical, just on account of of writing data one voxel at a time in text mode.

Discussed in https://github.com/nrc-cnrc/EGSnrc/discussions/688

Originally posted by MartinMartinov June 1, 2020 I've been running simulations computing 3ddoses in 512x512x117 phantoms, where the actual particle simulation runs pretty quickly, but the all the storeState/setState calls (I suspect) are bottlenecking the simulation significantly. I'm currently setting nbatch to 1 to avoid too many state calls which works well interactively, but when running several simulations in parallel accessing the same hard drive, it is still a huge efficiency killer when they all go to write egsdat files at once (once again, just a suspicion).

Would it be possible to write a parallel set of store/setState which save data in binary format (ie, storeBinState and setBinState) which usercodes that have massive arrays can use instead? I don't mind giving it a try myself, I'm just not sure how well it would fit into the current architecture, so I wanted to post here first.

ftessier avatar Jul 04 '22 15:07 ftessier