openPMD-api icon indicating copy to clipboard operation
openPMD-api copied to clipboard

Read: Close Oldest Iterations

Open ax3l opened this issue 2 years ago • 2 comments

In reads of very large data series, it is common to run into ulimits.

One convinient way to support scripting that we could offer is to implement a user-facing option on how many iterations to not call .close() on when opening a new iteration in non-streaming read mode.

That way, we could clean up the oldest iterations we opened when opening a new one while looping through series for analysis and the user does not need to keep track. Such an option would also make it simpler for openPMD-viewer et al. to implement iterations through series.

ax3l avatar Apr 08 '23 00:04 ax3l

Prerequisite: for reading, we currently cannot re-open closed() iterations yet #1606

ax3l avatar Apr 11 '23 17:04 ax3l

Related to #1592 #1606

ax3l avatar Jun 11 '24 18:06 ax3l