NetCDF reporter slows down OpenMM performance?

[nathanmlim]

I'm experiencing some slow downs in performance when using the NetCDF reporter from MDTraj versus the DCD reporter from OpenMM.

When using the NetCDF reporter the performance declines the longer the simulation goes on. It goes from ~170ns/day -> 83ns/day:

10:18:57 [OpenMM NPT simulation] OpenMM(7.1.1.dev-c1a64aa) simulation generated for OpenCL platform 10:18:57 [OpenMM NPT simulation] #"Progress (%)" "Step" "Time (ps)" "Speed (ns/day)" "Elapsed Time (s)" "Time Remaining" 1498.3880417346954 1:10:07:47 10:52:35 [OpenMM NPT simulation] 2.0% 1250000 4999.999999909413 171 2018.143259525299 1:10:20:11 ... 03:24:48 [OpenMM NPT simulation] 24.4% 15250000 61000.00001059233 84.2 61550.770728349686 2:5:51:24 03:49:02 [OpenMM NPT simulation] 24.8% 15500000 62000.00001079606 83.7 63004.653540849686 2:5:56:18

Running the same simulation with the DCD Reporter, it stays at a consistent speed of 176ns/day:

02:51:36 [OpenMM NPT simulation] 24.4% 15250000 61000.00001059233 176 29407.696353912354 1:1:43:54 02:59:38 [OpenMM NPT simulation] 24.8% 15500000 62000.00001079606 176 29889.26694726944 1:1:35:17 03:07:39 [OpenMM NPT simulation] 25.2% 15750000 63000.000010999785 176 30370.182510137558 1:1:26:40

Can any of you guys provide some insight on why this happens?

[peastman]

A similar problem has been reported in the NetCDF reporter from ParmEd, and someone traced that to a memory leak in the library it uses to write NetCDF files. I don't know if MDTraj uses the same library, but if so that's probably the reason.

[davidlmobley]

Sounds like this is a "check the ParmEd issue tracker" thing to check what library is used.

[peastman]

Or check the source. It looks like it uses scipy.io.netcdf.

[peastman]

Here's the relevant issue: https://github.com/ParmEd/ParmEd/issues/619. It suggests a workaround.

[slochower]

I encountered this issue as well. I installed netCDF4 via pip and still experienced gradual slowing with the NetCDFReporter imported from mdtraj but using parmed.openmm.reporters.NetCDFReporter worked as expected. I didn't follow all the PRs and linked Issues to see if simply installing netCDF4 is supposed to fix the problem with mdtraj's implementation.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.