yank icon indicating copy to clipboard operation
yank copied to clipboard

Published 20 hours ago • verified publisher icon choderalab

REMD Performance

Open shitanshu-cell opened this issue 3 years ago • 4 comments

I am working on REMD . I set up the simulation using the worked out example proposed in link : REMD Openmmtools I installed openmm,openmmtools and yank using conda .I am running it on gpu cluster equipped with GeForce RTX 2080 Ti card in each node.I dont know why its running extremely slow .Surely i am missing something or not configured correctly. code :

#********************* Replica exchange code *******************

#Test System :: Alanine dipeptide


import math
from simtk import unit
from openmmtools import testsystems, states, mcmc
import openmmtools as opmm
from simtk.openmm import app
import simtk.openmm as opnmm
import os
import tempfile    

testsystem = testsystems.AlanineDipeptideImplicit()

n_replicas = 4  # Number of temperature replicas.
T_min = 298.0 * unit.kelvin  # Minimum temperature.
T_max = 600.0 * unit.kelvin  # Maximum temperature.
temperatures = [T_min + (T_max - T_min) * (math.exp(float(i) / float(n_replicas-1)) - 1.0) / (math.e - 1.0) for i in range(n_replicas)]

thermodynamic_states = [states.ThermodynamicState(system=testsystem.system, temperature=T) for T in temperatures]

move = mcmc.GHMCMove(timestep=2.0*unit.femtoseconds,n_steps=50)
simulation =opmm.multistate.ReplicaExchangeSampler(mcmc_moves=move, number_of_iterations=1000)

storage_path = 'test_traj.nc'

print("storage path",storage_path)
reporter = opmm.multistate.MultiStateReporter(storage_path,checkpoint_interval=1)
simulation.create(thermodynamic_states=thermodynamic_states,sampler_states=states.SamplerState(testsystem.positions),storage=reporter)
simulation.run()

Test runs are submitted using script.

#!/bin/bash -l
#PBS -q gpu
#PBS -l select=1:ngpus=1:host=master
#PBS -N test_job_gpu
#PBS -o output.log
#PBS -e error.log
export PATH=/usr/local/cuda-10.0/bin:/usr/local/cuda-10.0/NsightCompute-1.0${PATH:+:${PATH}}

export LD_LIBRARY_PATH=/usr/local/cuda-10.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

export OPENMM_CUDA_COMPILER=/usr/local/cuda-10.0/bin/nvcc

cd $PBS_O_WORKDIR
conda activate myenv
python simulation.py

Thanks

shitanshu-cell avatar Dec 24 '20 08:12 shitanshu-cell

Hi @shitanshu-cell . If you have set the logging level to DEBUG, you should see timing information in the log. In case it's useful, there's a utility function in YANK for this: yank.utils.config_root_logger.

andrrizzi avatar Dec 25 '20 10:12 andrrizzi

Hi andrizzi , Thanks , I am able to get the timing information .But it seems like replicas are running in serial , So how i can run it in parallel means each replica will propogate in seperate node to get the maximum performance . I am sharing the log output for two iteration steps

DEBUG:openmmtools.multistate.multistatesampler:Iteration 1/10
DEBUG:openmmtools.multistate.multistatesampler:********************************************************************************
DEBUG:mpiplus.mpiplus:Single node: executing <function ReplicaExchangeSampler._mix_replicas at 0x2b64434b0dd0>
DEBUG:openmmtools.multistate.replicaexchange:Mixing replicas...
DEBUG:openmmtools.utils:Mixing of replicas took    0.004s
DEBUG:openmmtools.multistate.replicaexchange:Accepted 512/512 attempted swaps (100.0%)
DEBUG:openmmtools.multistate.multistatesampler:Propagating all replicas...
DEBUG:mpiplus.mpiplus:Running _propagate_replica serially.
DEBUG:mpiplus.mpiplus:Running _get_replica_move_statistics serially.
DEBUG:openmmtools.utils:Propagating all replicas took    0.458s
DEBUG:mpiplus.mpiplus:Running _compute_replica_energies serially.
DEBUG:openmmtools.utils:Computing energy matrix took    0.015s
DEBUG:mpiplus.mpiplus:Single node: executing <function MultiStateSampler._report_iteration at 0x2b644346b440>
DEBUG:mpiplus.mpiplus:Single node: executing <function MultiStateSampler._report_iteration_items at 0x2b644346b710>
DEBUG:openmmtools.utils:Storing sampler states took    0.003s
DEBUG:openmmtools.utils:Writing iteration information to storage took    0.014s
DEBUG:openmmtools.multistate.multistatesampler:Not enough iterations for online analysis (self.online_analysis_minimum_iterations = 200)
DEBUG:mpiplus.mpiplus:Single node: executing <function MultiStateSampler._online_analysis at 0x2b644346e290>
DEBUG:openmmtools.multistate.multistatereporter:Creating new NetCDF variable f_k with parameters: {'dtype': dtype('float64'), 'dims': ('dim_size4',), 'chunksizes': (4,)}
DEBUG:openmmtools.multistate.multistatereporter:Creating new NetCDF variable free_energy with parameters: {'dtype': <class 'numpy.float64'>, 'dims': ('dim_size2',), 'chunksizes': (2,)}
DEBUG:openmmtools.multistate.multistatereporter:Creating new NetCDF variable f_k_history with parameters: {'dtype': dtype('float64'), 'dims': ('iteration', 'dim_size4'), 'chunksizes': (1, 4)}
DEBUG:openmmtools.multistate.multistatereporter:Creating new NetCDF variable free_energy_history with parameters: {'dtype': <class 'numpy.float64'>, 'dims': ('iteration', 'dim_size2'), 'chunksizes': (1, 2)}
DEBUG:openmmtools.multistate.multistatesampler:*** ONLINE analysis free energies:
DEBUG:openmmtools.multistate.multistatesampler:        -0.0     1.0     1.1     0.8
DEBUG:openmmtools.utils:Computing online free energy estimate took    0.008s
DEBUG:openmmtools.multistate.multistatesampler:Iteration took 0.502s.
DEBUG:openmmtools.multistate.multistatesampler:Estimated completion in 0:00:04.522636, at Tue Dec 29 12:25:22 2020 (consuming total wall clock time 0:00:05.025151).
DEBUG:openmmtools.multistate.multistatesampler:********************************************************************************
DEBUG:openmmtools.multistate.multistatesampler:Iteration 2/10
DEBUG:openmmtools.multistate.multistatesampler:********************************************************************************
DEBUG:mpiplus.mpiplus:Single node: executing <function ReplicaExchangeSampler._mix_replicas at 0x2b64434b0dd0>
DEBUG:openmmtools.multistate.replicaexchange:Mixing replicas...
DEBUG:openmmtools.utils:Mixing of replicas took    0.000s
DEBUG:openmmtools.multistate.replicaexchange:Accepted 238/512 attempted swaps (46.5%)
DEBUG:openmmtools.multistate.multistatesampler:Propagating all replicas...
DEBUG:mpiplus.mpiplus:Running _propagate_replica serially.
DEBUG:mpiplus.mpiplus:Running _get_replica_move_statistics serially.
DEBUG:openmmtools.utils:Propagating all replicas took    0.147s
DEBUG:mpiplus.mpiplus:Running _compute_replica_energies serially.
DEBUG:openmmtools.utils:Computing energy matrix took    0.009s
DEBUG:mpiplus.mpiplus:Single node: executing <function MultiStateSampler._report_iteration at 0x2b644346b440>
DEBUG:mpiplus.mpiplus:Single node: executing <function MultiStateSampler._report_iteration_items at 0x2b644346b710>
DEBUG:openmmtools.utils:Storing sampler states took    0.002s
DEBUG:openmmtools.utils:Writing iteration information to storage took    0.011s
DEBUG:openmmtools.multistate.multistatesampler:Not enough iterations for online analysis (self.online_analysis_minimum_iterations = 200)
DEBUG:mpiplus.mpiplus:Single node: executing <function MultiStateSampler._online_analysis at 0x2b644346e290>
DEBUG:openmmtools.multistate.multistatesampler:*** ONLINE analysis free energies:
DEBUG:openmmtools.multistate.multistatesampler:        -0.0     1.4     1.5     0.7
DEBUG:openmmtools.utils:Computing online free energy estimate took    0.003s
DEBUG:openmmtools.multistate.multistatesampler:Iteration took 0.171s.
DEBUG:openmmtools.multistate.multistatesampler:Estimated completion in 0:00:02.696136, at Tue Dec 29 12:25:20 2020 (consuming total wall clock time 0:00:03.370171).

Thanks

shitanshu-cell avatar Dec 29 '20 12:12 shitanshu-cell

You'll have to run the python script with your mpi launcher. This depends on your system, but an example is

mpirun -n 4 python myscript.py

andrrizzi avatar Jan 01 '21 19:01 andrrizzi

Hii Thanks alot @andrizzi its working , But i am not getting significant speedup , I dont know why its slow possibly my system (Alanine Dipeptide in vaccum) size is small and due to communication b/w the nodes taking more time or something else i am not sure , How much we can expect in ns/day for such a small system ? I am attaching my log output file when its running using mpirun -n 4 python simulation .py 

WARNING:openmmtools.multistate.multistatereporter:Warning: The openmmtools.multistate API is experimental and may change in future releases
DEBUG:openmmtools.multistate.multistatereporter:Initial checkpoint file automatically chosen as test_traj_5ns_checkpoint.nc
DEBUG:mpiplus.mpiplus:MPI initialized on node 3/4
DEBUG:mpiplus.mpiplus:Node 3/4: waiting for broadcast of <bound method MultiStateReporter.storage_exists of <openmmtools.multistate.multistatereporter.MultiStateReporter object at 0x7f2af4e9d2d0>>
WARNING:openmmtools.multistate.multistatesampler:Warning: The openmmtools.multistate API is experimental and may change in future releases
WARNING:openmmtools.multistate.multistatesampler:Warning: The openmmtools.multistate API is experimental and may change in future releases
WARNING:openmmtools.multistate.multistatereporter:Warning: The openmmtools.multistate API is experimental and may change in future releases
DEBUG:openmmtools.multistate.multistatereporter:Initial checkpoint file automatically chosen as test_traj_5ns_checkpoint.nc
WARNING:openmmtools.multistate.multistatereporter:Warning: The openmmtools.multistate API is experimental and may change in future releases
DEBUG:openmmtools.multistate.multistatereporter:Initial checkpoint file automatically chosen as test_traj_5ns_checkpoint.nc
WARNING:openmmtools.multistate.multistatesampler:Warning: The openmmtools.multistate API is experimental and may change in future releases
WARNING:openmmtools.multistate.multistatereporter:Warning: The openmmtools.multistate API is experimental and may change in future releases
DEBUG:openmmtools.multistate.multistatereporter:Initial checkpoint file automatically chosen as test_traj_5ns_checkpoint.nc
DEBUG:mpiplus.mpiplus:MPI initialized on node 4/4
DEBUG:mpiplus.mpiplus:MPI initialized on node 2/4
DEBUG:mpiplus.mpiplus:Node 4/4: waiting for broadcast of <bound method MultiStateReporter.storage_exists of <openmmtools.multistate.multistatereporter.MultiStateReporter object at 0x7fbb63a37190>>
DEBUG:mpiplus.mpiplus:Node 2/4: waiting for broadcast of <bound method MultiStateReporter.storage_exists of <openmmtools.multistate.multistatereporter.MultiStateReporter object at 0x7fa683702d90>>
DEBUG:mpiplus.mpiplus:MPI initialized on node 1/4
DEBUG:mpiplus.mpiplus:Node 1/4: executing <bound method MultiStateReporter.storage_exists of <openmmtools.multistate.multistatereporter.MultiStateReporter object at 0x7fa23bc3b590>>
DEBUG:mpiplus.mpiplus:Node 1/4: waiting for broadcast of <bound method MultiStateReporter.storage_exists of <openmmtools.multistate.multistatereporter.MultiStateReporter object at 0x7fa23bc3b590>>
DEBUG:mpiplus.mpiplus:Node 4/4: waiting for barrier after <function MultiStateSampler._initialize_reporter at 0x7fbb78b39320>
DEBUG:mpiplus.mpiplus:Node 1/4: executing <function ReplicaExchangeSampler._display_citations at 0x7fa250c99dd0>
DEBUG:mpiplus.mpiplus:Node 1/4: executing <function MultiStateSampler._display_citations at 0x7fa250d3d050>
DEBUG:mpiplus.mpiplus:Node 1/4: executing <function MultiStateSampler._initialize_reporter at 0x7fa250d3d320>
DEBUG:mpiplus.mpiplus:Node 2/4: waiting for barrier after <function MultiStateSampler._initialize_reporter at 0x7fa69378e320>
DEBUG:mpiplus.mpiplus:Node 3/4: waiting for barrier after <function MultiStateSampler._initialize_reporter at 0x7f2b09f9c320>
DEBUG:openmmtools.multistate.multistatereporter:Serialized state thermodynamic_states/0 is  3241B | 3.165KB | 0.003MB
DEBUG:openmmtools.utils:Storing thermodynamic states took    0.094s
DEBUG:openmmtools.multistate.multistatesampler:Storing general ReplicaExchange options...
DEBUG:mpiplus.mpiplus:Node 1/4: executing <function MultiStateSampler._report_iteration at 0x7fa250d3d440>
DEBUG:mpiplus.mpiplus:Node 1/4: executing <function MultiStateSampler._report_iteration_items at 0x7fa250d3d710>
DEBUG:openmmtools.utils:Storing sampler states took    0.088s
DEBUG:openmmtools.utils:Writing iteration information to storage took    0.183s
DEBUG:mpiplus.mpiplus:Node 1/4: waiting for barrier after <function MultiStateSampler._initialize_reporter at 0x7fa250d3d320>
DEBUG:mpiplus.mpiplus:Node 1/4: execute _compute_replica_energies(0)
DEBUG:mpiplus.mpiplus:Node 2/4: execute _compute_replica_energies(1)
DEBUG:mpiplus.mpiplus:Node 3/4: execute _compute_replica_energies(2)
DEBUG:mpiplus.mpiplus:Node 4/4: execute _compute_replica_energies(3)
DEBUG:mpiplus.mpiplus:Node 1/4: sending results of _compute_replica_energies to 0
DEBUG:mpiplus.mpiplus:Node 3/4: sending results of _compute_replica_energies to 0
DEBUG:mpiplus.mpiplus:Node 4/4: sending results of _compute_replica_energies to 0
DEBUG:mpiplus.mpiplus:Node 2/4: sending results of _compute_replica_energies to 0
DEBUG:openmmtools.utils:Computing energy matrix took    0.587s
DEBUG:openmmtools.utils:Computing energy matrix took    0.587s
DEBUG:openmmtools.utils:Computing energy matrix took    0.587s
DEBUG:openmmtools.utils:Computing energy matrix took    0.587s
DEBUG:mpiplus.mpiplus:Node 1/4: executing <bound method MultiStateReporter.write_energies of <openmmtools.multistate.multistatereporter.MultiStateReporter object at 0x7fa23bc3b590>>
DEBUG:openmmtools.multistate.multistatesampler:********************************************************************************
DEBUG:openmmtools.multistate.multistatesampler:********************************************************************************
DEBUG:openmmtools.multistate.multistatesampler:********************************************************************************
DEBUG:openmmtools.multistate.multistatesampler:Iteration 1/100
DEBUG:openmmtools.multistate.multistatesampler:Iteration 1/100
DEBUG:openmmtools.multistate.multistatesampler:Iteration 1/100
DEBUG:openmmtools.multistate.multistatesampler:********************************************************************************
DEBUG:openmmtools.multistate.multistatesampler:********************************************************************************
DEBUG:openmmtools.multistate.multistatesampler:********************************************************************************
DEBUG:mpiplus.mpiplus:Node 3/4: waiting for broadcast of <function ReplicaExchangeSampler._mix_replicas at 0x7f2b09f64dd0>
DEBUG:mpiplus.mpiplus:Node 4/4: waiting for broadcast of <function ReplicaExchangeSampler._mix_replicas at 0x7fbb78b01dd0>
DEBUG:mpiplus.mpiplus:Node 2/4: waiting for broadcast of <function ReplicaExchangeSampler._mix_replicas at 0x7fa693756dd0>
DEBUG:openmmtools.multistate.multistatesampler:********************************************************************************
DEBUG:openmmtools.multistate.multistatesampler:Iteration 1/100
DEBUG:openmmtools.multistate.multistatesampler:********************************************************************************
DEBUG:mpiplus.mpiplus:Node 1/4: executing <function ReplicaExchangeSampler._mix_replicas at 0x7fa250d05dd0>
DEBUG:openmmtools.multistate.replicaexchange:Mixing replicas...
DEBUG:openmmtools.utils:Mixing of replicas took    0.003s
DEBUG:openmmtools.multistate.replicaexchange:Accepted 512/512 attempted swaps (100.0%)
DEBUG:mpiplus.mpiplus:Node 1/4: waiting for broadcast of <function ReplicaExchangeSampler._mix_replicas at 0x7fa250d05dd0>
DEBUG:openmmtools.multistate.multistatesampler:Propagating all replicas...
DEBUG:openmmtools.multistate.multistatesampler:Propagating all replicas...
DEBUG:openmmtools.multistate.multistatesampler:Propagating all replicas...
DEBUG:openmmtools.multistate.multistatesampler:Propagating all replicas...
DEBUG:mpiplus.mpiplus:Node 1/4: execute _propagate_replica(0)
DEBUG:mpiplus.mpiplus:Node 3/4: execute _propagate_replica(2)
DEBUG:mpiplus.mpiplus:Node 2/4: execute _propagate_replica(1)
DEBUG:mpiplus.mpiplus:Node 4/4: execute _propagate_replica(3)
DEBUG:mpiplus.mpiplus:Node 1/4: sending results of _propagate_replica to 0
DEBUG:mpiplus.mpiplus:Node 3/4: sending results of _propagate_replica to 0
DEBUG:mpiplus.mpiplus:Node 2/4: sending results of _propagate_replica to 0
DEBUG:mpiplus.mpiplus:Node 2/4: execute _get_replica_move_statistics(1)
DEBUG:mpiplus.mpiplus:Node 3/4: execute _get_replica_move_statistics(2)
DEBUG:mpiplus.mpiplus:Node 4/4: execute _get_replica_move_statistics(3)
DEBUG:mpiplus.mpiplus:Node 1/4: execute _get_replica_move_statistics(0)
DEBUG:mpiplus.mpiplus:Node 1/4: allgather results of _get_replica_move_statistics
DEBUG:mpiplus.mpiplus:Node 4/4: allgather results of _get_replica_move_statistics
DEBUG:mpiplus.mpiplus:Node 2/4: allgather results of _get_replica_move_statistics
DEBUG:mpiplus.mpiplus:Node 3/4: allgather results of _get_replica_move_statistics
DEBUG:openmmtools.utils:Propagating all replicas took    0.944s
DEBUG:openmmtools.utils:Propagating all replicas took    0.944s
DEBUG:openmmtools.utils:Propagating all replicas took    0.944s
DEBUG:openmmtools.utils:Propagating all replicas took    0.944s
DEBUG:mpiplus.mpiplus:Node 2/4: execute _compute_replica_energies(1)
DEBUG:mpiplus.mpiplus:Node 3/4: execute _compute_replica_energies(2)
DEBUG:mpiplus.mpiplus:Node 1/4: execute _compute_replica_energies(0)
DEBUG:mpiplus.mpiplus:Node 4/4: execute _compute_replica_energies(3)
DEBUG:mpiplus.mpiplus:Node 2/4: sending results of _compute_replica_energies to 0
DEBUG:mpiplus.mpiplus:Node 4/4: sending results of _compute_replica_energies to 0
DEBUG:mpiplus.mpiplus:Node 3/4: sending results of _compute_replica_energies to 0
DEBUG:mpiplus.mpiplus:Node 1/4: sending results of _compute_replica_energies to 0
DEBUG:openmmtools.utils:Computing energy matrix took    0.005s
DEBUG:openmmtools.utils:Computing energy matrix took    0.005s
DEBUG:openmmtools.utils:Computing energy matrix took    0.005s
DEBUG:openmmtools.multistate.multistatesampler:Not enough iterations for online analysis (self.online_analysis_minimum_iterations = 200)
DEBUG:openmmtools.multistate.multistatesampler:Not enough iterations for online analysis (self.online_analysis_minimum_iterations = 200)
DEBUG:openmmtools.multistate.multistatesampler:Not enough iterations for online analysis (self.online_analysis_minimum_iterations = 200)
DEBUG:mpiplus.mpiplus:Node 4/4: waiting for broadcast of <function MultiStateSampler._online_analysis at 0x7fbb78b3d290>
DEBUG:openmmtools.utils:Computing energy matrix took    0.005s
DEBUG:mpiplus.mpiplus:Node 2/4: waiting for broadcast of <function MultiStateSampler._online_analysis at 0x7fa693792290>
DEBUG:mpiplus.mpiplus:Node 3/4: waiting for broadcast of <function MultiStateSampler._online_analysis at 0x7f2b09fa0290>
DEBUG:mpiplus.mpiplus:Node 1/4: executing <function MultiStateSampler._report_iteration at 0x7fa250d3d440>
DEBUG:mpiplus.mpiplus:Node 1/4: executing <function MultiStateSampler._report_iteration_items at 0x7fa250d3d710>
DEBUG:openmmtools.utils:Storing sampler states took    0.001s
DEBUG:openmmtools.utils:Writing iteration information to storage took    0.006s
DEBUG:openmmtools.multistate.multistatesampler:Not enough iterations for online analysis (self.online_analysis_minimum_iterations = 200)
DEBUG:mpiplus.mpiplus:Node 1/4: executing <function MultiStateSampler._online_analysis at 0x7fa250d41290>
DEBUG:openmmtools.multistate.multistatereporter:Creating new NetCDF variable f_k with parameters: {'dtype': dtype('float64'), 'dims': ('dim_size4',), 'chunksizes': (4,)}
DEBUG:openmmtools.multistate.multistatereporter:Creating new NetCDF variable free_energy with parameters: {'dtype': <class 'numpy.float64'>, 'dims': ('dim_size2',), 'chunksizes': (2,)}
DEBUG:openmmtools.multistate.multistatereporter:Creating new NetCDF variable f_k_history with parameters: {'dtype': dtype('float64'), 'dims': ('iteration', 'dim_size4'), 'chunksizes': (1, 4)}
DEBUG:openmmtools.multistate.multistatereporter:Creating new NetCDF variable free_energy_history with parameters: {'dtype': <class 'numpy.float64'>, 'dims': ('iteration', 'dim_size2'), 'chunksizes': (1, 2)}
DEBUG:openmmtools.multistate.multistatesampler:*** ONLINE analysis free energies:
DEBUG:openmmtools.multistate.multistatesampler:        -0.0    -0.0    -0.1    -1.8
DEBUG:openmmtools.utils:Computing online free energy estimate took    0.003s
DEBUG:mpiplus.mpiplus:Node 1/4: waiting for broadcast of <function MultiStateSampler._online_analysis at 0x7fa250d41290>
DEBUG:openmmtools.multistate.multistatesampler:Iteration took 0.962s.
DEBUG:openmmtools.multistate.multistatesampler:Iteration took 0.963s.
DEBUG:openmmtools.multistate.multistatesampler:Iteration took 0.963s.
DEBUG:openmmtools.multistate.multistatesampler:Iteration took 0.963s.
DEBUG:openmmtools.multistate.multistatesampler:Estimated completion in 0:01:35.220806, at Wed Jan  6 12:42:40 2021 (consuming total wall clock time 0:01:36.182632).
DEBUG:openmmtools.multistate.multistatesampler:Estimated completion in 0:01:35.331601, at Wed Jan  6 12:42:40 2021 (consuming total wall clock time 0:01:36.294546).
DEBUG:openmmtools.multistate.multistatesampler:Estimated completion in 0:01:35.329665, at Wed Jan  6 12:42:40 2021 (consuming total wall clock time 0:01:36.292591).
DEBUG:openmmtools.multistate.multistatesampler:Estimated completion in 0:01:35.332144, at Wed Jan  6 12:42:40 2021 (consuming total wall clock time 0:01:36.295094).
DEBUG:openmmtools.multistate.multistatesampler:********************************************************************************
DEBUG:openmmtools.multistate.multistatesampler:Iteration 2/100
DEBUG:openmmtools.multistate.multistatesampler:********************************************************************************
DEBUG:openmmtools.multistate.multistatesampler:********************************************************************************
DEBUG:openmmtools.multistate.multistatesampler:********************************************************************************
DEBUG:openmmtools.multistate.multistatesampler:********************************************************************************
DEBUG:openmmtools.multistate.multistatesampler:Iteration 2/100
DEBUG:openmmtools.multistate.multistatesampler:Iteration 2/100
DEBUG:openmmtools.multistate.multistatesampler:Iteration 2/100
DEBUG:mpiplus.mpiplus:Node 1/4: executing <function ReplicaExchangeSampler._mix_replicas at 0x7fa250d05dd0>
DEBUG:openmmtools.multistate.multistatesampler:********************************************************************************
DEBUG:openmmtools.multistate.multistatesampler:********************************************************************************
DEBUG:openmmtools.multistate.multistatesampler:********************************************************************************
DEBUG:openmmtools.multistate.replicaexchange:Mixing replicas...
DEBUG:mpiplus.mpiplus:Node 3/4: waiting for broadcast of <function ReplicaExchangeSampler._mix_replicas at 0x7f2b09f64dd0>
DEBUG:mpiplus.mpiplus:Node 4/4: waiting for broadcast of <function ReplicaExchangeSampler._mix_replicas at 0x7fbb78b01dd0>
DEBUG:mpiplus.mpiplus:Node 2/4: waiting for broadcast of <function ReplicaExchangeSampler._mix_replicas at 0x7fa693756dd0>
DEBUG:openmmtools.utils:Mixing of replicas took    0.000s
DEBUG:openmmtools.multistate.replicaexchange:Accepted 204/512 attempted swaps (39.8%)
DEBUG:mpiplus.mpiplus:Node 1/4: waiting for broadcast of <function ReplicaExchangeSampler._mix_replicas at 0x7fa250d05dd0>
DEBUG:openmmtools.multistate.multistatesampler:Propagating all replicas...
DEBUG:openmmtools.multistate.multistatesampler:Propagating all replicas...
DEBUG:openmmtools.multistate.multistatesampler:Propagating all replicas...
DEBUG:openmmtools.multistate.multistatesampler:Propagating all replicas...
DEBUG:mpiplus.mpiplus:Node 1/4: execute _propagate_replica(0)
DEBUG:mpiplus.mpiplus:Node 3/4: execute _propagate_replica(2)
DEBUG:mpiplus.mpiplus:Node 2/4: execute _propagate_replica(1)
DEBUG:mpiplus.mpiplus:Node 4/4: execute _propagate_replica(3)
DEBUG:mpiplus.mpiplus:Node 1/4: sending results of _propagate_replica to 0
DEBUG:mpiplus.mpiplus:Node 2/4: sending results of _propagate_replica to 0
DEBUG:mpiplus.mpiplus:Node 3/4: sending results of _propagate_replica to 0
DEBUG:mpiplus.mpiplus:Node 4/4: sending results of _propagate_replica to 0
DEBUG:mpiplus.mpiplus:Node 2/4: execute _get_replica_move_statistics(1)
DEBUG:mpiplus.mpiplus:Node 3/4: execute _get_replica_move_statistics(2)
DEBUG:mpiplus.mpiplus:Node 4/4: execute _get_replica_move_statistics(3)
DEBUG:mpiplus.mpiplus:Node 1/4: execute _get_replica_move_statistics(0)
DEBUG:mpiplus.mpiplus:Node 1/4: allgather results of _get_replica_move_statistics
DEBUG:mpiplus.mpiplus:Node 3/4: allgather results of _get_replica_move_statistics
DEBUG:mpiplus.mpiplus:Node 4/4: allgather results of _get_replica_move_statistics
DEBUG:mpiplus.mpiplus:Node 2/4: allgather results of _get_replica_move_statistics
DEBUG:openmmtools.utils:Propagating all replicas took    0.289s
DEBUG:openmmtools.utils:Propagating all replicas took    0.289s
DEBUG:openmmtools.utils:Propagating all replicas took    0.289s
DEBUG:openmmtools.utils:Propagating all replicas took    0.289s
DEBUG:mpiplus.mpiplus:Node 2/4: execute _compute_replica_energies(1)
DEBUG:mpiplus.mpiplus:Node 1/4: execute _compute_replica_energies(0)
DEBUG:mpiplus.mpiplus:Node 3/4: execute _compute_replica_energies(2)
DEBUG:mpiplus.mpiplus:Node 4/4: execute _compute_replica_energies(3)
DEBUG:mpiplus.mpiplus:Node 4/4: sending results of _compute_replica_energies to 0
DEBUG:mpiplus.mpiplus:Node 3/4: sending results of _compute_replica_energies to 0
DEBUG:mpiplus.mpiplus:Node 2/4: sending results of _compute_replica_energies to 0
DEBUG:mpiplus.mpiplus:Node 1/4: sending results of _compute_replica_energies to 0
DEBUG:openmmtools.utils:Computing energy matrix took    0.018s
DEBUG:openmmtools.utils:Computing energy matrix took    0.018s
DEBUG:openmmtools.utils:Computing energy matrix took    0.018s
DEBUG:openmmtools.multistate.multistatesampler:Not enough iterations for online analysis (self.online_analysis_minimum_iterations = 200)
DEBUG:openmmtools.multistate.multistatesampler:Not enough iterations for online analysis (self.online_analysis_minimum_iterations = 200)
DEBUG:openmmtools.multistate.multistatesampler:Not enough iterations for online analysis (self.online_analysis_minimum_iterations = 200)
DEBUG:mpiplus.mpiplus:Node 4/4: waiting for broadcast of <function MultiStateSampler._online_analysis at 0x7fbb78b3d290>
DEBUG:mpiplus.mpiplus:Node 2/4: waiting for broadcast of <function MultiStateSampler._online_analysis at 0x7fa693792290>
DEBUG:mpiplus.mpiplus:Node 3/4: waiting for broadcast of <function MultiStateSampler._online_analysis at 0x7f2b09fa0290>
DEBUG:openmmtools.utils:Computing energy matrix took    0.018s
DEBUG:mpiplus.mpiplus:Node 1/4: executing <function MultiStateSampler._report_iteration at 0x7fa250d3d440>
DEBUG:mpiplus.mpiplus:Node 1/4: executing <function MultiStateSampler._report_iteration_items at 0x7fa250d3d710>
DEBUG:openmmtools.utils:Storing sampler states took    0.001s
DEBUG:openmmtools.utils:Writing iteration information to storage took    0.005s
DEBUG:openmmtools.multistate.multistatesampler:Not enough iterations for online analysis (self.online_analysis_minimum_iterations = 200)
DEBUG:mpiplus.mpiplus:Node 1/4: executing <function MultiStateSampler._online_analysis at 0x7fa250d41290>
DEBUG:openmmtools.multistate.multistatesampler:*** ONLINE analysis free energies:
DEBUG:openmmtools.multistate.multistatesampler:        -0.0    -0.0    -0.3    -3.0
DEBUG:openmmtools.utils:Computing online free energy estimate took    0.001s
DEBUG:mpiplus.mpiplus:Node 1/4: waiting for broadcast of <function MultiStateSampler._online_analysis at 0x7fa250d41290>
DEBUG:openmmtools.multistate.multistatesampler:Iteration took 0.314s.
DEBUG:openmmtools.multistate.multistatesampler:Iteration took 0.314s.
DEBUG:openmmtools.multistate.multistatesampler:Iteration took 0.314s.
DEBUG:openmmtools.multistate.multistatesampler:Iteration took 0.314s.
DEBUG:openmmtools.multistate.multistatesampler:Estimated completion in 0:01:02.520975, at Wed Jan  6 12:42:07 2021 (consuming total wall clock time 0:01:03.796914).
DEBUG:openmmtools.multistate.multistatesampler:Estimated completion in 0:01:02.574026, at Wed Jan  6 12:42:07 2021 (consuming total wall clock time 0:01:03.851047).
DEBUG:openmmtools.multistate.multistatesampler:Estimated completion in 0:01:02.575054, at Wed Jan  6 12:42:07 2021 (consuming total wall clock time 0:01:03.852096).
DEBUG:openmmtools.multistate.multistatesampler:Estimated completion in 0:01:02.575287, at Wed Jan  6 12:42:07 2021 (consuming total wall clock time 0:01:03.852334).

Thanks

shitanshu-cell avatar Jan 06 '21 07:01 shitanshu-cell