Support building HOOMD-Blue in RET

[paklui]

Is your feature request related to a problem? Please describe. To enable RET to build HOOMD-Blue and its dependencies in Miniconda3

Describe the solution you'd like Using the installation steps and implement in src/cmd and src/cmd_utils. Details of the installation steps to setup conda and its package requirements for HOOMD-Blue is described below.

Describe alternatives you've considered Consider to install directly via bash scripts below.

Additional context ret.log for install: ret-hoomd-install.log

Installation on HOOMD-Blue web site: https://hoomd-blue.readthedocs.io/en/stable/installation.html#compiling-from-source

Current installation instruction for HIP port:

Installation instructions for HOOMD-blue -- HIP branch on AMD servers
=====================================================================

12/02/2019 12/10/2019 12/13/2019 12/19/2019
[email protected], [email protected], [email protected]

Build Open MPI with ROCm-enabled UCX support

while HOOMD-blue (HIP branch) is under development, the documentation will be kept up to date here
https://hoomd-blue.readthedocs.io/en/hip/installation.html

(see there for dependencies, unless superseded by this doc)

see here for latest progress
https://github.com/glotzerlab/hoomd-blue/pull/541

# for multi-node MPI https://github.com/openucx/ucx/wiki/OpenMPI-and-OpenSHMEM-installation-with-UCX
# export LD_LIBRARY_PATH=/home/rocmhoomds/ompi/ompiinstall/lib:$LD_LIBRARY_PATH

# Checkout ROCm dependencies
sudo apt-get install hipsparse
sudo apt-get install rocfft
sudo apt-get install hipcub # removed -y
sudo apt-get install rocrand # removed -y
sudo apt-get install rocthrust # removed -y
sudo apt-get install roctracer-dev # new

# there may be a conflict between NVIDIA thrust and rocThrust if the NVIDIA CUDA toolkit is simultaneously installed in /usr.
# (in that case uninstall the NVIDIA toolkit)

# host compilers I tested the HIP branch with: gcc-7.4 and clang++-10

# Install gcc 7.4.0 from source and apt-get
# try running ./contrib/download_prerequisites.sh from the gcc source dir. It worked for me (for the current version of gcc though (gcc-4.7)

# checkout HOOMD-blue (hip branch)
git clone https://github.com/glotzerlab/hoomd-blue
cd hoomd-blue
git checkout next #hip
git submodule update --init

# set up python environment
cd $HOME
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
sh Miniconda3-latest-Linux-x86_64.sh
reply yes to Do you wish the installer to initialize Miniconda3 by running conda init

# close and reopen shell (login logout)
conda install -y -c anaconda python=3.8
conda update -y --all
conda create -y -n myenv python=3.8 # or rename myenv to something like hoomd-single-mpi, to differentiate and switch between builds
echo "conda activate myenv" >> ~/.bashrc
source ~/.bashrc
conda activate myenv

# newest version >= 3.14.0
conda install -y cmake 
conda install -y numpy
conda install -y pybind11
conda install -y eigen
conda install -y -c conda-forge cereal 
conda install -y -c conda-forge signac-flow

# build hoomd in single precision (md component only)
mkdir build; cd build
echo "export CMAKE_PREFIX_PATH=$CONDA_PREFIX" >> ~/.bashrc

# several build alternatives hexagon and depletion need -D SINGLE_PRECISION=OFF 
cmake -D PYTHON_EXECUTABLE=`which python3` -D ENABLE_GPU=ON -D SINGLE_PRECISION=ON -D ENABLE_MPI=OFF ../hoomd-blue/
cmake -D PYTHON_EXECUTABLE=`which python3` -D ENABLE_GPU=ON -D SINGLE_PRECISION=OFF -D ENABLE_MPI=OFF ../hoomd-blue/
cmake -D PYTHON_EXECUTABLE=`which python3` -D ENABLE_GPU=ON -D SINGLE_PRECISION=OFF -D ENABLE_MPI=ON ../hoomd-blue/
cmake -D PYTHON_EXECUTABLE=`which python3` -D ENABLE_GPU=ON -D SINGLE_PRECISION=ON -D ENABLE_MPI=ON ../hoomd-blue/ 

# Note: For ROCm-enabled MPI, use the ENABLE_MPI_CUDA=ON flag
cmake -D PYTHON_EXECUTABLE=`which python3` -D ENABLE_GPU=ON -D SINGLE_PRECISION=ON -D ENABLE_MPI=ON -D ENABLE_MPI_CUDA=ON -D CMAKE_INSTALL_PREFIX=${HOME}/miniconda3/envs/myenv -D PYTHON_SITE_INSTALL_DIR=${HOME}/miniconda3/lib/python3.8/site-packages/hoomd ../hoomd-blue/

make -j 64 install

# clone hoomd-benchmarks and dependencies (next branch)
cd $HOME
git clone https://[email protected]/glotzerlab/hoomd-benchmarks
cd hoomd-benchmarks
git checkout next

# run LJ liquid benchmark

# default size 1M particles
mpirun -np 1 python project.py run lj_liquid-benchmark-gpu_np1 # 1 GPU
mpirun -np 4 python project.py run lj_liquid-benchmark-gpu_np1 # 4 GPUs
mpirun -np 8 python project.py run lj_liquid-benchmark-cpu_np8 # 8 CPU cores

# custom size N=50^3 particles
cd lj_liquid
python init.py 50
cd ..
python project.py run -o lj_liquid-equilibrate # equilibrate
python project.py run -o lj_liquid-benchmark-gpu_np1 -f n 50

# or, run **all** benchmarks (lj_liquid, patchy_protein, hexagon, microsphere, quasicrystal, depletion, spce)
python project.py run

# retrieve performance (TPS or MPS=N*TPS) from data base
signac document -f benchmark lj_liquid

# - due to a bug in HIP (likely related to https://github.com/ROCm-Developer-Tools/HIP/pull/1698) this command may abort after
#   some time with a message saying it could not find a kernel. This is likely because a shared library is lazily loaded
#   in between GPU kernel calls. In that case, execute benchmarks seprately

# - if a benchmark fails with an unknown HSA error, or one like the following, 
#   ### HCC STATUS_CHECK Error: HSA_STATUS_ERROR_INVALID_ISA (0x100f) at file:mcwamp_hsa.cpp line:1191
#   this is probably due to the issue addressed in
#   https://github.com/ROCm-Developer-Tools/HIP/pull/1676


Issue with PyThread_tss_alloc, because OMPI is not build with thread support?