Install Request: JAGURS on Myriad [IN06022091]

[balston]

https://github.com/jagurs-admin/jagurs/tree/master

Dependencies on PROJ.4, netCDF and netCDF-Fortran amongst others. We don't currently have a suitable set of them.

[balston]

We now have netCDF and netCDF-FORTRAN that can be used to build JAGURS.

[balston]

Existing PROJ.4 build should be OK I think.

[balston]

Need these modules I think:

module load beta-modules
module -f unload compilers mpi gcc-libs
module load gcc-libs/10.2.0
module load compilers/gnu/10.2.0
module load numactl/2.0.12
module load binutils/2.36.1/gnu-10.2.0
module load ucx/1.9.0/gnu-10.2.0
module load mpi/openmpi/4.0.5/gnu-10.2.0
module load sqlite/3.36.0/gnu-10.2.0
module load proj.4/8.1.1/gnu-10.2.0
module load fftw/3.3.9/gnu-10.2.0
module load hdf/5-1.10.6/gnu-10.2.0
module load netcdf/4.9.2/gnu-10.2.0
module load netcdf-fortran/4.6.1/gnu-10.2.0

[balston]

I will test a build using the Makfile.gfotran and the above set of modules tomorrow.

[balston]

Trying a build using ./jagurs-JAGURS-D_V0516/src/Makefile.gfortran with at least the following changes:

sed -i.bak1 '/^PROJ4_DIR/s|$(HOME)/local|/shared/ucl/apps/PROJ.4//8.1.1|' Makefile

sed -i.bak2 '/^FFTW3_INCLUDE_DIR/s|$(HOME)/local/include|$(FFTWINCLUDE)|' Makefile

sed -i.bak3 '/^NETCDF/s|$(HOME)/local|/shared/ucl/apps/NetCDF/4.9.2-gnu-10.2.0|' Makefile

[balston]

Getting errors that imply the version of PROJ is not recent enough. Need to build a newer version first. Latest version is 9.2.0

[balston]

Building PROJ 9.2.0.

[balston]

Build of PROJ has failed during the installation step - hasn't picked up the install location correctly. This due to the switch from GNU configure to CMAKE for the build process. Will sort and re-run.

[balston]

That was actually my silly mistake - I had forgotten to add:

-D CMAKE_INSTALL_PREFIX=${INSTALL_PREFIX}

to the CMAKE config step.

[balston]

PROJ 9.2.0 is now built but it turns out that I don't need the new version but a very only one! That is ready too now.

[balston]

I now have a set of modules that JAGURS can build with:

module load beta-modules
module -f unload compilers mpi gcc-libs
module load gcc-libs/10.2.0
module load compilers/gnu/10.2.0
module load numactl/2.0.12
module load binutils/2.36.1/gnu-10.2.0
module load ucx/1.9.0/gnu-10.2.0
module load mpi/openmpi/4.0.5/gnu-10.2.0
module load sqlite/3.36.0/gnu-10.2.0
module load proj.4/4.9.1
module load fftw/3.3.9/gnu-10.2.0
module load hdf/5-1.10.6/gnu-10.2.0
module load netcdf/4.9.2/gnu-10.2.0
module load netcdf-fortran/4.6.1/gnu-10.2.0
module load szip/2.1

I'm using the template Makefile.gfortran:

rm Makefile
cp Makefile.gfortran Makefile

with the following mods:

sed -i.bak1 '/^PROJ4_DIR/s|$(HOME)/local|/shared/ucl/apps/PROJ.4/4.9.1|' Makefile

sed -i.bak2 '/^FFTW3_INCLUDE_DIR/s|$(HOME)/local/include|$(FFTWINCLUDE)|' Makefile

sed -i.bak3 '/^NETCDF/s|$(HOME)/local|/shared/ucl/apps/NetCDF-fortran/4.6.1-gnu-10.2.0|' Makefile

The make runs to completion without errors. I've no idea if it works - that's the next step. See if it runs with some sample data.

[balston]

I've founds some sample date and it includes a generic job script. Getting it onto Myriad now.

[balston]

I ran the example data using a modified form of the MPI/OpenMP hybrid example job script:

https://www.rc.ucl.ac.uk/docs/Example_Jobscripts/#hybrid-mpi-plus-openmp-jobscript-example

and it seems to work correctly.

I've now got a build script ready to do an installation in /shared/ucl/apps.

[balston]

build script run as ccspapp:

cd /shared/ucl/apps/build_scripts
module -f unload compilers mpi gcc-libs
./JAGURS-V0516_install 2>&1 | tee ~/Software/JAGURS-V0516_install.log

with no errors. Now to produce a module file and module bundle.

[balston]

module file and module bundle created and uploaded to Myriad. Test job submitted.

[balston]

Seems to be working OK. Using a job script like this:

#!/bin/bash -l

# Run the MPI version of JAGURS on Myriad with sample data from here:
#
# https://www.dropbox.com/sh/oiypdwufu31za1g/AADQ2QBRnu5BShO4Gh4989S4a?dl=0

# Request 1 hour of wallclock time (format hours:minutes:seconds).
#$ -l h_rt=1:0:0

# Request 4 gigabyte of RAM per core (must be an integer)
#$ -l mem=4G

# Request 15 gigabytes of TMPDIR space per node (default is 10 GB - remove if cluster is diskless)
#$ -l tmpfs=15G

# Set the name of the job.
#$ -N JAGURS_MPI_1

# Select the MPI parallel environment and 4 cores.
#$ -pe mpi 4

# Set the working directory to somewhere in your scratch space. 
# This directory must exist.
#$ -wd /home/ccaabaa/Scratch/JAGURS_examples

export OMP_NUM_THREADS=2

module load beta-modules
module -f unload compilers mpi gcc-libs
module load jagurs/mpi/recommended

s=/home/ccaabaa/Scratch/Software/JAGURS/Sample_data
mkdir run_$JOB_ID
cd run_$JOB_ID
cp $s/bathy.grd .
cp $s/disp.Uz.grd .
cp $s/gridfile.dat .
cp $s/test_tgs.dat .
cp $s/tsun.par .

gerun jagurs par=tsun.par

This what the sample data needed.

[balston]

Users informed.