Add example scripts for PBS cluster manager

[Mugglea]

Dear sir, While I was installing the informME, I encountered a problem. Could you give me any advice? Thank you very much.

#path
IPATH="/**/program/informME-master"
MATLICE="/**/program/matlab/dir_install/licenses/license_standalone.lic"
PATH1="/**/program/informME-master/others/eigen/install"
PATH2="/**/program/informME-master/others/mpfr-3.1.6"

Firstly, I finished the installing steps of 1, 2 and 4, and then I ran the informME-master/SubmissionScripts/MexSource/runCompileMEX.sh. An error occurred:

qsub: No default queue specified MSG=cannot locate queue
for COMPILE submitted on Wed Dec 20 20:51:12 CST 2017

So I gave it the queue id of "-q Biocc.ZY" and it liked this:

# Run compile.sh script
JOB=`qsub << EOJ
#
#$ -wd ${WORKDIR}
#$ -j y
#$ -r y
#$ -S /bin/bash
#$ -N INFM_CMP
#$ -q Biocc.ZY    ##########here
#$ -l mem_free=15G
#$ -l h_vmem=15G
#$ -l h_fsize=100G
#$ -o ${SOUTDIR}
# 
${SCRIPTDIR}compile.sh "${EIGEN}" "${MPFRINCL}" "${MPFRLIB}" "${MATLICE}"
EOJ
`
echo "${JOB} for COMPILE submitted on `date`"

But it made no sense. Therefore, I submited the job with PBS file named "install.pbs" by the qsub command. As follows:

#!/bin/bash
##############################################################
# the name of job
#PBS -N output-7

# set the queue
#PBS -q Biocc.ZY

# nodes: number of nodes requested by job
# ppn  : the number of processors per node requested by job
#PBS -l nodes=1:ppn=8

#PBS -l walltime=1000:00:00
##############################################################
IPATH="/**/program/informME-master"
MATLICE="/**/program/matlab/dir_install/licenses/license_standalone.lic"
PATH1="/**/program/informME-master/others/eigen/install"
PATH2="/**/program/informME-master/others/mpfr-3.1.6"

# Non-user specified directories
# (trailing slash)
EIGEN="$PATH1/"
MPFRINCL="$PATH2/include/"
MPFRLIB="$PATH2/lib/"
SCRIPTDIR="$IPATH/SubmissionScripts/MexSource/"
SOUTDIR="$IPATH/SubmissionScripts/STDouts/"
WORKDIR="$IPATH/MexSource/"

${SCRIPTDIR}compile.sh "${EIGEN}" "${MPFRINCL}" "${MPFRLIB}" "${MATLICE}"

The error is like this:

#tips for error
Error identifier: MATLAB:UndefinedFunctionError message: Undefined function 'compile' for input arguments of type 'char'.

#output file 
starting command: compile('/**/program/informME-master/others/eigen/install/','/**/program/informME-master/others/mpfr-3.1.6/include/','/**/program/informME-master/others/mpfr-3.1.6/lib/');

                            < M A T L A B (R) >
                  Copyright 1984-2015 The MathWorks, Inc.
                  R2015b (8.6.0.267246) 64-bit (glnxa64)
                              August 20, 2015


To get started, type one of these: helpwin, helpdesk, or demo.
For product information, visit www.mathworks.com.

Job Running
error in MATLAB
tic;compile('/**/program/informME-master/others/eigen/install/','/**/program/informME-master/others/mpfr-3.1.6/include/','/**/program/informME-master/others/mpfr-3.1.6/lib/');toc;

Any suggestions would be very helpful. Thank you very much.

[Mugglea]

I have found the solution. "cd /xxx/program/informME-master/MexSource" should be added to the "install.pbs" . But another problem occured.

Error identifier: MATLAB:mex:ErrorError message: /**/program/informME-master/MexSource/computeZ.cpp:22:20: fatal error: mpreal.h: No such file or directory
 #include "mpreal.h"
                    ^
compilation terminated.

I will check it tomorrow. Thank you.

[GarrettJenkinson]

Hello and thank you for your interest in our pipeline.

I think you are running into problems because PBS is subtlely different from SGE, and the example submission scripts are designed for SGE. See here for a comparison: https://kb.iu.edu/d/avgl which demonstrates the problem you ran into where our '-wd' flag to set the working directory does not exist in PBS, and so you had to add a change directory command to your script. The version on master, v0.2.2, has only been tested on SGE, so you are blazing a new trail here.

The new error you are getting is due to the C++ header file "mpreal.h" not being found in the inclusion path. You can download this file here: http://www.holoborodko.com/pavel/mpfr/ and put it in your eigen folder, and the compiler should find it (and you won't need root permissions to put it in a default inclusion path).

Since you have the added difficulty of using PBS, let me propose another option for you: try v0.3.0 that is currently a pull request in this repository https://github.com/GarrettJenkinson/informME/tree/v0.3.0. Jordi and I have been working for months on this total revamp of the software, with one of the main goals being ease of installation. I wrote an interactive installation script that will automatically do many of the things that you are having to do manually in v0.2.2. The version v0.3.0 has been tested on SLURM and SGE clusters, and we are quite happy with it. There might be some minor tweaks coming to the Rscripts postprocessing code before we merge that branch to master and tag it as a release, but you could always just use the v0.2.2 Rscripts for now. Otherwise, I consider the v0.3.0 branch to be ready for use, and it will be much easier to get up and running, especially for someone who has the added complexity of figuring out a different cluster submission system (i.e., PBS).

Let me know if you have any further questions or problems.

[Mugglea]

Hi, Thank you very much for your help. I will use the v0.3.0, because there are so many jobs I need to submit.

You have done a awesome job, and I feel honoured to use your tools. I'm interested in how the environment affects biological processes, and your group have made a great contribution to this direction. Thank you for the great effort you have made to develop and maintain this pipeline.

[GarrettJenkinson]

Thank you so much for the compliments! I hope v0.3.0 can get you up and running. Please let me know if you run into troubles there and I will try to help as best as I can while not having access to a PBS cluster. We just recently got access to a SLURM cluster to add that functionality to the pipeline. Perhaps if you end up modifying our SGE scripts to make working PBS scripts you could consider making a pull request. We would be happy to have a community contribution that other scientists using PBS might find useful.

I'm going to keep this issue open as a place for debugging installation issues on a PBS cluster. I will tag it for now as an enhancement since we would like to support as many cluster architectures as possible.

[Mugglea]

I have a very shallow view of this problem. I will be very grateful if you can point out the crucial factor that I've overlooked. I think that maybe you can leave this problem to the users and you don't need to consider which type of server the user have. Because perhaps many users may even not have a computer cluster, just a computer with the Linux system. What you need to consider is to complete the pipeline in the shell environment. The user will choose how they can submit the jobs. If it needs a large amount of calculation, prompt the users and they would know what to do. I believe they can use their computer cluster if they have one. So, don't worry.

By the way, I'm not so familiar with the PBS cluster. What I know is how to submit the job and how to get the result. The script for submiting the job is like this:

#!/bin/bash
##############################################################
# the name of job
#PBS -N output

# set the queue, #####this is Mandatory and use your queue id
#PBS -q Biocc.ZY

# nodes: number of nodes requested by job
# ppn  : the number of processors per node requested by job
#PBS -l nodes=1:ppn=8
#PBS -l walltime=1000:00:00
##############################################################

###### Replace the code from here to submit different "runXXXXX.sh" tasks

IPATH="/**/program/informME-master"
MATLICE="/**/program/matlab/dir_install/licenses/license_standalone.lic"
PATH1="/**/program/informME-master/others/eigen/install"
PATH2="/**/program/informME-master/others/mpfr-3.1.6"

# Non-user specified directories
# (trailing slash)
EIGEN="$PATH1/"
MPFRINCL="$PATH2/include/"
MPFRLIB="$PATH2/lib/"
SCRIPTDIR="$IPATH/SubmissionScripts/MexSource/"
SOUTDIR="$IPATH/SubmissionScripts/STDouts/"
WORKDIR="$IPATH/MexSource/"

cd $WORKDIR
${SCRIPTDIR}compile.sh "${EIGEN}" "${MPFRINCL}" "${MPFRLIB}" "${MATLICE}"

This script can successfully finish the compiling task as long as all the prerequisites are sastified. Other jobs can also be submitted based on this templet and what users need to do is to replace it with the code in the runXXXX.sh file(for example: runCompileMEX.sh).

Note:

1. Users have to set the queue, for example, "#PBS -q Biocc.ZY"
2. Users need to change to

cd $WORKDIR
${SCRIPTDIR}compile.sh "${EIGEN}" "${MPFRINCL}" "${MPFRLIB}" "${MATLICE}"

from this part

# Run compile.sh script
JOB=`qsub << EOJ
#
#$ -wd ${WORKDIR}
#$ -j y
#$ -r y
#$ -S /bin/bash
#$ -N INFM_CMP
#$ -l mem_free=15G
#$ -l h_vmem=15G
#$ -l h_fsize=100G
#$ -o ${SOUTDIR}
# 
${SCRIPTDIR}compile.sh "${EIGEN}" "${MPFRINCL}" "${MPFRLIB}" "${MATLICE}"
EOJ
`
echo "${JOB} for COMPILE submitted on `date`"

in the runCompileMEX.sh.

All I know is this. I don't think I competent to consider all the different situations and parameters needed. It's so complicated. Therefore, I'm so sorry I can't help you.