auxprogs/filterBam/src/Makefile

[mathog]

bamtools not where the default build expects. Had to change this Makefile as shown below. Note in particular the addition of "/include" in the INCLUDES line.

diff auxprogs/filterBam/src/Makefile auxprogs/filterBam/src/Makefile.dist
11,13c11,12
< BAMTOOLS = /usr/common/src/bamtools
< LDFLAGS += -L/usr/common/src/bamtools/lib
< INCLUDES = -I$(BAMTOOLS)/include -Iheaders -I./bamtools
---
> BAMTOOLS = /usr/include/bamtools
> INCLUDES = -I$(BAMTOOLS) -Iheaders -I./bamtools

[mathog]

Following up on this, it was necessary to edit 5 Makefiles to build 3.3.3 on CentOS 7. It really should be set up so that only the top Makefile is ever touched, with TOOLDIR, BAMTOOLS, SAMTOOLS and BOOST_INCLUDE set there. Here is a patch (diff -au) of all of these changes:

--- auxprogs/bam2hints/Makefile.dist    2019-09-20 10:27:15.034992433 -0700
+++ auxprogs/bam2hints/Makefile 2019-09-20 10:46:40.324047660 -0700
@@ -8,7 +8,8 @@
 #      Last modified:  09-October-2015 by Katharina J. Hoff
 
 # Variable definition
-INCLUDES = /usr/include/bamtools
+INCLUDES = /usr/common/src/bamtools/include
+LDFLAGS += -L/usr/common/src/bamtools/lib
 LIBS = -lbamtools -lz
 SOURCES = bam2hints.cc 
 OBJECTS = $(SOURCES:.cc=.o)
--- auxprogs/filterBam/src/Makefile.dist        2019-09-20 10:31:04.690943216 -0700
+++ auxprogs/filterBam/src/Makefile     2019-09-20 10:51:55.497471478 -0700
@@ -8,8 +8,9 @@
                        printElapsedTime.cc sumMandIOperations.cc sumDandIOperations.cc PairednessCoverage.cc
 PROGRAM = filterBam
 OBJECTS = $(SOURCES:.cc=.o)
-BAMTOOLS = /usr/include/bamtools
-INCLUDES = -I$(BAMTOOLS) -Iheaders -I./bamtools
+BAMTOOLS = /usr/common/src/bamtools
+LDFLAGS += -L/usr/common/src/bamtools/lib
+INCLUDES = -I$(BAMTOOLS)/include -Iheaders -I./bamtools
 LIBS = -lbamtools -lz
 CFLAGS = -std=c++0x
 VPATH = functions
--- auxprogs/bam2wig/Makefile.dist      2019-09-20 10:33:46.257721953 -0700
+++ auxprogs/bam2wig/Makefile   2019-09-20 10:34:45.292737133 -0700
@@ -15,6 +15,7 @@
 PROGRAM = bam2wig
 SOURCES = $(PROGRAM)
 OBJECTS = $(SOURCES:.c=.o)
+TOOLDIR = /usr/common/src
 SAMTOOLS=$(TOOLDIR)/samtools
 HTSLIB=$(TOOLDIR)/htslib
 BCFTOOLS=$(TOOLDIR)/bcftools
--- auxprogs/utrrnaseq/Debug/src/subdir.mk.dist 2019-09-20 10:38:35.345696799 -0700
+++ auxprogs/utrrnaseq/Debug/src/subdir.mk      2019-09-20 10:38:38.435750034 -0700
@@ -38,7 +38,7 @@
 src/%.o: ../src/%.cpp
        @echo 'Building file: $<'
        @echo 'Invoking: GCC C++ Compiler'
-       g++ -I/usr/include/boost -O0 -g3 -pedantic -Wall -c -fmessage-length=0 -MMD -MP -MF"$(@:%.o=%.d)" -MT"$(@)" -o "$@" "$<"
+       g++ -I/usr/include/boost169 -O0 -g3 -pedantic -Wall -c -fmessage-length=0 -MMD -MP -MF"$(@:%.o=%.d)" -MT"$(@)" -o "$@" "$<"
        @echo 'Finished building: $<'
        @echo ' '
 
--- Makefile.dist       2019-09-20 10:41:02.127224899 -0700
+++ Makefile    2019-09-20 10:41:02.142225157 -0700
@@ -12,18 +12,18 @@
        cd src && ${MAKE} clean
        cd auxprogs && ${MAKE} clean
 
-INSTALLDIR = /opt/augustus-$(AUGVERSION)
+INSTALLDIR = /usr/common/modules/el7/x86_64/software/augustus/$(AUGVERSION)-CentOS-vanilla
 
 install:
        install -d $(INSTALLDIR)
        cp -a config bin scripts $(INSTALLDIR)
-       ln -sf $(INSTALLDIR)/bin/augustus /usr/local/bin/augustus
-       ln -sf $(INSTALLDIR)/bin/etraining /usr/local/bin/etraining
-       ln -sf $(INSTALLDIR)/bin/prepareAlign /usr/local/bin/prepareAlign
-       ln -sf $(INSTALLDIR)/bin/fastBlockSearch /usr/local/bin/fastBlockSearch
-       ln -sf $(INSTALLDIR)/bin/load2db /usr/local/bin/load2db
-       ln -sf $(INSTALLDIR)/bin/getSeq /usr/local/bin/getSeq
-       ln -sf $(INSTALLDIR)/bin/espoca /usr/local/bin/espoca
+#      ln -sf $(INSTALLDIR)/bin/augustus /usr/local/bin/augustus
+#      ln -sf $(INSTALLDIR)/bin/etraining /usr/local/bin/etraining
+#      ln -sf $(INSTALLDIR)/bin/prepareAlign /usr/local/bin/prepareAlign
+#      ln -sf $(INSTALLDIR)/bin/fastBlockSearch /usr/local/bin/fastBlockSearch
+#      ln -sf $(INSTALLDIR)/bin/load2db /usr/local/bin/load2db
+#      ln -sf $(INSTALLDIR)/bin/getSeq /usr/local/bin/getSeq
+#      ln -sf $(INSTALLDIR)/bin/espoca /usr/local/bin/espoca

 # for internal purposes:
 release:

[mathog]

Sorry, I clicked "comment" too soon. The top Makefile assumes that the installation should put links in /usr/local/bin. That is not the case for a "module" based install.

[zx-morningstar]

Struggling with bam2wig for five hours, please do me a favor. when I type 'make' in bam2wig, it ended like: samtools/libbam.a(bam_aux.o)：In function ‘bam_parse_region’： samtools/bam_aux.c:71：Undefined reference to ‘bam_name2id’ samtools/bam_aux.c:76：Undefined reference to ‘bam_name2id’

what should i do, i've been searching many website, and i tried to modify like you and others, but found no one getting in trouble like me, my AUGUSTUS is 3.3.3. THANKS FOR YOUR HELP!!!

[mathog]

On Wed, 20 May 2020, bam2wigKillMe wrote:

Struggling with bam2wig for five hours, please do me a favor. when I type 'make' in bam2wig, it ended like: samtools/libbam.a(bam_aux.o)：In function ‘bam_parse_region’： samtools/bam_aux.c:71：Undefined reference to ‘bam_name2id’ samtools/bam_aux.c:76：Undefined reference to ‘bam_name2id’

what should i do, i've been searching many website, and i tried to modify like you and others, but found no one getting in trouble like me, my AUGUSTUS is 3.3.3. THANKS FOR YOUR HELP!!!

This is how I built Augustus 3.3.3. The module script is here: https://saf.bio.caltech.edu/pub/software/linux_or_unix_tools/module_generate_from_directory.sh and extract is from drm_tools on sourceforge (but you can probably figure out what it is doing (rtds = replace text delete string, rtss = replace text substitute string.)

(Sorry if this wraps.)

#PACKAGE: augustus package=augustus alt_pkg=Augustus pversion=3.3.3

TOPDIR=/usr/common/modules/el8/x86_64/software/${package}/${pversion}-CentOS-vanilla

THELUA=/usr/common/modules/el8/x86_64/modules/all/${package}/${pversion}-CentOS-vanilla.lua #as modules on 2020/04/22 on CentOS 8 #remove CentOS 7 version rm -rf /usr/common/modules/el7/x86_64/software/augustus rm -rf /usr/common/modules/el7/x86_64/modules/all/augustus rm -rf /usr/common/src/${package}* cd /usr/common/src wget https://github.com/Gaius-${alt_pkg}/${alt_pkg}/archive/v3.3.3.tar.gz gunzip -c v${pversion}.tar.gz | tar -xf - rm v${pversion}.tar.gz mv ${alt_pkg}-${pversion} ${package}-${pversion} ln -s ${package}-${pversion ${package} cd ${package} mkdir -p $TOPDIR cp auxprogs/bam2hints/Makefile auxprogs/bam2hints/Makefile.dist extract -in auxprogs/bam2hints/Makefile.dist
-out auxprogs/bam2hints/Makefile
-if '^INCLUDES'
-fmt 'INCLUDES = /usr/common/src/bamtools/include/bamtools\nLDFLAGS += -L/usr/common/src/bamtools/lib' cp auxprogs/filterBam/src/Makefile auxprogs/filterBam/src/Makefile.dist extract -in auxprogs/filterBam/src/Makefile.dist
-if '^BAMTOOLS'
-fmt 'BAMTOOLS = /usr/common/src/bamtools\nLDFLAGS += -L/usr/common/src/bamtools/lib'
| extract -if '^INCLUDES'
-out auxprogs/filterBam/src/Makefile
-fmt 'INCLUDES = -I$(BAMTOOLS)/include/bamtools -Iheaders -I./bamtools' cp auxprogs/bam2wig/Makefile auxprogs/bam2wig/Makefile.dist extract -in auxprogs/bam2wig/Makefile.dist
-out auxprogs/bam2wig/Makefile
-if '^SAMTOOLS'
-fmt 'TOOLDIR = /usr/common/src\nSAMTOOLS=$(TOOLDIR)/samtools' cp auxprogs/utrrnaseq/Debug/src/subdir.mk auxprogs/utrrnaseq/Debug/src/subdir.mk.dist extract -in auxprogs/utrrnaseq/Debug/src/subdir.mk.dist
-out auxprogs/utrrnaseq/Debug/src/subdir.mk
-if '/usr/include/boost'
-fmt '[rtdsboost:rtssboost169:1,]' cp Makefile Makefile.dist extract -in Makefile.dist
-if '^INSTALLDIR'
-fmt "INSTALLDIR = $TOPDIR"
| extract -if 'ln -sf' -fmt '#[1,]' -out Makefile

make -j 4 2>&1 | tee build_2020_04_22.log make install 2>&1 | tee install_2020_04_22.log

module_generate_from_directory.sh
$package
$pversion
CentOS/vanilla
$TOPDIR
"AUGUSTUS is a program that predicts genes in eukaryotic genomic sequences."
"http://bioinf.uni-greifswald.de/augustus/" cat >>$THELUA <<'EOD' -- added manually prepend_path("PATH", pathJoin(root, "scripts")) setenv("AUGUSTUS_CONFIG_PATH",pathJoin(root,"config")) if (mode() == 'load') then LmodMessage ("WARNING: The default AUGUSTUS_CONFIG_PATH is a directory you cannot write.") LmodMessage ("In some contexts you will need to change this to a directory you control.") end EOD

Regards,

David Mathog

[zx-morningstar]

On Wed, 20 May 2020, bam2wigKillMe wrote: Struggling with bam2wig for five hours, please do me a favor. when I type 'make' in bam2wig, it ended like: samtools/libbam.a(bam_aux.o)：In function ‘bam_parse_region’： samtools/bam_aux.c:71：Undefined reference to ‘bam_name2id’ samtools/bam_aux.c:76：Undefined reference to ‘bam_name2id’ what should i do, i've been searching many website, and i tried to modify like you and others, but found no one getting in trouble like me, my AUGUSTUS is 3.3.3. THANKS FOR YOUR HELP!!! This is how I built Augustus 3.3.3. The module script is here: https://saf.bio.caltech.edu/pub/software/linux_or_unix_tools/module_generate_from_directory.sh and extract is from drm_tools on sourceforge (but you can probably figure out what it is doing (rtds = replace text delete string, rtss = replace text substitute string.) (Sorry if this wraps.) #PACKAGE: augustus package=augustus alt_pkg=Augustus pversion=3.3.3 TOPDIR=/usr/common/modules/el8/x86_64/software/${package}/${pversion}-CentOS-vanilla THELUA=/usr/common/modules/el8/x86_64/modules/all/${package}/${pversion}-CentOS-vanilla.lua #as modules on 2020/04/22 on CentOS 8 #remove CentOS 7 version rm -rf /usr/common/modules/el7/x86_64/software/augustus rm -rf /usr/common/modules/el7/x86_64/modules/all/augustus rm -rf /usr/common/src/${package}* cd /usr/common/src wget https://github.com/Gaius-${alt_pkg}/${alt_pkg}/archive/v3.3.3.tar.gz gunzip -c v${pversion}.tar.gz | tar -xf - rm v${pversion}.tar.gz mv ${alt_pkg}-${pversion} ${package}-${pversion} ln -s ${package}-${pversion ${package} cd ${package} mkdir -p $TOPDIR cp auxprogs/bam2hints/Makefile auxprogs/bam2hints/Makefile.dist extract -in auxprogs/bam2hints/Makefile.dist \ -out auxprogs/bam2hints/Makefile \ -if '^INCLUDES' \ -fmt 'INCLUDES = /usr/common/src/bamtools/include/bamtools\nLDFLAGS += -L/usr/common/src/bamtools/lib' cp auxprogs/filterBam/src/Makefile auxprogs/filterBam/src/Makefile.dist extract -in auxprogs/filterBam/src/Makefile.dist \ -if '^BAMTOOLS' \ -fmt 'BAMTOOLS = /usr/common/src/bamtools\nLDFLAGS += -L/usr/common/src/bamtools/lib' \ | extract -if '^INCLUDES' \ -out auxprogs/filterBam/src/Makefile \ -fmt 'INCLUDES = -I$(BAMTOOLS)/include/bamtools -Iheaders -I./bamtools' cp auxprogs/bam2wig/Makefile auxprogs/bam2wig/Makefile.dist extract -in auxprogs/bam2wig/Makefile.dist \ -out auxprogs/bam2wig/Makefile \ -if '^SAMTOOLS' \ -fmt 'TOOLDIR = /usr/common/src\nSAMTOOLS=$(TOOLDIR)/samtools' cp auxprogs/utrrnaseq/Debug/src/subdir.mk auxprogs/utrrnaseq/Debug/src/subdir.mk.dist extract -in auxprogs/utrrnaseq/Debug/src/subdir.mk.dist \ -out auxprogs/utrrnaseq/Debug/src/subdir.mk \ -if '/usr/include/boost' \ -fmt '[rtdsboost:rtssboost169:1,]' cp Makefile Makefile.dist extract -in Makefile.dist \ -if '^INSTALLDIR' \ -fmt "INSTALLDIR = $TOPDIR" \ | extract -if 'ln -sf' -fmt '#[1,]' -out Makefile make -j 4 2>&1 | tee build_2020_04_22.log make install 2>&1 | tee install_2020_04_22.log module_generate_from_directory.sh \ $package \ $pversion \ CentOS/vanilla \ $TOPDIR \ "AUGUSTUS is a program that predicts genes in eukaryotic genomic sequences." \ "http://bioinf.uni-greifswald.de/augustus/" cat >>$THELUA <<'EOD' -- added manually prepend_path("PATH", pathJoin(root, "scripts")) setenv("AUGUSTUS_CONFIG_PATH",pathJoin(root,"config")) if (mode() == 'load') then LmodMessage ("WARNING: The default AUGUSTUS_CONFIG_PATH is a directory you cannot write.") LmodMessage ("In some contexts you will need to change this to a directory you control.") end EOD Regards, David Mathog

Thank you for your answer very much! But I'm not a root user, so I can't rm files in /usr/ . And I later just commented out the "cd bam2wig; make" in augustus/Makefile, it seemed to work! Hope it won't crash in the future.

Thank you again for your precious time, Mr. Mathog. Regards.

PS: I can't open your "module_generate_from_directory.sh" maybe caused by restrictions in my country. but in the page, the line: ln -s ${package}-${pversion ${package} seems to miss a "}".

[mathog]

On 2020-05-22 20:33, bam2wigKillMe wrote:

Thank you for your answer very much! But I'm not a root user, so I can't rm files in /usr/ .

Anybody who uses Augustus may cause it to store files within its directory. It either has to have directory protections to allow that or an environmental variable must be set to relocate those files.

And I later just commented out the "cd bam2wig; make" in augustus/Makefile, it seemed to work! Hope it won't crash in the future. Whatever works.

Thank you again for your precious time, Mr. Mathog. No problem - silly to make somebody else solve the same problem again.

PS: I can't open your "module_generate_from_directory.sh" maybe caused by restrictions in my country.

As in you cannot download it? There might be an issue with the "Let's Encrypt" certificate on the server and your machine not having the corresponding root certificate. That file may also be downloaded with ftp:

ftp://newsaf.bio.caltech.edu/pub/software/linux_or_unix_tools/module_generate_from_directory.sh

which should not have that problem. Again, you only need that script if you are using Lmod and need to set up a module for a package.

but in the page, the line: ln -s ${package}-${pversion ${package} seems to miss a "}".

That was a typo on my part, as you say, it is missing the "}" after pversion. Sorry.

Regards,

David Mathog [email protected] Manager, Sequence Analysis Facility, Biology Division, Caltech

[SchwarzEM]

The bugfixes in this issue solved compilation failures that were driving me to tears.

Thanks to @mathog for this!

AUGUSTUS developers: if you have the opportunity, please consider streamlining the way that programs such as bam2hints are compiled, so that bugfixing of Makefiles guided by GitHub ticket archaeology is no longer necessary.

[hmehlan]

Thanks for the feedback. As @mathog suggested, all INCLUDES and LIBRARY paths can now be set in commons.mk alone. Help with problems or hints for installing necessary packages as a non-root user can be found in docs/INSTALL.md