manta icon indicating copy to clipboard operation
manta copied to clipboard

Published 20 hours ago • verified publisher icon Illumina

Build failed with internal htslib but succeed with external one

Open ghost opened this issue 4 years ago • 1 comments

Hi everyone,

I am trying to build manta from source by git clone https://github.com/Illumina/manta. After making a build directory named build and executing configure, I executed the following command (I'm using something like "${MYHOST}" to hide my paths):

(base) ["${MYHOST}"]::[~/usr/src/manta/build] (master)
$ CC=clang
(base) ["${MYHOST}"]::[~/usr/src/manta/build] (master)
$ CXX=clang++
(base) ["${MYHOST}"]::[~/usr/src/manta/build] (master)
$ make
make[1]: Entering directory '"${MYHOME}"/usr/src/manta/build'
make[2]: Entering directory '"${MYHOME}"/usr/src/manta/build'
make[2]: Leaving directory '"${MYHOME}"/usr/src/manta/build'
make[2]: Entering directory '"${MYHOME}"/usr/src/manta/build'
[  1%] Updating buildtime config info
-- Detected workflow version: 1.6.0-3-g75b5c38d
make[2]: Leaving directory '"${MYHOME}"/usr/src/manta/build'
[  1%] Built target manta_buildtime_config
make[2]: Entering directory '"${MYHOME}"/usr/src/manta/build'
make[2]: Leaving directory '"${MYHOME}"/usr/src/manta/build'
make[2]: Entering directory '"${MYHOME}"/usr/src/manta/build'
make[2]: Leaving directory '"${MYHOME}"/usr/src/manta/build'
[  1%] Built target manta_cxx_buildtime_config
make[2]: Entering directory '"${MYHOME}"/usr/src/manta/build'
make[2]: Leaving directory '"${MYHOME}"/usr/src/manta/build'
make[2]: Entering directory '"${MYHOME}"/usr/src/manta/build'
make[2]: Leaving directory '"${MYHOME}"/usr/src/manta/build'
[  1%] Built target manta_source_check
make[2]: Entering directory '"${MYHOME}"/usr/src/manta/build'
make[2]: Leaving directory '"${MYHOME}"/usr/src/manta/build'
make[2]: Entering directory '"${MYHOME}"/usr/src/manta/build'
[  1%] Building htslib library
Failed to build htslib, see "${MYHOME}"/usr/src/manta/build/redist/htslib-1.9/htslib.build.error.log for details.
make[2]: *** [redist/CMakeFiles/manta_samtools.dir/build.make:85: redist/htslib-1.9/libhts.a] Error 1
make[2]: Leaving directory '"${MYHOME}"/usr/src/manta/build'
make[1]: *** [CMakeFiles/Makefile2:1166: redist/CMakeFiles/manta_samtools.dir/all] Error 2
make[1]: Leaving directory '"${MYHOME}"/usr/src/manta/build'
make: *** [Makefile:158: all] Error 2

Then I viewed the log file with:

(base) ["${MYHOST}"]::[~/usr/src/manta/build] (master)
$ cat "${MYHOME}"/usr/src/manta/build/redist/htslib-1.9/htslib.build.error.log
In file included from hts.c:41:
In file included from ./cram/cram.h:45:
In file included from ./cram/cram_samtools.h:61:
"${MYHOME}"/usr/local/include/htslib/sam.h:204:5: error: unknown type name 'hts_pos_t'
    hts_pos_t pos;
    ^
"${MYHOME}"/usr/local/include/htslib/sam.h:214:5: error: unknown type name 'hts_pos_t'
    hts_pos_t mpos;
    ^
"${MYHOME}"/usr/local/include/htslib/sam.h:215:5: error: unknown type name 'hts_pos_t'
    hts_pos_t isize;
    ^
"${MYHOME}"/usr/local/include/htslib/sam.h:341:1: error: unknown type name 'HTSLIB_EXPORT'
HTSLIB_EXPORT
^
"${MYHOME}"/usr/local/include/htslib/sam.h:342:10: error: expected ';' after top level declarator
sam_hdr_t *sam_hdr_init(void);
         ^
"${MYHOME}"/usr/local/include/htslib/sam.h:355:1: error: unknown type name 'HTSLIB_EXPORT'
HTSLIB_EXPORT
^
"${MYHOME}"/usr/local/include/htslib/sam.h:356:10: error: expected ';' after top level declarator
sam_hdr_t *bam_hdr_read(BGZF *fp);
         ^
"${MYHOME}"/usr/local/include/htslib/sam.h:367:1: error: unknown type name 'HTSLIB_EXPORT'
HTSLIB_EXPORT
^
"${MYHOME}"/usr/local/include/htslib/sam.h:373:1: error: unknown type name 'HTSLIB_EXPORT'
HTSLIB_EXPORT
^
"${MYHOME}"/usr/local/include/htslib/sam.h:383:1: error: unknown type name 'HTSLIB_EXPORT'
HTSLIB_EXPORT
^
"${MYHOME}"/usr/local/include/htslib/sam.h:384:10: error: expected ';' after top level declarator
sam_hdr_t *sam_hdr_dup(const sam_hdr_t *h0);
         ^
"${MYHOME}"/usr/local/include/htslib/sam.h:406:1: error: unknown type name 'HTSLIB_EXPORT'
HTSLIB_EXPORT
^
"${MYHOME}"/usr/local/include/htslib/sam.h:407:10: error: expected ';' after top level declarator
sam_hdr_t *sam_hdr_parse(size_t l_text, const char *text);
         ^
"${MYHOME}"/usr/local/include/htslib/sam.h:417:1: error: unknown type name 'HTSLIB_EXPORT'
HTSLIB_EXPORT
^
"${MYHOME}"/usr/local/include/htslib/sam.h:418:10: error: expected ';' after top level declarator
sam_hdr_t *sam_hdr_read(samFile *fp);
         ^
"${MYHOME}"/usr/local/include/htslib/sam.h:426:1: error: unknown type name 'HTSLIB_EXPORT'
HTSLIB_EXPORT
^
"${MYHOME}"/usr/local/include/htslib/sam.h:433:1: error: unknown type name 'HTSLIB_EXPORT'
HTSLIB_EXPORT
^
"${MYHOME}"/usr/local/include/htslib/sam.h:434:7: error: expected ';' after top level declarator
size_t sam_hdr_length(sam_hdr_t *h);
      ^
"${MYHOME}"/usr/local/include/htslib/sam.h:446:1: error: unknown type name 'HTSLIB_EXPORT'
HTSLIB_EXPORT
^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
make[3]: *** [Makefile:99: hts.o] Error 1

Then I cd into the htslib directory and try to build it separately, which turn showed the same error:

(base) ["${MYHOST}"]::[~/usr/src/manta/build] (master)
$ cd "${MYHOME}"/usr/src/manta/build/redist/htslib-1.9/
(base) ["${MYHOST}"]::[~/usr/src/manta/build/redist/htslib-1.9] (master)
$ make
"${MYHOME}"/usr/local/bin/clang -Wall -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem "${MYHOME}"/miniconda3/include -I. -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -isystem "${MYHOME}"/miniconda3/include -c -o hts.o hts.c
In file included from hts.c:41:
In file included from ./cram/cram.h:45:
In file included from ./cram/cram_samtools.h:61:
"${MYHOME}"/usr/local/include/htslib/sam.h:204:5: error: unknown type name 'hts_pos_t'
    hts_pos_t pos;
    ^
"${MYHOME}"/usr/local/include/htslib/sam.h:214:5: error: unknown type name 'hts_pos_t'
    hts_pos_t mpos;
    ^
"${MYHOME}"/usr/local/include/htslib/sam.h:215:5: error: unknown type name 'hts_pos_t'
    hts_pos_t isize;
    ^
"${MYHOME}"/usr/local/include/htslib/sam.h:341:1: error: unknown type name 'HTSLIB_EXPORT'
HTSLIB_EXPORT
^
"${MYHOME}"/usr/local/include/htslib/sam.h:342:10: error: expected ';' after top level declarator
sam_hdr_t *sam_hdr_init(void);
         ^
"${MYHOME}"/usr/local/include/htslib/sam.h:355:1: error: unknown type name 'HTSLIB_EXPORT'
HTSLIB_EXPORT
^
"${MYHOME}"/usr/local/include/htslib/sam.h:356:10: error: expected ';' after top level declarator
sam_hdr_t *bam_hdr_read(BGZF *fp);
         ^
"${MYHOME}"/usr/local/include/htslib/sam.h:367:1: error: unknown type name 'HTSLIB_EXPORT'
HTSLIB_EXPORT
^
"${MYHOME}"/usr/local/include/htslib/sam.h:373:1: error: unknown type name 'HTSLIB_EXPORT'
HTSLIB_EXPORT
^
"${MYHOME}"/usr/local/include/htslib/sam.h:383:1: error: unknown type name 'HTSLIB_EXPORT'
HTSLIB_EXPORT
^
"${MYHOME}"/usr/local/include/htslib/sam.h:384:10: error: expected ';' after top level declarator
sam_hdr_t *sam_hdr_dup(const sam_hdr_t *h0);
         ^
"${MYHOME}"/usr/local/include/htslib/sam.h:406:1: error: unknown type name 'HTSLIB_EXPORT'
HTSLIB_EXPORT
^
"${MYHOME}"/usr/local/include/htslib/sam.h:407:10: error: expected ';' after top level declarator
sam_hdr_t *sam_hdr_parse(size_t l_text, const char *text);
         ^
"${MYHOME}"/usr/local/include/htslib/sam.h:417:1: error: unknown type name 'HTSLIB_EXPORT'
HTSLIB_EXPORT
^
"${MYHOME}"/usr/local/include/htslib/sam.h:418:10: error: expected ';' after top level declarator
sam_hdr_t *sam_hdr_read(samFile *fp);
         ^
"${MYHOME}"/usr/local/include/htslib/sam.h:426:1: error: unknown type name 'HTSLIB_EXPORT'
HTSLIB_EXPORT
^
"${MYHOME}"/usr/local/include/htslib/sam.h:433:1: error: unknown type name 'HTSLIB_EXPORT'
HTSLIB_EXPORT
^
"${MYHOME}"/usr/local/include/htslib/sam.h:434:7: error: expected ';' after top level declarator
size_t sam_hdr_length(sam_hdr_t *h);
      ^
"${MYHOME}"/usr/local/include/htslib/sam.h:446:1: error: unknown type name 'HTSLIB_EXPORT'
HTSLIB_EXPORT
^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
make: *** [Makefile:99: hts.o] Error 1

However, after I cloned the official htslib directory (which appears to be at version 1.10) into this directory and built again, the error disappeared:

(base) ["${MYHOST}"]::[~/usr/src/manta/build/redist/htslib-1.9] (master)
$ cd ..
(base) ["${MYHOST}"]::[~/usr/src/manta/build/redist] (master)
$ cp ~/usr/src/htslib htslib-1.9 -r
(base) ["${MYHOST}"]::[~/usr/src/manta/build/redist] (master)
$ cd htslib-1.9
(base) ["${MYHOST}"]::[~/usr/src/manta/build/redist/htslib-1.9] (develop)
$ autoheader
(base) ["${MYHOST}"]::[~/usr/src/manta/build/redist/htslib-1.9] (develop)
$ autoconf
(base) ["${MYHOST}"]::[~/usr/src/manta/build/redist/htslib-1.9] (develop)
$ cd "${MYHOME}"/usr/src/manta/build
(base) ["${MYHOST}"]::[~/usr/src/manta/build] (master)
$ make
make[1]: Entering directory '"${MYHOME}"/usr/src/manta/build'
make[2]: Entering directory '"${MYHOME}"/usr/src/manta/build'
make[2]: Leaving directory '"${MYHOME}"/usr/src/manta/build'
make[2]: Entering directory '"${MYHOME}"/usr/src/manta/build'
[  1%] Updating buildtime config info
-- Detected workflow version: 1.6.0-3-g75b5c38d
make[2]: Leaving directory '"${MYHOME}"/usr/src/manta/build'
[  1%] Built target manta_buildtime_config
make[2]: Entering directory '"${MYHOME}"/usr/src/manta/build'
make[2]: Leaving directory '"${MYHOME}"/usr/src/manta/build'
make[2]: Entering directory '"${MYHOME}"/usr/src/manta/build'
make[2]: Leaving directory '"${MYHOME}"/usr/src/manta/build'
[  1%] Built target manta_cxx_buildtime_config
make[2]: Entering directory '"${MYHOME}"/usr/src/manta/build'
make[2]: Leaving directory '"${MYHOME}"/usr/src/manta/build'
make[2]: Entering directory '"${MYHOME}"/usr/src/manta/build'
make[2]: Leaving directory '"${MYHOME}"/usr/src/manta/build'
[  1%] Built target manta_source_check
make[2]: Entering directory '"${MYHOME}"/usr/src/manta/build'
make[2]: Leaving directory '"${MYHOME}"/usr/src/manta/build'
make[2]: Entering directory '"${MYHOME}"/usr/src/manta/build'
[  1%] Building htslib library
[  1%] Unpacking samtools package
[  2%] Building samtools package
Failed to build samtools, see "${MYHOME}"/usr/src/manta/build/redist/samtools-1.9/samtools.build.error.log for details.
make[2]: *** [redist/CMakeFiles/manta_samtools.dir/build.make:80: redist/samtools-1.9/samtools] Error 1
make[2]: *** Deleting file 'redist/samtools-1.9/samtools'
make[2]: Leaving directory '"${MYHOME}"/usr/src/manta/build'
make[1]: *** [CMakeFiles/Makefile2:1166: redist/CMakeFiles/manta_samtools.dir/all] Error 2
make[1]: Leaving directory '/gpfsdata/home/kangjiangn/usr/src/manta/build'
make: *** [Makefile:158: all] Error 2

htslib was successfully built with samtools failed, just as expected.

I would like to know the reason for the first error, is it due to the htslib distributed Manta? I'm using clang 11.1.0 under CentOS.

It might also be a good idea to support external pre-built htslib by commands like, ./configure --with-htslib=blablabla.

Thanks!

ghost avatar Jun 14 '20 18:06 ghost

I'm sorry, but I would like to add a few comments:

Firstly, samtools was successfully built after using an external one;

Secondly, I've retried building Manta under tag 1.6.0 with internal htslib, but I got the same error;

Thirdly, I've tried to build Manta under GCC version 7.5.0 with internal htslib, but I got the same error.

Thanks!

ghost avatar Jun 14 '20 18:06 ghost