Minimac4 icon indicating copy to clipboard operation
Minimac4 copied to clipboard

Published 20 hours ago • verified publisher icon statgen

CentOS 6 install failed

Open yutoubb opened this issue 6 years ago • 14 comments

Scanning dependencies of target minimac4 [ 8%] Building CXX object CMakeFiles/minimac4.dir/src/Analysis.cpp.o [ 16%] Building CXX object CMakeFiles/minimac4.dir/src/AnalysisChunks.cpp.o [ 25%] Building CXX object CMakeFiles/minimac4.dir/src/DosageData.cpp.o [ 33%] Building CXX object CMakeFiles/minimac4.dir/src/HaplotypeSet.cpp.o [ 41%] Building CXX object CMakeFiles/minimac4.dir/src/Imputation.cpp.o [ 50%] Building CXX object CMakeFiles/minimac4.dir/src/ImputationStatistics.cpp.o [ 58%] Building CXX object CMakeFiles/minimac4.dir/src/Main.cpp.o [ 66%] Building CXX object CMakeFiles/minimac4.dir/src/MarkovModel.cpp.o [ 75%] Building CXX object CMakeFiles/minimac4.dir/src/MarkovParameters.cpp.o [ 83%] Building CXX object CMakeFiles/minimac4.dir/src/Unique.cpp.o [ 91%] Building CXX object CMakeFiles/minimac4.dir/src/Estimation.cpp.o [100%] Linking CXX executable minimac4 CMakeFiles/minimac4.dir/src/Analysis.cpp.o: In function Analysis::CheckGeneticMapFile()': Analysis.cpp:(.text+0x1abe): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' CMakeFiles/minimac4.dir/src/Analysis.cpp.o: In function Analysis::readm3vcfFileChunk(int, HaplotypeSet&)': Analysis.cpp:(.text+0x2871): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' CMakeFiles/minimac4.dir/src/Analysis.cpp.o: In function Analysis::InitializeRefFileStream(String&)': Analysis.cpp:(.text+0x32ff): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' Analysis.cpp:(.text+0x3340): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits<char>, std::allocator<char> >&)' Analysis.cpp:(.text+0x378b): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' CMakeFiles/minimac4.dir/src/Analysis.cpp.o:Analysis.cpp:(.text+0x40b0): more undefined references to InputFile::readLine(std::basic_string<char, std::char_traits<char>, std::allocator<char> >&)' follow CMakeFiles/minimac4.dir/src/HaplotypeSet.cpp.o: In function HaplotypeSet::GetVariantInfofromVCFFile(String&, String, AllVariable&)': HaplotypeSet.cpp:(.text+0xad0f): undefined reference to VcfRecordFilter::getString(int)' CMakeFiles/minimac4.dir/src/HaplotypeSet.cpp.o: In function HaplotypeSet::readm3vcfFile(String, String, int, int, int)': HaplotypeSet.cpp:(.text+0xc406): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits<char>, std::allocator<char> >&)' HaplotypeSet.cpp:(.text+0xc52c): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' HaplotypeSet.cpp:(.text+0xcfd2): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits<char>, std::allocator<char> >&)' HaplotypeSet.cpp:(.text+0xdb78): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' CMakeFiles/minimac4.dir/src/Estimation.cpp.o: In function Estimation::readm3vcfFileChunk(int, HaplotypeSet&)': Estimation.cpp:(.text+0x1de9): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' CMakeFiles/minimac4.dir/src/Estimation.cpp.o:Estimation.cpp:(.text+0x5bcd): more undefined references to `InputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' follow collect2: ld returned 1 exit status make[2]: *** [minimac4] Error 1 make[1]: *** [CMakeFiles/minimac4.dir/all] Error 2 make: *** [all] Error 2 Binary created at /release-build/minimac4

yutoubb avatar Nov 23 '18 05:11 yutoubb

I've been struggling to install Minimac4 for a couple of days. After installed python virtualenv for cget and changed cmake and other shared libs, I got above error in make step.

By the way, would it be possible to provide linux executable files like Minimac3? For users without root privileges and using an old cluster like me, it's very difficult to get every thing right for compiling.

yutoubb avatar Nov 23 '18 05:11 yutoubb

Can you please provide the OS version you are running?

The problem is that libStatGen cannot be found for linking. I'm assuming you ran the install.sh script and there are error messages in that script's log files. I would recommend not using that script. Instead you can run the following from the Minimac4 source directory:

cget install -f ./requirements.txt
rm -rf build && mkdir build && cd build
cmake -DCMAKE_TOOLCHAIN_FILE=../cget/cget/cget.cmake .. 
make

Alternatively, there are binary packages in the releases section. There are system dependent ways to extract into a non-root directory. For Debian it will look something like dpkg -x pkg.deb dest_dir.

Side note: you do not need a virtual environment to install cget without root. Instead run pip install --user cget.

jonathonl avatar Nov 23 '18 15:11 jonathonl

Thank you for your reply! The OS should be a customized version of Centos 6. Our system default python is 2.6, so I used a python 2.7 which installed by other user. I have tried to install the libStatGen in another folder (by git clone & make all if I remember correctly) and got no error. And when I ran the install.sh I have checked the install.log but it seems fine.

Anyway, I find the .rpm in the release folder, and use follow command to get the executable file. rpm2cpio ctags-5.8-2.el6.x86_64.rpm |cpio -idvm and then use glibc 2.17 to run it.

yutoubb avatar Nov 24 '18 10:11 yutoubb

$ cget install -f ./requirements.txt Downloading https://github.com/statgen/libStatGen/archive/HEAD.tar.gz [######################################################################] 100% Extracting archive /some_path/Minimac4/cget/cget/build/tmp-62ee1d7ab3794245974f551b30cb3092/HEAD.tar.gz ... -- The C compiler identification is GNU 4.4.7 -- The CXX compiler identification is GNU 4.4.7 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Configuring done -- Generating done CMake Warning: Manually-specified variables were not used by the project:

BUILD_TESTING
CGET_CMAKE_DIR
CGET_CMAKE_ORIGINAL_SOURCE_FILE

-- Build files have been written to: /some_path/Minimac4/cget/cget/build/tmp-62ee1d7ab3794245974f551b30cb3092/build Scanning dependencies of target libStatGen [100%] Builing libStatGen ... make[3]: warning: jobserver unavailable: using -j1. Add +' to parent make rule. [100%] Built target libStatGen [100%] Builing libStatGen ... make[3]: warning: jobserver unavailable: using -j1. Add +' to parent make rule. [100%] Built target libStatGen Install the project... -- Install configuration: "Release" -- Installing: /some_path/Minimac4/cget/cget/pkg/statgen__libStatGen/install/include/SamHeaderTag.h -- Installing: /some_path/Minimac4/Minimac4/cget/cget/pkg/statgen__libStatGen/install/include/SamRecordPool.h

...too many lines..........

-- Installing: /some_path/Minimac4/cget/cget/pkg/statgen__libStatGen/install/include/VcfGenotypeField.h -- Installing: /some_path/Minimac4/cget/cget/pkg/statgen__libStatGen/install/lib/libStatGen.a Successfully installed statgen/libStatGen

$ rm -rf build && mkdir build && cd build $ cmake -DCMAKE_TOOLCHAIN_FILE=../cget/cget/cget.cmake .. -- The C compiler identification is GNU 4.4.7 -- The CXX compiler identification is GNU 4.4.7 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Try OpenMP C flag = [-fopenmp] -- Performing Test OpenMP_FLAG_DETECTED -- Performing Test OpenMP_FLAG_DETECTED - Success -- Try OpenMP CXX flag = [-fopenmp] -- Performing Test OpenMP_FLAG_DETECTED -- Performing Test OpenMP_FLAG_DETECTED - Success -- Found OpenMP: -fopenmp -- Found ZLIB: /zfssz5/BC_PS/huangshujia/biosoft/local/lib/libz.so (found version "1.2.8") -- Configuring done -- Generating done -- Build files have been written to: /some_path/Minimac4/build

(mini) zhouyuwen@software-install:/some_path/Minimac4/build$ make Scanning dependencies of target minimac4 [ 8%] Building CXX object CMakeFiles/minimac4.dir/src/Analysis.cpp.o [ 16%] Building CXX object CMakeFiles/minimac4.dir/src/AnalysisChunks.cpp.o [ 25%] Building CXX object CMakeFiles/minimac4.dir/src/DosageData.cpp.o [ 33%] Building CXX object CMakeFiles/minimac4.dir/src/HaplotypeSet.cpp.o [ 41%] Building CXX object CMakeFiles/minimac4.dir/src/Imputation.cpp.o [ 50%] Building CXX object CMakeFiles/minimac4.dir/src/ImputationStatistics.cpp.o [ 58%] Building CXX object CMakeFiles/minimac4.dir/src/Main.cpp.o [ 66%] Building CXX object CMakeFiles/minimac4.dir/src/MarkovModel.cpp.o [ 75%] Building CXX object CMakeFiles/minimac4.dir/src/MarkovParameters.cpp.o [ 83%] Building CXX object CMakeFiles/minimac4.dir/src/Unique.cpp.o [ 91%] Building CXX object CMakeFiles/minimac4.dir/src/Estimation.cpp.o [100%] Linking CXX executable minimac4 CMakeFiles/minimac4.dir/src/Analysis.cpp.o: In function Analysis::AppendtoMainLooVcfFaster(int, int)': Analysis.cpp:(.text+0x210a): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' CMakeFiles/minimac4.dir/src/Analysis.cpp.o: In function Analysis::AppendtoMainVcfFaster(int, int)': Analysis.cpp:(.text+0x31c1): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' Analysis.cpp:(.text+0x3586): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits<char>, std::allocator<char> >&)' CMakeFiles/minimac4.dir/src/Analysis.cpp.o: In function Analysis::readm3vcfFileChunk(int, HaplotypeSet&)': Analysis.cpp:(.text+0x3d96): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits<char>, std::allocator<char> >&)' CMakeFiles/minimac4.dir/src/Analysis.cpp.o: In function Analysis::InitializeRefFileStream(String&)': Analysis.cpp:(.text+0x7cd0): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits<char>, std::allocator<char> >&)' CMakeFiles/minimac4.dir/src/Analysis.cpp.o:Analysis.cpp:(.text+0x7cff): more undefined references to InputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' follow CMakeFiles/minimac4.dir/src/HaplotypeSet.cpp.o: In function HaplotypeSet::GetVariantInfofromVCFFile(String&, String, AllVariable&)': HaplotypeSet.cpp:(.text+0x43a2): undefined reference to VcfRecordFilter::getString(int)' CMakeFiles/minimac4.dir/src/HaplotypeSet.cpp.o: In function HaplotypeSet::GetSummary(InputFile*)': HaplotypeSet.cpp:(.text+0x6417): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' CMakeFiles/minimac4.dir/src/HaplotypeSet.cpp.o: In function HaplotypeSet::ReadM3VCFChunkingInformation(String&, std::basic_string<char, std::char_traits<char>, std::allocator<char> >)': HaplotypeSet.cpp:(.text+0x6d9d): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' CMakeFiles/minimac4.dir/src/HaplotypeSet.cpp.o: In function HaplotypeSet::DetectFileType(String)': HaplotypeSet.cpp:(.text+0x8b64): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' CMakeFiles/minimac4.dir/src/HaplotypeSet.cpp.o: In function HaplotypeSet::readm3vcfFile(String, String, int, int, int)': HaplotypeSet.cpp:(.text+0x92bb): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' HaplotypeSet.cpp:(.text+0x93e2): undefined reference to InputFile::readLine(std::basic_string<char, std::char_traits<char>, std::allocator<char> >&)' CMakeFiles/minimac4.dir/src/HaplotypeSet.cpp.o:HaplotypeSet.cpp:(.text+0x9b8e): more undefined references to InputFile::readLine(std::basic_string<char, std::char_traits, std::allocator >&)' follow collect2: ld returned 1 exit status make[2]: *** [minimac4] Error 1 make[1]: *** [CMakeFiles/minimac4.dir/all] Error 2 make: *** [all] Error 2

yutoubb avatar Dec 06 '18 06:12 yutoubb

Actually I did run each steps separately before, and just repeated the process again and still get error at the last make step.

yutoubb avatar Dec 06 '18 06:12 yutoubb

I spun up a CentOS 6 VM and figured out what's going. There's a silent error due to a bug in gcc v4.4 when building libStatGen.a and the archive is left incomplete. I made a fix to the libStatGen repo. Remove the cget directory and rerun the cget install command to pull the latest libStatGen:

rm -rf cget/
cget install -f requirements.txt
cd build && make

jonathonl avatar Dec 06 '18 15:12 jonathonl

I removed everthing and tried again, but still get the same error. I don't understand why in the log 'The C/CXX compiler identification is GNU 4.4.7 but anctually I set a higher gcc in .bashrc, and if use 'gcc --version' it is 5.2.0.

$gcc --version
gcc (GCC) 5.2.0
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Our os version is confirmed to be CentOS release 6.9 (Final).

yutoubb avatar Dec 07 '18 03:12 yutoubb

I believe the binary that cmake finds first is cc and c++ (run cc --version and c++ --version), which I suspect are still the default compilers on your system.

I've tested this on CentOS 6 with default compiler and it builds fine. If it still doesn't work for you, I would suggest building libStatGen (https://github.com/statgen/libStatGen) manually and specifying it's location to Minimac4 either by cmake variables on command line or by editing the CMakelists.txt file. The following edits would work:

# Add this line after project() function to set libStatGen include path
include_directories(/path/to/libStatGen/include

# Remove this line
find_library(STATGEN_LIBRARY StatGen)

# Update this line to set absolute path to library archive
target_link_libraries(minimac4 /path/to/libStatGen/libStatGen.a ${ZLIB_LIBRARIES})

jonathonl avatar Dec 07 '18 15:12 jonathonl

I changed CMakelists.txt, removed cget and build folder, and ran follow commands:

mkdir build && cd build
cmake ..
make

And I got same error. Is there still something to be changed in dep/libstatgen.cmake or some where? or should I just put libStatgen/include under src folder?

yutoubb avatar Dec 10 '18 06:12 yutoubb

libstatgen.cmake is not being used with this approach. The libStatGen include and lib can be located anywhere. Did libStatGen build without error?

jonathonl avatar Dec 10 '18 18:12 jonathonl

Yes, the libStatGen was built without error. Commands I used:

git clone https://github.com/statgen/libStatGen.git 
cd libStatGen
make

yutoubb avatar Dec 11 '18 01:12 yutoubb

It doesn't make sense to me that you're still getting the same error. Can you please attach the Minimac4 CMakelists.txt file that you edited?

jonathonl avatar Dec 11 '18 02:12 jonathonl

The structure is:

/software/imputation/Minimac4/
├── libStatGen
│   ├── bam
│   ├── copyrights
│   ├── Doxyfile
│   ├── fastq
│   ├── general
│   ├── glf
│   ├── include
│   ├── libStatGen.a
│   ├── Makefile
│   ├── Makefiles
│   ├── README.md
│   ├── samtools
│   └── vcf
└── Minimac4
    ├── build
    ├── CMakeLists.txt
    ├── dep
    ├── install.sh
    ├── LICENSE
    ├── README.md
    ├── requirements.txt
    └── src

Content of CMakelists.txt

cmake_minimum_required(VERSION 3.2)
project(minimac4 VERSION 1.0.1)

include_directories(/software/imputation/Minimac4/libStatGen/include)

set(CMAKE_CXX_STANDARD 11)
execute_process(COMMAND date OUTPUT_VARIABLE DATE OUTPUT_STRIP_TRAILING_WHITESPACE)
execute_process(COMMAND whoami OUTPUT_VARIABLE USER OUTPUT_STRIP_TRAILING_WHITESPACE)

add_definitions(-DVERSION="${PROJECT_VERSION}" -DUSER="${USER}" -DDATE="${DATE}")

find_package(OpenMP)
if (OPENMP_FOUND)
    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${OpenMP_C_FLAGS}")
    set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${OpenMP_CXX_FLAGS}")
endif()

find_package(ZLIB REQUIRED)
add_executable(minimac4
        src/Analysis.cpp
        src/Analysis.h
        src/AnalysisChunks.cpp
        src/DosageData.cpp
        src/DosageData.h
        src/HaplotypeSet.cpp
        src/HaplotypeSet.h
        src/Imputation.cpp
        src/Imputation.h
        src/ImputationStatistics.cpp
        src/ImputationStatistics.h
        src/Main.cpp
        src/MarkovModel.cpp
        src/MarkovModel.h
        src/MarkovParameters.cpp
        src/MarkovParameters.h
        src/MyVariables.h
        src/Unique.cpp
        src/Unique.h
        src/Estimation.cpp
        src/Estimation.h)

target_link_libraries(minimac4 /software/imputation/Minimac4/libStatGen/libStatGen.a ${ZLIB_LIBRARIES})

install(TARGETS minimac4 RUNTIME DESTINATION bin)

yutoubb avatar Dec 14 '18 06:12 yutoubb

I have one last theory. I bet libStatGen is being build with g++ (gcc v5.2.0) and Minimac4 is being built with c++ (gcc v4.4.7). Edit Minimac4/build/CMakeCache.txt and set CMAKE_C_COMPILER and CMAKE_CXX_COMPILER to point to the 5.2 versions of gcc and g++ respectively. Then rebuild Minimac4. There's an ABI breakage in std::string between versions 4 and 5 of g++ that would cause such errors when linking.

jonathonl avatar Dec 14 '18 15:12 jonathonl