IncludeOS icon indicating copy to clipboard operation
IncludeOS copied to clipboard

How to build IncludeOS in centos7?

Open zhang2639 opened this issue 6 years ago • 28 comments

I tried to build IncludeOS in centos7. But I failed. Could you please help me?

zhang2639 avatar Jun 13 '18 02:06 zhang2639

I use ./etc/install_from_bundle.sh. But It failed.

See:

/usr/local/bin/clang
/usr/local/bin/clang++


>>> Best guess for compatible compilers: clang++ / clang


>>> Building IncludeOS
~/IncludeOS/build_x86_64 /home/zsq169608/IncludeOS
-- Target CPU x86_64
-- Target triple x86_64-pc-linux-elf
CMake Error at /usr/local/share/cmake-3.3/Modules/CMakeDetermineSystem.cmake:104 (message):
  Could not find toolchain file: /root/IncludeOS//cmake/elf-toolchain.cmake
Call Stack (most recent call first):
  CMakeLists.txt:24 (project)


CMake Error: Error required internal CMake variable not set, cmake may be not be built correctly.
Missing variable is:
CMAKE_C_COMPILER_ENV_VAR
CMake Error: Error required internal CMake variable not set, cmake may be not be built correctly.
Missing variable is:
CMAKE_C_COMPILER
CMake Error: Could not find cmake module file: /root/IncludeOS/build_x86_64/CMakeFiles/3.3.2/CMakeCCompiler.cmake
CMake Error: Error required internal CMake variable not set, cmake may be not be built correctly.
Missing variable is:
CMAKE_CXX_COMPILER_ENV_VAR
CMake Error: Error required internal CMake variable not set, cmake may be not be built correctly.
Missing variable is:
CMAKE_CXX_COMPILER
CMake Error: Could not find cmake module file: /root/IncludeOS/build_x86_64/CMakeFiles/3.3.2/CMakeCXXCompiler.cmake
CMake Error at CMakeLists.txt:24 (project):
  No CMAKE_C_COMPILER could be found.

  Tell CMake where to find the compiler by setting the CMake cache entry
  CMAKE_C_COMPILER to the full path to the compiler, or to the compiler name
  if it is in the PATH.


CMake Error at CMakeLists.txt:24 (project):
  No CMAKE_CXX_COMPILER could be found.

  Tell CMake where to find the compiler by setting the CMake cache entry
  CMAKE_CXX_COMPILER to the full path to the compiler, or to the compiler
  name if it is in the PATH.


CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage
CMake Error: CMAKE_CXX_COMPILER not set, after EnableLanguage
-- Configuring incomplete, errors occurred!

zhang2639 avatar Jun 13 '18 07:06 zhang2639

Looks like some envs are off. ./install.sh sets these up, including installing all the necessary dependencies. I don't know anything about centos7 tho I'm afraid!

Try export INCLUDEOS_SRC=path/to/repo and see if you can get any further.

AndreasAakesson avatar Jun 13 '18 07:06 AndreasAakesson

You will need to install clang-5.0 or later, and then set CC and CXX to point to clang-5.0 and clang++-5.0 respectively. Since you are on centOS we dont install any pre-requisites for you. You will have to install all packages manually yourself. Such as: gcc, g++-multilib, clang-5.0 (or later), curl, cmake, nasm, qemu, python-pip etc.

IncludeOS is a CMake project, so its possible to build it manually as well once all the bare minimum dependencies are satisfied.

The full list of deps should be listed when running install.sh

fwsGonzo avatar Jun 13 '18 07:06 fwsGonzo

Thank you! @fwsGonzo @AndreasAakesson install.sh doesn't work. See:

$./install.sh
>>> Sorry <<<
Currently only Debian testing/jessie backports, Ubuntu, Fedora, Arch,
and OSX are actively supported for *building* IncludeOS.
On other Linux distros it shouldn't be that hard to get it to work - take
a look at ./etc/install_from_bundle.sh

I will try to use clang-5.0 again.

zhang2639 avatar Jun 13 '18 08:06 zhang2639

I have just downloaded CentOS 7 ISO, but I have no experience using it.. I will report back progress :P

fwsGonzo avatar Jun 13 '18 08:06 fwsGonzo

If you can somehow manage to install all our pre-requisites I have detailed a way to manually install the OS here: https://github.com/hioa-cs/IncludeOS/issues/1804#issuecomment-396869742 Its not as good as running install.sh, which also builds and install several other cmake projects: vmbuild, chainloader, diskbuilder etc. That said, vmbuild and diskbuilder are both native programs which should be easy to build and install.

fwsGonzo avatar Jun 13 '18 09:06 fwsGonzo

Yes, cmake did succeed using #1804 . See:

-- Target CPU x86_64
-- Target triple x86_64-pc-linux-elf
-- The C compiler identification is GNU 4.8.5
-- The CXX compiler identification is GNU 4.8.5
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using vanilla CPU features: SSE3. CAPABS = -msse3 -mfpmath=sse
-- Init git submodule: mod/GSL
-- Init git submodule: mod/http-parser
-- Init git submodule: mod/uzlib
-- Init git submodule: mod/rapidjson
-- Init git submodule: NaCl
-- The ASM_NASM compiler identification is unknown
-- Found assembler: /usr/bin/nasm
-- Init git submodule: lib/protobuf
-- Configuring done
-- Generating done
-- Build files have been written to: /home/zsq169608/IncludeOS/build_x86_64
------------------------------------------------------------------------------------

I use clang-6.0 and clang++-6.0 Then I use make, It failed again. See:

Scanning dependencies of target solo5_repo
Scanning dependencies of target diskbuilder
Scanning dependencies of target botan
Scanning dependencies of target nacl_bin
Scanning dependencies of target openssl_bundle
Scanning dependencies of target vmbuild
Scanning dependencies of target crti
Scanning dependencies of target crtn
Scanning dependencies of target boot_logger
Scanning dependencies of target run
Scanning dependencies of target hotswap64
Scanning dependencies of target default_stdout
[  0%] Creating directories for 'diskbuilder'
[  0%] Creating directories for 'solo5_repo'
[  1%] Creating directories for 'nacl_bin'
[  2%] Creating directories for 'vmbuild'
[  3%] Creating directories for 'botan'
[  3%] Creating directories for 'openssl_bundle'
[  4%] Building ASM_NASM object src/arch/x86_64/CMakeFiles/crti.dir/crti.asm.o
[  4%] Building ASM_NASM object src/arch/x86_64/CMakeFiles/crtn.dir/crtn.asm.o
[  4%] Generating bootloader
[  4%] Building CXX object src/drivers/CMakeFiles/boot_logger.dir/bootlog.cpp.o
[  4%] Generating hotswap64.bin
[  5%] Building CXX object src/drivers/CMakeFiles/default_stdout.dir/default_stdout.cpp.o
c++: error: unrecognized command line option ‘-std=c++17’
c++: error: unrecognized command line option ‘-std=c++17’
make[2]: *** [src/drivers/CMakeFiles/boot_logger.dir/bootlog.cpp.o] Error 1
make[2]: *** [src/drivers/CMakeFiles/default_stdout.dir/default_stdout.cpp.o] Error 1
make[1]: *** [src/drivers/CMakeFiles/boot_logger.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [src/drivers/CMakeFiles/default_stdout.dir/all] Error 2
[  5%] Linking CXX static library libcrti.a
[  5%] Linking CXX static library libcrtn.a
[  5%] Built target hotswap64
[  5%] Built target run
[  5%] No download step for 'diskbuilder'
[  5%] Performing download step (git clone) for 'solo5_repo'
[  5%] No download step for 'vmbuild'
[  5%] Performing download step (download, verify and extract) for 'nacl_bin'
[  5%] Performing download step (download, verify and extract) for 'openssl_bundle'
[  6%] Performing download step (download, verify and extract) for 'botan'
-- downloading...
     src='https://github.com/includeos/NaCl/releases/download/v0.1.0/nacl_bin.tar.gz'
     dst='/home/zsq169608/IncludeOS/build_x86_64/nacl_bin/src/nacl_bin.tar.gz'
     timeout='none'
-- downloading...
     src='https://github.com/fwsGonzo/OpenSSL_bundle/releases/download/v1.0/openssl_bundle.tar.gz'
     dst='/home/zsq169608/IncludeOS/build_x86_64/openssl/src/openssl_bundle.tar.gz'
     timeout='none'
CMake Error at nacl_bin-stamp/download-nacl_bin.cmake:27 (message):
  error: downloading
  'https://github.com/includeos/NaCl/releases/download/v0.1.0/nacl_bin.tar.gz'
  failed

    status_code: 1
    status_string: "Unsupported protocol"
    log: Protocol "https" not supported or disabled in libcurl

  Closing connection -1





-- downloading...
     src='https://github.com/includeos/botan/releases/download/inc-3.0/botan-includeos-x86_64.tar.gz'
     dst='/home/zsq169608/IncludeOS/build_x86_64/botan/src/botan-includeos-x86_64.tar.gz'
     timeout='none'
CMake Error at openssl_bundle-stamp/download-openssl_bundle.cmake:27 (message):
  error: downloading
  'https://github.com/fwsGonzo/OpenSSL_bundle/releases/download/v1.0/openssl_bundle.tar.gz'
  failed

    status_code: 1
    status_string: "Unsupported protocol"
    log: Protocol "https" not supported or disabled in libcurl

  Closing connection -1





make[2]: *** [nacl_bin/src/nacl_bin-stamp/nacl_bin-download] Error 1
CMake Error at botan-stamp/download-botan.cmake:27 (message):
  error: downloading
  'https://github.com/includeos/botan/releases/download/inc-3.0/botan-includeos-x86_64.tar.gz'
  failed

    status_code: 1
    status_string: "Unsupported protocol"
    log: Protocol "https" not supported or disabled in libcurl

  Closing connection -1





[  7%] No patch step for 'diskbuilder'
make[1]: *** [CMakeFiles/nacl_bin.dir/all] Error 2
[  7%] No update step for 'diskbuilder'
make[2]: *** [openssl/src/openssl_bundle-stamp/openssl_bundle-download] Error 1
[  7%] No patch step for 'vmbuild'
make[1]: *** [CMakeFiles/openssl_bundle.dir/all] Error 2
[  7%] No update step for 'vmbuild'
make[2]: *** [botan/src/botan-stamp/botan-download] Error 1
make[1]: *** [CMakeFiles/botan.dir/all] Error 2
[  7%] Built target crtn
[  7%] Built target crti
Cloning into 'solo5_repo'...
[  7%] Performing configure step for 'diskbuilder'
[  8%] Performing configure step for 'vmbuild'
-- The C compiler identification is GNU 4.8.5
-- The C compiler identification is GNU 4.8.5
-- The CXX compiler identification is GNU 4.8.5
-- The CXX compiler identification is GNU 4.8.5
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting CXX compile features - done
-- Configuring done
-- Configuring done
-- Generating done
-- Build files have been written to: /home/zsq169608/IncludeOS/diskimagebuild/build
-- Generating done
-- Build files have been written to: /home/zsq169608/IncludeOS/vmbuild/build
[  8%] Performing build step for 'diskbuilder'
[  8%] Performing build step for 'vmbuild'
Scanning dependencies of target diskbuilder
Scanning dependencies of target elf_syms
Scanning dependencies of target vmbuild
[ 25%] Building CXX object CMakeFiles/diskbuilder.dir/main.cpp.o
[ 50%] Building CXX object CMakeFiles/diskbuilder.dir/filetree.cpp.o
[ 75%] Building CXX object CMakeFiles/diskbuilder.dir/writer.cpp.o
c++: error: unrecognized command line option ‘-std=c++14’
c++: error: unrecognized command line option ‘-std=c++14’
make[5]: *** [CMakeFiles/diskbuilder.dir/main.cpp.o] Error 1
make[5]: *** Waiting for unfinished jobs....
c++: error: unrecognized command line option ‘-std=c++14’
make[5]: *** [CMakeFiles/diskbuilder.dir/writer.cpp.o] Error 1
make[5]: *** [CMakeFiles/diskbuilder.dir/filetree.cpp.o] Error 1
make[4]: *** [CMakeFiles/diskbuilder.dir/all] Error 2
make[3]: *** [all] Error 2
make[2]: *** [diskbuilder-prefix/src/diskbuilder-stamp/diskbuilder-build] Error 2
make[1]: *** [CMakeFiles/diskbuilder.dir/all] Error 2
[ 20%] Building CXX object CMakeFiles/vmbuild.dir/vmbuild.cpp.o
c++: error: unrecognized command line option ‘-std=c++14’
make[5]: *** [CMakeFiles/vmbuild.dir/vmbuild.cpp.o] Error 1
make[4]: *** [CMakeFiles/vmbuild.dir/all] Error 2
make[4]: *** Waiting for unfinished jobs....
[ 40%] Building CXX object CMakeFiles/elf_syms.dir/elf_syms.cpp.o
[ 60%] Building CXX object CMakeFiles/elf_syms.dir/home/zsq169608/IncludeOS/src/util/crc32.cpp.o
c++: error: unrecognized command line option ‘-std=c++14’
make[5]: *** [CMakeFiles/elf_syms.dir/elf_syms.cpp.o] Error 1
make[5]: *** Waiting for unfinished jobs....
c++: error: unrecognized command line option ‘-std=c++14’
make[5]: *** [CMakeFiles/elf_syms.dir/home/zsq169608/IncludeOS/src/util/crc32.cpp.o] Error 1
make[4]: *** [CMakeFiles/elf_syms.dir/all] Error 2
make[3]: *** [all] Error 2
make[2]: *** [vmbuild/src/vmbuild-stamp/vmbuild-build] Error 2
make[1]: *** [CMakeFiles/vmbuild.dir/all] Error 2
Note: checking out '2765e0f5f090c0b27a8d62a48285842236e7d20f'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b new_branch_name

HEAD is now at 2765e0f... Merge pull request #220 from Weichen81/arm64-refactor
[  9%] No patch step for 'solo5_repo'
[  9%] No update step for 'solo5_repo'
[  9%] Performing configure step for 'solo5_repo'
[  9%] Performing build step for 'solo5_repo'
make[4]: *** read jobs pipe: Invalid argument.  Stop.
make[4]: *** Waiting for unfinished jobs....
make[3]: *** [ukvm] Error 2
make[2]: *** [precompiled/src/solo5_repo-stamp/solo5_repo-build] Error 2
make[1]: *** [CMakeFiles/solo5_repo.dir/all] Error 2
make: *** [all] Error 2

Perhaps It is c++ error?

zhang2639 avatar Jun 13 '18 12:06 zhang2639

Yes it looks like your default C++ compiler is not recent enough to support C++14. If you lost CC and CXX environment variables before you started building, CMake will revert to detecting whatever compiler is default on your system. I suggest either upgrading that compiler, making clang-5.0 the default or making sure that CMake detects clang-5.0 during CMakes configuration phase. From your logs:

[  8%] Performing build step for 'vmbuild'
c++: error: unrecognized command line option ‘-std=c++14’

This means the compiler is called "c++" which is a local alias for something else. You can find out what compiler this is by doing c++ --version. And it doesnt support c++14 yet, so its too old. To reconfigure you will have to delete the build folders in all the subprojects that need it (vmbuild, chainload, diskbuilder)

fwsGonzo avatar Jun 13 '18 20:06 fwsGonzo

I see now, CMake is detecting a really old GCC:

-- The C compiler identification is GNU 4.8.5
-- The CXX compiler identification is GNU 4.8.5

My local GCC version here is 7.3.0 and clang is 6.0. I recommend building with clang-5 or clang-6. We dont yet support building with GCC. I have it on my TODO list to support GCC. In any case, any GCC below 6.1 is probably too old to build modern C++.

fwsGonzo avatar Jun 13 '18 20:06 fwsGonzo

Im going to create a manual_build.sh script that will help people who for any reason need to build the OS and install it manually :) In the end, its all about making CMake detect a new enough compiler, but it doesnt help that if you misconfigured cmake you actually have to delete all the build folders to try again.

fwsGonzo avatar Jun 13 '18 21:06 fwsGonzo

I build with clang-6,but make failed. See:

$make -j20
[  0%] Performing build step for 'solo5_repo'
[  0%] Performing download step (download, verify and extract) for 'nacl_bin'
[  1%] Performing download step (download, verify and extract) for 'botan'
[  1%] Performing build step for 'vmbuild'
[  2%] Performing build step for 'diskbuilder'
[  2%] Built target crti
[  2%] Performing download step (download, verify and extract) for 'openssl_bundle'
[  2%] Built target crtn
[  2%] Built target run
[  2%] Building CXX object src/drivers/CMakeFiles/boot_logger.dir/bootlog.cpp.o
[  2%] Built target hotswap64
[  3%] Building CXX object src/drivers/CMakeFiles/default_stdout.dir/default_stdout.cpp.o
-- downloading...
     src='https://github.com/includeos/NaCl/releases/download/v0.1.0/nacl_bin.tar.gz'
     dst='/home/zsq169608/IncludeOS/build_x86_64/nacl_bin/src/nacl_bin.tar.gz'
     timeout='none'
-- downloading...
     src='https://github.com/includeos/botan/releases/download/inc-3.0/botan-includeos-x86_64.tar.gz'
     dst='/home/zsq169608/IncludeOS/build_x86_64/botan/src/botan-includeos-x86_64.tar.gz'
     timeout='none'
CMake Error at nacl_bin-stamp/download-nacl_bin.cmake:27 (message):
  error: downloading
  'https://github.com/includeos/NaCl/releases/download/v0.1.0/nacl_bin.tar.gz'
  failed

    status_code: 1
    status_string: "Unsupported protocol"
    log: Protocol "https" not supported or disabled in libcurl

  Closing connection -1





CMake Error at botan-stamp/download-botan.cmake:27 (message):
  error: downloading
  'https://github.com/includeos/botan/releases/download/inc-3.0/botan-includeos-x86_64.tar.gz'
  failed

    status_code: 1
    status_string: "Unsupported protocol"
    log: Protocol "https" not supported or disabled in libcurl

  Closing connection -1





-- downloading...
     src='https://github.com/fwsGonzo/OpenSSL_bundle/releases/download/v1.0/openssl_bundle.tar.gz'
     dst='/home/zsq169608/IncludeOS/build_x86_64/openssl/src/openssl_bundle.tar.gz'
     timeout='none'
make[4]: *** read jobs pipe: Invalid argument.  Stop.
make[4]: *** Waiting for unfinished jobs....
make[2]: *** [nacl_bin/src/nacl_bin-stamp/nacl_bin-download] Error 1
make[2]: *** [botan/src/botan-stamp/botan-download] Error 1
CMake Error at openssl_bundle-stamp/download-openssl_bundle.cmake:27 (message):
  error: downloading
  'https://github.com/fwsGonzo/OpenSSL_bundle/releases/download/v1.0/openssl_bundle.tar.gz'
  failed

    status_code: 1
    status_string: "Unsupported protocol"
    log: Protocol "https" not supported or disabled in libcurl

  Closing connection -1





make[1]: *** [CMakeFiles/nacl_bin.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [CMakeFiles/botan.dir/all] Error 2
make[2]: *** [openssl/src/openssl_bundle-stamp/openssl_bundle-download] Error 1
make[1]: *** [CMakeFiles/openssl_bundle.dir/all] Error 2
[ 40%] Built target vmbuild
[100%] Built target diskbuilder
[100%] Built target elf_syms
[  3%] Linking CXX static library libdefault_stdout.a
[  4%] Linking CXX static library libboot_logger.a
make[3]: *** [ukvm] Error 2
[  4%] Performing install step for 'diskbuilder'
make[2]: *** [precompiled/src/solo5_repo-stamp/solo5_repo-build] Error 2
[  4%] Performing install step for 'vmbuild'
make[1]: *** [CMakeFiles/solo5_repo.dir/all] Error 2
[  4%] Built target default_stdout
[  4%] Built target boot_logger
[100%] Built target diskbuilder
[ 40%] Built target vmbuild
[100%] Built target elf_syms
Install the project...
Install the project...
-- Install configuration: ""
-- Install configuration: ""
[  5%] Completed 'diskbuilder'
[  5%] Completed 'vmbuild'
[  6%] Built target diskbuilder
[  8%] Built target vmbuild
make: *** [all] Error 2

It seems like download error. Any suggestions?

zhang2639 avatar Jun 14 '18 09:06 zhang2639

error is: log: Protocol "https" not supported or disabled in libcurl

some hints in there: https://stackoverflow.com/questions/6884669/curl-1-protocol-https-not-supported-or-disabled-in-libcurl

I am almost done with the manual-build script, Im just testing it now The manual build script is here now: https://github.com/hioa-cs/IncludeOS/pull/1807

fwsGonzo avatar Jun 14 '18 09:06 fwsGonzo

Thank you, You are warm-hearted! I will try again.

zhang2639 avatar Jun 14 '18 09:06 zhang2639

It's strange. See:

$curl -V
curl 7.51.0 (x86_64-pc-linux-gnu) libcurl/7.51.0 OpenSSL/1.0.2k zlib/1.2.7
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: Largefile NTLM NTLM_WB SSL libz UnixSockets

curl supports https. And I download nacl_bin.tar.gzbotan-includeos-x86_64.tar.gz and openssl_bundle.tar.gz. Put them in place. But I fail again. Then I install curl again. But it doesn't work.

zhang2639 avatar Jun 14 '18 10:06 zhang2639

Ok, I will try to build myself of CentOS 7. I already have it installed, Im just not very good with using yum

so far so good... had to build a few things from source, and others I got via devtoolset-7 I got a custom repo for clang-5.0.0 but Im having trouble forcing it to use a modern C++ std library..

fwsGonzo avatar Jun 14 '18 17:06 fwsGonzo

Ok, I can reproduce your problem now - but Im getting another error - just failed downloads and mismatching MD5 hashes. :weary:

fwsGonzo avatar Jun 14 '18 19:06 fwsGonzo

@zhang2639 I managed to fix the downloading issue: When you are building CMake from source, you have to enable OpenSSL otherwise CMake will just silently fail all the downloads because its using an "internal curl".

I built CMake like this: ./bootstrap -- -DCMAKE_USE_OPENSSL=ON I think with this we can make a CentOS building guide. Building CMake is fairly simple, and all the other things can be downloaded with yum more or less.

I then built a service:

mkdir build
cmake ..
make -j

To run a service was a bit harder, because I havent installed clang properly so I had to build the OS with GCC and those services dont really run properly. But anyways, it built and I could start the demo_example service:

/usr/libexec/qemu-kvm --enable-kvm -cpu host -kernel $INCLUDEOS_PREFIX/includeos/chainloader -initrd IncludeOS_example -nographic

screenshot from 2018-06-14 21-43-11

fwsGonzo avatar Jun 14 '18 19:06 fwsGonzo

Yeah, It works! I am making now. perhaps It may be succeed. After cmake ,using make -j20 occurs error, see:

[  0%] Performing build step for 'solo5_repo'
[  0%] Performing build step for 'diskbuilder'
[  1%] Built target botan
[  2%] Performing build step for 'vmbuild'
[  5%] Built target nacl_bin
[  6%] Built target crti
[  6%] Built target run
[  6%] Built target openssl_bundle
[  7%] Built target boot_logger
[  7%] Built target hotswap64
[  8%] Built target crtn
[  8%] Built target default_stdout
make[4]: *** read jobs pipe: Invalid argument.  Stop.
make[4]: *** Waiting for unfinished jobs....
make[3]: *** [ukvm] Error 2
make[2]: *** [precompiled/src/solo5_repo-stamp/solo5_repo-build] Error 2
[100%] Built target diskbuilder
[ 40%] Built target vmbuild
make[1]: *** [CMakeFiles/solo5_repo.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[100%] Built target elf_syms
[  8%] Performing install step for 'diskbuilder'
[  8%] Performing install step for 'vmbuild'
[100%] Built target diskbuilder
[ 40%] Built target vmbuild
[100%] Built target elf_syms
Install the project...
Install the project...
-- Install configuration: ""
-- Install configuration: ""
[  9%] Completed 'diskbuilder'
[  9%] Completed 'vmbuild'
[ 10%] Built target diskbuilder
[ 12%] Built target vmbuild
make: *** [all] Error 2

It failed because make[4]: *** read jobs pipe: Invalid argument. Stop. Then I use make, It works! I will report the latest progress later. In the end,thank you for helping me.

zhang2639 avatar Jun 15 '18 02:06 zhang2639

As you said before. I install it successfully. But to run a service was a bit harder. See:

[ 40%] Built target config_json
[100%] Built target service
/home/zsq169608/includeos/includeos/bin/vmbuild: error while loading shared libraries: libc++.so.1: cannot open shared object file: No such file or directory
make[2]: *** [CMakeFiles/prepend_bootloader] Error 127
make[1]: *** [CMakeFiles/prepend_bootloader.dir/all] Error 2
make: *** [all] Error 2

Now I am busy. I will try qemu-kvm later. Thank you.

zhang2639 avatar Jun 15 '18 03:06 zhang2639

My build process is:

  1. install cmake 3.4.3 or higher (clang 6.0 use cmake3.4.3 at least) note: when use ./bootstrap , append -- -DCMAKE_USE_OPENSSL=ON ,like ./bootstrap -- -DCMAKE_USE_OPENSSL=ON
  2. install clang6.0
  3. install other things like curl make nasm bridge-utils qemu and so on.
  4. cd IncludeOS
  5. open ~/.bashrc and append:
    export INCLUDEOS_SRC= path to repo
    export INCLUDEOS_PREFIX=/home/your_username/includeos
    export CC=/usr/local/bin/clang
    export CXX=/usr/local/bin/l=clang++
    export PATH=$PATH:/home/your_username/includeos/bin
  1. git submodule update --init --recursive
     export PLATFORM=x86_pc
     export ARCH=x86_64
  1. mkdir build_x86_64 && cd build_x86_64
  2. cmake .. -DCMAKE_INSTALL_PREFIX=/home/your_username/includeos
  3. make && make install (use make -jN may error)

Before install the includeos, there are many warnings when make,see:

                                            ……………………………………
/root/IncludeOS/src/posix/pthread.cpp:69:75: warning: unused parameter 'attr' [-Wunused-parameter]
int pthread_mutex_init(pthread_mutex_t* mutex, const pthread_mutexattr_t* attr)
                                                                          ^
/root/IncludeOS/src/posix/pthread.cpp:96:44: warning: unused parameter 'mutex' [-Wunused-parameter]
int pthread_mutex_destroy(pthread_mutex_t *mutex)
                                           ^
/root/IncludeOS/src/posix/pthread.cpp:112:39: warning: unused parameter 'cond' [-Wunused-parameter]
int pthread_cond_wait(pthread_cond_t* cond, pthread_mutex_t* mutex)
                                      ^
/root/IncludeOS/src/posix/pthread.cpp:112:62: warning: unused parameter 'mutex' [-Wunused-parameter]
int pthread_cond_wait(pthread_cond_t* cond, pthread_mutex_t* mutex)
                                                             ^
/root/IncludeOS/src/posix/pthread.cpp:117:44: warning: unused parameter 'cond' [-Wunused-parameter]
int pthread_cond_broadcast(pthread_cond_t* cond)
                                           ^
/root/IncludeOS/src/posix/pthread.cpp:122:41: warning: unused parameter 'cond' [-Wunused-parameter]
int pthread_cond_signal(pthread_cond_t* cond)
                                        ^
/root/IncludeOS/src/posix/pthread.cpp:127:44: warning: unused parameter 'cond' [-Wunused-parameter]
int pthread_cond_timedwait(pthread_cond_t* cond,
                                           ^
/root/IncludeOS/src/posix/pthread.cpp:128:45: warning: unused parameter 'mutex' [-Wunused-parameter]
                           pthread_mutex_t* mutex,
                                            ^
/root/IncludeOS/src/posix/pthread.cpp:129:51: warning: unused parameter 'abstime' [-Wunused-parameter]
                           const struct timespec* abstime)
                                                  ^
/root/IncludeOS/src/posix/pthread.cpp:134:42: warning: unused parameter 'cond' [-Wunused-parameter]
                                             ………………………………

Is it normal?

To start a service:

/usr/libexec/qemu-kvm --enable-kvm -cpu host -kernel $INCLUDEOS_PREFIX/includeos/chainloader -initrd IncludeOS_example -nographic

But I dont find chainloader. And I use IncludeOS/etc/build_chainloader.sh to create chainloader. It use

export PLATFORM=x86_nano
export ARCH=i686

which is inconsistent with x86_pc and x86_64. Then I change the build_chainloader.sh, and export the PLATFORM and ARCH. It always failed, see:

-- Building for arch i686, platform x86_nano
-- The C compiler identification is Clang 6.0.0
-- The CXX compiler identification is Clang 6.0.0
-- Check for working C compiler: /usr/local/bin/clang
-- Check for working C compiler: /usr/local/bin/clang -- broken

It doesn't work. Otherwise, I use boot to start a service.

    $ cp -r seed/service ~/my_service
    $ cd ~/my_service
    $ mkdir build && cd build
    $ cmake ..
    $ make
    $ boot my_service

It failed because ImportError: No module named jsonschema.See:

[root@da12bb03e9ea my_service]# boot my_service
Traceback (most recent call last):
  File "/root/includeos//bin/boot", line 126, in <module>
    from vmrunner import vmrunner
  File "/root/includeos//includeos/vmrunner/vmrunner.py", line 10, in <module>
    import validate_vm
  File "/root/includeos//includeos/vmrunner/validate_vm.py", line 2, in <module>
    from jsonschema import Draft4Validator, validators
ImportError: No module named jsonschema

I install the python34-jsonschema.noarch use yum. It doesn't work. Is it because my install process is wrong?

zhang2639 avatar Jun 16 '18 16:06 zhang2639

Not sure what python version is used. Maybe you need a python2 version jsonschema. You installeed a python3 version of it.

JohnCoconut avatar Jun 18 '18 08:06 JohnCoconut

@zhang2639 manual_build.sh downloads and installs a chainloader for you, and you dont have to use our boot python script (which really should have just been written as a Go static binary).

I am running the demo service like this on CentOS 7:

/usr/libexec/qemu-kvm --enable-kvm -cpu host -kernel $INCLUDEOS_PREFIX/includeos/chainloader -initrd IncludeOS_example -nographic

It wont have networking, but at least it will boot. To get networking we will have to add the proper network card (virtionet, vmxnet3 or e1000e).

fwsGonzo avatar Jun 18 '18 09:06 fwsGonzo

I'm sorry for the late reply. @mywtfmp3 Yes! Python2 is OK. @fwsGonzo I use /usr/libexec/qemu-kvm --enable-kvm -cpu host -kernel $INCLUDEOS_PREFIX/includeos/chainloader -initrd IncludeOS_example -nographic to start demo_service. But there is a very strange phenomenon. See:


                           #include<os> // Literally

================================================================================
     [ Kernel ] Stack: 0x1ffe48
     [ Kernel ] Boot magic: 0x2badb002, addr: 0x9500
     [ Kernel ] Registering plugin Autoconf plugin
     [ x86_64 ] Initializing paging
                * Supported page sizes: 4.000_KiB, 2.000_MiB, 1.000_GiB
                * Adding 512 1GiB entries @ 0x0 -> 0x8000000000
                * Marking page 0 as not present
                * Allowing execute on 0x201000 -> 0x42d0f8
                * Passing page tables to CPU
     [ Kernel ] Booted with multiboot
                * Boot flags: 0x4f
                * Valid memory (130687 Kib):
                  0x00000000 - 0x0009fbff (639 Kib)
                  0x00100000 - 0x07ffffff (130048 Kib)

                * Booted with parameters @ 0x8000: IncludeOS_example
                * Multiboot provided memory map  (6 entries @ 0x9000)
                  0x0000000000 - 0x000009fbff FREE (639 Kb.)
                  0x000009fc00 - 0x000009ffff RESERVED (1 Kb.)
                  0x00000f0000 - 0x00000fffff RESERVED (64 Kb.)
                  0x0000100000 - 0x0007ffdfff FREE (130040 Kb.)
                  0x0007ffe000 - 0x0007ffffff RESERVED (8 Kb.)
                  0x00feffc000 - 0x00feffffff RESERVED (16 Kb.)

    [ Statman ] Initialized with 585 stats capacity
     [ Kernel ] Assigning fixed memory ranges (Memory map)
     [ Kernel ] Assigning heap
================================================================================

                           #include<os> // Literally

================================================================================
     [ Kernel ] Stack: 0x1ffe48
     [ Kernel ] Boot magic: 0x2badb002, addr: 0x9500
     [ Kernel ] Registering plugin Autoconf plugin
     [ x86_64 ] Initializing paging
                * Supported page sizes: 4.000_KiB, 2.000_MiB, 1.000_GiB
                * Adding 512 1GiB entries @ 0x0 -> 0x8000000000
                * Marking page 0 as not present
                * Allowing execute on 0x201000 -> 0x42d0f8
                * Passing page tables to CPU
     [ Kernel ] Booted with multiboot
                * Boot flags: 0x4f
                * Valid memory (130687 Kib):
                  0x00000000 - 0x0009fbff (639 Kib)
                  0x00100000 - 0x07ffffff (130048 Kib)

                * Booted with parameters @ 0x8000: IncludeOS_example
                * Multiboot provided memory map  (6 entries @ 0x9000)
                  0x0000000000 - 0x000009fbff FREE (639 Kb.)
                  0x000009fc00 - 0x000009ffff RESERVED (1 Kb.)
                  0x00000f0000 - 0x00000fffff RESERVED (64 Kb.)
                  0x0000100000 - 0x0007ffdfff FREE (130040 Kb.)
                  0x0007ffe000 - 0x0007ffffff RESERVED (8 Kb.)
                  0x00feffc000 - 0x00feffffff RESERVED (16 Kb.)

    [ Statman ] Initialized with 585 stats capacity
     [ Kernel ] Assigning fixed memory ranges (Memory map)
     [ Kernel ] Assigning heap
================================================================================

                           #include<os> // Literally

================================================================================
     [ Kernel ] Stack: 0x1ffe48
     [ Kernel ] Boot magic: 0x2badb002, addr: 0x9500
     [ Kernel ] Registering plugin Autoconf plugin
     [ x86_64 ] Initializing paging
                * Supported page sizes: 4.000_KiB, 2.000_MiB, 1.000_GiB
                * Adding 512 1GiB entries @ 0x0 -> 0x8000000000
                * Marking page 0 as not present
                * Allowing execute on 0x201000 -> 0x42d0f8
                * Passing page tables to CPU
     [ Kernel ] Booted with multiboot
                * Boot flags: 0x4f
                * Valid memory (130687 Kib):
                  0x00000000 - 0x0009fbff (639 Kib)
                  0x00100000 - 0x07ffffff (130048 Kib)

                * Booted with parameters @ 0x8000: IncludeOS_example
                * Multiboot provided memory map  (6 entries @ 0x9000)
                  0x0000000000 - 0x000009fbff FREE (639 Kb.)
                  0x000009fc00 - 0x000009ffff RESERVED (1 Kb.)
                  0x00000f0000 - 0x00000fffff RESERVED (64 Kb.)
                  0x0000100000 - 0x0007ffdfff FREE (130040 Kb.)
                  0x0007ffe000 - 0x0007ffffff RESERVED (8 Kb.)
                  0x00feffc000 - 0x00feffffff RESERVED (16 Kb.)

    [ Statman ] Initialized with 585 stats capacity
     [ Kernel ] Assigning fixed memory ranges (Memory map)
     [ Kernel ] Assigning heap
             ………………………………………………………………………………………………………………

It's cycling all the time. Failed to start the service.

zhang2639 avatar Jun 22 '18 10:06 zhang2639

@zhang2639 in this instance it means either the OS or the Service was built with GCC instead of clang. Right now we dont support GCC, but I will work towards supporting it very soon. You can use ccmake to check which compiler it detected for any build folder:

ccmake ..

Then press T to toggle extra information, and look at CMAKE_CXX_COMPILER.

EDIT: Ive done some more research and Ive found that you can build the service with gcc/g++, but the OS has to be built with clang-5.0 or later. So, can you check what compiler you have used to build the OS?

fwsGonzo avatar Jun 22 '18 10:06 fwsGonzo

@fwsGonzo Thank you sir. I'm sorry for the late reply. I use ccmake . The results are as follows. ccmake in /IncluedeOS/build_x86_64 :

CMAKE_CXX_COMPILER               /usr/local/bin/clang++
 CMAKE_CXX_FLAGS
 CMAKE_CXX_FLAGS_DEBUG            -g
 CMAKE_CXX_FLAGS_MINSIZEREL       -Os -DNDEBUG
 CMAKE_CXX_FLAGS_RELEASE          -O3 -DNDEBUG
 CMAKE_CXX_FLAGS_RELWITHDEBINFO   -O2 -g -DNDEBUG
 CMAKE_C_COMPILER                 /usr/local/bin/clang

ccmake in build of service:

CMAKE_CXX_COMPILER               /usr/local/bin/clang++
 CMAKE_CXX_FLAGS
 CMAKE_CXX_FLAGS_DEBUG            -g
 CMAKE_CXX_FLAGS_MINSIZEREL       -Os -DNDEBUG
 CMAKE_CXX_FLAGS_RELEASE          -O3 -DNDEBUG
 CMAKE_CXX_FLAGS_RELWITHDEBINFO   -O2 -g -DNDEBUG
 CMAKE_C_COMPILER                 /usr/local/bin/clang

/usr/local/bin/clang++ is clang++ 6.0, see:

$ /usr/local/bin/clang++ --version
clang version 6.0.0 (tags/RELEASE_600/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/local/bin

/usr/local/bin/clang is clang 6.0,see:

$ /usr/local/bin/clang --version
clang version 6.0.0 (tags/RELEASE_600/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /usr/local/bin

It seems that I build OS and service using clang instead of gcc.

zhang2639 avatar Jun 25 '18 06:06 zhang2639

In that case I dont know whats wrong. Building with clang has always just worked, and there shouldnt be much more to it. Perhaps the VM is being configured in a weird manner? We usually use boot locally which just passes a few standard qemu arguments. Which service did you build and boot?

fwsGonzo avatar Jun 27 '18 09:06 fwsGonzo

I build demo_service. Maybe I should build os and service again.

zhang2639 avatar Jun 27 '18 10:06 zhang2639

I just tried to build and run with clang-5 myself in CentOS and it just wouldn't boot, even though it built just fine. So instead now I'm building with gcc-7, and I get the same problem. Here is the new issue: https://github.com/hioa-cs/IncludeOS/issues/1835

fwsGonzo avatar Jun 29 '18 09:06 fwsGonzo