generator

generator copied to clipboard

#+TITLE: Build Generator #+AUTHOR: Jan Moringen #+EMAIL: [email protected]

#+LINK: jenkins https://jenkins.io/ #+LINK: sbcl http://sbcl.org #+LINK: quicklisp https://www.quicklisp.org/beta/

#+ATTR_HTML: :alt "build status image" :title Build Status :align right [[https://travis-ci.org/RDTK/generator][https://travis-ci.org/RDTK/generator.svg]]

• Introduction

  The build generator is a program for building the components of software systems from declarative recipes describing software projects and software systems (and a few other things). This can be done in several ways. One way is automatically installing and configuring a [[jenkins][Jenkins]] instance, then automatically generating Jenkins jobs which do the actual building. Other ways include generating [[https://docs.docker.com/engine/reference/builder/][Dockerfiles]], generating [[https://www.gnu.org/software/make/manual/make.html#Introduction][Makefiles]] and building in the local filesystem under the control of the build generator process.

• Obtaining a Build Generator Binary

  One way to obtain a build generator binary is [[*Building][building from source]]. Furthermore, the following pre-built binaries are available:

  • Binaries for released versions can be downloaded in the [[https://github.com/rdtk/generator/releases]["Releases" section of the GitHub project]].

  • Binaries for the master version can be downloaded from the [[https://jenkins-cse.bob.ci.cit-ec.net/view/build-generator-nightly-ci-docker/job/build-generator-master-ci-docker-build-generator-nightly][continous integration server]].

• Dependencies

  • Common Lisp Implementation :: At the moment, the code is almost but not quite portable and can only be used with the [[sbcl][SBCL]] implementation of Common Lisp.

  • System Libraries :: For secure communication with the Jenkins server as well as archive retrieval via HTTPS, the build generator relies on the OpenSSL library, specifically =libssl.so.*=. In Debian-derived systems, the =openssl= package provides this library.

  • Lisp dependencies :: The following Lisp dependencies cannot be installed automatically via [[quicklisp][Quicklisp]], yet:

    • The [[https://github.com/scymtym/model.transform.trace][model.transform.trace]] project

    • The [[https://github.com/scymtym/language.yaml][language.yaml]] project

    • The [[https://github.com/scymtym/text.source-location][text.source-location]] project

    • The [[https://code.cor-lab.org/projects/rosetta][rosetta]] project

      • The [[https://code.cor-lab.org/projects/rosetta][rosetta-project]] project

• Building

  The following steps are intended to achieve the following

  • building the master version of the build generator

  • without elevated privileges

  • without polluting system or user directories outside of a =build= sub-directory within the project directory

  The last two points make things a little more complicated but hopefully more useful overall.

  1. Start in the project directory

     #+BEGIN_SRC bash git clone https://github.com/RDTK/generator cd generator mkdir -p build #+END_SRC

  2. Installing SBCL

     Not all Debian-like systems provide a reasonably recent version of the [[sbcl][SBCL]] implementation of Common Lisp. It is therefore better to install a binary release from the project website:

     #+BEGIN_SRC bash ( cd build curl -L "https://prdownloads.sourceforge.net/sbcl/sbcl-1.5.0-x86-64-linux-binary.tar.bz2" | tar -xj cd sbcl-* INSTALL_ROOT="$(pwd)/../sbcl" sh install.sh ) #+END_SRC

  3. Installing Quicklisp

     [[quicklisp][Quicklisp]] is a package manager and installer for Common Lisp. See [[quicklisp]] for detailed instructions including cryptographic verification of the downloaded file.

     A quick way to install Quicklisp is:

     #+BEGIN_SRC bash ( cd build curl -o quicklisp.lisp "https://beta.quicklisp.org/quicklisp.lisp" SBCL_HOME="$(pwd)/sbcl/lib/sbcl"
     sbcl/bin/sbcl --noinform --non-interactive
     --load quicklisp.lisp
     --eval '(quicklisp-quickstart:install :path "quicklisp")' ) #+END_SRC

     This creates the directory =build/quicklisp=. Dependencies will be downloaded into sub-directories of this directory.

  4. Installing non-Quicklisp dependencies

     A quick way to install the [[Dependencies][non-Quicklisp dependencies]] is

     #+BEGIN_SRC bash ( cd build/quicklisp/local-projects
     && git clone -b future https://github.com/scymtym/model.transform.trace
     && git clone -b future https://github.com/scymtym/language.yaml
     && git clone -b future https://github.com/scymtym/text.source-location
     && git clone https://code.cor-lab.de/git/rosetta.git.cl
     && git clone https://code.cor-lab.de/git/rosetta.git.project-cl ) #+END_SRC

  5. Building the executable

     #+BEGIN_SRC bash SBCL_HOME="$(pwd)/build/sbcl/lib/sbcl"
     ASDF_OUTPUT_TRANSLATIONS="/:$(pwd)/build/fasl-cache/"
     CL_SOURCE_REGISTRY="$(pwd)//:"
     build/sbcl/bin/sbcl --noinform --non-interactive
     --load build/quicklisp/setup.lisp
     --eval '(ql:quickload :build-generator.commandline-interface)'
     --eval '(asdf:make :build-generator.commandline-interface)' #+END_SRC

  This should produce a binary named =build-generator= in the project root directory.

• Running Tests

  Assuming the steps in [[Building]] have been completed, the following runs the unit tests of the project

  #+BEGIN_SRC bash SBCL_HOME="$(pwd)/build/sbcl/lib/sbcl"
  ASDF_OUTPUT_TRANSLATIONS="/:$(pwd)/build/fasl-cache/"
  CL_SOURCE_REGISTRY="$(pwd)//:"
  build/sbcl/bin/sbcl --noinform --non-interactive
  --load build/quicklisp/setup.lisp
  --eval '(ql:quickload :build-generator/test)'
  --eval '(asdf:test-system :build-generator)' #+END_SRC

• Contributing

  If you want to contribute to this project, please

  • Submit your intended changes as coherent pull requests.

  • Rebase onto the master branch and squash any fixups and corrections.

  • Make sure the unit tests pass.

• Acknowledgments

  The development of this software has been supported as follows:

  • The development of this software was supported by CoR-Lab, Research Institute for Cognition and Robotics Bielefeld University.

  • This work was supported by the Cluster of Excellence Cognitive Interaction Technology ‘CITEC’ (EXC 277) at Bielefeld University, which is funded by the German Research Foundation (DFG).