Set up continuous integration builds on jenkins.ros.org

[esteve]

The Open Source Robotics Foundation is willing to host continuous integration builds for meta-ros on http://jenkins.ros.org. Automated builds will make incorporating changes easier and more robust.

[bulwahn]

As a first step, I suggest to test the following operation continuously:

• Build environment/setup: always latest version of bitbake, OE-core, meta-oe and meta-ros
• local.conf: MACHINE = "qemuarm"
• always clean directories sstate and tmp-eglibc before test operation
• test operation: bitbake packagegroup-ros-world (or even bitbake core-image-ros-world)
• fails if any build error occurs
• invoke on any change of the git repositories above

This test is very likely to be executed often, fail often, and take quite some execution time (due to clean builds). From our observations how much execution time it needs, how often it is invoked and the continuous observation of which git repositories is beneficial, we might come up with a refined setup.

[KristofRobot]

Great stuff!

Additionally, we could perhaps leverage this mechanism to speed up/assist validating acceptance of new patches/commits.

E.g. we could create an additional meta-ros branch "unstable" (or whatever). New commits are first accepted in that branch after code review, then built on jenkins to fully validate them. If they pass a full build on jenkins, the commit can then be moved to the "stable" branch.

In that way, the Jenkins server could take over the time-consuming "fresh-build-to-validate" step from the meta-ros maintainers, without ever breaking the "stable" version.

[esteve]

@KristofRobot Yep, that's what we're aiming for with this. I read Lukas's comment on a patch I submitted (https://github.com/bmwcarit/meta-ros/pull/216#issuecomment-32472819) and I realized that we at the Open Source Robotics Foundation could help with that. The overall plan is to have every pull request built in Jenkins and report back to GitHub if the changes break anything, eventually we can provide more stuff, like daily builds.

[bulwahn]

Hi Esteve, has there been any progress on this matter?

[esteve]

@bulwahn We're experimenting with Docker at OSRF to isolate our build environments, you can track our progress at https://github.com/esteve/jenkins_scripts/tree/docker We already have a script for building meta-ros inside a Docker container, but we haven't integrated the scripts with Jenkins yet.

I'm fairly confident that next week we'll have the base infrastructure in place and start integrating it with Jenkins, hopefully the week after we'll be ready to integrate GitHub pull request notifications with our Jenkins buildfarm and the Docker containers.

[esteve]

@bulwahn I just pushed a new revision of the meta-ros build scripts we're working on. I'll update the instructions tomorrow, but if you want to give it a try, install Docker (http://docs.docker.io/en/latest/installation/), python-empy and type the following from a local copy of the jenkins_scripts repo:

$ python generate_meta_ros_job.py ubuntu precise i386 /tmp/temporary_workspace PATH_TO_LOCAL_ANGSTROM_SETUP_SCRIPTS PATH_TO_LOCAL_META_ROS beagleboard

where PATH_TO_LOCAL_ANGSTROM_SETUP_SCRIPTS and PATH_TO_LOCAL_META_ROS, point to a copy of Angstrom's setup-script repository (https://github.com/Angstrom-distribution/setup-scripts) and a copy of a meta-ros repository or branch, respectively.

The goal of the scripts is to have fully isolated environments where meta-ros can be built. I'll update the scripts tomorrow to spit out the built files to a given directory on the host system, so we could eventually provide automated builds for certain machine types (Beagleboard, Raspberry Pi, etc.)

[KristofRobot]

@esteve I tried this, using the following command:

python generate_meta_ros_job.py ubuntu saucy /tmp/temporary_workspace /home/kristof/meta-ros-jenkins/setup-scripts /home/kristof/OE/meta-ros beagleboard

(Note: I found that specifying the architecture ('i386') would generate an error, so omitted that)

That seems to run largely correctly, apart from following error at the end:

fatal: empty ident name (for <rosbuild@2e82f226bc94.(none)>) not allowed

Although this error is called "fatal", a correct image seems to be generated in /tmp/temporary_workspace/output/deploy/eglibc/images/beagleboard

However, it seems that, in the process, no meta-ros packages are actually build - it seems that this builds a core-image-minimal instead.

Shouldnt we let this build core-image-ros-world.bb instead?

Thanks,

Kristof

[esteve]

@KristofRobot Sorry, I missed your comment.

That seems to run largely correctly, apart from following error at the end:

fatal: empty ident name (for <rosbuild@2e82f226bc94.(none)>) not allowed Although this error is called "fatal", a correct image seems to be generated in > /tmp/temporary_workspace/output/deploy/eglibc/images/beagleboard

It looks like a git error, caused by an identity misconfiguration, did you run this on a fresh machine? (i.e. no git config --global user.email and git config --global user.name) It'd be weird though, because you already have git configured (you wouldn't be using meta-ros without git, after all ;-)) This might be the git process running inside the Docker container, I'll check if that's the case.

Shouldnt we let this build core-image-ros-world.bb instead?

I wanted to have it as minimal as possible to test the most basic packages until we refine the script, but I agree with you it should build core-image-ros-world.

[KristofRobot]

@esteve

This might be the git process running inside the Docker container, I'll check if that's the case.

I suspect that is the case - I ran this on my normal build machine.

Thanks for looking into this!