Continuous Integration tests

[evanhunter]

trafficstars

Hi Steve,

Here is a patch for the use of the Travis CI server - It ensures that new code does not have memory leaks and ensures that new code has test coverage.

Evan

[msteveb]

A few comments:

• I think it would be better to move these special purpose targets out of the Makefile and just have some shell scripts in a directory (e.g. coverage/) along with a README about how to use them.
• I don't think that we need an explicit --ci-test option
• Use cc-check-progs when checking for executables such as valgrind and lcov
• Add your cflags to CCOPTS, not CFLAGS
• lcov_parse.pl in perl?!

[evanhunter]

I don't think that we need an explicit --ci-test option

How about changing it to --enable-everything or similar? That was mostly the point - to have a mode where the most amount of code gets tested...

I think it would be better to move these special purpose targets out of the Makefile and just have some shell scripts in a directory (e.g. coverage/) along with a README about how to use them.

How about an additional makefile which gets included from the main one?

Also, if the CI server is running these tests, then they need to be easy, or even automatic for contributors to run also... otherwise they'll get failed builds which tested fine on their machine.

The other items I'll fix up.

[evanhunter]

BTW: I would have used '--full' except that it doesn't do what it says on the tin... It doesn't include references or docs. In fact I've noticed that due to dependencies, docs are not generated with just 'make' even if enabled. It is only done when 'make install' happens - so contributors may accidentally break the documentation without realising.

[msteveb]

How about changing it to --enable-everything or similar? That was mostly the point - to have a mode where the most amount of code gets tested...

--full --maintainer already does that. I suggest putting the exec false fix under JIM_MAINTAINER

It doesn't include references or docs.

Even configure without --full enables both references and docs.

$ ./configure
...
Enabling references

How about an additional makefile which gets included from the main one?

I still think that is overkill. A script or two and a single 'coverage' target in the Makefile should be fine.

Also, if the CI server is running these tests, then they need to be easy, or even automatic for contributors to run also... otherwise they'll get failed builds which tested fine on their machine. In fact I've noticed that due to dependencies, docs are not generated with just 'make' even if enabled

I disagree. Most people who build Jim will never contribute. Why make them have asciidoc installed just so they can build a local copy? After all, that's the purpose of CI. If they submit a pull request, CI will verify it. I do agree, though, that CI should be building the docs.

[evanhunter]

I have reworked the Continuous Integration and coverage tests into shell scripts, expanded their scope a little and converted the perl script to TCL - now there are no changes to the makefiles/configure-script.