Complete Testing Framework

[IanTheEngineer]

• [x] Add ROS Actions to test scripts
• [x] Create a test_srvs package of complex ROS Services
• [x] Add ROS Actions to test scripts
• [ ] Build in automated Jenkins testing hooks to invoke these tests on a new pull request to either rosjs or gennodejs

[NickZ]

I notice you guys have set up Travis CI; I think this would be fairly easy to implement on there?

[chris-smith]

Should be - setting up the environment and actually running the tests there has been on my list for a while...

[louisgv]

@chris-smith What else does this issue need to close?

[chris-smith]

We do automatically run tests against pull requests/commits through travis...

It'd be good if we tested against future versions of Ubuntu - I think we need docker for this because travis only supports 16.04. @IanTheEngineer was planning to look into this recently but I'm not sure if he got an opportunity.

[louisgv]

I have been investigating about init Docker via Travis and reading their documentation. After further planning, I will submit a PR that have Travis spin up docker for several distro and run the test.

https://docs.travis-ci.com/user/docker/

@chris-smith Can you give me a list of target distros we would like to test against?

[chris-smith]

I think 14.04, 16.04, and 18.04 would be good - support for 14.04 will probably drop ~soon (I think OSRF is dropping 14.04/Indigo next April) but we should continue supporting for now.

[louisgv]

I went with Travis's build matrix as it seems that it is supporting later Ubuntu versions. However, it seems that xenial and bionic cannot locate ros indigo... And for some reason trusty is failing on some dpkg call :\ ...

[IanTheEngineer]

@louisgv thanks so much for looking into this! Yes, the newer versions of Ubuntu are not supported by Travis, which means we need to use Docker to create the correct build matrix. I modeled our sns-ik repo’s Travis ci off of MoveIt’s, and that seems to work really well. I’ll add this to my todo, but it would be wonderful if you’d like to look into Dockerizing the CI as well :)

[chris-smith]

I think we'll need to use different ROS versions for xenial and bionic too (probably kinetic and melodic respectively)

[louisgv]

@IanTheEngineer I would love to dive into dockerizing the CI and try my best to help. Would you happen to have a list of material / convention / dockerizing method that you would like me to go through and follow for this issue?

I was looking at maybe we can use docker compose to ease the scripting process :-?

[louisgv]

I managed to set custom ROS version per distro, and per the CI log above, it succeed for trusty and xenial, but failed on bionic. The error message on bionic is:

E: Unable to locate package ros-melodic-catkin

Seems like the pipeline to work with catkin changed in melodic?

It is also seems travis does not play well with multidimensional matrix, thus I went ahead and manually specify all the nodejs version and Ubuntu distro combination. The latest travis build should reflect that.

https://travis-ci.org/RethinkRobotics-opensource/rosnodejs/builds/415271564?utm_source=github_status&utm_medium=notification

LMK if you have any feedback! @IanTheEngineer @chris-smith

[chris-smith]

I'm not immediately sure why ros-melodic-catkin can't be found... I'm a little surprised the 16.04 and 18.04 builds start trying to install packages though. As far as I'm aware, travis doesn't actually support either of them. @IanTheEngineer seems to have made different branches for each release in sns_ik, along with docker images for each. I'd prefer not to have to maintain different branches for each ROS release if possible though.

[louisgv]

I inspected the log again and apparently Travis does not have 18.04 for their matrix yet, and thus it fallback to 12.04:

I will setup matrix for env + docker using https://hub.docker.com/r/rfc1149/rose-dev/ and we will see if that works.