Permit extending multiple workspaces

[mikepurvis]

It was pointed out to me a while ago that it's possible to extend more than one catkin workspace, simply by manually mangling the environment vars to the union of the parents, as if one was extending the next.

There are obvious commercial applications for this, eg, where it's desirable to ship packages in a standalone binary workspace, without making assumptions about the location of the base ROS install, eg:

sudo apt-get install ros-indigo-ros-base
sudo apt-get install vendor1-ros-indigo-secret-thing
sudo apt-get install vendor2-ros-indigo-other-secret-thing
cd path/to/my/workspace
catkin config --extend /opt/ros/indigo --extend /opt/vendor1/ros/indigo --extend /opt/vendor2/ros/indigo
catkin build

It would be sweet if catkin_tools could do this.

[mikepurvis]

I can't promise immediate attention, but with some design guidance I might be able to move this along. Do we have a sense what the timeline is for 0.4 and 0.5 releases?

[jbohren]

I can't promise immediate attention, but with some design guidance I might be able to move this along. Do we have a sense what the timeline is for 0.4 and 0.5 releases?

The 0.4.0 release is imminent, and is just pending some additional documentation now that all the issues in #293 have been resolved. It brings a lot of changes so we're going to make a release announcement too. The timeline for 0.5.0 is going to depend on how much we actually want to include with it, but like it says in the roadmap, it's going to be focused on optimization and scalability features such as this one.

[stonier]

I'd also like this. We've been using this feature in our own tooling for a long time, but would like to align with these tools.

[mikepurvis]

I'm going on parental leave in January and may have some time to look at this, but I'm discouraged from putting in a lot of effort given the lack of attention to #391, #400, #414. All in all, this repo appears to have been abandoned by its maintainers.

[wjwwood]

I understand your frustration, I'm going to be spending time on this again soon. I'll also look into having more maintainers to avoid this issue in the future.

Also congrats :)

[mikepurvis]

Thanks, and yay! I :heart: catkin_tools.

[ahoarau]

any news on this feature ?

[mikepurvis]

@ahoarau Most of the guts of the needed "merge" functionality is present in the merge_envs function which I contributed in #449. Currently, this functionality is used only internally to make --isolate-install and --isolate-devel do the right thing (to build a package with multiple build dependencies which were built isolated, you need an environment which is the union of all the dependencies' environments).

So yeah— the pieces are here, but there's work remaining to break it out as a user-facing feature. I'd absolutely support that and would help with reviewing and testing it, but I will also concede that merging arbitrary resultspaces is a more complex problem than merging "isolated" ones which are part of the same catkin_tools workspace.

That isn't to say that this shouldn't be a feature, only that I will expect the implementation to be imperfect and require caution in its use. The major pitfalls should be documented, and any vendors who experiment with shipping "parallel" workspaces as suggested in the original post should be encouraged to explore ways of making those as safe as possible, for example building static binaries to avoid shipping fork versions of packages that shadow things from upstream.