verbs: workspace (ws) verb for managing multiple workspaces

[jbohren]

Catkin tools is the perfect place to add tools for analyzing and managing one or more catkin workspaces. This verb could have sub-verbs for different workspace-management commands.

Rough sketch of ideas for the workspace or ws verb:

• catkin ws create [<workspace-name>] Create a new workspace with default directories and an optional identifier
• catkin ws create --extend <workspace-name> Create a new workspace that explicitly extends another workspace either by name or by path
• catkin ws save [--default] <workspace-name> Save the current workspace to a persistent file with an identifier, or set an already saved workspace as the default.
• catkin ws get <workspace-name> Print the path to the workspace identified by <workspace-name>
• catkin ws load [<workspace-name>] Load either the default or a named workspace environment from a persistent file. This could go into your shell profile so each new shell gets the workspace that you're currently using.
• catkin ws list List the saved workspaces
• catkin ws clean Remove the appropriate build and devel directories (prevents people from having to use rm -rf in their workspaces
• catkin ws info <workspace-name> Show a workspace's dependencies, which known workspaces depend on it, number of packages, if it's been built, etc
• catkin ws discover <path> Find all catkin workspaces under some path (by looking for marker file introduced here https://github.com/ros-infrastructure/catkin_pkg/pull/95)

For workspace names, there could be some defaults like ros-hydro for /opt/ros/hydro but people will also use project names for different workspaces.

The persistent file could look something like /.catkin_workspaces:

workspaces:
  hydro: '/opt/ros/hydro'
  indigo: '/opt/ros/indigo'
  overlay: '/home/jbohren/ws/overlay/devel'
  jhu: '/home/jbohren/ws/jhu/devel'
  nasa: '/home/jbohren/ITAR/nasa/devel'

A catkin ws command should be concerned solely with managing catkin workspaces as they pertain to the catkin buildsystem. It shouldn't be concerned with rosdep, wstool, or other external tools.

I'd love feedback on this idea / @davetcoleman @isucan

[wjwwood]

@ahendrix

[trainman419]

I threw together a spec for a similar tool a while back, but I haven't had time to implement it or try to incorporate it into the new catkin tool: https://github.com/trainman419/catkin_workspace

[jack-oquin]

Seems like a good idea to me, but I probably don't understand all the implications.

For a long time I have wanted to be able to define a workspace environment without using so many shell variables, making it easy to move between one workspace and another.

[jbohren]

Seems like a good idea to me, but I probably don't understand all the implications.

For a long time I have wanted to be able to define a workspace environment without using so many shell variables, making it easy to move between one workspace and another.

Yeah, I'm not sure there's a way around that, since catkin tries to just work with everyone else's variables. So this approach would just give identifiers to workspaces on your computer, and any other information on those workspaces would come from inspecting / parsing them in some way or another.

[wjwwood]

So, maybe I'm missing something, but when do you ever set any environment variables manually? For myself, I always use the setup.*sh files. Even if I wanted to do it manually, the only environment variable required to build a workspace as far as I know is CMAKE_PREFIX_PATH and maybe the PYHTONPATH. The PYTHONPATH is only needed if your build uses some python code in underlay's, which most don't (exceptions are message generators, but they are usually in /opt/ros/*).

[jbohren]

So, maybe I'm missing something, but when do you ever set any environment variables manually? For myself, I always use the setup.*sh files.

I also only ever use the setup.*sh files, except when unsetting CMAKE_PREFIX_PATH. When I mention "inspecting / parsing" I'm talking about for introspection / visualization, not for actually loading them. For that, catkin already does (I think) a pretty good job of managing the environment variables.

All I was saying is that a catkin ws verb wouldn't try to do what the catkin setup files already do, it would just make it easier to tell which setup files you're sourcing and when.

[jbohren]

I threw together a spec for a similar tool a while back, but I haven't had time to implement it or try to incorporate it into the new catkin tool: https://github.com/trainman419/catkin_workspace

Your proposal looks like a great integration tool, and i think it really looks like the new rosws. In other words, it could integrate features from wstool, catkin, and rosdep. I see catkin ws * as having a much more limited scope, though.

[wjwwood]

I'm not sure about unsetting either, I would just open a new shell, it's really cheap, and that way you are sure that your env is clean.

[jack-oquin]

Beginners have a lot of trouble setting up their environment correctly.

Even experienced people are unsure whether to source /opt/ros/... as well as their workspace. IIUC, the answer is "no", but things don't work right unless the correct underlay was sourced when the workspace was initially created. Sourcing another workspace in the same shell generally fails in some ugly manner.

The semantics are not sufficiently clear.

[wjwwood]

Also, I want to point out that I designed this package so that tools like this can be developed externally. I would really like to avoid this package depending directly on rosdep, wstool, bloom, etc...

See: http://catkin-tools.readthedocs.org/en/latest/development/extending_the_catkin_command.html

[jack-oquin]

+1 to minimal dependencies

[jbohren]

Also, I want to point out that I designed this package so that tools like this can be developed externally. I would really like to avoid this package depending directly on rosdep, wstool, bloom, etc...

I agree completely. A catkin ws command should be concerned solely with managing catkin workspaces as they pertain to the catkin buildsystem.

[jbohren]

@wjwwood

I'm not sure about unsetting either, I would just open a new shell, it's really cheap, and that way you are sure that your env is clean.

I (and everyone I know) puts source /path/to/something.sh in their shell profile, though. When running systems I spawn more shells than , so I don't want to waste time sourcing things manually.

@jack-oquin

Beginners have a lot of trouble setting up their environment correctly.

Even experienced people are unsure whether to source /opt/ros/... as well as their workspace. IIUC, the answer is "no", but things don't work right unless the correct underlay was sourced when the workspace was initially created. Sourcing another workspace in the same shell generally fails in some ugly manner.

The semantics are not sufficiently clear.

Yeah, students and novices waste many hours on problems like these. It's about time that we have a reliable go-to solution for debugging / inspecting / switching workspace configurations.

[wjwwood]

Sourcing another workspace in the same shell generally fails in some ugly manner.

I think catkin does a pretty good job of managing the environment variables which it sets, and I have never seen this cause problems, but I don't recommend it either since we cannot control what people do in their environment hooks.

Personally I follow these rules:

• Always start with a clean shell and never put sourcing lines in my bashrc
  • Instead use bash aliases to do groovy/hydro/indigo common sourcing
• Avoid workspaces with multiple parents (it makes reproducing it harder)
• Source the (hopefully single) parent workspace before building a workspace
• Source the resulting setup.*sh file in a new shell to use the workspace

They aren't strictly required, and I think people would complain if we recommended taking the sourcing of setup files from their bashrc, but they work pretty well for me.

I understand the problems new users may run into, and so I support a tool for helping to manage these workspaces. There is, however, a technical problem (in my opinion) with a tool which switches workspaces, because Python cannot modify the environment it is run in permanently. The best it could do is create a setup file and tell the user to source it or maybe run a new instance of bash or zsh and drop the user in it, but I'm unsure about both of those options. I'm open to suggestion on how to do that, but up until now we have always tried to lean on the tools provided by Linux and the System shell to manage these things.

If the users have no understanding of the shell and how it's environment works then we have more fundamental problem. I'm not saying we should give up on supporting these people, but there is a tipping point on us building increasingly complex tools and the users learning about the system they are running on.

[jbohren]

@wjwwood

They aren't strictly required, and I think people would complain if we recommended taking the sourcing of setup files from their bashrc, but they work pretty well for me.

Yeah, I can't stand having to source something in each new window. I juggle a bunch of workspaces for different projects, and a new shell might not be anywhere near the relevant workspace. I use a bunch of different workspaces mostly because of terrible nightmares caused by catkin_make's merged buildspace, so maybe it isn't as important to be able to do this with catkin build.

@wjwwood

If the users have no understanding of the shell and how it's environment works then we have more fundamental problem. I'm not saying we should give up on supporting these people, but there is a tipping point on us building increasingly complex tools and the users learning about the system they are running on.

I think it's reasonable to expect users to understand how shells, enviornments, and environemnt variables work. What I don't think we should expect them to know is how catkin, in particular, uses CMAKE_PREFIX_PATH to chain workspaces. When learning how to use ROS there are already enough things to remember without this.

The real issue with workspace management is that the moment a user goes from having one workspace to having two workspaces, they're unlikely to either expect that those two workspaces could interfere with each-other, and even less-likely to know how to debug the problems that result from it.

When helping students I've often seen things like:

CMAKE_PREFIX_PATH=/home/user1/catkin_ws:/opt/ros/hydro:/home/user1/another_ws

So I see the need for something like catkin ws status and catkin ws list which shows you which workspaces (i.e. the chain of workspaces) you currently have loaded, which ones are on your system, etc.

@wjwwood

I understand the problems new users may run into, and so I support a tool for helping to manage these workspaces. There is, however, a technical problem (in my opinion) with a tool which switches workspaces, because Python cannot modify the environment it is run in permanently. The best it could do is create a setup file and tell the user to source it or maybe run a new instance of bash or zsh and drop the user in it, but I'm unsure about both of those options. I'm open to suggestion on how to do that, but up until now we have always tried to lean on the tools provided by Linux and the System shell to manage these things.

I agree that we should probably avoid forking sub-shells. This is definitely a technical challenge, but we could probably solve these with shell functions.

For example, catkin could actually be defined as a shell-function which intercepts and either passes its arguments on to the catkin python program or runs something in-line.

For example ( without error handling etc):

catkin() {
  if [[ $1 == "ws" && $2 == "load" ]]; then
      source $(catkin ws get "$3"); 
  else
    return catkin "$@"; 
  fi;
}

This could be put in /usr/share/catkin_tools/catkin_tools.sh or something, and you source that in your bashrc if you want to use the workspace management stuff.

---

As an aside, I currently use sh aliases like:

alias wspwd='export ROS_WORKSPACE=$(pwd)'
alias wssave='echo -e "$(pwd)" > ~/.curws'
alias wsload='source $(cat ~/.curws)/setup.zsh'
alias wscd='cd $(cat ~/.curws)/../src'

I just put wsload in my .zshrc and whatever path is in the ~/.curws file gets loaded for each new shell.

[davetcoleman]

This sounds useful, but I don't know enough about the under workings of catkin to weight in much. I do know I often am frustrated by the way overlaying workspaces seems to get their path ordering mixed up, requiring me to rebuild all of ros from scratch because I want to change some downstream workspace. Having tools to view and edit those overlays seems useful, though maybe this would not do that exactly.

[davetcoleman]

Complete off-topic: I'd love a shortcut for adding a build/run dependency to a catkin package without having to copy it 4 times in two different files.

[tkruse]

Regarding workspace setup: To me, it is still one of the big design blunders of catkin that the current environment of a shell is cached in a file when a user creates a new workspace. I think no other tool in the world behaves like that, for good reason. The sane approach would be that for chaining, a user has to explicitly state the workspace(s) chained to.

The design makes typing for chaining a little less effort (of a command that an average user types twice a year on average), while dramatically increasing the learning curve of catkin, because so many things can go wrong. Having to start a fresh shell every time is not the solution, it is the least painful workaround for a bad design approach, and it does not match well with the common idiom in the ROS ecosystem of sourcing a setup.sh in a profile file.

Anything that catkin_tools can make to get rid of that behavior gets a +1 by me.

[jack-oquin]

+1 from me, also, @tkruse

[wjwwood]

@jbohren I think the shell script could work, it kind of sucks to implement and maintain, but I think it could be ok.

Complete off-topic: I'd love a shortcut for adding a build/run dependency to a catkin package without having to copy it 4 times in two different files.

Use REP-140 and catkin_simple?

To me, it is still one of the big design blunders of catkin that the current environment of a shell is cached in a file when a user creates a new workspace. I think no other tool in the world behaves like that, for good reason.

Maybe I misunderstand your point, but I think that CMake writes information and paths to the CMake Cache file based on the current environment.

The sane approach would be that for chaining, a user has to explicitly state the workspace(s) chained to.

Recursively or just the "leaf" workspaces?

(of a command that an average user types twice a year on average)

Which command? That seems made up.

it does not match well with the common idiom in the ROS ecosystem of sourcing a setup.sh in a profile file

Should that be one setup file or many? The biggest issue is that setup files don't just extend the CMAKE_PREFIX_PATH (semantically the CATKIN_PREFIX_PATH), they also extend the PKG_CONFIG_PATH, PYTHONPATH, LD_LIBRARY_PATH, etc...

The simple scenario of having source /opt/ros/hydro/setup.bash in my ~/.bashrc, opening a new terminal, creating a workspace for indigo desktop entirely from source (not passing any workspaces so implicitly parentless like was suggested), and then building that workspace becomes problematic.

[tkruse]

Sure CMake writes "information based on the current environment". Most CMake users don't even realize this. That's because it does not affect them, as it is not an intended usage pattern of cmake that you'd custom-tune your cmake environment each time you configure any of the 300 projects on your harddisk before running the cmake command. Cmake caches information that it assumes to be stable, thus that the concept of "caching" is as close to the truth as possible (the cached value being likely to represent the "true" value that would be read any next time if it had not been cached). Else it is not "caching" anymore, but customizing. and for customizing, cmake expects explicit input. catkin mixes up the concepts of "implicit caching" and "explicit customizing" when it does "implicit customizing".

I have no strong opinion of how workspaces should be chained best, but I believe to support ROS it would be sufficient to support only one underlay and one overlay with the provided toolset. Anything else increases the learning curve dramatically for almost zero added benefit.

Average users do not create workspaces often during a year. They create one with the current distro, and later maintain sometimes 2 workspaces, one with a last distro, one with the next. Creating a workspace thus is a event that happens very rarely. If users will delete their build folders (and the valuable configuration) often out of sheer desperation over broken builds, that is not a usability virtue. ROS users are supposed to be busy making robots do circus tricks, not setting up build environments over and over again.

Obviously the command they run currently is "catkin_make", but I am talking about the abstract command concept ('create a workspace against the given underlay' vs. 'compile my code'). The first one, 'create a workspace against the given underlay' must not be keystroke-optimized, it must be transparency optimized. Ideally a user would run this command only once per ros distro. It does not matter then whether it takes him 10 seconds or 40 seconds to type it. You can burden the user with having to explicitly type out the chaining configuration for this command.

I don't think you, William, should source any ROS distro in your bashrc. But you need to take into account that most ROS users do not need more than one ROS distro at a time, and for them it is most convenient to have the sourcing command in their bashrc. It is acceptable if they will get errors when sourcing hydro in bashrc and then sourcing an indigo overlay or something. That is a concept even a novice user can grasp easily.

[wjwwood]

I believe to support ROS it would be sufficient to support only one underlay and one overlay with the provided toolset

So, where does the problem come in? In the simplest case, you put source /opt/ros/hydro/setup.bash into your ~/.bashrc and you have one workspace which you call catkin_make on. Even if they source the result and invoke catkin_make again, the chance for harm is pretty low. In fact until you have more than one custom workspace, this whole issue is sort of moot.

you need to take into account that most ROS users do not need more than one ROS distro at a time

I do take that into account, I've never suggested that people not do it, I was simply saying what I do, and I do that mostly because I have complex scenarios. For the common case (source opt/ros and build one workspace) you can easily source something in your bashrc and be fine.

I don't have time to discuss this endlessly, lets try to narrow down what we're talking about here, is there a concrete proposal on the table for how to change the system?

It seems to me that there are two proposals:

• Change catkin to require a PARENT_WORKSPACES CMake variable as input and it should ignore the CMAKE_PREFIX_PATH with respect to workspace discovery.
• Leave catkin alone and build a tool to help manage the complexity.

If so, we should try to enumerate the current workflows and use cases and try to describe how they are improved or hindered by these changes.

[tkruse]

It seems to me that there are two proposals:

• Change catkin to require a PARENT_WORKSPACES CMake variable as input and it should ignore the CMAKE_PREFIX_PATH with respect to workspace discovery.
• Leave catkin alone and build a tool to help manage the complexity.

Given that the current catkin maintainer is not helpful in making such changes to catkin, I would recommend not wasting good will and motivation on trying to change catkin. If an obvious easy way to create catkin workspaces without reading environment-variables presents itself, it should be used. But else focus on other things first, establish catkin_tools as a replacement for catkin, then revisit the topic.

[jack-oquin]

Catkin build has already been helpful for me.

I work on so many diverse packages, that I find myself with eight or ten catkin_make workspaces, a big time waster. They frequently get mangled in ways that are hard for me to diagnose, so I often scrap and recreate them.

With catkin build I believe I can do most work in a single workspace per distro, like other users. That is a big improvement in my work flow.

[jbohren]

Catkin build has already been helpful for me.

I work on so many diverse packages, that I find myself with eight or ten catkin_make workspaces, a big time waster. They frequently get mangled in ways that are hard for me to diagnose, so I often scrap and recreate them.

With catkin build I believe I can do most work in a single workspace per distro, like other users. That is a big improvement in my work flow.

I think this is a really good point, and something I've also found with catkin build. By decoupling packages at configuration time, it means they're lies likely to conflict. It's refreshing to be able to build single packages and only their dependencies like we so often did with rosbuild.

I think now, the most common use case for multiple workspaces is if you need to use different versions of the same packages.

[jbohren]

Now that the workspace extension discussion has been more focused in #48, I wanted to return to the potential implementation of a catkin ws verb.

@wjwwood

I agree that we should probably avoid forking sub-shells. This is definitely a technical challenge, but we could probably solve these with shell functions.

I was just looking into something and I thought I'd mention this. With login shells, it looks like bourne-compatible shells source all the files in /etc/profile.d but non-login ("normal") shells do not. It would be nice if we could install some shell scripts that were immediately available without having to modify .bashrc or .zshrc, but I don't see any way around that.

I think once we have a prototype interface for explicitly controlling catkin workspace chaining, we can revisit the catkin ws adverb idea.

[jbohren]

Ok... so I've been having issues for about 15 minutes now where my overlay wasn't working. I am not sure how this happened, but I echoed $CMAKE_PREFIX_PATH and I saw this:

/opt/ros/hydro:/home/jbohren/ws/hydro/rrm/devel

So. There is a failure of the catkin workspace rollback behavior.

[dirk-thomas]

Since several people have had this kind of issue before there is definitely something wrong. But the problem is that we need a reproducible example to act upon. None of us was ever able to reproduce it. May be your bash history can help to find the command sequence which led to it?

[jbohren]

Since several people have had this kind of issue before there is definitely something wrong. But the problem is that we need a reproducible example to act upon. None of us was ever able to reproduce it. May be your bash history can help to find the command sequence which led to it?

Got it. This happens both with bash and zsh.

unset CMAKE_PREFIX_PATH
echo $CMAKE_PREFIX_PATH
# >
source /opt/ros/hydro/setup.bash
echo $CMAKE_PREFIX_PATH
# > /opt/ros/hydro
mkdir -p /tmp/ws/src
cd /tmp/ws/src
catkin_create_pkg foo
cd /tmp/ws
catkin_make
source devel/setup.bash
echo $CMAKE_PREFIX_PATH
# > /tmp/ws/devel:/opt/ros/hydro
# ...
# user tries running something, and it doesn't work as expected
# as a last resort he or she blows away build and devel in an attempt to start fresh
cd /tmp/ws
rm -rf build devel
# To be sure that things are clean, the user re-sources /opt/ros/hydro.bash
# THIS is the mistake! By re-sourcing this, it prepends it to the $CMAKE_PREFIX_PATH without removing the workspace
# After this point, the workspace list is inverted
source /opt/ros/hydro/setup.bash
echo $CMAKE_PREFIX_PATH
# > /opt/ros/hydro:/tmp/ws/devel
# the user then tries to re-build, without having noticed $CMAKE_PREFIX_PATH getting busted
catkin_make
source devel/setup.bash
echo $CMAKE_PREFIX_PATH
# > /opt/ros/hydro:/tmp/ws/devel

The only way to resolve this is to unset CMAKE_PREFIX_PATH remove build and devel and re-build after sourcing /opt/ros/hydro/setup.bash.

[davetcoleman]

+1 great find @jbohren