Support invocation in subfolders of the workspace

[koenlek]

If I run colcon build in ~/ros2_ws/src it will create the output folders in the ~/ros2_ws/src folder, i.e.:

• ~/ros2_ws/src/build
• ~/ros2_ws/src/log
• ~/ros2_ws/src/install

Instead, you would expect them to end up in the top level directory, like catkin_tools does (so in ~/ros2_ws instead of ~/ros2_ws/src). Catkin tools marks the top level directory using the .catkin_tools folder (like git does it by "going up" until it finds a .git folder). I see that colcon doesn't have such a folder yet, so it doesn't have a way to determine the workspace top-level. Sooner or later I guess such a folder (e.g. ~/ros2_ws/.colcon) should be added anyway, to store things like profiles/configs (like catkin tools does).

Maybe a good feature to add?

[dirk-thomas]

colcon is intended to be invoked in the root of the workspace. It currently doesn't support being invoked in the src directory. While that would be possible it is currently not planned to be worked on.

[rotu]

I would appreciate this too.

I often accidentally run colcon in a subfolder or sometimes in a superfolder of my workspace. I think it would be a fair change of behavior if colcon build took an optional positional argument. If provided, it would create a workspace there and build it. If absent, it would recursively crawl parent folders until it found a workspace (identified by its install/.colcon_install_layout file) and build there. Not finding one would show an error message and exit.

[dirk-thomas]

I think it would be a fair change of behavior if colcon build took an optional positional argument. If provided, it would create a workspace there and build it.

The logic is a little bit more complicated since this will affect other arguments which are currently specified as paths relative to the current directory. E.g. if you specify a custom --install-base base it is relative to the cwd. But the default value install would need to be relative to the proposed optional positional argument.

If absent, it would recursively crawl parent folders until it found a workspace (identified by its install/.colcon_install_layout file) and build there.

The same complexity for interpreting default values / relative paths exists here.

[deitry]

After some use of colcon I just start to run all commands from src folder and saw no difference at all except my paths became two steps shorter. Was: /home/user/workspace/colcon_proj_ws/src/my_project/ Become: /home/user/workspace/my_project

[dirk-thomas]

After some use of colcon I just start to run all commands from src folder and saw no difference at all except my paths became two steps shorter.

@deitry That might work for your specific use case (assuming the packages under my_project are in subdirectories or you don't mind the build / install directory being in the source directory). The recommended layout ensures build / install and src are clearly separated and can be easily deleted in all cases.

[jacobperron]

An alternative is to have a version of colcon that is workspace-aware. E.g. a tool colcon_ws that wraps common colcon commands, but behaves more similarly to catkin_tools. I do miss the ROS 1 workflow of quickly changing to a package, making a small edit, and invoking catkin bt.

[dirk-thomas]

Please see my previous comment about the complexity around command line arguments representing relative paths. So just cd into the root and running the same command isn't sufficient.

[rhaschke]

It would be great to have such a feature, because the need to cd to the top-level folder significantly interrupts the development workflow. As far as I understand, colcon doesn't use any config cache, e.g. build parameters, package selection, etc. as catkin tools has done (which is a pity as well). Thus, it doesn't have a notion of the top-level build folder, does it? @dirk-thomas, what is your suggestion to implement such a feature? Is the configuration mechanism a suitable starting point?

[dirk-thomas]

Is the configuration mechanism a suitable starting point?

I don't see how the configuration mechanisms are related to this?

what is your suggestion to implement such a feature?

Obviously the root of the workspace needs to be marked somehow on the first invocation. That part should be trivial.

Then subsequent invocations have to check if they are called from a recursive subdirectory of any marked workspace root. And if that is the case the execution should happen as if the command was called in the workspace root - except that any explicitly provided arguments from the CLI which contain relative path must be updated accordingly if e.g. the cwd is changed to achieve the behavior to run from the root.

The challenge here will be to know what arguments are coming from the CLI and which ones are coming from configuration files and/or default values since they need to be treated differently.

[rhaschke]

I think adding a configuration mechanism that relies on cached information stored in the workspace root (á la catkin tools) would be a nice addition to colcon. Don't you agree?

[dirk-thomas]

I do agree that profiles would be a nice feature. But they are orthogonal to the feature described in this ticket and tracked in a separate one: see #168.

[rhaschke]

Agreed. For me, the features are (loosely) related, as the profiles would mark the workspace root.

[dirk-thomas]

the features are (loosely) related, as the profiles would mark the workspace root.

It depends on the implementation choice for profiles. Maybe they are stored in a user config directory instead.

[andrejpan]

After 6 years this is still impossible/low level priority ticket?

[rhaschke]

There are several plugins provided by the community to address this issue, for example: https://github.com/rhaschke/colcon-top-level-workspace

[andrejpan]

Tnx a lot @rhaschke. I guess we can also close this ticket.

[Flova]

There are several plugins provided by the community to address this issue, for example: https://github.com/rhaschke/colcon-top-level-workspace

I think this is based on the official one: https://github.com/colcon/colcon-top-level-workspace

But the official one seems pretty wip.

[rhaschke]

There are several plugins provided by the community to address this issue, for example: https://github.com/rhaschke/colcon-top-level-workspace

I think this is based on the official one: https://github.com/colcon/colcon-top-level-workspace

But the official one seems pretty wip.

Yes, my version was inspired by the "official" one. However it is a full rewrite. There is an open PR, namely https://github.com/colcon/colcon-top-level-workspace/pull/2, but @cottsay didn't have time to look into that yet.

[rhaschke]

I would like to keep this issue open until the official package has merged my PR.

[agyoungs]

Is there any movement on this issue? This looks to be pretty useful