[WIP] Add jazzy CI, migrate rolling to noble

[luca-della-vedova]

[maspe36]

Run sudo pip3 install git+https://github.com/colcon/colcon-cargo.git
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
    
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.
    
    If you wish to install a non-Debian packaged Python application,
    it may be easiest to use pipx install xyz, which will manage a
    virtual environment for you. Make sure you have pipx installed.
    
    See /usr/share/doc/python3.12/README.venv for more information.

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

Seems to be a new thing as of Python 3.11. Now with Jazzy targeting 24.04, we will need to figure out how we want to handle this... probably change our README instructions as well.

Not sure why Rolling is still on 3.10, guess its still based on Ubuntu 22.04.

[luca-della-vedova]

Ah yes I forgot about that, there is a break-system-packages flag that we can use, i.e. https://github.com/ros-infrastructure/ros_buildfarm/pull/1026.

There has been a fair bit of activity on the setup-ros side, the current status it that to get Jazzy up we should set a flag to use testing repos, instead of adding it and reverting it in 10 days I'd probably just leave it to be done when the actual stable release is out and leave this in draft for the time being.

[esteve]

@maspe36 @luca-della-vedova we could use pipx. pipx is available on Debian/Ubuntu via the pipx package

[esteve]

@luca-della-vedova I hope you don't mind, I've pushed a change to your fork that replaces sudo pip3 install with pipx. I wanted to submit a PR to your fork, but I pushed to the wrong remote. Let me know if you'd rather I submit a PR to your fork.

[arjo129]

PIPX won't work with colcon-cargo-ros as that is installed as a library/plugin and pipx is for installing standalone executables.

[luca-della-vedova]

I can patch it up by putting a --break-system-packages that only gets set for noble (since the argument doesn't exist in 22.04 pip3 versions). I suspect that the real solution however, would be to distribute colcon-cargo and colcon-ros-cargo as binaries through the buildfarm, was there any reason this wasn't done in the past? They seem quite conventional Python packages (except colcon-cargo that includes minimal Rust test code) so I wonder if we can go that direction and remove the pip3 install, since it is in general not a great practice to recommend.

[mxgrey]

I suspect that the real solution however, would be to distribute colcon-cargo and colcon-ros-cargo as binaries through the buildfarm, was there any reason this wasn't done in the past?

I agree with this, but I think there may still be some loose ends to tie up before moving forward with this, like getting workspaces to build properly. I'm attempting to build a workspace with colcon-ros-cargo but it fails because the individual crates don't have an associated Cargo.lock file (it doesn't seem to know that it should defer to the workspace's Cargo.lock file).

If we can get that working, I'd be comfortable opening a PR to get the packages into rosdistro.

But until then, I'm definitely having some irritating problems with getting these colcon extensions to work in Ubuntu 24.04. It seems it's necessary to create a virtual environment and do a complete install of colcon and its extensions into the virtual environment, and build the colcon workspace with the virtual environment sourced. All of the dependencies needed by everything in the workspace also need to be manually installed into the virtual environment.

[esteve]

Are you sure the colcon extensions built by the buildfarm? I can't find colcon in https://build.ros2.org/ , I think they are released separate from the rest of ROS. And in fact, they are available via packagecloud, which does not contain any other ROS package.

@cottsay do you have more info about releasing colcon extensions as Debian packages? Thanks.

[mxgrey]

The colcon installation page indicates that debians for colcon get distributed from both the ROS 1 and ROS 2 apt repo.

The packagecloud repo is meant for users that don't want to depend on the ROS ecosystem.

[esteve]

@mxgrey I can't find colcon in the buildfarm nor in the rosdistro repo. Some of the colcon packages are however referenced in https://github.com/ros/rosdistro/blob/master/rosdep/python.yaml#L5312-L5319, but I'd say they are there just so rosdep can resolve the keys.

Ah, I just found an ticket I filed exactly for this years ago:

https://github.com/colcon/colcon-core/issues/436#issuecomment-921230641

And yeah, that confirms what I wrote about the extensions not being released via the buildfarm and instead via packagecloud, or at least at the time that's how it worked.

More info about releasing Debian packages for colcon can be found at:

https://colcon.readthedocs.io/en/released/developer/release.html#publishing-a-release

[cottsay]

The colcon packages published to packagecloud are imported into the ROS 1 and ROS 2 repositories: https://github.com/ros-infrastructure/reprepro-updater/blob/master/config/colcon.ubuntu.upstream.yaml

That said, colcon-rust has simply never built a deb before: https://packagecloud.io/app/dirk-thomas/colcon/search?q=rust&filter=all&dist=

[mxgrey]

I spent another day trying to work with colcon+cargo in 24.04 (Python 3.12). It seems the changes to how pip works are really problematic for colcon extensions. Here are the key bullet points:

• To use colcon-cargo or colcon-ros-cargo (without resorting to the --break-system-packages flag) users will have to create a Python virtual environment to install the extensions into
• If you create your virtual environment with the --system-site-packages flag, then you can use the system-installed colcon while your virtual environment is active, but colcon will not be able to discover the colcon-cargo or colcon-ros-cargo extensions which were installed to the virtual environment
• Combining the above two bullet points means that anyone who wants to use colcon-cargo or colcon-ros-cargo must manually pip install colcon and all the desired extensions into the same virtual environment in order for everything to work

Now that the latest main branch of cargo-ament-build was published to crates.io, the cargo extensions for colcon are able to build workspaces correctly. In my mind this was the only crucial blocker for releasing, so I would support releasing debs of the extensions now.

If no one has any objections, I'll begin that process later this week.

[esteve]

@mxgrey

If no one has any objections, I'll begin that process later this week.

Did you manage to build Debian packages for the colcon cargo extensions? It'd be nice to revisit this PR and add support for newer distributions.

[ljburtz]

thank you @luca-della-vedova @mxgrey @esteve and all is there an update on this / the official way of installing ros2 rust on Jazzy?

[mxgrey]

@ljburtz We've been getting a lot of help from @cottsay to improve the quality and functionality of colcon-cargo to a level where it can be a common extension for colcon.

The main push on this right now is this PR.

[mxgrey]

As for how to install on jazzy, I personally just add the --break-system-packages flag since it's functionally no different than how we were doing it on earlier distros.

I think people who are more concerned about the health of their machine should consider using a Python virtual environment to install all of colcon (if you install only the extension in the virtual environment while colcon is installed to the system, then colcon won't find the extension). Or use a docker / virtual machine while building ros2-rust.

[lyinch]

Hello, what is the status on this? Is there anything other than the python dependency question blocking this from moving forward?

I also advocate for --break-system-packages as it's only done in a docker container which is already an isolated environment on its own. I think that having CI for jazzy is more important than solving this issue perfectly, especially since the isolated python packages is an unsolved problem of the whole ROS(2) community.

For more context, I was looking into the venv topic myself a while ago and didn't find good solution yet. There are some options out there:

• There is somewhere an equivalent of https://github.com/locusrobotics/catkin_virtualenv out there for ROS2/colcon that injects itself into the setup.py build hooks and creates a venv for each package. I can't find the repo right now, but it's a very ugly and brittle method, and that repo is outdated and doesn't work w/o a bunch of fixes
• There are nicer approaches such as https://github.com/UrbanMachine/colcon-poetry-ros or https://github.com/nzlz/colcon-uv . Instead of hijacking the setup.py build, they add a new "type" and use colcon-extensions. I think that this is the right way to solve the problem, but these projects currently are either outdated or immature, and still have some fundamental design questions that need to be answered before wide-spread adoption can happen.
• A simple alternative solution is to use one big "venv", as colcon seems to respect it if you activate it: https://github.com/ros2/ros2/issues/1094#issuecomment-2916700723 or alternatively change the python interpreter to use the one from the venv: https://github.com/ros2/ros2/issues/1094#issuecomment-2816646246 . But what do we gain from this if we already use a container?

There are more threads or projects out there, but I think that this list captures the main directions how venvs can be achieved.

[luca-della-vedova]

This was done some time ago (check here, I believe this PR is not necessary anymore. The discussion on when we can move away from using --break-system-packages depends on when colcon-cargo and colcon-ros-cargo will be released, but that is out of the scope of this so will close it.

[lyinch]

looks like I missed a lot of context. Sorry for reviving an old thread. I actually came from: https://github.com/ros2-rust/ros2_rust/issues/399 maybe that can be closed as well @luca-della-vedova ?

[luca-della-vedova]

Yap it can be but I don't have the permissions for it :)