rosbag2 icon indicating copy to clipboard operation
rosbag2 copied to clipboard

Update CI scripts to use Ubuntu Noble distros and bump action scripts to latest versions

Open MichaelOrlov opened this issue 1 year ago • 6 comments

  • Use Ubuntu Noble distros for ci jobs on rolling.
  • Bump actions-ros-lint to v0.1.3 and actions/checkout to v4.
  • Exclude cppcheck from CI due to a known issue that the new versions are very slow. See https://github.com/ament/ament_lint/pull/345 for details.

MichaelOrlov avatar Jun 12 '24 06:06 MichaelOrlov

@clalancette No. the build_and_test CI job is not expected to fail. It fails with the following error messages

024-06-12T07:33:28.6183572Z [36;1msource /opt/ros/rolling/setup.sh && colcon test --mixin linters-skip --packages-select ${rosbag2_packages} --packages-skip rosbag2_performance_benchmarking --event-handlers console_cohesion+ --return-code-on-test-failure --ctest-args "-L xfail" --pytest-args "-m xfail"[0m
2024-06-12T07:33:28.6188804Z shell: bash --noprofile --norc -e -o pipefail {0}
2024-06-12T07:33:28.6189206Z ##[endgroup]
2024-06-12T07:33:31.0730529Z usage: colcon [-h] [--log-base LOG_BASE] [--log-level LOG_LEVEL]
2024-06-12T07:33:31.0731792Z               {build,coveragepy-result,extension-points,extensions,graph,info,lcov-result,list,metadata,mixin,test,test-result,version-check}
2024-06-12T07:33:31.0732754Z               ...
2024-06-12T07:33:31.0733154Z colcon: error: Mixin 'linters-skip' is not available for 'test'
2024-06-12T07:33:31.1123681Z ##[error]Process completed with exit code 2.
2024-06-12T07:33:31.1170348Z ##[group]Run rosbag2_path=$(colcon list -p --packages-select rosbag2)/..
2024-06-12T07:33:31.1171017Z [36;1mrosbag2_path=$(colcon list -p --packages-select rosbag2)/..[0m
2024-06-12T07:33:31.1171490Z [36;1msudo pip uninstall -y mypy[0m
2024-06-12T07:33:31.1171928Z [36;1msudo apt update && sudo apt -y install mypy=0.942-1ubuntu1[0m
2024-06-12T07:33:31.1172357Z [36;1msource install/setup.sh[0m
2024-06-12T07:33:31.1172750Z [36;1mstubgen -p rosbag2_py -o ${rosbag2_path}/rosbag2_py[0m
2024-06-12T07:33:31.1173154Z [36;1mcd ${rosbag2_path}[0m
2024-06-12T07:33:31.1173438Z [36;1mgit diff --exit-code[0m
2024-06-12T07:33:31.1173854Z shell: bash --noprofile --norc -e -o pipefail {0}
2024-06-12T07:33:31.1174204Z ##[endgroup]
2024-06-12T07:33:33.0851536Z error: externally-managed-environment
2024-06-12T07:33:33.0851942Z 
2024-06-12T07:33:33.0852173Z × This environment is externally managed
2024-06-12T07:33:33.0853015Z ╰─> To install Python packages system-wide, try apt install
2024-06-12T07:33:33.0854037Z     python3-xyz, where xyz is the package you are trying to
2024-06-12T07:33:33.0854756Z     install.
2024-06-12T07:33:33.0855135Z     
2024-06-12T07:33:33.0855881Z     If you wish to install a non-Debian-packaged Python package,
2024-06-12T07:33:33.0856986Z     create a virtual environment using python3 -m venv path/to/venv.
2024-06-12T07:33:33.0857902Z     Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
2024-06-12T07:33:33.0858959Z     sure you have python3-full installed.
2024-06-12T07:33:33.0859337Z     
2024-06-12T07:33:33.0860325Z     If you wish to install a non-Debian packaged Python application,
2024-06-12T07:33:33.0860886Z     it may be easiest to use pipx install xyz, which will manage a
2024-06-12T07:33:33.0861426Z     virtual environment for you. Make sure you have pipx installed.
2024-06-12T07:33:33.0861830Z     
2024-06-12T07:33:33.0862151Z     See /usr/share/doc/python3.12/README.venv for more information.
2024-06-12T07:33:33.0862660Z 
2024-06-12T07:33:33.0863678Z note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.

MichaelOrlov avatar Jun 12 '24 15:06 MichaelOrlov

Add --break-system-packages to the pip command.

christophebedard avatar Jun 12 '24 15:06 christophebedard

Add --break-system-packages to the pip command.

Oh, you'll also have to update the apt install mypy command to get rid of the version number, since it is different in Noble.

clalancette avatar Jun 12 '24 15:06 clalancette

@clalancette We have to pin mypy to some version it could be the same version as by default in Noble. However it should be pinned to have a consistency with readme instruction in rosbag2_py package for generating and verifying .*pyi files.

I've pinned mypy to the 1.9.0-4ubuntu1 version as mentioned in the https://packages.ubuntu.com/search?keywords=mypy for Ubuntu 24.04 noble. Will see if CI will pass.

MichaelOrlov avatar Jun 12 '24 21:06 MichaelOrlov

@clalancette We have to pin mypy to some version it could be the same version as by default in Noble. However it should be pinned to have a consistency with readme instruction in rosbag2_py package for generating and verifying .*pyi files.

The thing is, you know for certain it is going to be that version; your Dockerfile is using Ubuntu 24.04, and the only supported Ubuntu for Rolling currently is Ubuntu 24.04.

That said, you can pin it if you want, I just think it is unnecessary.

clalancette avatar Jun 12 '24 21:06 clalancette

The new version of mypy causing a lot of discrepancies in checking existing *.pyi files. On the other hand the old version of the mypy can't be installed on the Ubuntu 24.04 Noble since apt throws an error E: Version '0.942-1ubuntu1' for 'mypy' was not found

@r7vme I would appreciate your assistance with the resolution of issue with the mypy installation and *.pyi files verification on CI.

MichaelOrlov avatar Jun 13 '24 06:06 MichaelOrlov

@MichaelOrlov you can pull two commits from this PR https://github.com/ros2/rosbag2/pull/1763 to fix issues with new mypy version.

  1. Re-generated pyi files with mypy 1.9 (default Ubuntu 24.04 version)
  2. Updated readme to allow developers on Ubuntu 22.04 to install mypy 1.9
  3. Unpinned mypy version in CI

r7vme avatar Jul 26 '24 13:07 r7vme

Have to rebase on the latest rolling since the build_and_test job was failing due to treating warnings as errors and using deprecated rcpputils::fs::create_temp_directory from tests.

MichaelOrlov avatar Aug 09 '24 18:08 MichaelOrlov

The CI is green, merging without running CI on the build farm since changes only in git workflow, readme file and generated pyi files.

MichaelOrlov avatar Aug 09 '24 19:08 MichaelOrlov

https://github.com/Mergifyio backport jazzy

MichaelOrlov avatar Aug 09 '24 19:08 MichaelOrlov

backport jazzy

✅ Backports have been created

mergify[bot] avatar Aug 09 '24 19:08 mergify[bot]