ci
ci copied to clipboard
ros2
Python warnings are not reported by CI
Neither colcon test nor Jenkins report warnings coming from Python code. For example, deprecated API was recently introduced in https://github.com/ros2/ros2cli/pull/463 but CI passes without reported the warnings. I would expect an unstable build so that we notice things like deprecated Python APIs and can address them.
Here is an example build testing ros2node:
It is unstable for a different reason. The point is there are 61 warnings visible in the console, but they are not reported by Jenkins: https://ci.ros2.org/job/ci_linux/9731/consoleFull#console-section-223
Specifically for tests, I tried passing a pytest argument to treat warnings as errors (https://github.com/ros2/ci/pull/416), but this results in a failed build, instead of unstable, which I think is not desired.
Related colcon PR to report pytest warnings: https://github.com/colcon/colcon-core/pull/236
Current aim is to try and figure out whether the existing Jenkins plugins can be configured to pick up python warnings in stderr (e.g. using a regex) or if a custom plugin will be required.
I did some checking and it doesn't appear that a python warning parser exists as part of the Jenkins analysis-model nor is there a separate plugin which provides one. I did manage to write a regexp-based parser to extract Python warnings from Pytest warnings summaries from colcon test stderr logs.
https://github.com/ros2/ci/pull/447 is a PR which adds a parser for the warnings as they appear in our colcon logs.
Neither colcon test nor Jenkins report warnings coming from Python code.
The pytest warnings are being redirected to stderr since colcon/colcon-core#338. This is matching the behavior of e.g. C++ compiler warnings.
It is up to Jenkins to extract those and change the build status based on it. See ros2/ci#509 for a draft PR doing that.
