autoware.universe icon indicating copy to clipboard operation
autoware.universe copied to clipboard

Improve and document the point cloud pipeline performance evaluation methods

Open xmfcx opened this issue 1 year ago • 0 comments

Checklist

  • [X] I've read the contribution guidelines.
  • [X] I've searched other issues and no duplicate issues were found.
  • [X] I've agreed with the maintainers that I can plan this task.

Description

Currently in Autoware, many developers rely on ros2 topic delay pointcloud_topic_name to measure the point cloud pipeline performance. This method is suitable for small messages, but for larger messages such as point clouds or images, it introduces significant overhead and reports exaggerated delays.

The primary reason is that our sensing/perception nodes run within composable node containers and utilize intra-process communication. External subscriptions to these messages, like using ros2 topic delay or rviz2, add additional delays. Even slow down existing pipelines just by subscribing from outside.

A more efficient method would be for the nodes themselves to report the delays.

Purpose

The purpose of this issue is to improve the accuracy and reliability of point cloud pipeline performance evaluation in Autoware. By developing a method where nodes internally report delays, we can achieve a more precise measurement of performance, essential for optimizing and benchmarking our systems.

Possible approaches

  1. Extend existing tier4_autoware_utils::DebugPublisher instances within nodes to report accumulated delays (current time - header time).
    auto accumulated_time =
      std::chrono::duration<double, std::milli>(
        std::chrono::nanoseconds((this->get_clock()->now() - input->header.stamp).nanoseconds()))
        .count();
    
    debug_publisher_->publish<tier4_debug_msgs::msg::Float64Stamped>(
      "debug/accumulated_time_ms", accumulated_time);
    
  2. Update the Autoware documentation to reflect these changes
  3. Add a tutorial page for the evaluation of the point cloud pipeline performance
  4. Create additional tools to assist in the evaluation

Definition of done

  • [x] Nodes within Autoware Universe are updated to report their own delays.
  • [x] Documentation is updated with the new methods for evaluating point cloud pipeline performance.
  • [ ] New tools for pipeline performance evaluation are developed and integrated.

cc. @kminoda @miursh

xmfcx avatar Jan 09 '24 07:01 xmfcx