Cannot play MCAP files with different topics having different serialization format

[brian386]

Description

** clear and concise description of the bug is **

When I run ros2 bag play bag_name.mcap, I get this error message: Topics with different rwm serialization format have been found. All topics must have the same serialization format. According to this issue, different topics having different serialization methods is supposed to be supported.

Edit: I also tried playing an mcap file with only protobuf serialization, and I get this error [ERROR] [1676265641.870980365] [rosbag2_cpp]: Could not initialize RMWImplementedConverter: No RMW implementation found supporting serialization format protobuf. Is there any way around this?

Additional context

** Add any other context about the problem here ** I am trying to play the NuScenes dataset as a ROS2 bag. I used the nuscenes2mcap repository to convert NuScenes data to MCAP files. Below is the info about the file. Most of the topics use serialization format protobuf, but /odom and /imu use JSON.

Files:             NuScenes-v1.0-mini-scene-0655.mcap
Bag size:          475.1 MiB
Storage id:        mcap
Duration:          19.844s
Start:             Aug 27 2018 15:51:32.104 (1535385092.104)
End:               Aug 27 2018 15:51:51.949 (1535385111.949)
Messages:          35471
Topic information: Topic: /diagnostics | Type: diagnostic_msgs/DiagnosticArray | Count: 22922 | Serialization Format: ros1
                   Topic: /markers/car | Type: foxglove.SceneUpdate | Count: 41 | Serialization Format: protobuf
                   Topic: /markers/annotations | Type: foxglove.SceneUpdate | Count: 41 | Serialization Format: protobuf
                   Topic: /gps | Type: foxglove.LocationFix | Count: 41 | Serialization Format: protobuf
                   Topic: /pose | Type: foxglove.PoseInFrame | Count: 41 | Serialization Format: protobuf
                   Topic: /CAM_FRONT_LEFT/annotations | Type: foxglove.ImageAnnotations | Count: 41 | Serialization Format: protobuf
                   Topic: /CAM_FRONT_LEFT/lidar | Type: foxglove.ImageAnnotations | Count: 237 | Serialization Format: protobuf
                   Topic: /CAM_FRONT_LEFT/camera_info | Type: foxglove.CameraCalibration | Count: 237 | Serialization Format: protobuf
                   Topic: /CAM_FRONT_LEFT/image_rect_compressed | Type: foxglove.CompressedImage | Count: 237 | Serialization Format: protobuf
                   Topic: /CAM_BACK/annotations | Type: foxglove.ImageAnnotations | Count: 41 | Serialization Format: protobuf
                   Topic: /CAM_BACK/lidar | Type: foxglove.ImageAnnotations | Count: 230 | Serialization Format: protobuf
                   Topic: /CAM_BACK/camera_info | Type: foxglove.CameraCalibration | Count: 230 | Serialization Format: protobuf
                   Topic: /CAM_FRONT/image_rect_compressed | Type: foxglove.CompressedImage | Count: 237 | Serialization Format: protobuf
                   Topic: /LIDAR_TOP | Type: foxglove.PointCloud | Count: 396 | Serialization Format: protobuf
                   Topic: /RADAR_BACK_RIGHT | Type: foxglove.PointCloud | Count: 262 | Serialization Format: protobuf
                   Topic: /RADAR_BACK_LEFT | Type: foxglove.PointCloud | Count: 269 | Serialization Format: protobuf
                   Topic: /RADAR_FRONT_RIGHT | Type: foxglove.PointCloud | Count: 262 | Serialization Format: protobuf
                   Topic: /RADAR_FRONT_LEFT | Type: foxglove.PointCloud | Count: 265 | Serialization Format: protobuf
                   Topic: /RADAR_FRONT | Type: foxglove.PointCloud | Count: 270 | Serialization Format: protobuf
                   Topic: /drivable_area | Type: foxglove.Grid | Count: 41 | Serialization Format: protobuf
                   Topic: /tf | Type: foxglove.FrameTransform | Count: 3177 | Serialization Format: protobuf
                   Topic: /semantic_map | Type: foxglove.SceneUpdate | Count: 1 | Serialization Format: protobuf
                   Topic: /map | Type: foxglove.Grid | Count: 1 | Serialization Format: protobuf
                   Topic: /odom | Type: Pose | Count: 984 | Serialization Format: json
                   Topic: /imu | Type: IMU | Count: 1975 | Serialization Format: json
                   Topic: /CAM_FRONT/camera_info | Type: foxglove.CameraCalibration | Count: 237 | Serialization Format: protobuf
                   Topic: /CAM_FRONT/lidar | Type: foxglove.ImageAnnotations | Count: 237 | Serialization Format: protobuf
                   Topic: /CAM_FRONT/annotations | Type: foxglove.ImageAnnotations | Count: 41 | Serialization Format: protobuf
                   Topic: /CAM_FRONT_RIGHT/image_rect_compressed | Type: foxglove.CompressedImage | Count: 236 | Serialization Format: protobuf
                   Topic: /CAM_FRONT_RIGHT/camera_info | Type: foxglove.CameraCalibration | Count: 236 | Serialization Format: protobuf
                   Topic: /CAM_FRONT_RIGHT/lidar | Type: foxglove.ImageAnnotations | Count: 236 | Serialization Format: protobuf
                   Topic: /CAM_FRONT_RIGHT/annotations | Type: foxglove.ImageAnnotations | Count: 41 | Serialization Format: protobuf
                   Topic: /CAM_BACK_RIGHT/image_rect_compressed | Type: foxglove.CompressedImage | Count: 237 | Serialization Format: protobuf
                   Topic: /CAM_BACK_RIGHT/camera_info | Type: foxglove.CameraCalibration | Count: 237 | Serialization Format: protobuf
                   Topic: /CAM_BACK_RIGHT/lidar | Type: foxglove.ImageAnnotations | Count: 237 | Serialization Format: protobuf
                   Topic: /CAM_BACK_RIGHT/annotations | Type: foxglove.ImageAnnotations | Count: 41 | Serialization Format: protobuf
                   Topic: /CAM_BACK_LEFT/image_rect_compressed | Type: foxglove.CompressedImage | Count: 235 | Serialization Format: protobuf
                   Topic: /CAM_BACK_LEFT/camera_info | Type: foxglove.CameraCalibration | Count: 235 | Serialization Format: protobuf
                   Topic: /CAM_BACK_LEFT/lidar | Type: foxglove.ImageAnnotations | Count: 235 | Serialization Format: protobuf
                   Topic: /CAM_BACK_LEFT/annotations | Type: foxglove.ImageAnnotations | Count: 41 | Serialization Format: protobuf
                   Topic: /CAM_BACK/image_rect_compressed | Type: foxglove.CompressedImage | Count: 230 | Serialization Format: protobuf

When I run ros2 bag play NuScenes-v1.0-mini-scene-0655.mcap, I get Topics with different rwm serialization format have been found. All topics must have the same serialization format.

To Reproduce

** Steps to reproduce the behavior, e.g.

1. Download the NuScenes dataset here. We need Full dataset (v1.0) Mini, Map expansion and CAN bus expansion
2. Follow the instructions on to convert the dataset to MCAP. The data directory should be as follows.

.
├── can_bus
├── maps
│   ├── basemap
│   ├── expansion
│   └── prediction
├── samples
│   ├── CAM_BACK
│   ├── CAM_BACK_LEFT
│   ├── CAM_BACK_RIGHT
│   ├── CAM_FRONT
│   ├── CAM_FRONT_LEFT
│   ├── CAM_FRONT_RIGHT
│   ├── LIDAR_TOP
│   ├── RADAR_BACK_LEFT
│   ├── RADAR_BACK_RIGHT
│   ├── RADAR_FRONT
│   ├── RADAR_FRONT_LEFT
│   └── RADAR_FRONT_RIGHT
├── sweeps
│   ├── CAM_BACK
│   ├── CAM_BACK_LEFT
│   ├── CAM_BACK_RIGHT
│   ├── CAM_FRONT
│   ├── CAM_FRONT_LEFT
│   ├── CAM_FRONT_RIGHT
│   ├── LIDAR_TOP
│   ├── RADAR_BACK_LEFT
│   ├── RADAR_BACK_RIGHT
│   ├── RADAR_FRONT
│   ├── RADAR_FRONT_LEFT
│   └── RADAR_FRONT_RIGHT
└── v1.0-mini

The resulting MCAP files should be in output/ after the script has been run 3. Copy the MCAP files to a ROS2 environment and install rosbag2 dependencies with sudo apt-get install ros-rolling-ros2bag ros-rolling-rosbag2* 4. Run ros2 bag play <filename>.mcap

System (please complete the following information)

• OS: Ubuntu 20.04.4 LTS
• ROS 2 Distro: rolling
• Version: release

[emersonknapp]

The problem here is that rosbag2 has no idea how to understand your data in order to play it back - it doesn't have a deserializer for protobuf data. I'm also noticing that the "type" field is foxglove types, like foxglove.ImageAnnotations, which isn't a ROS 2 message type.

This is kind of a big can of worms, actually!

Beyond expecting that there isn't an error printed - what is the goal you're trying to achieve by running ros2 bag play nuscenes.mcap? Other ros2 nodes subscribing to the data, viewing in RViz, piping through the foxglove bridge to foxglove studio? Something else? The more we understand about what is trying to be accomplished the better we can do to determine where/what functionality is missing.

EDIT: Just want to clarify a point that maybe isn't immediately clear. rosbag2 recording can create a rosbag2-compatible MCAP file that can be played back, but not all MCAP files are data that rosbag2 can use. MCAP is a generic data file format, that happens to be used (now) as a backing store for ros2 recording data.

[brian386]

Thanks for your response!

My goal is that I have other ros2 nodes that need to subscribe to the data. We are trying to test our autonomous vehicle stack on the nuscenes dataset.

[emersonknapp]

Yeah, that seems like a useful case. Just packing the nuscenes data into an MCAP file is not going to be sufficient to achieve it, unfortunately. There will need to be some sort of conversion from protobuf data to equivalent ROS 2 message types (like sensor_msgs/msg/PointCloud etc). Either at playback runtime (potentially costly for big bandwidth), or offline in a conversion step.

I'm not sure at the moment what exactly that process would entail, without more research.

There is the rosbag2 SerializationFormatConverter plugin framework, for this purpose, but we haven't actually built any such plugins to date!

Given that it's a limited set of data types, if you have e.g. python code to parse the protobuf message fields, you could write a script that reads them in, constructs ROS 2 messages, and writes them to a bag using rosbag2_py API. That wouldn't be fully general but may meet your needs in the short term.

[Erically]

@brian386 I face the same problem,have you resolved it?

[brian386]

@Erically I made a fork of the nuscenes2mcap repo to convert nuscenes data into ros2bag-playable MCAP files here. Not sure if you had the same use case, but hope this helps!

[Erically]

@brian386 nice job

[MichaelOrlov]

• Closing this issue as wouldn't be fixed. Rosbag2 works as designed. There are general ROS 2 limitations that it is support only one rmw layer during runtime.