Failed to play: No storage could be initialized. Abort

[AbhishekTiwari07]

Description

I encountered an issue when attempting to convert a single bag file into multiple compressed bag files. After the conversion, I am unable to play the compressed bag files and encounter an error.

Expected Behavior

I should be able to play the file without any error.

Actual Behavior

I encounter an error while trying to read the 4th bag file.

[ERROR] [1726217343.678629849] [rosbag2_storage]: Could not open 'split_bag/split_bag_3.db3' with 'sqlite3'. Error: Error when processing SQL statement. SQLite error: (11): database disk image is malformed
[ERROR] [1726217343.680619999] [rosbag2_storage]: Could not load/open plugin with storage id 'sqlite3'
[ERROR] [1726217343.743596884] [rosbag2_player]: Failed to play: No storage could be initialized. Abort

To Reproduce

1. Download the attached "reprod_files.zip" and extract "/input_bag" and "output.yaml".
2. Run ros2 bag convert -i <path to input_bag> -o output.yaml
3. Run ros2 bag play split_bag
4. Logs print error while attempting to read 4th bag file (split_bag_3.db3)

System (please complete the following information)

• OS: Ubuntu Noble (24.04)
• ROS 2 Distro: Jazzy
• Install Method: release
• Version: release

Additional context

reprod_files.zip

[AbhishekTiwari07]

I observed this issue in ROS2 Jazzy and found that it also occurs in Iron. However, it works correctly in Humble.

[karthiknit1]

@r7vme @MichaelOrlov - Can you take a look at this, I am also seeing this issue on windows with Jazzy and Iron installation

[r7vme]

@karthiknit1 I was able to reproduce it with Jazzy. Looks like this is smth to do with max_bagfile_size, when compression is enabled. When max_bagfile_size parameter is commented out everything works as expected.

@MichaelOrlov any ideas off the top of your head?

[karthiknit1]

@r7vme In the below function, when I pass empty string instead of message_definition.encoded_message_definition, Its able to convert db3 to zstd and decompress all the bags while playing with no error. I am not sure if the message definition is causing any issue here.

void SqliteStorage::create_topic( const rosbag2_storage::TopicMetadata & topic, const rosbag2_storage::MessageDefinition & message_definition) { std::lock_guardstd::mutex db_lock(db_read_write_mutex_); if (topics_.find(topic.name) == std::end(topics_)) { auto insert_topic = database_->prepare_statement( "INSERT INTO topics" "(name, type, serialization_format, offered_qos_profiles, type_description_hash) " "VALUES (?, ?, ?, ?, ?)"); insert_topic->bind( topic.name, topic.type, topic.serialization_format, rosbag2_storage::serialize_rclcpp_qos_vector(topic.offered_qos_profiles), topic.type_description_hash); insert_topic->execute_and_reset(); topics_.emplace(topic.name, static_cast(database_->get_last_insert_id())); } // TODO(morlov): Add topic.type_description_hash when it will be really calculated or getting // from service. Currently dummy hashes causing tests failure std::string topic_type_and_hash = message_definition.topic_type; if (!topic_type_and_hash.empty() && msg_definitions_.find(topic_type_and_hash) == std::end(msg_definitions_)) { std::string type_description_hash = message_definition.type_hash.empty() ? topic.type_description_hash : message_definition.type_hash; auto insert_msg_definition = database_->prepare_statement( "INSERT INTO message_definitions (topic_type, encoding, encoded_message_definition, " "type_description_hash) VALUES (?, ?, ?, ?)"); insert_msg_definition->bind( message_definition.topic_type, message_definition.encoding, message_definition.encoded_message_definition, type_description_hash); insert_msg_definition->execute_and_reset(); msg_definitions_.emplace( topic_type_and_hash, static_cast(database_->get_last_insert_id())); } db_file_size_ = db_page_size_ * read_total_page_count_locked(); }

[MichaelOrlov]

@AbhishekTiwari07 I am not able to reproduce this issue the way how you did it because don't have message definition for the vizualization_msgs.
Could you please attach what you have got after ros2 bag play split_bag? i.e. content of the split_bag folder.

[MichaelOrlov]

@AbhishekTiwari07 Never mind, I was able to reproduce this issue. Will need to debug and make a more comprehensive analysis.
It seems the 3rd DB file is really corrupted after conversion.

[MichaelOrlov]

There is something wrong with the compression operation during bag conversion. Without compression, all bags are not corrupted and successfully replayed.

[MichaelOrlov]

• Closing as fixed in the https://github.com/ros2/rosbag2/pull/1975. Feel free to reopen if you still observe this issue.