Record exception :SQLite error (1299): constraint failed

[cz000]

ubuntu 22.04, ros2 Humble, ros-humble-rosbag2-transport 0.15.13-1jammy.20241128.101511

1. I just record some topics and get this exception. and this exception is occasionally.
2. This node had reocrd same topics for many times and broke down for this time.
3. There is a .db3 file in the bag directory and no metadata.yaml. Use ros2 bag info with the db3 file, there are many topics count 0 and I can't distinguish which topic may have the bad message.

[cz000]

@MichaelOrlov

[cz000]

@fujitatomoya , Help me, please

[fujitatomoya]

it is hard to determine what happened to you with current information, it would be more helpful to share more details like complete procedure or reproducible examples so that we community could see what possibly happens here?

[MichaelOrlov]

Hi @cz000,

• Please refer to this closed issue https://github.com/ros2/rosbag2/issues/395 and this advise https://github.com/ros2/rosbag2/issues/1029#issuecomment-1497442077

[creatorofuniverses]

I encountered the same problem, and I'm glad there's a way to recover a corrupted bag. However, this doesn't address the issue of interrupted recordings that must be restarted.

In my case, the recording fails between 30 and 300 seconds after it begins, and make whole bag record utility completely unusable for me.

[MichaelOrlov]

@creatorofuniverses We have a graceful handling for the SIGINT and SIGTERM signals in the Rosbag2 recorder.
When you interrupt recording with the CTRL+C, it should gracefully finish the bag write.
The problems could be caused by abnormal recording termination, for instance, a power outage, a system crash or other reasons, like sending SIGINT to the Docker container where the Rosbag2 recorder is running.

If you want to be more resilient for abnormal recording termination, I would recommend using either the MCAP storage plugin, i.e., -s mcap option, or the resilient option for -- storage-preset-profile setting with sqlite3 backend. See details in the https://github.com/ros2/rosbag2/blob/rolling/rosbag2_storage_sqlite3/README.md#storage-configuration-file

$ ros2 bag record -s sqlite3 -o my_bag --all --storage-preset-profile resilient

[MichaelOrlov]

@cz000 As regards to the original error

what(): Error when processing SQL statement. SQLITE error (1299): constraint failed

Can we have more knowledge of how to reproduce this error?

[cz000]

Can we have more knowledge of how to reproduce this error?

I'm sorry for I have no way to reproduce this error now, it occurs low frequency, this error occurs in our product environment. We record same topics for many times and most time works well.

The problems could be caused by abnormal recording termination

In our log, after this error, other nodes still work normally.

Is there a problem with the sending and receiving of messages? The SQLITE error (1299) seems write incorrect message. Maybe the recorder write same data in sqlite? @MichaelOrlov

[creatorofuniverses]

@creatorofuniverses We have a graceful handling for the SIGINT and SIGTERM signals in the Rosbag2 recorder. When you interrupt recording with the CTRL+C, it should gracefully finish the bag write. The problems could be caused by abnormal recording termination, for instance, a power outage, a system crash or other reasons, like sending SIGINT to the Docker container where the Rosbag2 recorder is running.

If you want to be more resilient for abnormal recording termination, I would recommend using either the MCAP storage plugin, i.e., -s mcap option, or the resilient option for -- storage-preset-profile setting with sqlite3 backend. See details in the https://github.com/ros2/rosbag2/blob/rolling/rosbag2_storage_sqlite3/README.md#storage-configuration-file

$ ros2 bag record -s sqlite3 -o my_bag --all --storage-preset-profile resilient

Thank you for your suggestion. I'll give it a try.

An important point: the issue doesn't occur after stopping the recorder or container (I use a Docker container based on the ros:humble-ros-base image). Literally, I do nothing after starting the recording; it just crashes. The container remains running, and everything else works fine.

I'm attempting to record Ouster LiDAR data (including point clouds and IMU data) as well as camera images (CompressedImage). Sometimes it works well for about an hour without problems, but other times, like now, it fails after just a minute of operation.

I realized that when the system was functioning smoothly for an hour, it was using per-file zstd compression. And the LiDAR clouds had a frequency of about 3 Hz (which was my mistake). Now, I have tried running it without compression and with an honest 9-10 Hz for the pointcloud topic.

Could this be related to the issue mentioned earlier?

[MichaelOrlov]

@cz000 @creatorofuniverses When an issue is difficult to reproduce, like this one, but it causes the process to crash, the only meaningful way to try to triage it is to get a core dump after the crash. Could you please try to capture a core dump after the crash on your system and share it with the community for further analysis?

Note. You may need to perform some extra steps to instruct your local system to produce core dumps in case of a process crash.

[creatorofuniverses]

@MichaelOrlov I tried to get such dump, unsure if I have done it correctly, but here it is.

core.418.ros2.gz

[MichaelOrlov]

@creatorofuniverses The core dump collected on the aarch64. I don't have binaries for the aarch64 to be able to parse it and debug. I've tried to cross-compile but unfortunately got a lot of errors from vendor packages since they are not handling cross-compilation properly. Can you reproduce thi issue and collect coredump on a x86-64 architecture?