Add external time reference restamping [enable VIO with RS435 IR frames]

[m0oz]

Adds class ExternalTimestamping that allows to subscribe to a topic with timestamps that were captured using an external micro controller. Incoming frames are matched to hw timestamps, re-stamped and then published.

The matching works both ways: An incoming timestamp will be buffered if no frame was yet received or vice versa. We wait until we receive frame and timestamp within a specified time interval. Then, we use seq IDs if frame and stamps to determine an offset between the two streams. Any subsequent stamp/frame pairs will be correlated using this seq offset. If we drop a frame or stamp or if the two for some reason do not arrive within the specified time interval we reset the sync mode and wait to re-initialize based on the next incoming pair.

The interval within that we expect stamp and frame to arrive can be specified using the variables kMaxStampAge and kMaxFrameAge.

[andre-nguyen]

@m0oz Is this functional or was this a proof of concept to start a discussion?

[m0oz]

@andre-nguyen I used the feature branch for testing and had it running with the occasional quirk. However, I did not rebase or maintain the feature as it does not seem like it would get merged any time soon as there is no feedback at all from the intel team on this PR

Honestly, I would recommend to get yourself one of the new D435i cameras, which have an onboard IMU with hw time synchronization and spare yourself from setting it up and testing it yourself ;)

[andre-nguyen]

@m0oz Actually I rebased this on top of master and I'm currently using it. Curiously enough I sometimes have dropped frames where I have a time stamp but no frame to go with it. @doronhi I think this would be a very interesting feature to merge in provided we rebase it and parameterize the streams in the launch file which should be processed by the timestamp.

[doronhi]

@m0oz I want to apologize for not providing a feedback much sooner. The truth of the matter is that your original PR seemed very difficult to check without an appropriate external micro controller. It also seemed very difficult to validate in terms of accuracy of synchronization and frame drops. As it didn’t seem like a common request it got pushed down in priority. From what I read in the later posts, it seems that the D435i answers the original problem. Is that true? Also, not long ago, a global time stamp domain was added. It stamps the frames with a system time which can then be synced with any other device connected to the system. Is this a way to tackle the original issue?

[mhkabir]

@doronhi Are there any docs on the global timestamp domain stuff? How is the camera's time synchronised to the host system time?

[mhkabir]

The original intent of this PR was to allow timestamping of image frames with sub-millisecond accuracy against the host PC clock.

[doronhi]

The global time algorithm assumes a linear equation between the device’s HW clock and system clock. Each frame’s HW timestamp is converted to system timestamp using this equation. A separate thread occasionally samples the device for its clock reading, taking the reply time into consideration, and the algorithm updates the equation’s parameters accordingly. I measured the reported time difference between 2 frames taken simultaneously by 2 different cameras. Using this method I measured less then 1 millisecond difference in 99.9% of frames and less then 2 milliseconds in the rest of the frames. In the current librealsense release, 2.24.0, the algorithm still suffer from noise in the first couple of seconds, until it gathered enough samples and stabilized. This also will be corrected shortly #4396

[mhkabir]

Unless you're doing a two-way message exchange with the host computer, the one-way translation you describe can only yield timestamps which are accurate with respect to each other (i.e accurate delta times). If I'm not making a mistake, there will still be a (generally unoberservable) static offset with respect to the host timebase. This makes that method unsuitable for use where there are other sensors (e.g an IMU) correctly timestamped against the host timebase, unless you estimate the static offset by e.g a visual-inertial calibration.