astrobee icon indicating copy to clipboard operation
astrobee copied to clipboard

Add image metadata to recorded bagfiles

Open rsoussan opened this issue 3 years ago • 8 comments

It would be helpful to record some of the image configurations, such as gain and exposure, during activities for future analysis. This could be generalized for other important config files as well if needed.

rsoussan avatar Mar 08 '22 19:03 rsoussan

We had also talked before about publishing a "CameraInfo" message with the calibration data.

bcoltin avatar Mar 08 '22 19:03 bcoltin

Both of these make sense, but I think we are talking about different things:

  1. Calibration info which is hopefully static fits perfectly into the standard ROS sensor_msgs/CameraInfo message. (Typically publish once, latching.)
  2. We might also want to publish information that has the potential to be dynamic on a per-image basis. Examples would be focus, exposure, and white balance parameters that are often auto-adjusted dynamically by the camera or the driver. These are the kinds of things that camera hardware normally saves in the photo metadata (relevant metadata standards include EXIF, IPTC, and XMP). I'm not sure if ROS has established a standard message for that. Currently, we try to turn off most of the "auto" features for NavCam/DockCam, but I doubt we got everything. For example, if auto-white-balance is enabled, but we could record the color temperature changes on the fly, we could correct for the changes across multiple images when building maps and avoid color casting artifacts. And if we want to have the option of turning on auto-exposure in the future, Ryan already noted that if we publish the exposure parameters, that could help the localization maintain feature tracking through exposure changes.

trey0 avatar Mar 08 '22 20:03 trey0

Yeah, I agree, just figured we might want to do calibration if we're going to do these other parameters.

bcoltin avatar Mar 08 '22 21:03 bcoltin

Are all the parameters defined in rosparam? (I know that parameters that use config_server are) (https://github.com/nasa/astrobee/blob/master/shared/ff_util/src/config_server/config_server.cc#L177)

We could easily add a 'rosparam dump' to the logs (maybe by adding it to the data_bagger when starting the recording)

marinagmoreira avatar Mar 14 '22 23:03 marinagmoreira

No, I don't believe any of these are currently published at all.

bcoltin avatar Mar 15 '22 18:03 bcoltin

Could we change the config reader to publish to rosparam and record it? Might be a generalized solution that's future proof.

marinagmoreira avatar Mar 15 '22 18:03 marinagmoreira

I think the issue would be with Trey's #2 point above, parameters that can change dynamically.

bcoltin avatar Mar 15 '22 18:03 bcoltin

We have gain and exposure now but could use a more general metadata message

bcoltin avatar May 16 '22 22:05 bcoltin

We may want to add a more general message in the future (or add more information to the current message).

bcoltin avatar Aug 22 '22 22:08 bcoltin