foxglove
Image panel: support custom message containing nested image
I'd like to publish multiple images in one msg, but foxglove studio can only visualize the standard image msg type.
The custom msg may be like:
sensor_msgs/Image raw1
sensor_msgs/Image raw2
sensor_msgs/CompressedImage compressed1
sensor_msgs/CompressedImage compressed2
@chenjun00 Is there some limitation you have on-robot that prevents these from being sent as separate topics?
Theoretically there is no reason the Image panel shouldn't be able to pull out one field of a message as long as that field has the expected format/datatype.
@chenjun00 Is there some limitation you have on-robot that prevents these from being sent as separate topics?
I'm writing a node that using OpenCV to process image, so there are many debug images during the processing that I want to publish to visualize. If the images are sent as separate topics, I need to construct lots of publisher which I think is not convenient. But I have figured out a way that using a param to control which image to publish to a single topic. So if you think this feature is unnecessary, please close this issue.
And I'm looking forward to the feature https://github.com/foxglove/studio/issues/1662 to be supported.
Seems like a reasonable feature to support.
@defunctzombie @jtbandes how difficult would it be to show message fields that match a supported schemaName (in addition to topics that match a support schemaName) in the image selector dropdown?
Somewhat challenging - tho not impossible. We'd need to figure out if we provide the name through our parsed RosDatatypes structure.
I'm just not convinced the UI complexity is worth it tho.
What UI complexity? Wouldn't we just replace the dropdown in the Image panel with a MessagePathInput?
@jtbandes Adrian actually proposed keeping a dropdown and in the dropdown showing all the detected renderable topics and fields.
An alternative could be to use MessagePathInput and pre-filter its items to the topics and message paths that are renderable. That could also work well.
Yeah, I think just predetermine all the message paths, and add them to the existing dropdown. That way for existing users without nested images, there will be no UI change.
This could be a very useful feature. Our use case is that the image is published together with some meta information. e.g:
my_image_msg_type { meta_msg msg sensor_msgs/Image image }
Would it be possible to specify something like "my_msg.image" at the top of image panel? Just like described here - https://github.com/foxglove/studio/issues/2410#issuecomment-999082740
Would it be possible to specify something like "my_msg.image" at the top of image panel? Just like described here
Yep. Need to think through how we accomplish this in the implementation but I think the UX described above with a dropdown that supports selecting any topic or field that contains a matching datatype seems sensible.