data_tamer
data_tamer copied to clipboard
PickNikRobotics
Can't view mcap files in foxglove
It seems that mcap files generated by the mcap sink can't be viewed in Foxglove Studio. It gives an Unsupported encoding "data_tamer" error. I see that the profile field in the message is set to data_tamer, but i'm not sure if that's what's causing the problem or not.
Ok I have looked more into this and can confirm the reason foxglove can't view the data is because it's not using one of foxgloves supported encodings/schema's. It's currently using a custom datatamer schema which I assume only plotjuggler is setup to read
This is expected. The data tamer protocol is NOT supposed to work with foxglove.
Said that, if the foxglove team want to, I will be happy to support them with the development of a parser 😉
Would it maybe be better(or at least as good? idk) to add support for one of their supported protocols? I may be able to put some time in on that and see if I can get something working if you think it'd be worth it
The current protocol is very, very similar to ROS1. The only difference is that you should remove the ability of disabling a variable.
I can prepare a preliminary branch with the changes, but I need other people to test it, since I have little bandwidth
Yeah I just meant that it'd probably be easier to make data tamer log data in a format that foxglove natively supports rather than try and add something to foxglove to support a custom data tamer protocol. I could totally be wrong though as i'm not all that familiar with the specifics of either
I went ahead and created a foxglove websocket sink and an mcap sink that both work with foxglove here. If you all are interested in incorporating this functionality in data tamer itself let me know and I can incorporate it however you'd like and make any suggested changes. If not no problem I can keep it there. It doesn't support the active variable masking stuff though. It's still new but I've been testing with it and it has been functional for me.
If I understand correctly, looking at the code, the way this works is converting it to a brutally large JSON message...
https://gitlab.com/jlack/data_tamer_tools/-/blob/main/src/foxglove_mcap_sink.cpp?ref_type=heads#L75
So, even if I believe that it is good, this goes against the idea of "efficiency" in the library...
Maybe it is still useful, but I am uncertain about including this.
Yeah I actually very much agree with you. I like the data tamer API and very much want to use it but I want to use it with foxglove and the data tamer schema isn't supported by foxglove so I just picked one of the protocols that foxglove supported and went with that. I'm open to suggestions on other avenues to be more efficient though, so if you have some suggestions on what I could do to maximize efficiency in a way that foxglove still supports throw them my way 😊
@facontidavide Should I add a link to this in the README? I'm sure others would find this really useful. I'm unfamiliar with foxglove studio so I don't have any suggestions regarding efficiency.
I agree w/ @facontidavide about the inefficiencies. It should probably wait until I can figure out how to make it more efficient. I have opened a discussion on the foxglove discussion board where i'm trying to figure out how to make it more efficient given the schema parsing abilities foxglove already has. Ideally they would support the data tamer style schema even if through some other more standard mechanism, but I can't currently see how to make that work.
We can close this I think. I have spent a good amount of time making a data_tamer_tools package that has a number of tools that are built on data tamer that I hope folks may find useful. Most of the tools are targeted at using data tamer to log data in formats that foxglove can visualize, and I have expanded this json stuff to be much more efficient using protobuf as well as added rosout logging and some mechanisms for automatic log file rotation. Appreciate the discussions here though 🙏