parsec
parsec copied to clipboard
ICLDisco
PTT version converter initial pass at ptt converter
Usage:
python ptt_converter.py 2t1 2023-05-01_xsdk_ts200_N4800_dtd_lfq_normal_tv2.prof.h5 test2v1.prof.h5
I ran into an issue with this a while ago but never made a PR to start the conversation. Currently the biggest issue is that converting backwards (from version 2 to version 1) seems to cause all the datatypes in all the columns to become strings. Now, that was something I had to do to get a workaround for an error if I didn't do it, but surely there must be a better way. For the time being, here is the tool for people to review and comment on.
@omor1 you're a python guru and the author of trace version 2 (which my trace tools have migrated to) and I'm sure you would have lots of helpful feedback. If you want to spare the time, please feel free to comment here or message me about it.
The problem is probably that the v1 format has a giant table with a bunch of empty entries. Since integer-typed columns don't have a way to represent "no value" (unlike float, which can use NaN), it does something silly and converts everything to object—or to string, I guess? Anyway, the point is that types were inherently a problem in the v1 format because of this, and is part of the reason why I created the v2 format.
Well, good to know. It seems like we were running into the exact same issue with converting to "object" or "string". This makes converting between the ptt versions a bit tenuous since realistically there doesn't seem to be a great way to convert between them, just a "good enough" way. The main goal is just to make it work with my visualization tool, and that at least seems doable.