ImHex-Patterns icon indicating copy to clipboard operation
ImHex-Patterns copied to clipboard

patterns: Add json support for glb files

Open xZise opened this issue 6 months ago • 8 comments

This makes it possible to separate display the different buffer views, accessors and images (even visualizing them).

Unfortunately the data within the JSON gets sometimes corrupted and this is the reason, why it parses the JSON multiple times at some places.

xZise avatar Jun 22 '25 12:06 xZise

I've uploaded this as a draft, because it parses the JSON multiple times and preferably this shouldn't be necessary. I'm also not sure with what formatting it should work. Newer files seem to use UpperCamelCase for types and snake_case for fields. Currently the original part uses the C++ style with the _t suffix.

It also fails the unit test as the Json type is not available without ImHex.

xZise avatar Jun 22 '25 13:06 xZise

I've uploaded this as a draft, because it parses the JSON multiple times and preferably this shouldn't be necessary.

I am confused by this statement. Json doesn't get parsed multiple times. The pattern you wrote is the one that is repeating the operation 3 times and you are right, it is not needed, so remove the extra copies.

I'm also not sure with what formatting it should work. Newer files seem to use UpperCamelCase for types and snake_case for fields. Currently the original part uses the C++ style with the _t suffix.

Since you are extending an existing pattern the proper thing to do is to use the format that it was written originally so that the original code remains as unchanged as possible while making the pattern easier to read for everybody.

It also fails the unit test as the Json type is not available without ImHex.

Use the same check the evaluator uses to see if it is running with all the plugins and if it isn't don't use the json type. thats ok for unit tests since we dom't test the json part anyway.

paxcut avatar Jun 22 '25 13:06 paxcut

After going through the motions of adding a meshes struct to the group formed by images, buffer views and accessors, I realized that I was simply recreating something that already exists in the pattern.

Meshes can be accessed from the first json chunk so adding them again is simply not necessary. Naturally, I started to wonder what the purpose of your creating variables for buffer views and accessors since the ones in the json object already exist and already follow the strict format specifications. At first I thought they were simply created to facilitate the access to the binary data and my confusion now stems from the realization that we can access the binary data equally well or even better using the variable that already exists and read as the first gltf chunk.

paxcut avatar Jun 24 '25 08:06 paxcut

Can you maybe explain your last part of the comment a little bit?

[...] we can access the binary data equally well or even better using the variable that already exists and read as the first gltf chunk.

What do you mean with "access the binary data"? Access it to generate pattern data? Or access it to visualize it?

xZise avatar Jun 24 '25 16:06 xZise

Can you maybe explain your last part of the comment a little bit? What do you mean with "access the binary data"? Access it to generate pattern data? Or access it to visualize it?

In order to visualize data, and by that i mean to use the visualize attribute on a pattern holding the data you must first generate pattern data. Right now the binary part of the file is converted into a pattern that is an array of bytes. Yes, it is a pattern, but no, it isn't what the format specifies as being stored there.

paxcut avatar Jun 24 '25 19:06 paxcut

Okay in that case it's up to you, what you want to do. My usecase was to determine what parts of the binary data contains what data and this is accomplished by the pattern.

My two reasons for marking it as a draft (duplicate jsons and naming) are not present anymore which is why I marked them ready for review.

xZise avatar Jun 24 '25 20:06 xZise

I have tested your pattern on files that are not the unit test file and found it very slow. It is true that the reason for it is mostly one of the changes I proposed. I wasn't considering the effect on evaluation time but moving the match for types to the last moment makes it have to check for matches at every value. To make up for my mistake I am attaching a patch to put the code back to how it would have been without that change so you can just copy paste it.

Even with the patch the code is still very slow for want it does. In one case it goes from 20 seconds to 2 minutes to display images, bufferviews and accessors. While it may be worth it for images, looking at buffers of bytes dont really say much about the data being stored. Some of the data are indices, some are vertices and so on which can be seen in the 3d visualizer model code I have added to the gltf pattern to visualize models stored.

patch.txt

paxcut avatar Jun 28 '25 10:06 paxcut

Thanks a lot! @paxcut Do you think we can merge this?

WerWolv avatar Aug 31 '25 09:08 WerWolv