format/format_entries does not work with a template with non-type values (and an @)

[LennardF1989]

Originally reported here: https://github.com/WerWolv/ImHex-Patterns/issues/147 - but closed due to wrong repository.

Consider the following file:

10 00 00 00 18 00 00 00 04 00 00 00 FF FF FF FF FF FF FF FF FF FF FF FF 74 65 73 74 31 32 33 34 35 6C 6F 6C 31 33 33 37 08 00 00 00 04 00 00 00 0C 00 00 00 05 00 00 00 11 00 00 00 03 00 00 00 14 00 00 00 04 00 00 00

test.zip

With the following pattern:

fn entryName(auto e) {    
    return e.name;
};

struct Entry<auto offset> {
    u32 nameOffset;
    u32 nameSize;
    char name[nameSize] @ offset + nameOffset;
}/*[[format("entryName")]]*/;

struct Header {
    u32 blockOffset;
    u32 nameBlockSize;
    u32 numberOfElements;
    
    Entry<blockOffset> entries[numberOfElements] 
        @ blockOffset + nameBlockSize
        //[[format_entries("entryName")]]
        ;
};

Header header @ 0x0;

Basically, this file requires the entry to know where the lookup table for the strings is located. I've used a template for that. The original issue was about not being to use a template in a format-function, but auto solves that. However, this particular file and pattern causes an error[E0011] when you try to use either of the format attributes.

I'm pretty sure it's related to the @ sign throwing things off.

[paxcut]

i uncommented top one only and i dont get any errors. it is true that the format function is not used in that case because using @ inside a struct doesnt place the data in the pattern data window.

[LennardF1989]

because using @ inside a struct doesnt place the data in the pattern data window.

This is not true. Local variables are the only thing that aren't shown, unless you explicitedly tag them as visible.

Default view, no formatters:

Either of the formatters enabled:

[paxcut]

you are right. my input file got overwitten somehow which turned wrong result. still you should not use @ inside structcts and use $ instead.

[paxcut]

I revised the code to use $ to avoid effect of creating views

#include "std/string.pat"
fn entryName(auto e) {    
    return e.name;
};

struct Entry<auto offset> {
    u32 nameOffset;
    u32 nameSize;
    $ =  offset + nameOffset;
    std::print("size {}",nameSize);
    char name[nameSize];
}[[format("entryName")]];

struct Header {
    u32 blockOffset;
    u32 nameBlockSize;
    u32 numberOfElements;
    $ = blockOffset + nameBlockSize ;
    std::print("Elements {}",numberOfElements);
    Entry<blockOffset> entries[numberOfElements] ;// [[format_entries("entryName")]];
};

Header header @ 0x0;

the output produced is

I: Elements 4
I: size 4
I: size 1819241525
E: error[E0004]: Type error.
E:   --> <Source Code>:11:23
E: 11 |     char name[nameSize];
E:                            ^
E:                            Array expanded past end of the data.
E: 
E: 
I: Evaluation took 0.0361785s

[paxcut]

After ir reads the name "test" the 1234 gets assigned to the nameoffset and the namesize becomes a huge number that overflows.

[paxcut]

the problem was with going back and forth because the strings occur before the data needed to read them. Notice that only the first string offset is needed because the next offset is the previous plus the previous size. So i implemented the code this ways and if reads all strings

#include "std/string.pat"
#include "std/core.pat"
fn entryName(auto e) {    
    return e.name;
};

struct String<auto nameSize> {
    char name[nameSize];
} [[format("entryName")]];

struct Entry<auto offset> {
    u32 nameOffset;
    u32 nameSize;
};

struct Header {
    u32 blockOffset;
    u32 nameBlockSize;
    u32 numberOfElements;
    $ = blockOffset + nameBlockSize ;
    std::print("Elements {}",numberOfElements);
    Entry<blockOffset> entries[numberOfElements] ;// [[format_entries("entryName")]];
    $ = entries[0].nameOffset+blockOffset;
    String<entries[std::core::array_index()].nameSize> name[numberOfElements];
};

Header header @ 0x0;

[LennardF1989]

This is just an example, in my actual situation the strings can be in any order and doesn't have to sequential, from a maintainability perspective, I also need to names to be part of the entry, because the entry in my file has a lot more data than "just" the name.

But thank you regardless :) I like the array_index() trick you did there (even though the final example doesn't end up using the auto offset).

[paxcut]

You are right about auto offset not needed, nice catch! I totally missed that.

To answer the issues of maintainability and keeping the data in one place here is code that satisfies both and also works. I think what is causing the bug is that the format function needs to access a variable that is not at the offset obtained by adding the offsets of the structure to the sizes of the elements before it. When you turn the string into a normal variable like the following code the error goes away.

#include "std/string.pat"
#include "std/core.pat"
fn entryName(auto e) {    
    return e.name;
};


struct CharArray<auto nameSize> {
   char name[nameSize];
} [[format("entryName"),inline]];

struct Entry<auto offset> {
    u32 nameOffset;
    u32 nameSize;
    CharArray<nameSize> name @ nameOffset+offset;
};


struct Header {
    u32 blockOffset;
    u32 nameBlockSize;
    u32 numberOfElements;
    Entry<blockOffset> entries[numberOfElements] @  blockOffset + nameBlockSize;// [[format_entries("entryName")]];
};

Header header @ 0x0;

[paxcut]

I edited the previous post to something a bit more helpful or at least i hope it is.