add glob support

[joy4eg]

For instance, this will allow us to build images using one generic config:

image boot.vfat {
        vfat {
                files = {
                        "zImage",
                        "*.dtb",
                        "boot.scr"
                }
        }
        size = 6M
}

Also if none *.dtb were found, that should be not an error.

[michaelolbrich]

Hmm, it's not that simple. Just allowing shell expansion will change the behavior of "filename with spaces.suffix".

[joy4eg]

We can simply workaround that:

if (stat(pattern) == 0) {
    // file exists, process it as is
} else {
    glob_t gl;
    glob(pattern, GLOB_BRACE | GLOB_TILDE, NULL, &gl);
    // process as array of files, if any
}

[jameshilliard]

What might make more sense is a layering system sort of like how dtb includes work. That way you can have a base config with overlay configs with board specific changes.

[Harvie]

files={} is unfortunately specific to VFAT. I think it would make sense to convert this to be generic parameter available to all filesystems, so everyone can benefit from such improvements...

[michaelolbrich]

Unfortunately that not trivial to do in a reliable reproducible fashion. First we would need to copy everything because most tools take one directory tree as input. And plain C is not a very good language to do this. And how would we handle user/group settings? And permissions? If you're not extra careful then the directory permissions will actually depend on the currently active umask. And that can cause all kinds of problems. And not to mention timestamps if you want this really reproducible.

With vfat this is all rather simple. There are no permissions or ownership. Other filesystems are a lot more complex.

[Harvie]

And plain C is not a very good language to do this.

To be honest... Is C even good language for genimage? We end up calling lot of external tools (eg. mkfs) for performance critical stuff anyway. And for parsing configuration, general logic and image layout planing it seems to me that it would make sense to use something more python-ish... But that's just my opinion :-)

[michaelolbrich]

It depends on the image types. The stuff we do in image-hd.c is not something I'd like to implement in Python... If you dig deep enough in the history then you'll note that I didn't start this. I just took over from Sascha at some point. I might have picked another language. Maybe C++? I'm not sure. Nowadays I might consider rust but rewriting it now is not something I'd want to do either.