amber

amber copied to clipboard

Amber has buffer comparison algorithms (currently just RMSE, but more will probably be added). We should add a standalone tool that takes:

• the algorithm to run, and its parameters (probably as command line arguments)
• two PNG files (paths passed via command line arguments)

and outputs:

• Equal or not equal
• The input threshold value (e.g. the input tolerance RMSE value that would allow the comparison to pass)
• The output value from the comparison (e.g. the RMSE value)

I'm wondering if this needs to be a standalone tool. I'm guessing once we have textures working we'll want to be able to load real images into textures. (We've also previously had a request to load shaders from external files). So, I'm wondering if we add something like:

LOAD PNG img1 FROM path/to/img1.png FORMAT B32R32G32A32
LOAD PNG img2 FROM path/to/img2.png FORMAT B32R32G32A32

EXPECT img1 EQ_RMSE img2 TOLERANCE 0.2

(This is similar to the buffer comparison test we have which doesn't execute a pipeline either)

Or, something along those lines. The LOAD syntax needs work, I wanted to make sure we explicitly put the PNG in there as I could see us loading DATA or SPIRV or something else.

The LOAD command would create a buffer named img1 which has the same restrictions as a buffer around naming conflicts and the buffer is setup with a format from the FORMAT value.

Should the paths be relative? Absolute? Based off some config dir that we default to tests?

@dneto0 thoughts on this? Is my hypothesis that LOAD will be generally useful correct?

Yes I also wondered this (mentioned on chat). But would this require a Vulkan or Dawn device to be able to run the script?

Currently, it may, but I think that would be fixable. Something like, if there are no pipelines in the input file.... don't create the engine as there is nothing to do? Some kind of rule along those lines? Would have to look into the code before making a definitive answer.

Image comparison seems like a completely separable function. ?! It would be useful and quite usable as a separate utility. It seems overly fussy to force someone to write an Amberscript file to perform a comparison between two given files.

Now, the image comparison functionality should also be available as an expectation-check in an otherwise-normal AmberScript file.

So, like in @dj2' s example, you'd have an AmberScript that declare an image img1 and a pipeline to populate it, and an img2 similarly populated from a pipeline or a file load, then an EXPECT img1 EQ_RMSE img2 .... line.

I'd assume the amberscript version would only be used by people already creating amberscript for things and specifically want the root mean square error check, otherwise, I'd guess imagemagik would be used.

Discussed offline. There's an extra use case: while exploring new image comparison predicates, it's useful to check that new predicate on an existing library of pairs of images with known-outcomes. That motivates making a separate tool

In offline discussion @dj2 agrees.

Cool. How should we proceed here? We could add a directory image_tool/, or we could just start adding additional .cc files in samples/ and produce another binary from there. Would we need to build a library from png.cc (and perhaps other files), or can binaries (amber and image_tool) have overlapping .cc files?

Add it into samples as image_diff.cc. You can include the png and other files in multiple executable source sets it should be fine. If we get a third tool where we use them we can pull them out to their own support library at that point.