[UI Tests] Move image based tests to runtime tests

[jeromelaban]

What would you like to be added:

Many UI tests on Android and iOS are using "remote" screenshots which take a significant amount of time to run (1 second per screenshot in general) that do not validate actual UI interactions.

Moving these UI tests (most of which are using ImageAssert) to be Runtime Tests would help decrease significantly the CI build time.

Here's a (non-exhaustive) list of test that could be migrated:

• https://github.com/unoplatform/uno/blob/c1c6fdb534c0ff7762bd46dbecc1017b4c5392ac/src/SamplesApp/SamplesApp.UITests/Windows_UI_Xaml_Media/ImageBrushTests/ImageBrush_Stretch.cs#L20
• https://github.com/unoplatform/uno/blob/6e093f7bae3116d004e0b54a66467193dcd29a96/src/SamplesApp/SamplesApp.UITests/Windows_UI_Xaml_Controls/CanvasTests/Canvas_Measurement_Tests.cs#L17

Why is this needed:

More performant CI, with more platforms to be validated as part of the runtime tests.

[amelield]

When_Stretch() is using screenshots. So if the screenshot doesn't work in web assembly, should exclude Wasm from that test? I was trying to find a workaround that reproduce the same results(without much success). Or do anyone know a good way to make it works.

[jeromelaban]

That's correct. WebAssembly cannot make screenshots at this time, so we need to skip this test using #if !__WASM__.

[amelield]

In Skia, I can't find what kind of import or interface could make TakeScreenshot(String) works; I see there are possibilities (in Runtimes or others places) but non of those I tried fixed the TakeScreenshot not existing in context.

[jeromelaban]

Here are a few of those, using the RenderTargetBitmap API (an equivalent of TakeScreenshot, but in-app): https://github.com/unoplatform/uno/blob/aa9311da1926bd52df96ec152cb7585d3c8d74a3/src/Uno.UI.RuntimeTests/Tests/Windows_UI_Xaml/Given_FrameworkElement_Opacity.cs#L37-L72

[amelield]

Thank you

[amelield]

Do I need to put a link to an image in the Image_Brush and if so where can I find it?

[jeromelaban]

@amelield which image are you referring to?

[amelield]

@jeromelaban The image who create that comportment here. I am not sure where it come from in the code. It is also possible it is not needed.

[jeromelaban]

This is what await renderer.RenderAsync(SUT); does. You won't be able to "see" the image, but ImageAssert will be able to use it to make validations.

[amelield]

@jeromelaban I need some guidance. When I use var fill = new RawBitmap(renderer, X); With X being a Windows UI rectangle that have the ImageBrush as fill or whit a border around. Fill has an heigh and weight but has pixels = null and because of that failed any color comparison. Skia ImageBrush.ImageBrush is not a Framework element and is really hard to manipulate by itself. var fill = new RawBitmap(renderer, brush); Throws a not implemented error. S Windows.UI.Xaml.Media.ImageBrush does not throw exception but the pixels are still null. I tried to use Rect.Rect in lieu of the Windows UI rectangle but I did not find a way to include ImageBrush as a fill or a child.
When I try to create a new FrameworkElement that return a ImageBrush I receive init() not implemented. So I thought it was a dead end but it might be the way to go. What should I do next?

[jeromelaban]

RenderTargetBitmap only renders UIElements indeed. The best way to do this is to render a parent element (let's call it A), like a Border or a Grid, then use the control (called B) that applies the brush to get the its relative coordinates (using B.TransformToVisual(A)) and then do the pixels comparisons with those coordinates.

This method already does the coordinates conversion, so it may help: https://github.com/unoplatform/uno/blob/aa9311da1926bd52df96ec152cb7585d3c8d74a3/src/Uno.UI.RuntimeTests/Helpers/ImageAssert.cs#L43

[amelield]

HasColorAtChild(RawBitmap screenshot, UIElement child, double x, double y, Color expectedColor, byte tolerance = 0, [CallerLineNumber] int line = 0) use a UIElement and Brush is not one. Using A.Background or brush give a cannot convert Windows.UI.Xaml.Media.ImageBrush; into a Windows.UI.Xaml.Element; With the A element (grid or border), the test simply cannot fail, no matter the color difference. That method would be perfect to use for that if it was not the case.

I also tried to use B.TransformToVisual(A). In my case A is a Grid but I am not sure which control B would be (it is probably very simple). I thought that grid was the control that applied the background. I looked at helpers and documentation without finding the answer.

Edit: That might not be the problem (or the only problem) since all the pixels bytes are at 0 in all the RawBitmap and buffers. I will investigate that further.

[jeromelaban]

If all the pixel bytes are at 0, it probably means that the screenshot failed, somehow. Let's discuss this tomorrow :)