SkiaSharp
SkiaSharp copied to clipboard
mono
[FEATURE] Support for GPU Acceleration for WinUI3
GPU Acceleration is important in SkiaSharp for scenarios where you have large fill areas on high dpi monitors, as well as just general performance maximization. It's important that GPU acceleration is also supported for the WinUI3 platform.
It seems the challenge was lack of support for Angle. Is this still valid? Are there alternatives?
We are running into the same issue, our application is really struggling when drawing real time plots on high dpi monitors.
We are very interested in the GPU acceleration as well using Uno Platform with WinUI 3 main head.
This is especially important because I recently evaluated Win2D in WinUI3 and was very disappointed with the performance compared to SkiaSharp in SOFTWARE mode. Perhaps I was doing something wrong, But ATM it seems like SkiaSharp is the right answer for high performance 2D rendering in WinUI3
ANGLE needs to be patched to be used in WinUI 3, I have a sample project which uses a patched version of ANGLE in WinUI 3. https://github.com/levinli303/WinUIANGLE, the detail of the patch is in https://github.com/levinli303/vcpkg/tree/angle-winui, hopefully someone will find it useful.
Just for my understanding: SKXamlCanvas does not use GPU-Acceleration right now, correct? What needs to be done on your/my side to enable GPU-acceleration? I don't understand the comment from @levinli303 above and if this could help me. Do I need to compile ANGLE myself?
The current issue is that we require ANGLE to be rebuilt using the new WinUI types. The current implementation is just for UWP and the ANGLE team is not really interested in adding another Windows platform.
Since we are already using all the Google build tools such as gn/ninja, we can build it as part of the SkiaSharp build. However, someone needs to do the bit of work to update the type references and somehow (I am not sure yet) reference all the correct files in the WindowsAppSDK nuget package.
Once we get a build of ANGLE for WinUI 3, then I can uncomment the code that we already have for the SkiaSharp GL view.
As of today, none of the work is really SkiaSharp or native skia work.
All the work needs to be done is to fork (and hopefully PR back) the Google ANGLE code and make the required modifications: https://chromium.googlesource.com/angle/angle
The required code is probably all here: https://github.com/levinli303/vcpkg/tree/angle-winui
But instead of SkiaSharp depending on layers of patches and then using vcpkg to build those patches, we should just make those modifications in a normal way and upstream. Google is open to a nice PR and I am sure if we just get the code changes to them they will accept it. They are probably not really interested in building and testing the new platform - not sure they do that for UWP anyways.
I see the forked vcpkg does a lot of things to extract and generate files - and we can do that too in the ANGLE code using some python scripts. All doable, just I do not have the time to work on that.
Making this work will also help the Uno Platform apps ;) So if anyone who is a big Uno fan...
there's a slightly updated version of the patch for ANGLE on WinUI here (that should work with WinAppSDK 1.4.1) https://github.com/celestiamobile/vcpkg/commit/abac0a9a434f0b91b836c3ee6456f18aaa9f21c3
Can you work @levinli303 converted into a PR? It seems you know what you are doing - I don't understand whats Happening in your Patch and how all stitches together. Thank you!
i'm not familiar with SkiaSharp but i'm happy to provide more info and assist if anyone wants to make it work.
From my understanding Skiasharp does not have to be changed that much - am I right @mattleibow? As long as ANGLE gets support for WinUI it should be ok. So getting a PR done for ANGLE should mostly be it.
i don't think my patch can be upstreamed to Google's ANGLE easily. In addition to the patch being dirty (enabling the UWP path on Win32 in order to use the SwapChainPanel), https://github.com/celestiamobile/vcpkg/blob/56dcda1c419e9efefa36454f15e291a6ff9f558f/ports/angle/portfile.cmake#L155 makes the generated libs tied to the version of WindowsAppSDK specified. Some time ago I tried using the ANGLE built against WindowsAppSDK 1.2 in a WindowsAppSDK 1.3 project with SwapChainPanel, it does not work.
I would suggest keep a separate fork of google/angle
I've added a 50$ bounty on Gitpay for this issue to be solved - I hope that is at least a start. I digged into ANGLE for a moment but I am completely lost.
I suppose theoretically a new backend for Skia could be made that uses D3D11 or D3D12 directly instead of using ANGLE.
That sounds like perhaps more work than modifying ANGLE to work with the WinUI3 types.
"I recently evaluated Win2D in WinUI3 and was very disappointed with the performance compared to SkiaSharp in SOFTWARE mode. Perhaps I was doing something wrong"
@gmurray81 you were most definitely doing something wrong. There is absolutely no way whatsoever that can be true. Win2D is a very thin wrapper over D2D, which is extremely fast. You'd need to elaborate further (in some other issue) for others to understand what was going on there.
Are you capable of doing this? :)
I doubt it.
Have you considered taking a look at how Avalonia uses ANGLE?
I vaguely recall them mentioning that they maintain their own fork of ANGLE. They use it for their Skia implementation on Windows, as they find its performance is better than WGL.
@Sergio0694 unfortunately, It's a bit hard to decompose what I'm doing into individual areas that can be analyzed, since I'm attempting to implement a layer for our charting product that targets Win2d, and the framerates were just not acceptable compared to Skia. This may have been self inflicted, for example, perhaps I was creating and destroying resources that need to be cached.
BUT, I heavily suspect based on my past interactions with Direct2D both directly and via WPF that there are some inefficiencies how Direct2D tessellates a complex polyline to render it on the GPU. Perhaps this has been fixed in later versions of Direct2D, though.
Try this as an example. Create a polyline that oscillates up and down in the y direction entirely randomly and with high frequency. Render this in WPF and modify it with a high framerate. This will bring WPF to its knees. Again, could be fixed in later Direct2d (doesn't help WPF) but may help Win2d.
Anyhow, I don't know if it was scenarios like that which were causing the performance to be worse than software based Skia. But that's one of the scenarios I've experienced in the past that Skia handles much more gracefully than WPF (old Direct2d)
Funny story actually. The Chromium/Skia team actually broke this scneario when they first enabled the GPU support in Chrome, and I needed to report a bug, which they then fixed (not sure how, maybe they adjusted the way the prepped the lines for the GPU, or maybe they detect the scenario with heuristics and use a raster fallback)
https://bugs.chromium.org/p/chromium/issues/detail?id=102165&can=1&q=reporter%3Agmurray%40gmail.com
Anyhow, may not be that scenario, since I don't think I was testing anything related to that. But this is an example of the fact that what I'm testing is non-trivial polyline and polygon rendering. So perhaps Win2D is not optimized enough for certain types of shapes, etc, because traditionally Direct2D has definitely had some issues in this regard. To this day, you can feed a shape like this to WPF and it will just refuse to process it well.
Also note though, I've definitely had some issues from WinUI in the past also where it just seemed like there was too much tax being paid to bridge between managed/native. E.g. it seemed like all the data binding/dep prop stuff was pretty slow, presumably from being pretty chatty over that seam. The nature of my interaction with Win2D is also pretty chatty and from C# so there could be too much tax being paid to just funnel the information to Win2D for rendering. If managed/native bridging from WinUI to native libraries has been improved over time, then perhaps that will help also.
I haven't had the opportunity to test out the behavior of the Win2D layer I built against newer WinUI/Win2D versions in a while.
@gmurray81 Can you mock up a Skia drawing routine that you find performs poorly when not accelerated, and performs well when accelerated?
A method that takes in as a parameter SKPaintSurfaceEventArgs would do.
Are you capable of doing this? :)
Would you know where to start doing that?
I have no familiarity with Skia's codebase and am a bit lost delving into the repository so far.
"BUT, I heavily suspect based on my past interactions with Direct2D both directly and via WPF that there are some inefficiencies how Direct2D tessellates a complex polyline to render it on the GPU. Perhaps this has been fixed in later versions of Direct2D, though."
@gmurray81 I'm not entirely sure I'm following, are you sure we're talking about the same things here? You mention WPF, but that has its own DX9 renderer. I'm talking about Win2D, which runs on top of D2D, which uses DX11 behind the scenes. WPF does not use D2D in its renderer. Unless you specifically used Win2D (or D2D directly) from WPF (ie. you either referenced Win2D and used its canvas control via a XAML Island, or manually setup the Win32 interop to manage an HWND-based swapchain, or something else), then chances are you didn't actually use D2D at all. Is it possible there's a misunderstanding there about the various technologies? And if you did in fact use D2D, then yeah you most definitely just did something wrong, because as I said I just don't see how it could possibly be slower than Skia in software mode (or even if it was hardware accelerated), that just doesn't really make sense to me 😅
Eg. you talked about "creating a polyline". If this is the PolyLine you mean, yeah that's not Win2D/D2D at all.
This topic is quite important for my use-case - so I would love to see this tackled by anyone with knowledge in this space. I unfortunately do not have any clue what needs to be done.
But I raise the bounty for solving this issue to 500$.
I might take a look and see if I can build a demo when I have time. Assuming the work here required would be just building ANGLE for WinUI (which is already done sometime ago) and switch SkiaSharp to Microsoft.UI.Xaml.Controls.SwapChainPanel from Windows.UI.Xaml.Controls.SwapChainPanel
i've built the latest ANGLE against the latest WindowsAppSDK 1.4.231008000 here angle.zip
however i am struggling to make SkiaSharp compile (i have not made changes yet):
Error occurred while restoring NuGet packages: Invalid restore input. The original target frameworks value must match the aliases. Original target frameworks: netstandard2.0;netstandard2.1;net462;net7.0;net462;net7.0-windows10.0.19041.0, aliases: netstandard2.0;netstandard2.1;net462;net7.0;net7.0-windows10.0.19041.0. Input files: REDACTED\SkiaSharp\source\SkiaSharp.Views\SkiaSharp.Views.Desktop.Common\SkiaSharp.Views.Desktop.Common.csproj.
i've built the latest ANGLE against the latest WindowsAppSDK 1.4.231008000 here angle.zip
however i am struggling to make SkiaSharp compile (i have not made changes yet):
Error occurred while restoring NuGet packages: Invalid restore input. The original target frameworks value must match the aliases. Original target frameworks: netstandard2.0;netstandard2.1;net462;net7.0;net462;net7.0-windows10.0.19041.0, aliases: netstandard2.0;netstandard2.1;net462;net7.0;net7.0-windows10.0.19041.0. Input files: REDACTED\SkiaSharp\source\SkiaSharp.Views\SkiaSharp.Views.Desktop.Common\SkiaSharp.Views.Desktop.Common.csproj.
Maybe @mattleibow knows something about this?