wineopenxr: Verify proper number of swapchain formats is returned
If an OpenXR runtime gives an unrecognized Vulkan format (i.e. one that returns DXGI_FORMAT_UNKNOWN from map_format_vulkan_to_dxgi) for one of its swapchain formats, the formatCountOutput that is returned on the first call to xrEnumerateSwapchainFormats will differ from that of the second call, and apparently games (only tested with Blacktop Hoops) don't like this and will just end the OpenXR session. This is useful in particular for Monado, which advertises a few formats that aren't covered in map_format_vulkan_to_dxgi.
Made a few style adjustments.
Did you check that SteamVR's openxr indeed returns XR_ERROR_SIZE_INSUFFICIENT when output buffer size is non-zero but is not enough to fit the full list?
Besides that, the patch looks good to me from the functional point of view. I still have stylistic comments for nearly every line of the patch. I could quicker fix it myself rather than bouncing the comments and new patch iterations, but in case you want to do it yourselves would you please let me know, I will put the comments here.
Also, we need a real patch author's name in the patch. If you don't want to put it for some reason I can commit the final version under my name and reference this MR along with "Patch by Supreeeme [email protected]" in the patch description.
Originally based off the OpenXR spec but yes, SteamVR does return XR_ERROR_SIZE_INSUFFICIENT in that case.
Feel free to make the style changes yourself.
Updated the commit with my name.
I pushed the patch (edited by me) to Proton Experiment.