FB foveation extensions are missing some useful information

[Slin]

XR_FB_foveation_vulkan is not clear on what format the image in XrSwapchainImageFoveationVulkanFB has nor provides a way to set or query it. The vulkan specification only says: VK_FORMAT_FEATURE_FRAGMENT_DENSITY_MAP_BIT_EXT must be supported for the following formats if the fragment density map feature is enabled: VK_FORMAT_R8G8_UNORM Which I don't think excludes other formats?

Also there is no way to specify the desired foveation profile on swapchain creation, but at the same time I have not found any information on what the default is (I am assuming it's just no foveation at all). When implementing it, it seemed a bit strange that there is no way to unset it, but at the same time it can only be set after the swap-chain creation.

[bjornbytes]

Adding some notes to this:

• Vulkan currently provides 2 ways of using a fragment density map image -- VK_KHR_fragment_shading_rate and VK_EXT_fragment_density_map. The XR_FB_foveation_vulkan extension provides a means of retrieving a VkImage for the density map, but does not specify which extensions it is compatible with. One can sort of guess that VK_KHR_fragment_shading_rate is not supported because the Quest doesn't support it and the parent XR_FB_foveation extension only mentions the EXT extension, but I think spec language like "application must not use the density image with VkFragmentShadingRateAttachmentInfoKHR" would reduce ambiguity. (As a side note I think it would be cool if VK_KHR_fragment_shading_rate was supported somehow!)
• "The possible foveation levels are specified by the XrFoveationDynamicFB enumeration:" seems like a copy-paste typo from XrFoveationLevelFB.
• The first sentence of XR_FB_foveation_vulkan is "The Vulkan graphics API requires an image to be applied to the swapchain to apply a foveation pattern." I find this very confusing, even misleading. From an OpenXR perspective, images aren't applied to the swapchain. The extension lets you query an image associated with the OpenXR swapchain and use it as an attachment in a Vulkan render pass with one of Vulkan's variable-rate shading extensions.
• It's unclear how XR_FB_foveation_configuration+XR_FB_swapchain_update_state interacts with XR_FB_foveation_vulkan, if at all. Can an application use xrUpdateSwapchainFB to update the foveation profile of a swapchain created with XR_SWAPCHAIN_CREATE_FOVEATION_FRAGMENT_DENSITY_MAP_BIT_FB? If so, does the VkImage get modified, or should the application re-enumerate the swapchain images to get new density maps?
• Is it valid for an application to create a swapchain with XR_SWAPCHAIN_CREATE_FOVEATION_FRAGMENT_DENSITY_MAP_BIT_FB and later enumerate its images without XrSwapchainImageFoveationVulkanFB? I assume yes, but perhaps the spec should clarify.

[rpavlik-bot]

An issue (number 1604) has been filed to correspond to this issue in the internal Khronos GitLab (Khronos members only: KHR:openxr/openxr#1604 ), to facilitate working group processes.

This GitHub issue will continue to be the main site of discussion.

[s1ddok]

I know this is an old issue, but it is the only thing that GitHub search shows me. Did things get better since 2021? I want to implement eye tracked foveated rendering natively, using NDK and Vulkan for Quest Pro and I can't find any info about that in the whole internet. I would appreciate any directions!