KhronosGroup
Raytracing support
Hi, I was curious if there is any support for ray tracing. I am interested in prototyping raytracing applications on macOS with Metal 2 and then scaling them up for production on Nvidia's RTX GPUs.
Theoretically, it should be possible to support the VK_NV_ray_tracing extension on Mac, using the Metal Performance Shaders framework which uses a very similar technique to accelerate ray tracing. At this time, though, ray tracing isn't supported.
If anyone wants to take a run at implementing this...it would be welcome.
I might be interested, but I probably won't be free for a few weeks to a month for other priorities.
Why not go straight for the newest (vendor independent) VK_KHR_ray_tracing extension?
https://www.khronos.org/news/press/khronos-group-releases-vulkan-ray-tracing
Why not go straight for the newest (vendor independent)
VK_KHR_ray_tracingextension?
This is an older thread. But yes...we should now aim for the KHR version.
One of the dependencies of VK_KHR_ray_tracing is VK_KHR_buffer_device_address. Which requires the host to know the physical addresses (pointers) inside the device. However, it seems that Metal does not expose that information to the user.
Came here searching for VK_KHR_buffer_device_address references, as is a recommended extension for current VKD3D Proton 2.0.. let me point out that now raytracing on Metal (on iOS 14.0 and Mac 11.0) seems more similar to VK_KHR_ray_tracing having added support for raytracing shaders (intersection, etc..) (i.e. integrated into Metal shading language).. seems this new support requires on Mac: *RDNA GPU or later (no Vega or earlier support) *Intel Skylake GPU or later and on IOS: A13 or later.. recalling from memory so might be wrong..
@krupitskas so you will work on spirv cross raytracing metal support? also may be a good idea to look at https://github.com/ConfettiFX/The-Forge crossplatform raytracing support they support DXR Metal and Vulkan Rt.
seeing AMD RDNA2 6xxx GPUs now supported on MacOS 11.4 betas.. would be nice if someone can test if Metal Performance Shaders Raytracing demos get some "huge" speedups on these GPUs:
https://developer.apple.com/documentation/metalperformanceshaders/metal_for_accelerating_ray_tracing https://developer.apple.com/documentation/metalperformanceshaders/animating_and_denoising_a_raytraced_scene
this may indicate it's being mapped to new RDNA2 raytracing ISA..
anyway WWDC 21 is near and we (hope)surely will get some update on raytracing on Metal on MacOS..
NVIDIA Vulkan sample https://github.com/KhronosGroup/Vulkan-Samples/pull/274 both include use of Ray Queries
Metal Performance Shaders Raytracing is also available on Apple Silicon, with the added benefit of being able to run in a single rendering pass.
Since buffer device address now available in Metal 3, VK_KHR_buffer_device_address has been included in the latest version of MoltenVK. Now you should actually be able to implement VK_KHR_ray_tracing? Is there any information in this direction?
Since buffer device address now available in Metal 3, VK_KHR_buffer_device_address has been included in the latest version of MoltenVK. Now you should actually be able to implement VK_KHR_ray_tracing? Is there any information in this direction?
I have plans to work on this over the next six months. (Full disclosure: I'm being paid to do this.)
@cdavis5e I'm working on a Vulkan ray tracing library which uses most all features of Vulkan RT (SBT record data, AABB trees, etc). I'd be very interested to build off your work if I can. :)
Since buffer device address now available in Metal 3, VK_KHR_buffer_device_address has been included in the latest version of MoltenVK. Now you should actually be able to implement VK_KHR_ray_tracing? Is there any information in this direction?
I have plans to work on this over the next six months. (Full disclosure: I'm being paid to do this.)
Seeing as it’s been nearly a year since this comment was made, it might be worth hunting for another developer to work on this.
To implement raytracing, we need the dependencyVK_KHR_acceleration_structure first, and that extension needs VK_KHR_deferred_host_operations. If we were to implement acceleration structures, we could also implement VK_KHR_ray_query, so that could be some what easier.
I want to try a hand at implementing it, but I have no idea where to start, could someone give me some pointers. Thanks
Edit: I meant VK_KHR_deferred_host_operations not VK_KHR_deferred_operations
That would be a great start. VK_KHR_ray_query would also let you test things out without thinking about shader binding table details right away. Ultimately we'd want both queries and the full pipeline for portability reasons, but it's a good idea to break it all down.
I've never actually heard of VK_KHR_deferred_operations before. I see it's mentioned as VK_KHR_deferred_operation in some Intel RT slides, but other than that it's a bit of a dead end... Is this related to VK_KHR_deferred_host_operations? I seem to get more results for that.
https://registry.khronos.org/vulkan/specs/1.3-extensions/man/html/VK_KHR_deferred_host_operations.html I've talked to Josh Barczak from that man page before on twitter, he's nice and iirc open to DMs. You might reach out to him @JoshuaBarczak
To implement raytracing, we need the dependency
VK_KHR_acceleration_structurefirst, and that extension needsVK_KHR_deferred_host_operations. If we were to implement acceleration structures, we could also implementVK_KHR_ray_query, so that could be some what easier.I want to try a hand at implementing it, but I have no idea where to start, could someone give me some pointers. Thanks
Edit: I meant
VK_KHR_deferred_host_operationsnotVK_KHR_deferred_operations
You'd need to implement KHR_buffer_device_address first, at least the non-shader side of it.
Since buffer device address now available in Metal 3, VK_KHR_buffer_device_address has been included in the latest version of MoltenVK. Now you should actually be able to implement VK_KHR_ray_tracing? Is there any information in this direction?
I have plans to work on this over the next six months. (Full disclosure: I'm being paid to do this.)
Seeing as it’s been nearly a year since this comment was made, it might be worth hunting for another developer to work on this.
I've been offering to implement since 2021, but seems there's no funding available.
@devshgraphicsprogramming
I believe VK_KHR_buffer_address has been implemented already according to MoltenVK/Layers/MVKExtensions.def on the line
MVK_EXTENSION(KHR_buffer_device_address, KHR_BUFFER_DEVICE_ADDRESS, DEVICE, 13.0, 16.0)
@AntarticCoder is VK_KHR_deferred_host_operations actually required for VK_KHR_acceleration_structure?
I noticed that if I remove VK_KHR_deferred_host_operations, that my vulkan ray tracing application still works. I'm only using the device-side acceleration structure builds though, ie vkCmdBuildAccelerationStructuresKHR rather than vkBuildAccelerationStructuresKHR. But that's probably the most common usage.
vkBuildAccelerationStructuresKHR does consume a VkDeferredOperationKHR for the host-side accel build, but perhaps so long as no host commands are actually deferred, then perhaps this extension isn't required for VK_KHR_acceleration_structure?
Edit: ah, nvm, I get validation errors if I remove that extension.
@natevm You seem to be right, but would that not be going off the specification? Or are we allowed to deviate from the specification like this?
Some applications may use deferred operations as well as that, especially if they're building on the host, so that would also break some compatabillity, I believe.
@AntarticCoder if it violates spec, then it's likely required to be supported.
The extension seems simple enough itself. Would require defining an opaque VkDeferredOperationKHR handle which I believe is allocated by a vkCreateDeferredOperationKHR. Then, if this type is passed to a deferrable command (eg vkBuildAccelerationStructuresKHR) then that command might return VK_OPERATION_DEFERRED_KHR rather than the normal VK_SUCCESS or etc.
From there, the command can be executed using vkDeferredOperationJoinKHR. There's a vkGetDeferredOperationResultKHR, a vkDestroyDeferredOperationKHR, and then finally a vkGetDeferredOperationMaxConcurrencyKHR which could iiuc could just return the number of available CPU cores perhaps? Seems to depend on the operation passed in to be deferred...
Might be worth making a separate issue on this, then indicating that this issue depends on that one.