WIP acceleration structure API proposal

[tangmi]

Initial API proposal for just the acceleration structure part of adding ray tracing support. Once the API design is set, we can implement (and smoke test) and then move on to adding ray tracing pipelines and ray query feature detection.

There's a lot of open questions I have in the comments and I intend for this draft to be a place to track discussion. @kvark: I intend to drive discussion on individual pieces at #gfx:matrix.org, referring to this PR as a reference, so don't worry about reviewing all at once (I'll make sure we get through all of it)!

The main theme of nearly all my questions is about how far can we deviate from VK_KHR_acceleration_structure to better fit gfx (and potentially gain some API safety). Additionally, my intention is to make an API that is implemented trivially in Vulkan and pretty easily in DX12. Metal is an afterthought (although it is being thought about!), but I don't expect to make any significant concessions in the API to accommodate MPS.

Here's some notes on implementation differences I expect between DX12 and Vulkan, which may be interesting to discuss how we'll solve:

• B::AccelerationStructure
  • VK: vk::AccelerationStructureKHR
  • DX: Proabaly a ref to ID3D12Buffer, since there's no handle associated with an acceleration structure
• Serialize/Deserialize
  • VK:
    • Get serialized size: ash::extensions::khr::AccelerationStructure::write_acceleration_structures_properties with ash::vk::QueryType::ACCELERATION_STRUCTURE_COMPACTED_SIZE_KHR
  • DX:
    • Get serialized size: ID3D12GraphicsCommandList4::EmitRaytracingAccelerationStructurePostbuildInfo (or BuildRaytracingAccelerationStructure) and does not use a QueryPool. We can likely modify dx12's B::QueryPool to include a special case for the acceleration structure queries.
• Host operations
  • VK: vk::PhysicalDeviceAccelerationStructureFeaturesKHR::acceleration_structure_host_commands + methods on Device
  • DX: unsupported natively, could be emulated in gfx
• Specifically laid out structs for GPU buffers:
  • Transform matrix: 3x4 row matrix that corresponds to mint::RowMatrix3x4. Metal uses a 4x4 matrix (matrix_float4x4), which I believe maps to mint::RowMatrix4, given this "Working with Matrices" doc.
  • AABB positions: Both APIs use a float3 min, then max to specify the AABB. Metal puts a bounding box on every acceleration structure and doesn't have an AABB-specific variant.
  • TLAS instance struct:
    • VK: VkAccelerationStructureInstanceKHR
    • DX: D3D12_RAYTRACING_INSTANCE_DESC
    • These two structs have the same layout (not it Metal, though) and are probably intended to be written from compute shaders, which probably users probably need to know about it (either by docs or querying for it).
• Building acceleration structures
  • Primitive counts
    • VK: Separates the primitive counts and the geometry data in VkAccelerationStructureGeometryKHR and VkAccelerationStructureBuildRangeInfoKHR at descriptor and build time, respectively. The vkGetAccelerationStructureBuildSizesKHR takes a primitive count array that we can use for the DX case
    • DX: Requires the primitive count at descriptor time when creating D3D12_RAYTRACING_GEOMETRY_TRIANGLES_DESC or D3D12_RAYTRACING_GEOMETRY_AABBS_DESC, and doesn't require any additional data at build time. We will just add the primitive count array values to the geometry info structs as we build them.

Related: #2418

[kvark]

Merge remote-tracking branch 'upstream/master' into acceleration-stru…

Please try to rebase instead of merging, to keep history linear

[tangmi]

Merge remote-tracking branch 'upstream/master' into acceleration-stru…

Please try to rebase instead of merging, to keep history linear

I'll do this when I clean up this draft, please call me out if I forget then!

[tangmi]

@kvark the changes in hal are ready for review. Please note there's no rush since the backends depend on new releases of upstream deps and the usefulness of this API depends on ray-tracing-pipelines or ray-query being also defined and implemented.

Some things on my mind:

• I'm wondering if it's worth it to commit the API first, then the implementations or hold off until we have at least the Vulkan backend (which requires a new ash release >=0.32).
  • Note that I'm also not confident in the quality of what I've hacked together here for Vulkan.
• I will turn the leftover TODO comments into separate issues (i.e. work for host operations, capture replay)
• I'll definitely clean up the history (rebase) before the API is merged.
• I'm working on the ray-tracing-pipelines part of the API in a branch: https://github.com/tangmi/gfx/tree/ray-tracing-api
  • I don't have hardware to test ray-query, but I imagine there's very little work in hal to support it.

[kvark]

Thank you for moving this forward!

First of all, we've been trying to wrap up hal-0.7 release for the last month. I'd like to do this before merging the changes here. Also note how a lot of type signatures are simplified now: Borrow, ExactSizeIterator, and IntoIterator are all gone! If you haven't already, please consider updating this PR to reflect that change with the new API.

Please note there's no rush since the backends depend on new releases of upstream deps

We are totally cool with depending on github revisions from master. We only switch to releases when necessary, i.e. during a release. So this is not a blocker to merging the PR.

[kvark]

@tangmi what's the status of this work?

[tangmi]

@tangmi what's the status of this work?

The current state is that I have an API shape I'm pretty happy with and am in the middle of fixing some issues in the Vulkan impl (I think the last thing I was working on was overthinking mapping lists-of-lists of structs such that all the lifetimes agree).

Unfortunately, life and work has taken me away from this for the last couple months. Also, I (and everyone else) have been having a real hard time getting a hold of a GPU that can test ray query--although if ray tracing pipelines work, it's very likely that ray query will also work. I'm definitely still eager to get back into gfx-rs.

On the plus side, I believe ash as since released so I think we don't need to point at some random revision anymore!

[kvark]

@tangmi it would be good if gfx-rs organization created a "hardware fund" for cases like this...

[tangmi]

@kvark I'd happily contribute to a fund like that--sadly, I just can't find any ray tracing cards for sale at all this year!

(I know there's scalpers around but I feel icky supporting that!)

[kvark]

@tangmi now with Vulkan's RT support, aren't there any AMD cards on the market? Sorry, I'm not generally shopping for these.

[kvark]

It would be great to chat on #gfx:matrix.org (or a different platform, if you have a preference?) and see if we can unblock your progress.