Vulkan-Guide icon indicating copy to clipboard operation
Vulkan-Guide copied to clipboard

Somehow, somewhere, say something about synchronization in Ray Tracing

Open Tobski opened this issue 4 years ago • 1 comments

The way we've ended up making synchronization work for Vulkan ray tracing is not entirely obvious from the spec alone, and it's probably worth a bit of the guide to spell it out. I believe how it's intended to work is as follows:

  • For trace or query calls in a shader, use VK_ACCESS_ACCELERATION_STRUCTURE_READ_BIT_KHR with the relevant shader stage(s) for the acceleration structure
  • For accesses to the shader binding table in the ray tracing pipeline, use , VK_ACCESS_SHADER_READ_BIT with VK_PIPELINE_STAGE_RAY_TRACING_SHADER_BIT_KHR
  • For builds you use VK_PIPELINE_STAGE_ACCELERATION_STRUCTURE_BUILD_BIT_KHR with access bits corresponding to which resource is being accessed:
    • Destination AS uses VK_ACCESS_ACCELERATION_STRUCTURE_WRITE_BIT_KHR
    • Source AS (e.g. for updates) use VK_ACCESS_ACCELERATION_STRUCTURE_READ_BIT_KHR
    • Scratch buffers need both VK_ACCESS_ACCELERATION_STRUCTURE_WRITE_BIT_KHR and VK_ACCESS_ACCELERATION_STRUCTURE_READ_BIT_KHR
    • Vertex/Index/Instance/Transform buffers use VK_ACCESS_SHADER_READ_BIT
  • For acceleration structure copy commands you also use VK_PIPELINE_STAGE_ACCELERATION_STRUCTURE_BUILD_BIT_KHR, again with access flags dependent on sources
    • Destination AS uses VK_ACCESS_ACCELERATION_STRUCTURE_WRITE_BIT_KHR
    • Source AS uses VK_ACCESS_ACCELERATION_STRUCTURE_READ_BIT_KHR
    • Destination buffer uses VK_ACCESS_TRANSFER_WRITE_BIT
    • Source AS uses VK_ACCESS_TRANSFER_READ_BIT
  • For indirect trace calls, the indirect buffer is VK_PIPELINE_STAGE_DRAW_INDIRECT_BIT with VK_ACCESS_INDIRECT_COMMAND_READ_BIT
  • For indirect AS builds, the indirect buffer is VK_PIPELINE_STAGE_ACCELERATION_STRUCTURE_BUILD_BIT_KHR with VK_ACCESS_INDIRECT_COMMAND_READ_BIT

Of additional note is that VK_PIPELINE_STAGE_TRANSFER_BIT does not work for acceleration structure copy commands, breaking with other copy operations.

CC @dgkoch for confirmation

Tobski avatar Jan 13 '21 17:01 Tobski

The "Resource Usage and Synchronization" section in https://www.khronos.org/blog/vulkan-ray-tracing-final-specification-release says a lot of this, but it's not necessarily the most discoverable place for it either.

I haven't reviewed the above yet.

dgkoch avatar Jan 13 '21 17:01 dgkoch

See #217 for PR to add this

dgkoch avatar Jun 01 '23 01:06 dgkoch

Closed by #217

dgkoch avatar Jun 08 '23 14:06 dgkoch