Hardware raytracing acceleration

[mcclure]

This is probably a bad idea.

This is a feature branch in which I'm attempting to add RTX support to the Vulkan branch. Current status is it can check if raytracing is supported, enable it if so, and report back to Lua what it did.

This feature is problematic for reasons including RTX being questionably practical at VR framerates, unlikelihood of support on Android and raytracing being overall simply complex. But, I want to experiment with it. Long term, if we choose to add this feature it should probably have a shader-based fallback implementation for systems that lack RTX.

[shakesoda]

re questionably practical at vr framerates: i have to contest this on a couple points:

1. this is unlikely to be true for too much longer (i.e. a hardware generation or two if i'm being a bit more conservative)
2. vr eye buffer resolutions are only a subset of relevant use cases.
3. vr framerates at the bottom end of the scale are only around 45fps if we're accounting for tech like ASW - and i believe that's not outlandish on current gen hw

i agree that there should ideally be a fallback path, though, i'm going to have useful systems without RT hardware for quite some years to come where it'd still be very useful for experimentation.

[bjornbytes]

dev branch is probably in a mature-enough state for this feature now

[mcclure]

This is now rebased on the dev branch and "works" (can successfully report raytrace capability when run with https://github.com/mcclure/lovr-gpu-sample test "1-info").

[bjornbytes]

Poke, wondering if this is still active and should be kept open? Still excited about it and would be interested in contributing design/code at some point.

[mcclure]

Poke, wondering if this is still active and should be kept open? Still excited about it and would be interested in contributing design/code at some point.

I pushed my most recent efforts on the RT experiments branch (from I think… november or something?) And the sample programs with my experimental API are at https://github.com/mcclure/lovr-gpu-sample/tree/rt … for these to work I had to run them with

function lovr.conf(t)
        t.graphics.debug = true
        t.headset.drivers = { 'desktop' }
end

But I think if this were rebased on current master/gpu branch that would not be the case.

Current status is you can it can get feature and size information on raytracing from Vulkan, and it makes a call to create an acceleration structure. This fails because like… there's a lot of structures where the data is supplied by a data structure that is a union containing either a GPU memory pointer or a CPU memory pointer. I assumed this meant you could just pick either. No. It wants one or the other depending on mode (IE are you doing acceleration structure building sync on gpu or async in cpu, with the latter only being supported on some configurations) but either the docs were unclear about which to use when or I misread them. I built them in memory and I should have built them on GPU.

I think i came in here and asked about GPU "temp memory" and it turns out lovr-gpu is already well equipped for this so maybe it would have been easier to do the other way to start with.

Of course building an acceleration structure only gets you about 25% of the way there, plus my API is highly scaffoldy (I have Lua code building the in memory structures with luajit ffi and then the lovr api just wraps it thinly… I was thinking once I had example code it would be easy to move API into the C layer)

I have stalled out on this more or less completely, I have too many other things competing for my attention to work on this anytime soon, so if you want to try to do the following parts in my place, or just take the PR over completely, or toss this PR and start over with your own, that would make sense

My planned next steps:

The main problem is the lack of readily usable raytracing example code. The best "official" example code I found was all C++ and was that awful thing where instead of just showing you how to do it they provide a C++ wrapper class that does it for you, so it's hard to follow. I found a set of examples that I found easy to modify and was planning on using as the basis for the next pass
https://github.com/adrien-ben/vulkan-examples-rs
it is in Rust, but it is using Ash which is a Rust library whose point is that is is more or less 1:1 with the C API, so it might actually be easier to translate this sample code Rust->C than to decipher NVidia's overly-abstracted C++

Let me know if you want my file of vulkan rt doc/tutorial resources i was using to write this.