renderdoc icon indicating copy to clipboard operation
renderdoc copied to clipboard

DX11 DrawIndexedInstancedIndirect only allows debugging of final fragment touching a pixel

Open chrisbudd85 opened this issue 7 years ago • 7 comments

When attempting to debug a pixel shader no mater which fragment is selected in the pixel history it always starts debugging for the final fragment which writes to the render target in that pass.

I understand this might be a restriction rather than a bug but it would be really useful to be able to step through a selected fragment

chrisbudd85 avatar Oct 03 '17 10:10 chrisbudd85

Can you give more information about what the draw is doing? Are you able to show a capture that exhibits this?

The selection of a specific fragment from the pixel history is done by using the primitive ID, on the assumption that a single primitive only generates one fragment - that assumption though doesn't apply through if you're doing geometry shader or tessellation expansion, since then SV_PrimitiveID could be re-used for lots of fragments.

This actually came up recently with someone else who was doing geometry shader expansion to target multiple slices of a 3D texture (using SV_RenderTargetArrayIndex). The annoying thing is that the pixel data is there to be able to debug, but the code isn't currently set up to allow selection of a given fragment except by primitive ID and sample ID - which isn't always unique enough.

I wonder if your problem is similar - the selection isn't unambiguous enough - or if something else is going on. The former is solveable although unless I can find some other way to disambiguate it becomes a UI problem of how to let a human make a choice in between fetching all the fragments and debugging one of them. The latter is probably a bug but I'll need more info to investigate :smile:.

baldurk avatar Oct 03 '17 16:10 baldurk

I am currently a rendering programmer, due to NDAs etc I don't think I can share a capture.

In this situation we are instancing a single quad consisting of two primitives many times for rendering VFX. So I am guessing due to instancing the same problem arises. We have only 2 primitives but potentially many instances of each primitive touching the same fragment?

chrisbudd85 avatar Oct 04 '17 14:10 chrisbudd85

No worries, I'm used to working with people under NDA :smile:. I just thought it was worth asking in case you could.

I think you're right, that it's the instances that are aliasing and showing up as multiple fragments all with the same primitive ID. I think the problem is the same as geometry expansion then - there's not going to be any way to disambiguate between them automatically since I can't read SV_InstanceID in the pixel shader to know which fragment is the one you selected.

In that case the best solution I think is to display all the fragments to the user and ask which one should be debugged. It's clumsy, but without some unique identifier the only other realistic alternative would be to fetch all the necessary debugging initial conditions during the pixel history so that it's ready to debug - and that would I think be too heavy to be worth doing every time speculatively.

I could at least get it to only appear when there is actually some ambiguity, and it would help in the case of geometry expansion I mentioned before.

Maybe I could do something really wild like running N draws, with 0..N instances each. Each time hash the retrieved pixel inputs and take note of which new fragments appear at which instance ID - to kind of reverse engineer which instance a particular fragment came from. That sounds kind of brittle and quite likely to cause false positives if two fragments end up having the same inputs by coincidence.

baldurk avatar Oct 04 '17 16:10 baldurk

From what I've seen, is the issue mostly one of UI design? Right now the pixel history shows the results of each pixel shader for a given primitive, e.g. DrawInstanced(3, 3, ...) shows three fragments touching the pixel and allows you to click on each of those fragments. You might be able to reveal which instance you're choosing in this case by selecting the fragment you want to debug. Would this approach hit all the cases you need to cover? Or is there another case where this wouldn't allow you to fully disambiguate which instance of a shader you're trying to debug?

IAmVoran avatar Jul 16 '23 23:07 IAmVoran

No, it's not a UI design issue - you've glossed over the real problem which is that pixels are only distinguished for debugging via primitive not instance, because the primitive ID can be read by the pixel shader (albeit with some limitations) but the instance ID can't. This is true both in the pixel history and in the shader debugger launched from it.

If it's a different primitive touching the pixel in each instance then primitive ID can act as a substitute/proxy for instance ID, but this most commonly happens with e.g. particle systems or other cases where you have only a small number of primitives and are likely to see the same one write in multiple instances.

baldurk avatar Jul 16 '23 23:07 baldurk

Ah, I see. Even though the pixel history does show each instance separately, it does show them each with the same primitive ID, and there's no access to the instance ID. I suppose that's a bit deeper of a problem to solve then. Do you know what's needed to derive the instance ID from what's available?

IAmVoran avatar Jul 16 '23 23:07 IAmVoran

The pixel history shows each overwriting primitive separately and primitives in different instances are by definition different primitives, but that is effectively a coincidence and not a deliberate separation of instances. I discussed the problem in a comment above, there's no direct solution so it's more a question of which mitigation/workaround is best.

The feature request is clear and doesn't need any other clarification, it will hopefully be addressed in future but I don't have any estimated time for when that will happen.

baldurk avatar Jul 16 '23 23:07 baldurk