Support for Sparse/Bindless/NV_command_list OpenGL extensions

[FictionIO]

Hi,

We have a large application we would like to use Renderdoc to debug it. But it's using fair amount of OpenGL Extensions. We have no control in the application; what API or Context is used or not, due to it's Plug-in architecture. We can disable some of them. But, we have debug scenarios we need some components which (it appears) depends on unsupported GL API.

Because Renderdoc only supports Core Profile, and it's preventing extension function's load, once we run the application for debugging (LD_PRELOAD=.../librenderdoc.so app) it crashes.

I was wondering if there is a way to improve these situations.

• Is there a workaround for this?
• Would it be good idea to at least allow their load (thinking GetProcAddress as a pass through routine for these functions) even if their trace is not captured by Renderdoc?
• What was the motivation behind preventing extension function's load?
• How you would like to approach this in the future?

Many Thanks, Ali

[baldurk]

There's a few things to unpack here:

• Yes currently RenderDoc only supports GL 3.2 - 4.5 core profile, plus a few other extensions that are valid on modern GL and haven't been rolled into a core version.
• RenderDoc filters the list of reported extensions, to remove any that it doesn't support. This covers both old/legacy extensions as well as newer modern ones that are not supported yet, and even vendor-specific extensions.
• Currently the extension filter is just for information, it doesn't actively block anything. If the application ignores the list of extensions, then it can still use the functionality and it will go through to the GL implementation directly. Whether this goes through an unsupported function (which will print a message to renderdoc's log) or through unsupported parameters to an existing function.
• However, doing so will very likely produce a corrupted and incorrect replay, because you'll be missing important state setting or data that is necessary to replay the frame. In the worst case, it will cause crashes if you use some function in ways that renderdoc doesn't expect to be valid.

So to answer directly - there's nothing that directly prevents an extension function being loaded. It's just that a well-behaving application will check if the extension is supported first. The motivation is that these extensions are not supported by renderdoc so they will not work and will break captures - for applications with fallbacks or optional features, this means you can still use renderdoc even if the system reports some extensions I don't support.

If you wanted to you could remove the filtering code in renderdoc and let your application continue to use unsupported functionality I think it would work, but this doesn't really get you anywhere useful. While your application will run and renderdoc (probably!) won't crash, you very likely will not be able to load the replay and do anything useful with it. Being able to capture doesn't help you if the replay is completely broken.

As to how support can be improved, it depends a lot on the extension. I assume from the way you're describing it, you're not talking about something using very new extensions like bindless and sparse, more likely it's using old legacy GL like fixed function rendering.

In either case I'm not opposed to supporting it, but it's a question of prioritisation and allocation of resources. There is not much demand for either very old or very new features in GL, and so it's more important to spend time on supporting other APIs or analysis features that work on the more commonly used path. In theory once all of the other higher priority stuff is done I could work on this, however over time it will become even less important as more people move away from GL or legacy GL applications, or else update the code. And new features will arrive over time that will be higher priority.

In the end, I don't know when this will ever be important enough to work on, but in theory it could be something that a motivated third party could contribute. I wouldn't want to fully expose all of legacy GL's mess (e.g. somehow exposing the debugging of glBegin / glVertex / glEnd directly) but something that could capture legacy GL and map it to somewhat modern GL for display in the UI would be feasible.

[FictionIO]

Hi @baldurk,

Thank you so much for your quick and comprehensive response.

There's a few things to unpack here:

Yes currently RenderDoc only supports GL 3.2 - 4.5 core profile, plus a few other extensions that are valid on modern GL and haven't been rolled into a core version. RenderDoc filters the list of reported extensions, to remove any that it doesn't support. This covers both old/legacy extensions as well as newer modern ones that are not supported yet, and even vendor-specific extensions. Currently the extension filter is just for information, it doesn't actively block anything. If the application ignores the list of extensions, then it can still use the functionality and it will go through to the GL implementation directly. Whether this goes through an unsupported function (which will print a message to renderdoc's log) or through unsupported parameters to an existing function. However, doing so will very likely produce a corrupted and incorrect replay, because you'll be missing important state setting or data that is necessary to replay the frame. In the worst case, it will cause crashes if you use some function in ways that renderdoc doesn't expect to be valid.

Thanks a lot for clarifying these. For reference I also found that I overlooked https://github.com/baldurk/renderdoc/wiki/OpenGL

So to answer directly - there's nothing that directly prevents an extension function being loaded. It's just that a well-behaving application will check if the extension is supported first. The motivation is that these extensions are not supported by renderdoc so they will not work and will break captures - for applications with fallbacks or optional features, this means you can still use renderdoc even if the system reports some extensions I don't support.

The application in question is only running on a specific set of hardware and for some functionality there is no way to switch to the Core functions to support them. A good example is NVCommandList, or Bindless, SparseTexture APIs which you mentioned as well. I know supporting very specific hardware is probably slightly rare situation (although it maybe still common for some console APIs), but I still wanted ask to see if there is anything we can contribute on this end.

If you wanted to you could remove the filtering code in renderdoc and let your application continue to use unsupported functionality I think it would work, but this doesn't really get you anywhere useful. While your application will run and renderdoc (probably!) won't crash, you very likely will not be able to load the replay and do anything useful with it. Being able to capture doesn't help you if the replay is completely broken.

I would love to test this if it's possible. We are kind of stuck at the moment.

As to how support can be improved, it depends a lot on the extension. I assume from the way you're describing it, you're not talking about something using very new extensions like bindless and sparse, more likely it's using old legacy GL like fixed function rendering.

We don't have any Fixed Function calls. Even if we had I think we had an option to port them to the modern API. Sadly this is not the case it's generally modern extensions like I mentioned above.

In either case I'm not opposed to supporting it, but it's a question of prioritisation and allocation of resources. There is not much demand for either very old or very new features in GL, and so it's more important to spend time on supporting other APIs or analysis features that work on the more commonly used path. In theory once all of the other higher priority stuff is done I could work on this, however over time it will become even less important as more people move away from GL or legacy GL applications, or else update the code. And new features will arrive over time that will be higher priority.

I understand and I agree. There is a massive shift to VK. In our case we have to wait until the whole industry adopt it which will take very long time.

In the end, I don't know when this will ever be important enough to work on, but in theory it could be something that a motivated third party could contribute. I wouldn't want to fully expose all of legacy GL's mess (e.g. somehow exposing the debugging of glBegin / glVertex / glEnd directly) but something that could capture legacy GL and map it to somewhat modern GL for display in the UI would be feasible.

We don't need any legacy API support as I mentioned above. I think your view on this is 'it's possible, but not a priority'. Which is still promising for us. I would love to hear how you would approach to this problem so I can start some design and planning.

Many Thanks, Ali

[baldurk]

OK, sorry to misinterpret but mostly when people ask about unsupported OpenGL functionality it's usually legacy GL :smile:.

I would love to test this if it's possible. We are kind of stuck at the moment.

This would be easy enough to test, but I think you'll probably run into problems quickly with any of the features you mentioned. In gl_driver.cpp you'll find BuildGLExtensions - you can just add any extensions you want to there.

I think your view on this is 'it's possible, but not a priority'. Which is still promising for us. I would love to hear how you would approach to this problem so I can start some design and planning.

It depends a lot on an extension-by-extension basis. If you gave me an exact list of which extensions you need I could give you a better idea of how they would be to implement.

In general there's I guess two halfs to implementing some new API support:

1. The core capture/replay, just being able to capture the commands and replay them again to produce the same results.
2. The "analysis" or being able to meaningfully display the state and data from this API feature in the UI.

For the first, then as long as it's working, the code is clean, and it doesn't cause too much spaghetti elsewhere in the GL driver codebase then I'm open to PRs. This will be somewhat case-by-case but if generally the implementation is isolated to the implementation of each function it's probably fine. Depending on the function this might be easy or hard :smile:.

For the second it's more complex. Bear in mind RenderDoc is a debugger that works on multiple platforms, APIs and IHVs. A feature that is going to be used very rarely or only by a small amount of people must be weighed against any maintenance burden and UI complexity it adds. E.g. sparse textures are probably OK because they are supported by other APIs and generally are fairly invisible (although perhaps in future we'd want some sparseness visualiser, this is not required).

Specifically about the extensions you mentioned:

• ARB_sparse_texture: From memory last time I looked I thought it's probably doable but it's quite a complex extension since it affects a lot of different things. The main problem is the same as on vulkan/D3D12 where it's a core feature - because there's no way to readback the page table, you must track it 100% so it can be restored again.
• ARB_bindless_texture: I've looked at this before and it could be difficult. One problem is that while they added a 64-bit glVertexAttribL1ui64ARB and corresponding get function glGetVertexAttribLui64vARB, when they added glUniformHandleui64ARB there's no corresponding glGetUniformHandle function. There's a glGetUniformui64vARB in a different extension which could maybe be used but I don't know if it will always work. Likewise there's no program introspection query to tell if a shader uniform is bindless or not (the layout(bindless_sampler) part) Also since you'll need to remap/rewrite handle values between capture and replay as they will very likely change, this is possible for straight glUniformHandle calls but not feasible for any values that come through vertex attribs or even through plain uint64 values in plain memory in UBOs. So it would only work in certain cases.
• NV_command_list: I've not looked at this and I doubt it would be feasible, this extension hugely changes the API and adds whole new ways to perform draw commands. It doesn't map cleanly to any existing API concept and being a vendor extension that's very rarely used it's even less worthwhile than the others.

[Illation]

Hey, for the sake of visibility, I wanted to point out that it would also be really useful to me to get support for GL_ARB_bindless_texture, as my program now depends on it.

I might eventually try to write a workaround for this issue with texture arrays if the extension isn't supported, but for now I won't be able to use RenderDoc properly unfortunately.

[noizex]

Same here, RenderDoc crashes somewhere at the point where my sparse texture arrays are initialized. I'm using bindless & sparse textures in OpenGL and only API Trace works fine with these, most of other tools crash randomly. It would be enough for me to not have it crashing - it may not be able to view these resources. I suppose this is a problem to find out how to treat these because there is no information about the bindless/sparse nature of such texture and if RenderDoc tries to handle it like normal texture?

[FelixBrakel]

Here as well. When developing using and AMD card on Linux RenderDoc is really your only option for debugging frames as all their own debugging software has been discontinued (I even came across an AMD representative on their forums saying to just use RenderDoc instead of their own, old, software). GL_ARB_bindless_texture is something my program depends on as well so support for it would be really nice.

What is the current plan when it comes to adding more support for OpenGL extensions? Is the focus on Vulkan/DX12 right now or is this part of the software still being developed?

[TheCrustedOne]

Same here, I recently introduced bindless textures in my application and just discovered that it is not supported by Renderdoc. I'm currently developing on an AMD card as well and even on Windows you are basically the last hope of us that develop OpenGL applications on AMD as they dropped all support for their own tools (for OpenGL). So even though I understand that such features are not top priority for Renderdoc at the moment, it would be very nice if it could someday support newer extensions such as bindless textures.

Edit: I also implemented a debugging workaround which uses traditional OpenGL texture binding. In all my fragment shaders I set

#extension ARB_bindless_texture : enable layout(bindless_sampler) uniform;

which allows me to easily change between traditional texture binding and bindless textures without changing shader code at all. However, even if I use traditional texture binding in the application and do not call a single function introduced with GL_ARB_bindless_texture I cannot view the textures in Renderdoc as long as above layout is set in the shaders. It would be cool if Renderdoc could somehow "ignore" above layout regarding bindless texturing because it is also compatible with traditional texture binding and thus Renderdoc should be able to handle it right?

[tksuoran]

+1 for desired at least minimal support for GL_ARB_bindless_texture. Some idea:

• Keeping track and showing list of resident texture handles should be doable
• I would not mind making calls to some extension function from my application to provide extra information if that would be helpful