Feature Request: Ability to supply GLSLang the list of available extensions

[reduz]

Based on the question in #2594, and given in Godot we compile shaders in run-time instead of shipping them with the game (as it is a general purpose game engine), it would be immensely useful for us to be able to pass GLSLang (at least via API) the list of extensions that it should enable.

While I understand this can be made with our own API as macros, it completely beats the purpose of GLSL providing this mechanism and making it available in Vulkan. As far as I understand, GLSLang just assumes all extensions are enabled based on the base SPIR-V version they support, so we would like to make this information be faithful to what the hardware actually supports at the time of the shader being compiled.

I suppose this would be done by just passing GLSLang an optional list of const char* strings with the available extensions.

[greg-lunarg]

I am interested in making glslang more friendly to run-time compilation. However, my current level of funding practically does not enable me to implement something like this. So implementation would have to come from somewhere else. But I am willing to consult and review design and implementation. In fact, I reserve the right to deny a PR whose design is not "good". Therefore I highly recommend that any design be completely specified and reviewed by me before implementation starts.

That said, I think something like this could work and be beneficial, but I would like to see more specifics. Particularly:

• specification of the interface to the glslang API
• changes to GLSL to interact with this new feature

It would also be helpful to see examples of the API call as well as of shaders using this new feature.

[greg-lunarg]

Thinking about this for another minute, I just realized that changes to GLSL would have to go through the GLSL WG. Please consult with @gnl21 on how that could be accomplished.

I am going to close this issue until changes have been made to the GLSL spec to enable implementation to move forward, or it can be demonstrated that this proposal does not require GLSL changes. At that time please reopen or open another issue.

FWIW, if you don't want to go through the work and time of putting changes into the GLSL spec, as well as the glslang implementation work involved, I would strongly encourage you to reconsider my recommendation in #2594. This recommendation would not require changes to glslang or GLSL, and you could therefore start using it immediately.

[reduz]

Hi Greg, I don't understand why you would need changes to GLSL in order to achieve this. GLSL already provides everything necessary. As an example:

#if defined(GL_KHR_shader_subgroup_ballot)
#extension GL_KHR_shader_subgroup_ballot : enable

The problem lies entirely on the glslang side, because you can't tell glslang that this extension is not available. As an example SPIR-V supports this, but subgroups may not be available on the platform, or only for some shader stages in some hardware (as an example, on mobile subgroups only exist for compute, not fragment or vertex).

Because of this, glslang is forcing me to send yet another macro to tell that the extension is not available, when GLSL supports doing this just fine. With OpenGL this works as expected and as the spec intends it, but in Vulkan it does not, since you can't effectively use GLSL itself to indicate support for a feature, you have to do it manually.

[gnl21]

This seems like a generalisation of the device profiles that glslang already supports. I believe that you can currently supply a file with device limits to use during compilation. If you could supply supported extensions in there then it seems like it would solve the problem here?

I don't know if the limits can be specified from the glslang API, but I guess it should be easy enough to wire up if the core support is already there.

[greg-lunarg]

Re-opening for continuing conversation.

[reduz]

If you could supply supported extensions in there then it seems like it would solve the problem here?

Yeah, I suppose internally GLSlang enables extensions based on the core profile API, my suggestion is mostly just being to optionally pass the list of extensions I want enabled because the case where something is core but support does not exist (as I mentioned, Subgroups are a good example of this, they are SPIR-V 1.3, but not all hardware supports it, or not in all shader stages), I need to be able to tell the shader to not use them, or use something else, or simply pass which extensions I need enabled.

[greg-lunarg]

@reduz Please specify which glslang API function you wish to be changed.

Also please give an example call to this new function showing the exact new value that is passed.

Also please give an example of a full shader and how you wish the compiler to behave for that shader.

[reduz]

@greg-lunarg alright, I am not so familiar with the internals of GLSLang, but I was envisioning something along the lines of:

On the GLSlang side, I would add a function like:

TShader::restrictAvailableExtensions(const char* const* s, int n); // Only the supplied extensions will be supported

Which you would use like:

const char * available_extensions[]=
{
"GL_EXT_ray_tracing",
"GL_KHR_shader_subgroup_ballot",
};`

shader.restrictAvailableExtensions(available_extensions,2);

Then the shader would work like a normal shader, something along the lines of:

#version 450

#if defined(GL_KHR_shader_subgroup_ballot)
#extension GL_KHR_shader_subgroup_ballot : enable
#endif

#if defined(GL_EXT_ray_tracing)
#extension GL_EXT_ray_tracing : enable
#endif

void main() 
{


#if defined(GL_KHR_shader_subgroup_ballot)
//use version of code that optimizes when this is available
#else
//use slow version
#endif


#if defined(GL_EXT_ray_tracing)
//use raytraced reflections
#else
//use a reflection probe
#endif


}

And again, while I understand this can be manually done with my own macros, I feel its just a very dirty workaround to a feature GLSL already has, and which makes the code a lot cleaner.

[greg-lunarg]

Thank you. This is very helpful. I think I am seeing your problem now.

It appears that all available extension names based on the #version are "pre-defined", that is, the pre-processor defined() function is true for all available extensions at the top of the program whether they are explicitly enabled or otherwise. This is the part I didn't realize and was a little surprised by although I am starting to understand why this is true.

So the request is to have an option to only make available to glslang a specified set of extensions.

I think this is a reasonable request and addition to the compilation options, especially for run-time compilation. I think your proposed new interface is also reasonable. I agree that no change to GLSL is needed.

I will leave this as an open issue and invite someone to implement.

[greg-lunarg]

If someone starts to implement, please let me know so I can assign it to you and we can avoid duplicate effort.