glslang
glslang copied to clipboard
KhronosGroup
Define input-file format for holding multiple shaders in a single file.
It's not clear how one should name files.
For example, you outline that files would normally be named e.g. .frag for fragment shader. But the syntax of the file is glsl, so does .glsl also suit? You mention that the glslValidator uses the file extension to choose the appropriate internal logic, but it's not clear whether this can be overridden or this is establishing a standard.
In addition, it's not clear how the output files should be named. For example, we've been using $shader.frag.spv but this isn't ideal in cases where you are trying to determine the file type based purely on the file extension. It would be good to have a discussion here and then put forward a standard naming convention for shader files, in particular both glsl related files, and the spv counterparts.
The reason for this request is two fold:
1/ Set a baseline for engine and 3D software developers so that people can work consistently (e.g. resource loading systems). 2/ Make it clear what extensions should be recognised by IDEs, tools, e.g. for syntax highlighting, file icons, etc. e.g. https://github.com/hughsk/language-glsl/blob/5edd8bd17af5913f977bd7efe8c815786e6a4adf/grammars/glsl.cson#L1-L18
Thanks for your time.
I guess I'll add one more point. The file extension usually indicates the contents of the file, e.g. it's format, not the function. Ideally, I think it would make the most sense to use .glsl and compile that to .spv (e.g. how .c compiles to .o). But, that's counter the existing recommendations so I guess having a discussion here would be good.
Ideally, one could put multiple functions into the .glsl file, e.g. both the vertex and fragment functions, etc.
GLSL internally has no declaration of what stage a shader is meant for, so the information is external. Hence, a .glsl suffix doesn't help much for a single shader staying with GLSL syntax.
There is also no syntax for bundling multiple shaders into a single file. This could be layered on top of GLSL, in a tool like glslang or a wrapper around it, as GLSL proper has no concept of a file either.
It does establish a convention though for stage-specific file-name suffixes, which would be fine to follow, though there is standard to force that.
What does that leave? Just a command-line option request to explicitly set the stage and ignore the file suffix?
What does that leave? Just a command-line option request to explicitly set the stage and ignore the file suffix?
I see both pros and cons in this.
I'm also suggesting the the output file name be "suggested" or "recommended", e.g. foo.frag should canonically map to foo.frag.spv or something along those lines.
Note that a SPIR-V binary can have lots of stages and shaders in it. With a bit more functionality, glslang, or some other tool can turn input GLSL of shader.vert shader.frag into a single SPIR-V file, where the .vert and .frag know their stage only externally, but the .spv internally documents what stage each entry point is for.
@johnkslang Yes, I think that's a great thing and I'd like to see that working. I even tried to do this by specifying a single output path and multiple inputs but it didn't work correctly.
Even if the current behaviour is documented, to explain ideally how one uses the tool, and then perhaps working on the single file output with .spv extension going forward. Does that mean you'd have different functions in the .spv file corresponding to the main function for each stage?
Looking back at the original questions, and what you've said, would it make sense to define a new format, e.g. ending in .glsl, which has discrete functions, e.g. void vertex(...) and void fragment(...) that can be compiled to a single .spv file?
What does that leave? Just a command-line option request to explicitly set the stage and ignore the file suffix?
That would be useful. I use glslang as a syntax checker with emacs and flycheck. There is a lot of existing code with .glsl that it barfs on. Being able to specify allows some flexibility to infer or make it user definable from emacs.
I think this is getting addressed by PR #450, maybe the only question is how to handle lots of files simultaneously. Glslang can take lots of files across multiple stages and link them all together, so we need a way of specifying this per input file.
Perhaps it would make more sense to have a separate compiler and linker command - or to logically separate out these functions within the glgslang command, e.g. glslang -c (compile mode) vs glslang (link mode)
What I would recommend is creating an input file. Actually, it's essentially critical you have an input file because on windows at least there is a maximum number of arguments you can pass via the command line before the buffer fills up. We use cmake to generate an input file from proj files.
It's pretty awkward to have different file extensions for different shader stages. It's even more awkward for Vulkan seeing that a pipeline is a collection of shader stages, which seems logical to often want to put in one file. And, for HLSL, the .hlsl extension is actually natively understood and parsed in Visual Studio 2015 so you get alot of the code coloring and I think even some code completion stuff now.
However, generally speaking if you get really serious about compiling lots of shaders you'll wrap your own compiler exe which is what we do. We only use FXC (the standalone HLSL compiler) on spot checks for specific files.
@dankbaker What is an input file? Do you mean a file which lists other paths? Also, just on your point that having a lot of arguments is bad because there is a limit in a specific OS - while I agree that it might be pragmatic, this is a leaky abstraction.
It's pretty awkward to have different file extensions for different shader stages. And, for HLSL, the .hlsl extension is actually natively understood and parsed in Visual Studio 2015 so you get alot of the code coloring and I think even some code completion stuff now.
I had a couple of situations where this was a problem - *.glsl files have syntax highlighting in Atom, but the specific shader extensions don't by default. Agree with you here.
It's even more awkward for Vulkan seeing that a pipeline is a collection of shader stages, which seems logical to often want to put in one file.
In some cases, this might be true, but in other cases we do load and assemble a pipeline using the same vertex shader but different pixel shaders, etc.
However, generally speaking if you get really serious about compiling lots of shaders you'll wrap your own compiler exe which is what we do. We only use FXC (the standalone HLSL compiler) on spot checks for specific files.
That's a interesting approach. Do you publish this code open source somewhere?
That's a interesting approach. Do you publish this code open source somewhere?
No, It's part of the Nitrous Engine. We have our own meta language that sits on top of HLSL and manages more of the logistics of actually creating a material or shader and binding it to the engine somewhere, we call it OXSL for Oxide Shading language, but all the shader parts of it are just pasted into the target platforms compiler. We don't attempt to even parse them. Eventually, we might get to the point where we parse the syntax directly and that would be more elegant. GLSLang might be a good starting point for that customization.
Most game engines have similar systems, That is, complex material systems which layer on top of the shader language. It's easiest just to link in the compiler directly to compile the shaders. I find it pretty unlikely that someone would use the standalone executable of the standalone app in a major project.
That's an interesting approach. I appreciate the unix model where we have lots of small components that do one thing and one thing well, rather than a large vertical.
I did also in the past use a simple syntax which would parse out separate files, something like this:
@shader version=blah
vec3 shared_between_stages;
@vertex
void main() {
... vertex shader code
}
@fragment
void main() {
... fragment shader code
}
It wasn't a bad approach but I found we ended up duplicating a lot of code. Obviously we could do things like have custom directives (@ prefixed lines). This sounds similar to what you did but perhaps with more high level directives.
Glslang now has command line options to manage file endings. The main thing I see here is the meta-file format, which does not exist. Change the title for that.
If you don't plan on working on this yourself, we should close the issue as there are no plans to work on this in the foreseeable future.
We're willing to accept PRs that add new features as long as someone is actually using those features and they don't cause issues for the core functionality of glslang. If you are actually going to be using this, I think this would not be an unreasonable addition to glslang.