Shader System Rewrite Epic

[xen2]

Our shader system current implementation is slow, complex and difficult to maintain.

Terminology:

• XKSL: Xenko Shader Language

There was an underway effort to switch to something based on https://github.com/KhronosGroup/glslang and https://github.com/KhronosGroup/SPIRV-Cross :

• We only have to work on the part that interest us (XKSL) rather than the full shader stack (HLSL to GLSL, etc...)
• Maintenance is greatly reduced
• Much more things supported out of the box (CS, GS, new shader models, optimizers, etc...)

The design idea is:

• Customize glslang HLSL frontend to be able to parse XKSL to SPIRV (with some XKSL extensions)
• Rewrite a new XKSL "mixer" that will work at SPIRV level (rather than source AST level, making things much simpler)
• Leverage on SPIRV-Cross to output to any format we want: HLSL, GLSL, Metal, SPIR-V, etc...

So far we have a prototype for HLSL generation but there's still quite some work to do on it, and then need to add support for GLSL and others.

[tzachshabtay]

Have you considered ShaderGen as a possible alternative? https://github.com/mellinoe/ShaderGen

[tebjan]

@tzachshabtay interesting tool, that could help with code completion when writing shaders... if you add/use the right fake shader libraries to the c# file, the visual studio c# editor would provide all the comfort of code completion and so on... see also #157

[tebjan]

also this looks very interesting, maybe it can help, even has a gpu debugger: http://www.ilgpu.net/

[indigozero]

I just want to be clear on the design for this Epic.

• Take the code from https://github.com/KhronosGroup/glslang/tree/master/hlsl (or start from scratch) and extend a copy of it to create support for XKSL
• Use result to be able to use glslang to convert XKSL to SPIRV.
• Use resulting SPIRV with SPIRV-Cross to convert to any shader language depending on desired shader architecture.
• Implement compilation workflow in to existing tooling/compilation workflow of Xenko.
• Include support for other shader types to use conversion part of the workflow
• Include support for other shaders in existing tooling

The last two list items I'm not sure are part of the expectation of this Epic.

[xen2]

There is already such a fork at https://github.com/xenko3d/glslang Last time it was tested, D3D was working.

What needs to be done:

• Add proper unit testing on our glslang (there was manual tests but this needs to be reorganized)
• Bring up OpenGL (last time I tested there was a bunch of small issues, esp. Semantic Name to properly transmit)
• Bring up Vulkan
• Review Xenko integration

[berwyn]

Just out of curiosity, how open would the team be to supporting something like ShaderGen or ILGPU? Personally, as a newbie to gamedev but an old hat at C#, it really appeals to me being able to apply my existing knowledge and tooling to the problem of shader programming rather than having to learn a new syntax (and, to be fair to XKSL, this would apply equally to HLSL, GLSL, and MSL) and potentially having to deal with no tooling or at least tooling that may not be up to par with e.g. OmniSharp, Visual Studio, StyleCop, and even just the built-in things Roslyn itself has like quick fixes.

I think XKSL and SPIRV-Cross is definitely a good approach to re-using existing shader code people might have and to welcome folks that do have experience with shading languages, but a C# solution might be more friendly to people newer to the field, and just generally to new programmers at large (though honestly a visual shader setup like Unity and Unreal have is probably better for people entirely new to programming, but that's a whole different can of worms).

Collaborating on these solutions with Eric or Marcel could also be a way to share the burden of maintenance on that front as well, since they both already have vested interest in their respective libraries being capable and useful. ShaderGen, at least, already has very broad output support covering D3D11, the OpenGL families, Vulkan, and Metal (namely, all the things Eric's Veldrid rendering library supports).

Sorry that this turned into a bit of a long-winded thing, but it's a feature I'm super keen on.

[xen2]

@berwyn Interesting subject. I worked on such a C#/MSIL=>shader system many years ago too. I would be totally fine to have it as an additional approach to write shader code.

[Aminator]

@xen2 I am actually currently working on a MSIL to HLSL compiler in my little DirectX 12 engine where you can set shaders as material attributes at runtime. It is still in its early stages but it works quite well and even works with the new DirectX Shader Compiler which would allow cross-compiling to other shader languages. I tried to keep it very similar to how materials and shaders work in Xenko so i can maybe port it over later.

[xen2]

@Aminator9000 Great, please keep us posted! Can't wait to see what comes out of it.

BTW, if still discussing about MSIL=>HLSL, please make a new github issue on next answer.

[ykafia]

I guess it's better to write my questions here So what i understood of what we have is

• For the best case scenario : SDSL -> HLSL -> DXIL
• For Vulkan : SDSL -> HLSL -> Translation -> GLSL/MSL -> SPV

And the idea of the rework is cut down the translation and go through SPV-Cross ? Which would look like : SDSL -> SPV -> GLSL/MSL -> SPV

Does it also mean we would have to maintain a fork of the validator with the SDSL front end ?

If yes to all of this, would it still be decent to keep the SDSL -> HLSL translation at first and go spirv-cross then ?

[xen2]

The rework would look either like (depending on what stack we choose: (1) SDSL -> SPV(various transformations) -> HLSL/SPV/DXIL/GLSL/Metal (2) SDSL -> DXIL(various transformations) -> HLSL/SPV/DXIL/GLSL/Metal So far it was done on a fork of glslang: https://github.com/stride3d/xkslang But not sure if we could really restart from there

If yes to all of this, would it still be decent to keep the SDSL -> HLSL translation at first and go spirv-cross then ?

In the short term maybe, but I don't think it's worth the effort because it would still be a lot of work. Long answer: Currently, all the processing is done at the SDSL/HLSL source code level using AST (abstract syntax trees) but it takes a lot of time and complexity (cloning code, type analysis, transforming, merging, etc.). This is causing a huge amount of allocation, defensive copies, complex code analysis, etc. The whole idea is to do the actual transformations in bytecode space. It makes everything much simpler: type analysis is much easier, easy to clone code for change, mass replace, etc.

[ykafia]

In that case we could directly use Naga

We could create a front-end for sdsl and Naga can spits out some HLSL/MSL/GLSL

Once I get more time I'll take a look at it and try something, if no one has an issue working with rust. I was meaning to learn making a front end in Rust for sometime and now is the perfect occasion :D

Edit : forgot about the "mixer" thingy, I should think about it more

[Geenz]

SPV -> [Variants] is likely your best approach here from a cross platform perspective long-term. Though DXIL does pose an interesting option, but nowadays SPV has become so ubiquitous that it's likely better as a "cross platform" option especially once you factor in other non-PC platforms.

[ykafia]

Found that using spirv linkage would help us simplify the shader mixing process, there's also a spirv tool for this in the works.

The core idea is that we could use global variables and methods from another spirv module in a shader. And since some of our shaders with static methods could be candidate for being linked, we might not need to go through complex shader mixing processes, we'd just compile it once and link it anywhere. There's also the idea of using external HLSL/GLSL libraries that we could compile AOT.

[xen2]

Interesting. One large blocker I can think of is that our shader "generics" behaves more or less like C++ templates so we usually can't compile them without the type parameters specified.

[ykafia]

Yes of course, i imagined this would serve as a first idea for code reuse in a new implementation of the mixer

Currently, all the processing is done at the SDSL/HLSL source code level using AST (abstract syntax trees) but it takes a lot of time and complexity (cloning code, type analysis, transforming, merging, etc.). This is causing a huge amount of allocation, defensive copies, complex code analysis, etc. The whole idea is to do the actual transformations in bytecode space.

I'm getting a little bit more comfortable with the code base to understand what you meant in this comment. I imagine the idea is to have each Shader program generate bits of spirv bytecode and having the Shader Mixer merge all those spirv bits ?

The biggest headache would be to figure out how to track spirv identifiers since every types, methods and definitions has its own id for each spirv module. An idea i had was to create a global partitions of IDs, Stride's IDs would be set before hand (no management needed), user ones would be generated on the fly and managed (Dropped when unused for a long time etc) depending the needs ( i'm not sure any projects would reach 4 billions variable/methods/types but that's a possibility).

After that i think it would be a breeze to mix spirv code together, the language is very simple and straightforward