TSL Roadmap

[sunag]

Description

Roadmap for TSL (Three.js Shading Language)

Overview

This issue tracks planned improvements and changes. The goal is to enhance usability, performance, and maintainability while also laying the groundwork for future tutorials and courses.

https://github.com/mrdoob/three.js/wiki/Three.js-Shading-Language

Would love to hear thoughts and suggestions! 🚀

Tasks

• [ ] Improve error handling and validation https://github.com/mrdoob/three.js/pull/30811, https://github.com/mrdoob/three.js/pull/30783
  • Enhance syntax and value validation to provide clearer error messages.
• [x] Evaluate deprecation of append() https://github.com/mrdoob/three.js/pull/30956
  • Consider alternative approaches for chaining operations. It is still needed for some void functions.
• [x] Rename material.shadowPositionNode to material.receivedShadowPositionNode https://github.com/mrdoob/three.js/pull/30962
  • Intention for renaming, so that we have a property to define the projection of the shadow in the future.
• [x] Add uniformTexture() https://github.com/mrdoob/three.js/pull/31190
  • This is already possible with texture.value, but we consider having an alias to make its use more explicit.
• [ ] Remove the transformed prefix https://github.com/mrdoob/three.js/pull/31271
  • Remove the transformed prefix from transformedNormalView, transformedNormalWorld, and similar variables to improve readability and ease of use.
• [ ] Improve .assign() behavior
  • Automatically create a variable when possible, or provide a more user-friendly error message.
• [ ] Enhance documentation
  • Improve clarity, examples, and coverage of TSL concepts.
• [ ] Improve Transpiler
  • Add flow control and make the code less verbose
• [ ] Implement GLSL-to-WGSL transpilation
  • Enable automatic conversion of GLSL shaders to WGSL for WebGPU support.
• [ ] Add LegacyShaderMaterial
  • Introduce a compatibility layer for older shader materials.
• [ ] Migration guide
  • Creating a migration guide from *ShaderMaterial to TSL and NodeMaterial
• [ ] Replace Proxy with prototype
  • Investigate the feasibility of using prototypes instead of Proxy for better performance.

[boytchev]

In my opinion the biggest boost in spreading TSL would come if there is a documentation that is more friendly towards people who are new to TSL. Maybe the current TSL docs should be split in two - one more tutorial-oriened (with concepts, simple examples, and friendly explanations), and another more like a reference (with api, low-level details, etc). The first will attract users, the second will keep them engaged.

Writing documentation is cumbersome, and I would like to give a helping hand in writing parts of the tutorial-oriented documentation and its examples.

My biggest concern is that I barely know 10% of TSL and my time is somewhat limited. Most of these 10% come from trial-and-error. For example, today I used .debug() for the first time ... and found that Fn without layout gets inlined in the shader; while Fn with layout is compiled and called as a function.

[DefinitelyMaybe]

I agree with @boytchev's comments above. Adding a tutorialkit tutorial to my demo docs would be a good fit. To note, the current documentation issue #24984 would likely need to be resolved.

[mrdoob]

There are jsdocs ready to be published. We mostly need to add some js code to the current docs to redirect the current links and avoid breaking all the doc links out there.

[Bug-Reaper]

I have some loose ambition to ensure Attributes, Varyings, and Uniforms that are exposed via properties on THREEJS objects are also accessible in relevant TSL-World node override Fn contexts. Long-form ramble on what I mean here.

As a developer of custom-shader stuff I'd love if I could easily make new materials that behaved like vanilla-materials in that GPU-uniforms can be accessed via arbitrary top-level properties. It was something I never saw anybody do in GLSL and I think it hurt the adoption of a cool custom-material ecosystem. There's probably a real opportunity to create this generalized system in TSL.

I discussed the bundling/exporting of custom-shader-materials with @AndrewRayCode. He'd have better insight on the exact friction but from what I recall there was black-magic in material re-compilation triggers that was not-exposed and/or difficult to access

[Makio64]

Congrats on all the job on it @sunag , so many improvement in TSL since i start using it ! Big Thanks!

**Tutorials **

• Agree with @boytchev for easy tutorial, im working on it throught an interactive tsl-gallery showcasing reusable functions (noise/gradient/blend/fx/etc..) inspired by tsl-textures + basic tutorials—it's nearly ready and should nicely complement your official docs and examples you guys are working on!
• I also expect in the next years many similar projects & nodelib and tutorials coming re-enforce the TSL eco-system like The book of shaders did in glsl

Complex Shader

• When i write complex shader and i want fast iteration on it im using vite-plugin-tsl-operator and im very happy now.
• Also the transpiler is a big help on big shader from glsl but the result seems sometimes very "talkative" about the way it declare the function and i usually rewrite his code afterwards to feat my needs.

TSL TextureNode

• texture : It's a confusing node. It would make sense to merge it into unifom and using internally another node if the value is a sampler2D/3D or rename it so we understand it's an uniform instantly, like uniformTexture(...)
• it would give something like uniform( mytexture ).sample( uv() ) similar to uniform( mycolor ).mul(...)

NodeMaterial

• It would be great to have the uvNode to replace the uv() or as a shortcut to replace his context.
• I also think with the tsl ecosystem growing alternative NodeMaterial will emerge, with specialized customization like extends of RawShaderMaterial did before and I will be happy if the prebuilt nodes of threejs keep simple.

Again congrats & thanks for your fast support !

[AndrewRayCode]

@Bug-Reaper I'm less familiar with TSL overall so I might not be adding the right context. In the issue you linked - it takes a different approach than TSL. It uses a GLSL compiler to manipulate GLSL directly, making the shader node GLSL the source of truth for the inputs/outputs to the nodes. It finds uniforms/varyings/attributes by parsing the source code, and turns those into node inputs automatically. The end result relies on using onBeforeCompile (which I think is now deprecated / removed?) to fully hijack the Three.js material GLSL and basically overwrite what Three.js creates with the compiled Shaderfrog output. This allows some powerful composition techniques, for example a three.js material itself can be used as an input to another node, like this mixing a MeshPhysicalMaterial with a procedural fireball effect, which I don't think TSL currently supports.

Maybe what you're referring to is - Shaderfrog can overwrite any arbitrary part of a Three.js shader / GLSL. I think Three.js treats some uniforms as "private" - you aren't meant to set them directly - rather you are meant to go through the material properties. In Shaderfrog the whole shader source code is fair game. If TSL can analyze source with a parser/compiler then in theory it could also search for, and expose, uniforms/attributes/varyings as first class citizens to replace?

You might be generally talking about the challenges of using a DSL over a language (TSL over GLSL/WGSL). Shaderfrog has the "benefit" of users writing raw GLSL - maybe one option is to have a a raw code node in TSL so users could inject arbitrary code if the TSL abstraction doesn't support it as a first class fn?

[Bug-Reaper]

Learn from past experiences

Thanks so much for your insights @AndrewRayCode. I see TSL in an embryonic stage I felt it was apt to call upon you as a person who has:

1. Built a complex node-based material system for shaders that's compatible with three.js.
2. A longtime champion of trying to foster an ecosystem of people that customize three.js materials.

TSL does allow for some interesting gymnastics, I know I've played with an outputNode or similar that let's you re-route a material's final vec4 to additional shader logic (like another material I think ™️), and similarly you can override maps with additional shader-logic.

Question

I guess my question to you is beyond routing and varying/uniform/attribute access issues. Was there any material lifecycle black-magic or anything else that we should take-care to expose early-on in a new system? Is there anything you didn't plan for or cautionary stories you want to share from your own attempts at modular shader-logic?

Honestly I'm trying to scout for decisions that cascade into longterm headaches or poor final developer experience.

[Bug-Reaper]

Complex Shader

• When i write complex shader and i want fast iteration on it im using vite-plugin-tsl-operator and im very happy now.

• Also the transpiler is a big help on big shader from glsl but the result seems sometimes very "talkative" about the way it declare the function and i usually rewrite his code afterwards to feat my needs.

In my experience, the transpiler created some very verbose looking code but it was fine. It should be mentioned prominently in the docs (I spent a week manually transpiling 😂).

Even better if I had something that could transpile an entire CustomShaderMaterial from it's original declaration. I think we may need this for legacy compatibility.

---

TSL TextureNode

• texture : It's a confusing node. It would make sense to merge it into unifom and using internally another node if the value is a sampler2D/3D or rename it so we understand it's an uniform instantly, like uniformTexture(...)

• it would give something like uniform( mytexture ).sample( uv() ) similar to uniform( mycolor ).mul(...)

I agree the current setup is confusing and am generally more favorable to merge texture into uniform and just handle it internally. The usage would be more-clear and more-simple so win win IMO (sans any blocking technical issue?).

---

NodeMaterial

• It would be great to have the uvNode to replace the uv() or as a shortcut to replace his context.

• I also think with the tsl ecosystem growing alternative NodeMaterial will emerge, with specialized customization like extends of RawShaderMaterial did before and I will be happy if the prebuilt nodes of threejs keep simple.

Again congrats & thanks for your fast support !

Not sure I fully understand this one but I do vibe hard with desire for an ecosystem of custom materials with the power of ancestor RawShaderMaterial.

[DefinitelyMaybe]

an option for the texture vs uniformTexture would be to leave the name as is but reflect the type from the import i.e. import { texture } from "three/tsl/uniforms"

[sunag]

@boytchev @DefinitelyMaybe Thanks, the documentation and tutorials will definitely be one of the milestones.

@Makio64 Thanks, there are a lot of interesting suggestions, I added the uniformTexture() alias to the roadmap.

@Bug-Reaper We actually have more shared properties than before, like a simple normalWorld or TBN, previously you could see this rewritten in several custom shaders, attributes and uniforms are also there, you won't have problems with duplicated attributes or textures like you would in ShaderMaterial if you use attribute() or texture() methods. These new approaches are more self-intuitive for those who have used ShaderGraph, I think to answer your question it would be better if I made a migration guide, because it doesn't seem to be an issue or limitation, but documentation-wise, I just added a migration guide to the roadmap.

Any new suggestions are welcome.

[Bug-Reaper]

@sunag Glad to hear 🙂 and yea I agree the new TSL system already resolves many of the traditional issues we saw with shader-materials. To that end, massively appreciate all your good work. I have to do a bunch of migration (~50 scenes or so) can try to contribute to the migration guide you mention.

[Samsy]

Thanks for everything @sunag , a bit late to the party of this thread, but as @Makio64 said precedently,

I find myself often struggling with the UV's in the context of atlasing, a UVNode would be amazing to re-write the UV's in order to atlas a material map without over-riding the original behavior of the material

[boytchev]

A suggestion for a feature (which I could not find whether it exists or not):

TSL raycasting - i.e. raycasting against a mesh, which vertices are directly modified via TSL. A possible use-case is raycasting a mesh which material positionNode is a custom TSL function.