Make shaders define what they get used for.

[Garima3110]

This is a work in progress addressing the issue #https://github.com/processing/p5.js/issues/6759

PR Checklist

• [x] npm run lint passes
• [x] Inline documentation is included / updated
• [x] Unit tests are included / updated

[perminder-17]

Great job so far, Garima! The idea is that when you call shader(yourShader), it should only apply to fills, regardless of other settings, like the uStrokeShader uniform or whether lights() is used in the sketch.

Currently, methods like isLightShader()

https://github.com/processing/p5.js/blob/349b7d482d4fd2270fc19e7737a86292f9c48d59/src/webgl/p5.Shader.js#L983

check which uniforms are defined, and if any are defined, the function returns true. This allows us to apply fill shaders when lights() is used in sketches.

One possible solution would be to remove these methods that check whether uniforms and attributes are defined. Instead, we can directly set the appropriate user shader based on the method the user calls, such as strokeShader(shader) for stroke shaders.

Also, we should modify the function _getRetainedFillShader() written here:

https://github.com/processing/p5.js/blob/349b7d482d4fd2270fc19e7737a86292f9c48d59/src/webgl/p5.RendererGL.js#L1768

We could probably return the user-defined fill shader if present, or light shader if no fill is set. I guess this should work for fill shaders since _getLightShader() is returned when lighting is enabled or a texture is present. That's it for fill shaders, I can leave more reviews for strokeShaders and imageShaders as we proceed further but it's pretty similar for those methods also. Thankyou so much for working on it.

[Garima3110]

Thanks for the suggestions @perminder-17 ! I have made the suggested changes, and the fill shaders seem to be fixed. Please have a look , if everything seems fine I'll work further with the docs and other shaders too. Thankyou!

[perminder-17]

Great work @Garima3110 so far. I tested it, and it seems to be working well. But to be double sure, I'll test it again to make sure nothing is breaking. In the meantime, you should focus on completing the documentation for the fill shader to meet the first week's target.

The documentation should include: (these are just my thoughts, feel free to add your own explanations if you have any).

1. A description stating that the fill shader applies only to fills. (explain it in your way)
2. Reference links to other shaders (strokeShader, imageShader) with a brief explanation, just one line for each of them.
3. 2-3 example sketches for the tutorial, showing how when lights() are enabled, it returns this._getLightShader() (which is a light shader). When lights() are not enabled, it uses the fill shader and returns the fill. Don't go too deep into p5.js's internal system, but give users a basic idea of what’s happening at the backend as well.

Also, talking about the strokeShaders here's the method created

https://github.com/processing/p5.js/blob/12e6276d403eb36feb452acf95aa5cf01b4c7998/src/webgl/p5.Shader.js#L1020

You can remove the method and update the function here as well: (just you did for fill shaders)

https://github.com/processing/p5.js/blob/4a2f704e07c4c653661ba420a958d960b9889773/src/webgl/p5.RendererGL.js#L1710

and then we can move to the docs of strokeShader() as well.

Let me know if strokeShader() works for you.

I think we also need to alter some tests created for shaders, will update you for the tests as we completes the docs and code of this project.

Thanks again for your work. :)

[Garima3110]

Thanks for the follow up @perminder-17 I'll work on these and get back to you soon!

[Garima3110]

I have done all changes in docs and examples for fill shaders, please have a look @perminder-17 And just to mention I'm starting the work for strokeShader() too.

[perminder-17]

Great work @Garima3110 so far. I tested it, and it seems to be working well. But to be double sure, I'll test it again to make sure nothing is breaking. In the meantime, you should focus on completing the documentation for the fill shader to meet the first week's target.

Just an update, previously i mentioned the wrong block of code by mistake. I just saw when you pushed it you need to remove the isStrokeShader() code. I have edited the above comment now.

https://github.com/processing/p5.js/blob/12e6276d403eb36feb452acf95aa5cf01b4c7998/src/webgl/p5.Shader.js#L1020

[perminder-17]

As we're making breaking changes in this PR, we’ll end up with two distinct methods for applying shaders: strokeShader() and imageShader(). I'm considering whether we should also rename shader() to fillShader() for consistency. I'd like to get the WebGL stewards' thoughts on this. Tagging: @davepagurek @lukeplowden @stalgiag @aferriss @aceslowman, @ShenpaiSharma, @teragramgius, @JazerUCSB, @richardegil, @itsjoopark, @Gaurav-1306, @jeanetteandrews

The advantage of renaming it would be creating a clearer, more defined structure: fillShader() for fills, strokeShader() for strokes, and imageShader() for images. However, a potential downside is that users have grown accustomed to using shader() over the years, especially since fill shaders are used most often. This change might cause some confusion for long-time users. What are your thoughts on this?

Here's how it looks as of now:

Screencast from 15-09-24 01:13:59 AM IST.webm

[Garima3110]

Thank you for raising this discussion @perminder-17 !

Well, I would also like to add something to your suggestion, I see the potential value in renaming shader() to fillShader() for consistency alongside strokeShader() and imageShader(). However, I believe it would be better to keep the name shader() as it is. My reasoning is that in p5.js, we often retain a feature's basic name based on its default behavior, without explicitly naming it according to that behavior. The shader() function naturally applies to fills by default, and users are familiar with this behavior. Renaming it could introduce unnecessary complexity or confusion, especially for those who have been using p5.js for a long time. By keeping shader() intact, we maintain simplicity while introducing strokeShader() and imageShader() for more specific cases.

This approach aligns with how we’ve handled similar features, where the core functionality remains intuitively named, while specialized methods are introduced for distinct use cases. I’d love to hear your thoughts on this approach and whether this strikes a good balance between clarity and consistency. Thankyou!

[lukeplowden]

Looks good @perminder-17 !

I agree with @Garima3110, I don't think shader() should be changed to fillShader(). It's breaking, but I also think it's better to leave the name general because shader() may be used for things which aren't really 'fills'. For example you don't have to draw the results of a shader() call directly to the screen inside of p5.Framebuffer.draw(), it could be a post processing pass, or it could be related to instancing using endShape(). Eventually they might affect fill colour but I think the general function name is more intuitive.

With imageShader() is the idea to affect image() calls? I'm curious to see how that one will work!

[davepagurek]

I think I also agree with @lukeplowden and @Garima3110. Maybe the way to think about it is not in terms of fills and strokes, but in terms of materials and strokes. I suppose fill is actually just one component of an object's material, in addition to other properties like texture or shininess. Strokes are a separate system which adds something on top of the base material. In that view of everything, the material system is the main one you interact with and includes the most stuff, so treating it like the default in its naming feels right to me.

One thing that's not super clean about the model I just described is that images are kind of just a separate system, and regular objects that have a material and stroke can't go through the image system, and likewise, images don't get materials and strokes. I think that's fine, since we want images to always draw an image, so making overriding of image() very separate and explicit probably will help avoid accidental overrides and confusion. So I see image shaders as maybe more of a convenient way for developers to change small bits about image drawing and still take advantage of the fit and fill options of image() rather than a core part of the material/stroke system I guess.

[perminder-17]

Thankyou @lukeplowden @davepagurek and @Garima3110 for your thoughts.

With imageShader() is the idea to affect image() calls? I'm curious to see how that one will work!

We are now planning to create a method that will only affect image calls. In simple terms, we are separating our methods because in many scenarios, we have seen that sometimes fill gets applied to images and sometimes it doesn't, which causes problems and confusion see here: https://github.com/processing/p5.js/issues/6327 , https://github.com/processing/p5.js/issues/6564 . To avoid accidental shaders being applied and also to avoid confusion, we will create a new method called imageShader() that will specifically handle shaders for images called through the image() function.

I’d also like your thoughts on something related to the question, which may seem obvious but I want to confirm. Do you think the image() function should be influenced by shader()? Or, put simply, should shader() ignore image() calls?

[perminder-17]

Okay, @Garima3110 currently, the imageShader is not working. As we discussed on the call, we need to troubleshoot some functions.

For example, if you see the code https://github.com/processing/p5.js/blob/5fbde13db72c894941db5c92eba919af08ed8132/src/webgl/p5.Shader.js#L1374

we check if this.samplers.length > 0 so, when it returns true that means we have our image in our shader and we are currently using it (by calling uniform sampler2D uSampler and also using it).

For now, it’s worth disabling the fill shader effect for images, as @davepagurek's inclination was also towards not using fill shader for effecting image. For doing that, We can update the function https://github.com/processing/p5.js/blob/5fbde13db72c894941db5c92eba919af08ed8132/src/webgl/p5.RendererGL.js#L1768 by adding an extra condition which could handle isTextureShader().

Then, for imageShader() we can create a new function to return _getLightShader() when isTextureShader() is true.

Also, we could add example of by shaderHooks for only strokeShader() by studying this interesting tutorial. https://github.com/processing/p5.js/pull/7149#issuecomment-2336404893 .

[Garima3110]

Thank you for bringing this up! @perminder-17

I’d also like your thoughts on something related to the question, which may seem obvious but I want to confirm. Do you think the image() function should be influenced by shader()? Or, put simply, should shader() ignore image() calls?

From my perspective, the motivation behind introducing a separate imageShader() method is to bring greater clarity and control to the way shaders are applied in p5.js. Specifically, this new method would target images drawn using the image() function, thereby avoiding the confusion and unintended behavior that can arise when shader() affects image rendering.

Considering the issues mentioned (#6327, #6564), I would say that the shader() function should not influence image() calls. Instead, shader() should be reserved for fills, and strokeShader() should handle strokes. Meanwhile, the newly proposed imageShader() method, which me and @perminder-17 are currently working on, would exclusively manage shaders, for images drawn with the image() function. I would also like to know what others have to say on this one! Thankyou.

[davepagurek]

Ahh @perminder-17 you're right, those methods are going to clash, so we'll have to rename something. Maybe the hooks ones should be renamed to defaultStrokeShader() instead of just strokeShader()? It's a little verbose, maybe someone else has a better name idea?

[perminder-17]

defaultStrokeShader

How about lineShader() @davepagurek ? For hooks?

[davepagurek]

That also works for me. I guess the stroke one is the only naming conflict because the shader used internally for images is the material shader, and we're just using shader() to set that?

[lukeplowden]

defaultStrokeShader

How about lineShader() @davepagurek ?

I don't think I mind it, but it could be confusing to have two terms for stroke. I noticed that you actually wrote lineShader() here @davepagurek

However couldn't strokeShader() both set the active stroke shader and return it? Given that the hooks version doesn't take any parameters

[davepagurek]

oops good catch @lukeplowden, I guess if I can confuse the two already then maybe that's a sign they'd be too similar.

However couldn't strokeShader() both set the active stroke shader and return it? Given that the hooks version doesn't take any parameters

Normally when we do that, it's for something that acts as both a setter and a getter, so if we do that here, I'd expect it to return whatever the active stroke shader is, which might be something set by users. We probably will still want a reliable way to get the default one though. Maybe that could mean defaultStrokeShader() always returns the default, strokeShader() with no args returns the active stroke shader (or the default at first, when a custom one isn't set), and strokeShader(yourShader) would set it?

[lukeplowden]

That’s what I was imagining, but then again its also not ideal as it doesn’t align well with the other shader hook functions materialShader(), normal etc which can’t be set in the same way.

lineShader() may just be better

[davepagurek]

yeah I think if we rename this one we'd probably also want to rename the others to have the same naming convention, defaultColorShader() and defaultMaterialShader() maybe? Or to be less verbose, base instead of default, e.g. baseMaterialShader?

[Garima3110]

Well. I find all three lineShader() , defaultStrokeShader() and baseStrokeShader() suitable for the names, but my suggestion would be, maybe using baseStrokeShader would be a good name here?!!

[Garima3110]

Thanks @perminder-17 for the insights,I’m working on these changes ,will push the code super soon..!

[davepagurek]

btw @Garima3110 feel free to ignore the merge conflicts with the dev-2.0 branch for now. If things are working on this branch, once the project is done I can handle the merge 🙂

[Garima3110]

Thanks for all the suggestions to you @perminder-17 Special thanks to @davepagurek too for his valuable insights ! I'll work on these and aim to finish it super soon!

[perminder-17]

Just a question: Suppose we render something, like a rectangle with a red color, and then apply the filter(BLUR) function. Naturally, the result is a slightly blurred red rectangle. Now, if we also use a shader where gl_fragColor = vec4(green color), it would produce a blurred green rectangle.

But here's where it gets interesting: If we use imageShader(), it would render a completely red canvas. This happens because we're creating a framebuffer that matches the canvas size here : https://github.com/processing/p5.js/blob/c377b5358ef277949236799512e6274c0bfad678/src/webgl/p5.RendererGL.js#L1057 and then using the image() function to render the contents of the framebuffer (FBO) back onto the main canvas or target . https://github.com/processing/p5.js/blob/c377b5358ef277949236799512e6274c0bfad678/src/webgl/p5.RendererGL.js#L1170 This behavior also impacts panorama(img), likely because it uses filter shaders. So, I am thinking if we could disable imageShader() when applying filters?

Or maybe we could add some extra conditions to _getFillShader() by adding a boolean variable maybe (this._drawingFilter) at filter block, where when image() is called with filters, how imageShader() should effect without any effect via shader() anything else. If a model is rendered with filters() how it should react with imageShader() (should be ignored) and shader() should work.

@davepagurek, can you give a thumbs up if you agree?

[Garima3110]

I completely agree with the suggestion made above by @perminder-17 . The key issue lies in how imageShader() interacts with the framebuffer, especially when filters like filter(BLUR) are applied.

So according to me also, introducing a condition to disable imageShader() when filters are being applied, or adding a boolean like this._drawingFilter within the filter block in _getFillShader(), would help address this issue. It would allow filters to function as expected without interference from shaders, and maintain the correct interaction when both are used together.

[davepagurek]

Agreed, filters shouldn't be affected here.

If you're curious, I was talking a bit here https://github.com/processing/p5.js/pull/7270#issuecomment-2409005444 about refactoring how some of these things work so that there are "safe" functions we can call internally without user code leaking in. For now I think we just have to know to disable user shaders here.

[Garima3110]

@perminder-17 I have made all the changes, please have a look and let me know if any other thing needs to be updated. Thankyou!