[p5.strands] Implementing a noise function in strands

[lukeplowden]

Increasing access

This enhancement is part of the effort to integrate p5.strands more integrated into the core library (see parent issue #7849)

Most appropriate sub-area of p5.js?

• [ ] Accessibility
• [ ] Color
• [ ] Core/Environment/Rendering
• [ ] Data
• [ ] DOM
• [ ] Events
• [ ] Image
• [ ] IO
• [ ] Math
• [ ] Typography
• [ ] Utilities
• [x] WebGL
• [ ] Build process
• [ ] Unit testing
• [ ] Internationalization
• [ ] Friendly errors
• [ ] Other (specify if possible)

Feature enhancement details

A noise implementation in p5.strands would allow for a lot of interesting effects using p5.strands, for animating objects and rendering materials. This could be a GLSL implementation of simplex noise, or another algorithm.

In order to get this working, something similar to getTexture could be a good approach. That function is defined here in strands and also here in GLSL. This would mean making a new function call node in strands, and writing the GLSL function somewhere to be included in shaders. I would be inclined to make this a new GLSL file, since we might add more functions to it in the future.

The implementation can be different than the core library version of noise, with potential to add a noiseMode(GPU || CPU) in the future.

[davepagurek]

Some notes for working on this:

• Some noise approaches use a big table of noise values, but we probably don't want to use that for now if we're going to be adding this helper function to all shaders
  • Potentially in the future, we can conditionally add it only if a shader makes use of it, but if our implementation is small, it's OK not doing that yet
• If we're borrowing a GLSL noise implementation from the internet, that could be OK but we should be explicit about it and discuss what its software license is
• A lot of references to noise functions that you see only implement base noise. p5's noise() uses fractal noise, which looks something like this:

  for (int i = 0; i < 3; i++) {
    result += pow(2.0, -float(i)) * baseNoise(input * pow(2.0, float(i)));
  }
  

  In p5 the factors are controlled by noiseDetail, but it's also OK to just hardcode the default values for now and connect it to noiseDetail in a later update.

[LalitNarayanYadav]

Hi @davepagurek! Just leaving a comment here so you can assign the issue to me. Looking forward to working on this one too 🙂

[davepagurek]

Thanks! I'll assign it to you.

[LalitNarayanYadav]

Hi @davepagurek!

I've completed the implementation of noise(vec2) for p5.strands as discussed in #7897. The GLSL function is injected via fragmentDeclarations, and it's only active during shader generation. Let me know if you have any suggestions or would like any changes!

[davepagurek]

Thanks @LalitNarayanYadav, great work! I think the next steps for p5.strands noise would be:

• to add back octaves of noise (ok if it's hardcoded to just the defaults in p5 for now)
• try to match the function signatures that regular p5 noise uses (e.g. noise(x, y) in addition to noise(vec2 st), maybe in the js function that constructs the nodes?)
• to support 3D noise like outside of p5.strands
• to support something like noiseDetail (maybe automatically grabbing the noise detail values at the time the shader is constructed and embedding those as constants?)

Let me know if you're interested in working on any of those!

[LalitNarayanYadav]

Thanks so much @davepagurek! I'd love to continue contributing to p5.strands noise.

I'm happy to take on the next steps, starting with adding back octaves and matching the signature of noise(x, y) alongside noise(vec2). Supporting 3D noise and noiseDetail() also sounds really interesting — I’ll dig into how we might embed those constants from JS.

Looking forward to exploring this further!

[GregStanton]

Hi all! It's exciting to see the progress on strands. There are some related aspects of the existing noise features in p5 that may affect the development of the strand features.

noise()

@davepagurek:

try to match the function signatures that regular p5 noise uses (e.g. noise(x, y) in addition to noise(vec2 st), maybe in the js function that constructs the nodes?)

Thanks for catching this. Would it make sense to only add noise(x, [y], [z]) to strands for now, instead of noise(vec2)/noise(vec3)? Or are you thinking of adding a p5.Vector overload to the regular p5 noise() function too, for consistency? [^1] I think the current pattern for most p5 functions is to take individual rather than vector arguments. The only exception that immediately comes to mind is point().

In general, it might make sense to support more object arguments in the future, but maybe for now it'd make sense to only add new overloads that are required for consistency with existing APIs? I expect we'll start working out general guidelines for object arguments as part of the API audit, once that gets going.

Update: I noticed the noise(vec2) strands feature was just released (only five hours ago, as I write this). But since it's brand new and experimental, maybe it'd be okay to remove noise(vec2) and keep the signature with individual coordinate parameters? Or is there a reason to support a vector argument in the strands version but not the standalone version?

noiseMode()

@lukeplowden:

The implementation can be different than the core library version of noise, with potential to add a noiseMode(GPU || CPU) in the future.

Just FYI, there's been some previous discussion about a noiseMode() function in #7430. The specific use case you mentioned wasn't discussed in the linked issue, but considering that discussion should help ensure that the API is extensible and leaves room for other use cases.

noiseDetail()

@LalitNarayanYadav

noiseDetail() also sounds really interesting — I’ll dig into how we might embed those constants from JS.

Thanks for your work on this! Problems with noiseDetail() were detailed in #7430. In order to avoid propagating existing problems, maybe we could first try to reach a decision about whether to deprecate the old noiseDetail() API? I think we were close to a final design candidate already, based on the pattern established by splineProperty()/splineProperties() and textProperty()/textProperties(). And others have expressed interest in related features, like @shiffman in this enthusiastic comment!

Next steps?

Maybe if we can reach a consensus on an API to replace noiseDetail(), and ideally on the API for a new noiseMode() feature to go along with it, then noiseDetail() could be deprecated, work could start on a replacement, and the strands feature could be finalized? Then in a future 3.0 release, noiseDetail() could be removed. Thoughts @ksen0?

[^1]: In addition to overall API consistency, we may want to consider enhancing the design of p5.Vector before adding vector overloads in addition to separate-coordinate overloads. I think that in 1.x, p5.Vector would always create a vector of length three internally, setting any missing x, y, or z inputs to zero. Especially now that we've started to support n-dimensional vectors, it may be helpful to have a way of checking the dimension of a vector. For example, when passing a p5.Vector instance into a regular p5 function like noise(), it'd be possible to determine the intent of the user (whether it's intended as one-dimensional or three-dimensional, say). I suppose that might be important for some things, but I haven't thought enough about it yet, and adding a feature like this may be tricky due to the existing behavior.

[davepagurek]

I'm in favour of keeping vector overloads for p5.strands in general, although I also don't know that it needs to be backfilled in normal p5. I guess it comes down to goals for p5.strands: I don't see it as trying to abstract away shaders fully, but as a tool to introduce shaders more gently and be productive with them, including the advantages they have over regular JS. So I'm personally OK with having some shader-only features that you can learn about over time as long as you can also use the things you're familiar with from p5.

Generally I'm thinking about two main users of p5.strands:

• Users who do know shaders already, for whom p5.strands can be a more boilerplate-free, hassle-free shader programming experience. They are likely already familiar with the ins and outs of GLSL, but may be used to copy-and-pasting boilerplate functions from project to project. They likely will not use p5.strands if it slows them down, which it will if they now have to be writing vector operations one element at a time. So we're trying to both support common GLSL operators that aren't present in p5 (e.g. smoothstep()) as well as operators that are but have different names (e.g. mix(), which works the same as lerp()) and I think about supporting vector inputs to things as an extension of that -- while not a builtin GLSL feature, it is common practise to use vectors in places like this.
• Users who don't yet know shaders, but who know the rest of p5. For these users the main thing that matters is that the patterns they already know will continue to work (in this case, noise(x, y)). This also likely means that initially they will not using any shader-based vector operations (e.g. vec3(1, 2, 3) * 3) because those don't exist in regular JavaScript. But if they start looking into shaders more, they'll encounter vectors everywhere in GLSL examples, and I think starting to use them more will be part of their shader knowledge.

Not having vector overloads for noise in particular doesn't make a huge difference, but I think having shader-focused overloads in general (similar to how for color() you can pass in a CSS color string in addition to Processing-style channel values makes it a little more browser-native) makes it fit into its environment a little better.

[davepagurek]

For noiseMode I'm also in favour of the new API in that other issue, so I think if we want to add noiseMode in strands in the mean time, it's OK to do but we should just be aware that it may be refactored. Best case scenario, we actually don't need to reimplement mode switching, and we just need to record what the current mode is when we create a GLSL noise node in the shader graph, so if we refactor it in the future, we just need to make it read a slightly differently named state. But if it looks like it'll need a lot more work/reimplementation, then we may consider waiting.

[GregStanton]

Thanks @davepagurek! I'll reply about noise() overloads first.

It sounds like this is mostly an issue of reconciling conventions in two different environments (vector parameters in strands vs. number parameters in regular p5). Since it sounds like strands users may expect to have vector overloads already, it seems reasonable to have them. I guess that leaves a couple of options for reconciling the conventions, mainly for any users who start out in strands (perhaps coming from a shader background) and migrate to the rest of p5 (maybe for ergonomics, when shaders aren't necessary?).

Immediate solution: vector overloads in strands only

Consistently support vector overloads in strands but not regular p5, and provide overloads for separated coordinates in both. This is the path of least resistance, and it seems like it'd work reasonably well. This way, at least there's a pattern that strands users can pick up on. If they're transitioning from strands to regular p5, then they may initially expect to see vector overloads there too, but they'll learn pretty quickly that regular p5 doesn't (usually) have them. For users who are already familiar with regular p5 and then start using strands, the syntax they're already used to will be supported, as you noted.

Long-term solution (maybe): vector overloads in strands and regular p5

Consistently provide vector and separate-coordinate overloads, in both regular p5 and strands. Each type of overload has certain advantages, and this might actually be a good idea in the long run. It'd eliminate inconsistency; it'd allow for more readable, semantic code; and it'd allow for shorter parameter lists. For example, quad(2, 5, 2, 5, 2, 6, 7, 3, 4, 8, 9, 9) could become quad(v1, v2, v3, v4). This would require more design planning and development, though. For example, there are also cases like arc(x, y, w, h, start, stop, [mode], [detail]) that are more complex. And we don't currently have a good way of distinguishing 1D, 2D, and 3D vectors, which might be an issue (as far as I remember, all vectors are represented internally as 3D vectors, except for vectors with length greater than 3).

Edit: Replaced triangle() with quad(), since it illustrates the issue better.

[GregStanton]

@davepagurek:

For noiseMode I'm also in favour of the new API in that other issue, so I think if we want to add noiseMode in strands in the mean time, it's OK to do but we should just be aware that it may be refactored. Best case scenario, we actually don't need to reimplement mode switching, and we just need to record what the current mode is when we create a GLSL noise node in the shader graph, so if we refactor it in the future, we just need to make it read a slightly differently named state. But if it looks like it'll need a lot more work/reimplementation, then we may consider waiting.

I was thinking more about the API than the implementation, since refactoring can potentially happen later, as you mentioned.

Background

For anyone following along, the idea was to have different noise algorithms, each potentially with their own properties, similar to the following for splines:

// Using hobby curves with full config
splineType(HOBBY);
splineProperty('tension', 1);
splineProperty('curlStart', 1);
splineProperty('curlEnd', 1);

// Using Yuskel curves and changing just the interpolator from the defaults
splineType(YUSKEL);
splineProperty('interpolator', HYBRID);

The idea is that we'd have the following analogous patterns:

• splineType(), splineProperty()/splineProperties() (2.0 doesn't have splineType() yet, but it has the other two functions)
• textFont(), textProperty()/ textProperties() (these are all in 2.0 already)
• noiseType(), noiseProperty()/noiseProperties()

Note: I had forgotten about this, but I think "type" may be better than "mode" in this context (see the update at the end of the linked comment).

Usage conflict?

If noiseType(type) is really designed to set a specific algorithm, then that'd be different from setting whether the algorithm should run on the CPU or the GPU, which is the usage cited at the top of this issue. So I was thinking we might need to come up with a special API if we want users to be able to set where the algorithm should run. I suppose we might consider having a noiseMode(mode) function in addition to noiseType(type), but maybe it'd be too easy to confuse these with each other?

For the matrix proposals, I did have in mind something like computeMode(), which could be set like e.g. computeMode(SCIJS) or computeMode(STDLIB), so that users could choose their computational backend (that's a feature that might potentially be added in the long run, but not at first).

Edit: Added a little structure and clarified the main point.

[golanlevin]

Shucks, half the fun of GLSL programming is having to make one's own noise functions from scratch :)

[GregStanton]

@golanlevin Hah! I feel the same about so many things. The thrill of discovery and DIY projects is so important!

Matrix transforms are a good example. For those who already understand the math, it's fun to apply it by making graphics transforms from scratch. For those who aren't already familiar with the math, a high-level transform API can introduce them to the concepts; and then some users may start to wonder how those things are actually implemented. With a bit of support, this is an excellent way for learners to discover matrix multiplication themselves. Educators, or educational materials, can provide that support.

I think something similar can be said of splines, noise, fonts... But I'm curious to hear your take!

P.S. I recently made an exercise for a course I'm designing that asks learners to program their own square root function. Even though I'm familiar with that, it was still fun to write up the solution code. And I'm pretty glad that Math.sqrt() exists for when I'm in a hurry or need something optimized. So I guess that applies here too :)

[davepagurek]

I'm going to close this now that we've merged in 3D noise in strands, I've opened https://github.com/processing/p5.js/issues/8160 for support for changing noiseDetail, and when we decide on noiseMode or similar API changes, we'll know changes need to happen in p5.strands too.