Rewrite/v2.0 General Discussion Thread

[squarefeet]

Following incredibly valid points from @usnul and @cihadturhan, in order to get this library into the state that it deserves, a rewrite would appear to be necessary. This thread is for anyone to add their 2-cents/pennies/other-currency worth of thoughts on improvements, new features, etc. The more critical, the better (IMHO) - that's one of the major ways to ensure the new version is as easy-to-use and performant as possible.

I do ask that any requests/comments should follow the same format as below: one feature per bullet-point, and grouped into either Required or Nice-to-Have (or both, if you're feeling generous).

Thanks!

---

Required

• Sprite-sheet support for both animated textures and shared texture maps.
• < 1 particle per second.
• A much more intuitive API (at the moment, there are just far too many options. These need to be grouped, or...??)
• Dynamic, and efficient, creation/destruction of emitters.
  • Choice of immediate destruction vs. stopping the emitter and waiting for existing particles to die before being removed.
  • Removal of static emitters.
• Remove the 'Group' constructor, and have only one constructor for the entire library.
  • 'Group' constructor should be replaced by internal logic that groups emitters by texture.
  • A user shouldn't have to create a group manually.
• Runtime adjustment of all parameters.
• Efficient callback/event system that allows users access to individual particle parameters on a per-frame basis.
  • This could (read: probably will) have a negative performance impact, so will have to be thoroughly thought-out.
• "Spiral" emitter type
• An implementation of 'angleAlignVelocity' that actually works when particles are not starting from vec3(0,0,0) position.
• A more intuitive way of setting options. At the moment, it's pretty crappy.
• More efficient JS and GLSL code. It's currently not quite as performant as I'd like, and I know that there are places to squeeze!
• Improved documentation and examples. Maybe a few tutorials.
• It's own website to hold the documentation and examples. I'm not sure that gh-pages is public-facing enough.

Nice-to-haves

• Gravity-well or general "gravity" support.
• Vector-field support.
• Angular velocity and/or acceleration instead of/as well as the current angleStart/Middle/End properties.
• An incredibly difficult one, this: a way to generate fairly realistic smoke. Would require a very high particle count, and either vector-field support, particles interacting with other particles, or a dynamic 'wind' system. Maybe all three and yet more still...

[squarefeet]

@movitto, @alexeld, @AdrienMorgan, @DelvarWorld, @stemkoski - Just tagging you guys to let you know the existence of this thread. If you have anything to add, please do. Not expecting anything, but comments/criticism of the current version would be very handy.

Cheers.

[Usnul]

hey @squarefeet thanks for creating this thread. I read SuckerPunch presentation on their particle engine for their ps4 game "infamous". The slides are at: http://www.suckerpunch.com/images/stories/SPP_Particles_Full.pptx they are some 500MB thanks to high-res clips, so download at your own risk. Bill Rockenbeck names 180,000 as a high number of particles to manage (what they have as a limit it seems). I know for a fact that glsl can manage a lot more than that for simple systems. The relevance of that particular engine is their simulation of smoke, they use only a handful of particles to do that. For those particles to look nice you need quite a few things though:

• randomized (noisy) behaviour
• fluid motion

but this is only for the sake of behaviour of particles themselves, another thing is appearance and there things like reaction to lights/shadows and casting of shadows is apparently a lot more important, I'm quite certain it would be pretty hard to formulate without heavily coupling with some particular rendering pipeline, so I'm just leaving it out there.

Trouble with encoding things like "spiral" or "straight line" or whatever else is an interesting motion shape someone might want directly into the engine is lack of flexibility. I believe a great engine is:

• not one where each particle can be modified and inspected individually
• is aware of noise functions
• has "language" (configuration or model or whatever else is more comfortable to use as a term) to describe motion rules

if you could supply emitter with following position function:

pos = vec3(sine(time)*r, cos(time)*r);
r = log(time);

you could essentially give people freedom to define spirals, however it would give them a lot more than that. And maybe it is a good way, to just allow injection of a snippet of code into shader, where shader includes some useful function like noise inside of it already.

Another thing in terms of implementation that i'm quite certain of - it has to remove JS out of the loop almost entirely, best option for that right now seems to use 2 buffers and swap them between simulation steps, use one for lookup and other for writing new state. I believe there are better ways potentially, but they aren't supported by what's enabled in a browser by default (without toggling flags in chrome or equivalent elsewhere).

From my personal experience - keeping things as generic and as simple as possible usually pays out in the long run.

[cihadturhan]

I agree on what @Usnul said about injectible function. An injectable functions means fully generic system. Adding a gravity would be as simple as

pos = vec3(x0, y0 + g*time*time/2, z0);

and you can switch from a cylinder to a sphere easily by changing two parameters. If you've heard about superformula you'll definetely understand what I mean. This was also an issue I felt when I was coding Hydrogen atom probablity density viewer with SPE because I had a formula for probablity distribution and I had to inject it to place points to the related coordinates but the current library didn't help me about that and I had to change many parts.

This also may lead an issue about easiness of code because a developer should find out the right formula to generate desired shape of particle system. We should consider it.

[Usnul]

I don't really see this as an issue @cihadturhan, there is an option of creating a library of these parameterizable Emitters, something along the lines of:

var BasicPhysicsEmitter = function(options){
...
var velocity = options.velocity;
addUniform({ name: "velocity", value: velocity, type: "v3"});
...
"pos = pos+velocity*time" 
...
}

let people use these as much as they like, I'd imagine it would serve large portion of users.

PS: That hydrogen looks awesome :)

[cihadturhan]

Yep, on a second thought I think that won't be a problem.

Thanks @Usnul

[squarefeet]

This is all great stuff. I'd never thought of having custom position calculations... I'm assuming these will either be written in GLSL and injected at runtime (or, if an emitter is already running and its pos calculation is changed, it's shader's rebuilt)?

For the sake of playing devil's advocate, I have some issues with it:

1. GLSL isn't as easy to get to grips with as JS (it's userbase is much smaller). @cihadturhan - you mentioned this as well.
2. Changing the pos function during runtime could cause quite a few stutters.
3. Assuming it would be GLSL code injected into the shaders, this code would probably have to be written as strings... that feels a little clunky.

To argue against myself:

1. @Usnul: Your suggestion of a library of parameterizable emitters is a good call. That makes the issue of GLSL not being as well-known as JS a non-issue. We'd have the base library, then a bunch of general-use, pre-defined 'shape' emitters that would have their pos calculations filled in?
2. The use of multiple buffers (maybe even more than two) could alleviate this stuttering. Since we'd have fewer CPU cycles being used to calculate new pos/vel/accel/etc. values, these cycles could be used to swap buffers and re-compile shaders.
3. I can't think of another way around this problem... Chalk it up to a limitation of the platform?

I'm liking the idea of moving the functions that currently reside in SPE.utils into the shader, though. This would free up a lot of CPU cycles for the buffer swapping (which I'll probably have to quiz you quite a bit on, @Usnul!). I know neither of you explicitly stated that the SPE.utils functions would be moved to the shaders, but that does seem to be a logical extension of what you've both suggested so far...

What I would absolutely love to be able to do is get values back out of the shaders. At the moment (and please do correct me if I'm wrong) but I'm pretty sure that we can only whack values into a shader, but not get the result back into JS. WebCL should solve this problem, but I believe that is quite a way off from being included in browsers as standard. The reason I bring this up is that, in theory, we could do all of the setting up in JS, and then use multiple shaders to do different calculations (possibly even particle interactivity, eg. boids simulation). Oh, the possibilities!

Dreaming aside, though, this is a good step forward. @cihadturhan: That hydrogen density sim is really awesome - great work!

[squarefeet]

PS. Just done a v. quick lookup on the state of WebCL. The 1.0 spec was released in March this year, but Firefox, at least, isn't going to implement it. They seem to be erring on the side of OpenGL ES 3.1's compute shaders instead. I haven't yet found a timeline for ES 3.1's adoption.

[Usnul]

Hey @squarefeet, I like your reasoning. Let me try and address some issues:

• 2 buffers are to go around the issue of reading and writing at the same time into the same buffer, and the problem is - it causes glitches which were discussed at lengths throughout github on various particle-related projects. The clear consensus is to use one buffer for reading values and one for writing, thus eliminating race conditions that otherwise mess with your mojo.
• buffers store particle state, such as position, velocity and whatever else, I imagine there would need to be a provision for injecting arbitrary variables from users into the chain, as little beyond color and position are "required".
• since particle state is stored in a buffer and we know which one will be used for lookup and which for writing - we can extract current state for individual particle quite easily between render cycles and similarly inject new state information. I would imagine this pipeline completely in JS which would make manipulation of individual particles somewhat slow, but at the same time possible and wouldn't require much memory at all, since manipulations would be done directly on buffers.
• shader re-compilation. First of all, as weird as it sounds, graphics pipeline accepts strings, so it's not that strange to be working at that level. Second issue is frequency of recompilation and performance associated with that. I can not envision single emitter behaviour being manipulated too often, I'd imagine its code to stay he same, but maybe uniforms to change between frames, a usecase which would require recompiling of shader every frame is probably a bad fit for shaders in the first place. As for recompilation performance, there is a demo from 2010 by Evan Wallace on path tracing, where he does exactly that - recompile the shader at high frequency, and the results are rather encouraging showing no stutter whatsoever. http://madebyevan.com/webgl-path-tracing/

I could see every emitter class (not instance) use a separate shader, but at the same time - even hundreds wouldn't present much of a challenge, when thinking about games today that employ thousands of shaders at the same time.

[Usnul]

here's a promising thread from three.js on the similar concept of using double buffering: https://github.com/mrdoob/three.js/issues/1183

[cihadturhan]

A couple of things,

• I think, we need to implement gl_Vertex attribute to store the coordinates of each vertex in gpu. As we discussed on vector fields issue (https://github.com/squarefeet/ShaderParticleEngine/issues/37), we had a problem because gpu doesn't remember the previous changed position of the vertices. The problem was, it jumps the previous position after it leaves vector field because it just knows the first value defined. If we use gl_Vertex, gpu will remember the position and make computation according to current coordinate and that won't be a problem. gl_Vertex is a defined attribute in OpenGL already, but not WebGL. Therefore, we will implement this if we believe this is utilized well on vector fields or some other stuff. Also, in lightgl library, they implemented gl_Vertex somehow and they are using it in their library as well.
• on shader re-compliation, there is an existing library which recomplies shader immediately. There is also a demo where you can change shader properties and recompile on the runtime quickly.

[Usnul]

@cihadturhan I'm a little confused on this point. if we use an FBO - then vertex shader is pretty much excluded, you have a quad and that's it, each fragment however represents a particle, and looks up position of that particle from a texture (FBO). Having UV (0.5,1) for instance, we'd go to texture and lookup pixel at at that location, then we'd unpack state from it, including position of the particle. In a way UV only serves as ID of that particle. GLSL is quite new to me, and a lot of it i still don't understand, for instance I'm not sure how then this data can be used to place sprites or how it can be shared with things like THREE.js. On the other hand, if we have a vertex shader responsible for particle simulation, I'm not sure how FBO lookups would work, and how these vertices are going to be linked to specific texels.

[cihadturhan]

@Usnul I understand how you want to use FBO but I have no knowledge on how FBO works in deep. Will the coordinates change in the next frame like in gl_Vertex? This is what I don't know and I wonder much. If it always stores the new one then it's perfectly fine to use FBO.

[Usnul]

@cihadturhan here's a basic rundown:

set fbo1 as texture for lookup
set fbo2 as render target (where we draw to)
begin render
...
render done
swap fbo1 and fbo2 so that next time 1 is used for render target and 2 for lookup
rinse and repeat

okay, that's great, and here's what's happening inside fragment shader:

get pixel from texture based on U and V coordinates
do clever stuff with pixel data
set gl_FragColor with new pixel data so that it gets written to render target U V coordinates

the proviso here is that texture for lookup and render targets are identical in structure, this is why you can quite easily identify where to write, and why you can swap them too.

As you can see, particle state would correspond to a specific pixel, so it is preserved between render cycles. Inside the shader you have access to that state and also have the opportunity to use that "old" state when creating new one. A simple static shader, for instance, could just copy old state so that new state remains the same as the old one.

[cihadturhan]

@Usnul Thank you very much for explaining, I'm new to texture stuff. I see what you want to do and it's very clever :+1:

If the number of particles are n * m, then we'll use all of the pixels :)

PS: we can assign m = 1 everytime so that's not a problem though.

[movitto]

@squarefeet @Usnul this all seems reasonable / cool. Am also relatively new to GLSL so can't comment too much but agree it would be awesome to parameterize the shader w/ custom algorithms (if it's feasible / practical). Also agree on addressing the performance implications, perhaps in addition to the solutions presented above, we could support multiple modes w/ different shaders optimized for different scenarios.

In general would be good to try to do this as a pluggable architecture, where at the base we present simple classes / interface able to be extended/used for custom scenarios. Would be glad to help where/when I can.

[squarefeet]

@Usnul The FBO stuff does certainly seem to overcome our problem with particle state between render cycles - great call to use that. From my extremely limited understanding, though, wouldn't we only have 4 values per pixel to describe a particle's state (R/G/B/A)? I think I'm just missing the connection between how I describe a particle's state in my mind (pos, color, velocity, acceleration, opacity, angle), and how the shader would describe the particle's state using an FBO. Though I guess we would only really need to store the position? Maybe, if necessary, we could use more than one texel to describe a vertex...

As far as linking a particle/vertex to a specific texel co-ordinate, could we not just have a FloatArray that stores UV co-ords? Obv. two entries per vertex ([u,v,u,v, etc.]). Have that as a uniform, and store the start position in the FloatArray for each vertex as an attribute:

uniforms: {
    texels: new Float32Array(...)
}

attributes: {
    coords: { type: 'f', value: [...]
}

I know I'm probably bringing the discussion backwards a little here, but I just want to be sure I fully understand your proposal :)

---

@movitto Great to have you on board. I think it's more than feasible to have custom shader algorithms - as @Usnul mentioned, shader re-compilation doesn't seem to pose as much of a problem as we first thought. That, and the changing of shader algorithm(s) during runtime would probably be a small use-case anyway.

The idea of different shaders for different scenarios is a good idea - I reckon for ease of development, though (and for DRY purposes), it would be prudent to make our shader code as modular as possible; broken up into small-ish chunks that we could just tie together to make new shaders as and when necessary ( similar to what THREE.js does ). That said, I think that's what you might've meant when you talked about the 'pluggable architecture'!

[squarefeet]

Okay, been doing a bit of reading into the FBO stuff, and @Usnul - I followed your link to the double-buffering thread in the THREE.js repo. It helped quite a lot, thanks.

About the 4 values p/pixel issue: looking at this example of GPGPU boids/flocking (which is just stunning, btw) from here, it appears that @zz85 is using multiple textures per "particle" (one for position, another for velocity, etc.) Following this pattern would solve the issue of only 4 values p/pixel.

His SimulationRenderer.js file would be a good inspiration for the FBO-swapping...

[squarefeet]

Just got a quick practicality question:

• Where would be best for this rewrite to be hosted? I'm not keen on it being in a separate branch of this repo as it wouldn't make too much sense, but that said, nor am I keen on it being in a repo of its own.

What would you guys prefer? At this stage, it'd just be sketches, tests, etc.

[Usnul]

I'd say a SPE mark.2 would make sense :) As a possibility we could bind that engine to this one's API as well, though not as a primary API target I guess. But yeah - i see it as a clean slate which shares common goals with this project, but is, as you said - a re-write as opposed to an alteration.

[squarefeet]

So I've been getting my head around GPGPU/FBO/render-to-texture stuff, and I've re-written the THREE.js gpgpu birds example. It sounds a little strange to have done that but it gave me a good way to learn FBO stuff, as well as make a base "class" that we can use in the next version of the particle engine.

I've submitted this as a possible replacement for the existing SimulationRenderer for THREE.js here. Hopefully I'll get some useful feedback either here or on that thread and it can move forward into a useful little "class".

In short, this version of the Simulation Renderer takes care of generating data textures, swapping renderer targets / buffers, and automatically passing generated textures into shaders. It's fairly straightforward to use, so I haven't added many comments, but let me know if it's not as straightforward as I think it is and I'll comment the crap out of it :)

@Usnul: Once this SimulationRenderer is declared usable/useful, then I'll start up the SPE v2.0 repo.

[squarefeet]

I'm quite comfortable with FBO stuff now, so I'm starting to think about how we should organise what textures hold what data.

To start with, lets keep things simple and have the following per-particle attributes:

• Position (RGBA),
• Velocity (RGB),
• Acceleration (RGB),
• Opacity (?),
• Max age (?)

We could have one texture for each of these, but I'm wondering if we could pack some of them together. Like using the unused w component of the RGBA Position texture as the opacity value, etc. Or maybe even having the position and velocity textures combined, using an offset parameter to determine whether a pixel of the texture is a position value or a velocity value - Soulwire did something similar here.

We could also work around having one render-to-texture pass for each of the textures as well - maybe by using the velocity calculations from the existing SPE shaders that extrapolate velocity values based on particle age. These velocity calculations could be integrated into the position texture shader directly, so that's one less render pass to do. This may cause problems later on when we come to add vector fields and stuff like that, though.

So... Issues that need discussing:

• What data should textures hold?
• Can we combine two or more data sets into one texture?
• How much of a particle's current position can be extrapolated without using FBOs and without affecting the ability to add vector fields (etc.) at a later date?

[Usnul]

could make a mapping mechanism for variables, something along the lines of: int -> ... float -> ... v3 -> rgb v4 -> rgba let the mapping mechanism pack/unpack several floats into a single rgb/rgba etc as you said. Keep track of what FBOs we have and what fields in those aren't used yet. I'd say making it generic from the start would solve a few problems in the future.

[squarefeet]

I've been playing around with a few things tonight and wanted to get feedback on a possible "core" API. I'm not going to paste any of the internals I have going at the moment, because they're an absolute mess, but as far as using the thing goes, this is what I have:

var myParticles = new Particles();
myParticles.addProperty( 'acceleration', new THREE.Vector3( 0, 1, 0 ) );
myParticles.addCalculation( 'velocity', 'velocity += acceleration * delta' );
myParticles.addCalculation( 'position', 'position += velocity * delta' );

• Where the addProperty() method takes the following arguments: property name, property value
• Where the addCalculation() method takes the following arguments: property name, glsl calculation string.

For each addCalculation() call made, an FBO object is set up (double-buffered).

Note that I haven't defined any initial values for velocity or position. These default to vec3(0,0,0) if not defined.

Also, for each addCalculation() and each addProperty() call, a uniform is added to each of the FBOs created, so any of the properties can be accessed from any of the shaders that get created.

Oh, and one more thing, I haven't done anything re. emitter groups. Just this core API work.

Thoughts? I'm thinking that whilst this gives us the most freedom we need (particularly taking care of custom position calculations), it does seem to possibly be at the cost of usability.

[Usnul]

that looks pretty good @squarefeet. Regarding groups, i suggest using a hash of sprite URLs:

var sprites = {};
function Particles(url){
  var texture;
  if(sprites.hasOwnProperty(url)){
   texture = sprites[url];
  }else{
   texture = THREE.TextureUtils.load(url);
   sprites[url] = texture;
  }
  ...
}

also, regarding syntax: i do think it's verbose, but there are ways of reducing that. One way would be to write a glsl parser which would be a small investment and would probably be done later in the project. Another way would be to do something like this:

myParticles.properties.add('acceleration',new THREE.Vector3(0,1,0));
myParticles.calculations.add('velocity += acceleration * delta')
     .add('position += velocity * delta');

parsing this would be relatively easy, as all you'd have to do is tokenize it and exclude reserved tokens like float, int etc. and only keep ID tokens ([a-zA-Z_][a-zA-Z0-9_]*)

[squarefeet]

I'm not sure why we'd need a GLSL parser?

EDIT: Ah, wait. I think I understand why now... you've removed the first argument for the calculation.add() function. We'd need to parse the only argument you have given in order to find the property name that needs calculating?

[Usnul]

yup :) besides that, not all operations might end up as assignments, and some "variables" may only need to be temporary

[Usnul]

glsl parser project in JS: https://github.com/chrisdickinson/glsl-parser

[cihadturhan]

@squarefeet I really like this implementation. Maybe we can add chaining to all functions such as

myParticles.addProperty( 'acceleration', new THREE.Vector3( 0, 1, 0 ) );
                  .addCalculation( 'velocity += acceleration * delta' );
                  .addCalculation( 'position += velocity * delta' );

[squarefeet]

@Usnul: Thanks for that glsl parser link, I'll check it out as soon as I can. @cihadturhan: Chaining will be nice and easy to add.

I was also thinking of trying to figure out some sort of automatic way to find out what properties of an RGB/RGBA texture weren't being used, and flag those as usable. So if someone adds a velocity calculation, that could create an RGBA texture with the alpha channel unused, so that'll be flagged and taken up if a user adds an int or float property (opacity, for example). It might be quite tricky to implement but will save on render passes and texture generation.

[Usnul]

this is just a sketch, can be a lot simpler

vectors = [];
//prepare buckets
vectors[4] = [];
vectors[3] = [];
vectors[2] = [];
vectors[1] = [];
//
function addProperty(name,arity){
    var bucket = vectors[arity];
    bucket.push(name);
}
//
function getFBOs(){
    var fbos = [];
    //first make FBOs for 4 and 3 arity vectors
    vectors[4].forEach(function(name){ /* make RGBA FBO */ });
    vectors[3].forEach(function(name){/* make RGB FBO */});
    //pack the rest
    var stuff = [];
    Array.prototype.push.call(stuff,vectors[2].map(function(name){ return {arity:2, name:name}; }));
    Array.prototype.push.call(stuff,vectors[1].map(function(name){ return {arity:1, name:name}; }));
    //
    var available = 0,
        rgba = null,
        vector = null;
    while(stuff.length > 0){
        if(available <= 0){
            //depleted
            if(rgba!==null){
                fbos.push(rgba);
            }
            available= 4;
            rgba = /* make RGBA FBO */;
        }
        //find vector to pack
        var packet = -1;
        for(var i=0; i<stuff.length; i++){
            vector = stuff[i];
            if(vector.arity === available){ //exact fit
                packet = i;
                break;
            }else if(vector.arity < available && (packet<0 || stuff[packet].arity < vector.arity) ){
                packet = i;
            }
        }
        //
        if(packet >= 0){
            //we have a vector to place
            vector = stuff[packet];
            var index = ['r','g','b','a'][4-available];
            available -= vector.arity;
            stuff.splice(packet,1);
            rgba.reserve(index,vector.arity, vector.name);
        }else{
            //couldn't fit any vector in the remaining space, start new fbo
            available = 0;
        }
    }
    if(rgba!==null){
        fbos.push(rgba);
    }
    return fbos;
}