Instancing using instanced vertex attributes

[TimerErTim]

Summary

I want to implement a non-breaking method for instanced draw calls without the size limit of uniform arrays. For this, I'd like to use instanced arrays as vertex attributes.

The proposed API would look as follows:

let shader = Shader::from_vertex_file(ctx, "path/to/vertex_file")?;
shader.set_instanced_array(ctx, "a_offsets", offsets.as_slice()); // Offsets could be a Vec of any OpenGL attribute type
graphics::set_shader(ctx, &shader);
...
mesh.draw_instanced(ctx, 10_000, Vec2::new(16.0, 16.0));

It is very similar to the current uniform approach. Therefore offsets in this concrete example requires at least 10000 entries.

I am opening an issue for this, as I'd like to inform you about my intentions beforehand and because you may have some feedback or concerns regarding the API design or even overall idea.

Sidenote: As I am very new to OpenGL and barely having much freetime due to school, side job and my diploma thesis, this contribution may well take some time till it is complete.

Motivation/Examples

I'd need this change for a game prototype I am writing. I need to draw the same object 100s of 1000s of times. This leaves quite a performance impact. Instancing would be the ideal solution, but tetra currently only supports instancing using uniform arrays, which have quite a significant size limitation, only allowing some 1000s instances. Using instanced arrays, this limit can practically be (almost) completely removed, allowing even more performance gain due to even less draw calls.

One concrete use case would be very heavy particle effects, where half a million equal particles need to be drawn at the same time. Or for really suffisticated cellular automata simulations, where cellular states are better not to be stored as color in a texture but a more complex object.

Alternatives Considered

Two alternative API designs I've considered are the following:

let shader = Shader::from_vertex_file(ctx, "path/to/vertex_file")?;
graphics::set_shader(ctx, &shader);
...
let mut offset_attribute = Vec::new();
for i in 0..10_000 {
    offset_attribute.push(some_offset); // Push offsets to attribute
}
let mut attributes = Vec::new();
attributes.push((3, offset_attribut)); // Push attribute location and per instance values

graphics::set_instances(ctx, 10_000, attributes); // All draw calls inside will be instanced 10_000 times using the given attributes
mesh.draw(ctx, Vec2::new(16.0, 16.0)); // Will be drawn 10000 times
graphics::reset_instances(ctx); // All subsequent draw calls will be handled normally

But I think this is too long and stateful.

let shader = Shader::from_vertex_file(ctx, "path/to/vertex_file")?;
graphics::set_shader(ctx, &shader);
...
let mut offset_attribute = Vec::new();
for i in 0..10_000 {
    offset_attribute.push(some_offset); // Push offsets to attribute
}
let mut attributes = Vec::new();
attributes.push((3, offset_attribut)); // Push attribute location and per instance values

mesh.draw_instanced_array(ctx, 10_000, Vec2::new(16.0, 16.0), attributes); // Draws the mesh 10000 times using the given attributes

This is concise and effective, but in my humble opinion the originally proposed design better integrates into the current "workflow" for instancing.

[17cupsofcoffee]

I'm definitely on board with the idea of adding something like this, as Tetra's instancing support is fairly limited at the moment.

I'm not sure if Shader is the right place for it, though - adding instanced vertex data (as I understand it) involves binding another vertex buffer, which feels like more like something the Mesh API would handle to me? This is how it works in Love2D, which is what I based a lot of Tetra's API on:

https://love2d.org/wiki/love.graphics.drawInstanced#Examples

Looking at that, I think there's two missing pieces here:

• There's no way to create vertex buffers with non-Vertex data.
• There's no way to bind a buffer containing per-instance data to a Mesh.

So that might be the best way to break the problem down.

This does seem like it has potential to be a large change, so I should warn in advance that I don't know when I'll have the time/energy to review it, but feel free to give it a go if you'd still find it interesting!

[TimerErTim]

I have already conceptualized another API proposal. It addresses some of the problems you mentioned. I will go into further detail further down below.

I'd argue that Shader is the right place for it (at the very least with the new API proposal). Mainly because of three points:

• The shader code actually accesses and defines the used attributes (same as with uniforms, even though it is handled differently under the hood).
• Binding the buffer to the shader instead of the mesh allows more intuitive usage of the same shader program, which handles the instancing logic, with the same data while used with different meshes.
• Binding the buffer to the shader allows to easily set the same buffer and therefore data for different shader programs with eventually different attribute names.

One downside is the inconvenience arising for the user, who then has to manually switch data for the buffers (or the buffers themselves) when he wants to draw multiple instanced meshes with the same shader.

The new API would look as follows:

fn new(ctx: &mut Context) -> tetra::Result<GameState> {
  // Create the buffer with an initial capacity of 10,000 Vec2 and optimization for streamed drawing
  let offset_buffer: AttributeBuffer<Vec2<f32>> =
    AttributeBuffer::new(ctx, Vec::<Vec2<f32>>::with_capacity(10_000), BufferUsage::Stream);
  // Load the shader program
  let shader = Shader::from_vertex_file(ctx, "path/to/vertex/shader.vert")?;
  // Bind the buffer to an attribute defined in the shader program
  shader.set_vertex_attribute(ctx, "a_offsets", offset_buffer, Divisor::Instance(1));
  graphics::set_shader(ctx, &shader);
  
  // Optionally, the offset_buffer can be stored in the state for later modification during the game loop
  // This would not be necessary if we created a static buffer which is only bound during initialization
  Ok(GameState{
    offset_buffer
  }
}
// ...
fn draw(&mut self, ctx: &mut Context) -> tetra::Result {
  let mut offsets = Vec::new();
  // Push offsets here
  
  // Set data of offset_buffer
  self.offset_buffer.set_data(ctx, offsets);
  // Draw an instanced mesh, which is being processed by the currently active shader
  some_mesh.draw_instanced(ctx, offsets.len(), DrawParams::default());
}

The buffer creation:

let offset_buffer: AttributeBuffer<Vec2<f32>> =
    AttributeBuffer::new(ctx, Vec::<Vec2<f32>>::with_capacity(10_000), BufferUsage::Stream);

can be done using the three types of usage: Stream, Dynamic and Static. It can also be done with initial data by invoking the method with a Vec<Vec2<f32>> which actually contains values.

The Divisor part in this line:

shader.set_vertex_attribute(ctx, "a_offsets", offset_buffer, Divisor::Instance(1));

could be replaced with Divisor::Vertex, when the attribute should advance along the buffer once per vertex. Divisor::Instance(x) advances the attribute once every x instances.

Lastly setting the data in this line:

self.offset_buffer.set_data(ctx, offsets);

would automatically grow the buffer size if the Vec::len() of the given data exceeds the current buffer capacity. The new buffer capacity would be set to Vec::capacity() of that given data. However, that is just an implementation detail which could easily be changed.

[TimerErTim]

As I just got assigned a new project in school while barely having much free time at all I don't expect to be capable of working on this in the next upcoming semester :(