etherealengine
etherealengine copied to clipboard
EtherealEngine
[Epic] WebGPU extends the LOD system with Mesh Shaders like UE5 Nanite
ChatGPT wrote a ticket
Dependent on
- [ ] Amazing LOD system
Stories
Mesh shaders will expand the capabilities and performance of the geometry pipeline. Mesh shaders incorporate the features of Vertex and Geometry shaders into a single shader stage through batch processing of primitives and vertices data before the rasterizer. The shaders are also capable of amplifying and culling geometry.
Creating a complete code example for a dynamic level of detail (LOD) system like Unreal Engine 5's Nanite is a complex task that involves many parts of a rendering engine, including shaders, asset management, and runtime systems.
However, I can provide a high-level overview of how you can implement a simple LOD system in your Vulkan application using mesh shaders:
- Create a 3D model with multiple LODs. You can use a 3D modeling tool to generate the different LODs or use an automatic LOD generator.
- Load the 3D model and its LODs into your application. Store the LODs as separate vertex/index buffers.
- Calculate the LOD level based on the camera distance to the object. Use a simple heuristic, such as switching to a lower LOD when the object's screen space size is smaller than a certain threshold.
- Pass the selected LOD's vertex/index buffer to the mesh shader.
Here's a high-level example of a mesh shader that handles multiple LODs:
#version 460
#extension GL_NV_mesh_shader : enable
layout(local_size_x = 32, local_size_y = 1, local_size_z = 1) in;
layout(max_vertices = 3, max_primitives = 1, triangles) out;
layout(location = 0) out vec3 outColor;
layout(set = 0, binding = 0) buffer VertexBuffer {
vec4 positions[];
};
layout(set = 0, binding = 1) buffer ColorBuffer {
vec3 colors[];
};
layout(set = 0, binding = 2) buffer IndexBuffer {
uint indices[];
};
layout(push_constant) uniform Params {
uint lodOffset; // The starting index for the selected LOD in the index buffer
uint lodCount; // The number of vertices for the selected LOD
} params;
void main() {
uint vertexID = gl_LocalInvocationID.x;
if (vertexID < params.lodCount) {
uint index = indices[params.lodOffset + vertexID];
gl_Position = positions[index];
outColor = colors[index];
gl_PrimitiveID = 0;
gl_Layer = 0;
EmitVertex();
}
}
In this example, the shader takes the selected LOD's offset and count in the index buffer as input. The application calculates the appropriate LOD level based on the camera distance and passes it to the shader via push constants.
Keep in mind that this is a high-level example and there are several other components to implement in your Vulkan application, such as descriptor sets, vertex/index buffers, and push constants. Additionally, this example does not cover more advanced features of Unreal Engine 5's Nanite, such as virtual texturing or cluster culling. Implementing these features would require a more complex system and deeper integration with your rendering engine.
What is virtual texturing and cluster culling? Can you tell me how I could implement this with my own level of detail system?
Virtual texturing and cluster culling are advanced techniques that help to optimize rendering performance and memory usage in modern game engines. Let me briefly explain each technique and give you some guidance on how to implement them with your level of detail (LOD) system.
Virtual texturing: Virtual texturing, also known as "sparse texturing" or "mega-texturing," is a technique that allows efficient management of large textures by loading only the portions visible in the current view. The texture is divided into smaller tiles, and only the tiles needed for rendering are loaded into memory. This technique is especially useful when working with high-resolution textures or large environments where only a small portion of the texture data is visible at any given time.
To implement virtual texturing in your LOD system:
a. Create a virtual texture atlas by dividing your textures into smaller tiles and storing them in a texture array or a larger 2D texture. b. Modify your shaders to perform a virtual texture lookup, calculating the correct tile coordinates and the corresponding index in the texture array or coordinates in the larger 2D texture. c. In your application, determine which tiles are required for the current view and load them into memory. You can use a simple streaming system or more advanced techniques, such as a Least Recently Used (LRU) cache, to manage the tiles in memory. d. Update your shaders with the information about the loaded tiles, such as texture coordinates and tile indices.
Cluster culling: Cluster culling is a technique used to optimize the rendering of large scenes by reducing the number of draw calls and the amount of geometry processed by the GPU. In cluster culling, the scene geometry is divided into smaller clusters (e.g., an octree or BVH), and each cluster's visibility is determined using frustum culling, occlusion culling, or other visibility tests. Only the visible clusters are processed and rendered by the GPU.
To implement cluster culling in your LOD system:
a. Divide your 3D model into smaller clusters. You can use spatial data structures such as an octree, bounding volume hierarchy (BVH), or a simple grid. b. For each cluster, determine the appropriate LOD based on the camera distance. c. Perform visibility tests (e.g., frustum culling, occlusion culling) for each cluster to determine which clusters should be rendered. d. Update your draw calls to only render the visible clusters with the appropriate LOD.
Combining virtual texturing and cluster culling with your LOD system can significantly improve the performance and memory usage of your application, especially when dealing with large and complex scenes. Note that these techniques are advanced and may require a deep understanding of graphics programming and rendering engines.
How can I make a rendering engine for WebGL based on three js with a dynamic level of detail (LOD) system like Unreal Engine 5's Nanite using vertex shaders, Virtual texturing and cluster culling. Assume the web 3D game engine has a working asset management system and Entity Component System (ECS). Creating a WebGL rendering engine with a dynamic LOD system like Unreal Engine 5's Nanite involves multiple steps and components. This guide will give you a high-level overview of how to integrate LOD, virtual texturing, and cluster culling into a Three.js-based engine. Note that this is a complex task that requires a solid understanding of graphics programming, WebGL, and Three.js.
Dynamic LOD system: a. Prepare 3D models with multiple LODs. You can create these manually or use an automatic LOD generator. b. Load the 3D models and their LODs into your engine. Store each LOD in separate BufferGeometry objects. c. Create a custom LOD class that extends THREE.Object3D. This class should manage the LOD levels and update the visible geometry based on the camera distance. d. Replace the default THREE.Mesh instances in your scene with instances of your custom LOD class.
Virtual texturing: a. Divide your textures into smaller tiles and store them in a texture atlas or a larger 2D texture. You can use a tool to preprocess your textures and generate the necessary metadata. b. Modify your shaders to perform a virtual texture lookup, calculating the correct tile coordinates and the corresponding index in the texture atlas or the larger 2D texture. c. In your engine, determine which tiles are required for the current view and load them into memory. Use a simple streaming system or an LRU cache to manage the tiles in memory. d. Update your shaders with the information about the loaded tiles, such as texture coordinates and tile indices.
Cluster culling: a. Divide your 3D models into smaller clusters. Use spatial data structures such as an octree, BVH, or a grid. b. For each cluster, determine the appropriate LOD based on the camera distance. c. Perform visibility tests (e.g., frustum culling, occlusion culling) for each cluster to determine which clusters should be rendered. d. Update your draw calls to only render the visible clusters with the appropriate LOD.
Integrate the components: a. Modify your custom LOD class to use the cluster culling and virtual texturing techniques. b. Update your shaders to include the virtual texturing and LOD information. c. Adjust your rendering pipeline to handle the dynamic LOD, virtual texturing, and cluster culling. d. Optimize the performance by minimizing draw calls, using instancing, and other WebGL-specific techniques.
Keep in mind that this is a high-level overview, and implementing these features requires a deep understanding of graphics programming, WebGL, and Three.js. Each component can be quite complex, and integrating them into a working engine will likely involve solving various challenges and performance optimizations.
How do you transition rendering between 3 levels of detail of 3d object geometry using GLSL vertex shaders? Assume we are doing Cluster culling with a BVH. Can you write the javascript and GLSL code? To transition smoothly between three levels of detail (LODs) using GLSL vertex shaders, you can perform linear interpolation between LOD levels based on the distance from the camera. This technique is called geometric morphing. Note that this assumes your LODs have a matching number of vertices and a corresponding vertex mapping between different LOD levels.
Here's a high-level example of how to do this using WebGL, JavaScript, and GLSL:
JavaScript code for setting up the WebGL buffers and shaders:
// Load 3D models with 3 LODs
// Assume `modelLOD0`, `modelLOD1`, and `modelLOD2` are arrays of vertex positions
// Create WebGL buffers for LODs
const vertexBufferLOD0 = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vertexBufferLOD0);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(modelLOD0), gl.STATIC_DRAW);
const vertexBufferLOD1 = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vertexBufferLOD1);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(modelLOD1), gl.STATIC_DRAW);
const vertexBufferLOD2 = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vertexBufferLOD2);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(modelLOD2), gl.STATIC_DRAW);
// Create a shader program
const vertexShader = gl.createShader(gl.VERTEX_SHADER);
gl.shaderSource(vertexShader, vertexShaderSource); // vertexShaderSource is the GLSL code from step 2
gl.compileShader(vertexShader);
const fragmentShader = gl.createShader(gl.FRAGMENT_SHADER);
gl.shaderSource(fragmentShader, fragmentShaderSource);
gl.compileShader(fragmentShader);
const shaderProgram = gl.createProgram();
gl.attachShader(shaderProgram, vertexShader);
gl.attachShader(shaderProgram, fragmentShader);
gl.linkProgram(shaderProgram);
gl.useProgram(shaderProgram);
// Get attribute and uniform locations
const positionAttribLocation = gl.getAttribLocation(shaderProgram, "a_position");
const positionLOD1AttribLocation = gl.getAttribLocation(shaderProgram, "a_positionLOD1");
const positionLOD2AttribLocation = gl.getAttribLocation(shaderProgram, "a_positionLOD2");
const morphFactorUniformLocation = gl.getUniformLocation(shaderProgram, "u_morphFactor");
GLSL code for the vertex and fragment shaders:
// Vertex shader
precision mediump float;
attribute vec3 a_position;
attribute vec3 a_positionLOD1;
attribute vec3 a_positionLOD2;
uniform float u_morphFactor; // A value between 0.0 and 2.0, where 0.0 represents LOD0, 1.0 represents LOD1, and 2.0 represents LOD2
varying vec3 v_color;
void main() {
vec3 position;
if (u_morphFactor < 1.0) {
position = mix(a_position, a_positionLOD1, u_morphFactor);
} else {
position = mix(a_positionLOD1, a_positionLOD2, u_morphFactor - 1.0);
}
gl_Position = vec4(position, 1.0);
v_color = vec3(u_morphFactor * 0.5, 1.0 - u_morphFactor * 0.5, 0.0);
}
// Fragment shader
precision mediump float;
varying vec3 v_color;
void main() {
gl_FragColor = vec4(v_color, 1.0);
}
JavaScript code for updating the vertex attributes and geometric morphing:
function updateMorphFactor(distance) {
// Calculate the morph factor based on the distance to the camera
// This is a simple example; you can use more advanced techniques to determine the morph factor
const minDistance = 10.0;
const maxDistance = 50.0;
const morphFactor = 2.0 * (distance - minDistance) / (maxDistance - minDistance);
// Set the morph factor uniform value
gl.uniform1f(morphFactorUniformLocation, morphFactor);
}
function drawScene() {
// Set up the vertex attribute pointers
gl.bindBuffer(gl.ARRAY_BUFFER, vertexBufferLOD0);
gl.vertexAttribPointer(positionAttribLocation, 3, gl.FLOAT, false, 0, 0);
gl.enableVertexAttribArray(positionAttribLocation);
gl.bindBuffer(gl.ARRAY_BUFFER, vertexBufferLOD1);
gl.vertexAttribPointer(positionLOD1AttribLocation, 3, gl.FLOAT, false, 0, 0);
gl.enableVertexAttribArray(positionLOD1AttribLocation);
gl.bindBuffer(gl.ARRAY_BUFFER, vertexBufferLOD2);
gl.vertexAttribPointer(positionLOD2AttribLocation, 3, gl.FLOAT, false, 0, 0);
gl.enableVertexAttribArray(positionLOD2AttribLocation);
// Clear the canvas and draw the object
gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);
gl.drawArrays(gl.TRIANGLES, 0, modelLOD0.length / 3);
}
function render() {
// Calculate the distance from the camera to the object
// Assuming `cameraPosition` and `objectPosition` are vec3 objects
const distance = vec3.distance(cameraPosition, objectPosition);
// Update the morph factor based on the distance
updateMorphFactor(distance);
// Draw the scene with the updated morph factor
drawScene();
// Request the next frame
requestAnimationFrame(render);
}
// Start the rendering loop
render();
In this example, the updateMorphFactor function calculates the morph factor based on the distance between the camera and the object. The morph factor is a value between 0.0 and 2.0, where 0.0 represents LOD0, 1.0 represents LOD1, and 2.0 represents LOD2. The drawScene function sets up the vertex attribute pointers and draws the object with the updated morph factor. Finally, the render function calculates the distance to the object, updates the morph factor, draws the scene, and requests the next frame.
This example demonstrates the basic idea of transitioning between 3 levels of detail using GLSL vertex shaders and geometric morphing. Note that this technique assumes that the LODs have a matching number of vertices and a corresponding vertex mapping between different LOD levels. You may need to adjust this example to fit your specific use case and optimize it for performance, as well as integrate it with your BVH and cluster culling system.
What is the morph factor?
The morph factor is a scalar value used to determine the interpolation weight between two levels of detail (LODs) in a 3D object when using geometric morphing techniques. Geometric morphing is the process of smoothly transitioning between different LODs by interpolating vertex positions. The morph factor is used to control the blending between vertex positions of the two LODs.
The morph factor typically ranges from 0.0 to 1.0:
- A morph factor of 0.0 means that the first LOD (LOD0) is used exclusively, without any blending.
- A morph factor of 1.0 means that the second LOD (LOD1) is used exclusively, without any blending.
- A morph factor between 0.0 and 1.0 means that a mix of the two LODs is used, with the vertex positions interpolated based on the value of the morph factor.
In the provided example, the morph factor was extended to range from 0.0 to 2.0 to handle three LODs (LOD0, LOD1, and LOD2). The morph factor is used in the GLSL vertex shader to smoothly interpolate vertex positions between the LODs, creating a seamless transition between different levels of detail.
Relevant https://github.com/gpuweb/gpuweb/discussions/4236 Links to milestone: https://github.com/gpuweb/gpuweb/issues/3015
