Optimize texture allocation for voxels

[jjhembd]

VoxelPrimitive currently allocates a large Megatexture for each metadata property, which is frequently much larger than the input data (256 MB texture for a 20-30 MB tileset).

The sizing of the texture is constrained by several factors:

• The texture is 2D. The 3D metadata is stored in 2D slices.
• The texture is required to be square. This results in sub-optimal arrangement of the slices, since tiles can have different sizes along each axis.
• The size of the texture is required to be a power of two.

As a result, we are effectively choosing from a very sparse set of tile sizes, i.e., 4, 16, 64, or 256 MB. And with the inefficiencies of the slice arrangement, most real-world tilesets (including small 20-50 MB datasets) will usually allocate the maximum 256 MB texture.

A few peculiarities of the availableTextureMemoryBytes argument in the Megatexture constructor:

• The hardcoded maximum 512 MB is actually rounded down to 256 MB because of the requirement to be square.
• The default 128 MB is also not a square, so it is rounded down to 64 MB.

Options to improve:

1. Allow non-square textures. This would increase the number of possible sizes (for example: 128, 192, 384, 512)
2. Allow non-power-of-two sizes in a WebGL2 context. This could reduce the amount of padded space around the data.
3. Use 3D textures, and drop support for voxels in WebGL1. This could significantly reduce the amount of unused memory, and potentially also improve rendering times -- see https://github.com/CesiumGS/cesium/issues/11086

The first two options would require us to revisit the slice arrangement code, which is somewhat messy. The best ROI might be to jump straight to 3D textures. See https://github.com/CesiumGS/cesium/issues/12550

[Hiwen]

WebGL2 is now widely supported, I think Texture3D is an excellent choice.

[Hiwen]

@jjhembd @ggetz How about we first expand the basic 3D Texture support? And then consider how to modify the rendering of Voxel in the next step? This only requires modifying two files: packages\engine\Source\Renderer\createUniform.js packages\engine\Source\Renderer\createUniformArray.js And add a new file: packages\engine\Source\Renderer\Texture3D.js

[jjhembd]

Hi @Hiwen, doing it in smaller steps sounds like a good idea. However, if we implement 3D Texture support without using it in voxels, how will we test it? We usually want to verify the implementation in 2 ways:

1. A simple Sandcastle that uses the new feature, that we can use to confirm it works as expected.
2. An even simpler and smaller unit test, to make sure nothing breaks in the future.

If we follow your proposed structure for the file chnages, we will need to add another file Texture3DSpec. See https://github.com/CesiumGS/cesium/tree/main/Documentation/Contributors/TestingGuide#writing-tests.

[Hiwen]

@jjhembd Thanks for reply. And that makes sense. I'll take some time to learn about the rendering of Voxel. However, in any case, Texture3DSpec is necessary. Perhaps we can refer to the logic of TextureSpec to test Texture3D. And we can transplant an example of volume rendering in Three.js as the Sandcastle example.

[ggetz]

We should probably continue this discussion back in https://github.com/CesiumGS/cesium/issues/12550. I added a comment there that clouds could be a potential example use case that would be simpler to understand and more isolated than voxels.

[lilleyse]

@jjhembd Did https://github.com/CesiumGS/cesium/pull/12592 mean to target this issue?

[jjhembd]

@lilleyse Sorry, I tagged the wrong issue in an unrelated PR

[jjhembd]

I have a preliminary prototype in the voxel-texture3d branch that uses Texture3D for voxel rendering. Here is a testing Sandcastle that loads procedural data.

There are a few bugs that need to be worked out already. For example, the texture lookup is wrong if provider.maximumTileCount is not set. But the VoxelProvider interface allows maximumTileCount to be undefined.

I'm sure I'll find more issues as I test more datasets.

[Hiwen]

I have a preliminary prototype in the voxel-texture3d branch that uses Texture3D for voxel rendering. Here is a [testing Sandcastle that loads procedural data]

That's a big step!

[jjhembd]

Update: the problem with missing maximumTileCount is fixed. Remaining TODO:

• [x] Fix nearest sampling (currently linear sampling only)
• [ ] Test more datasets
• [ ] Get statistics for memory savings
• [x] Test for rendering framerate improvement -- no significant change
• [x] Fix spec for Megatexture.get3DTextureDimension: stub ContextLimits to get a consistent value on different machines
• [x] Use VoxelProvider.prototype.availableLevels to guess maximumTileCount where missing. This will help limit memory allocation for more datasets.
• [x] ~Fix clamping of tile coordinates when nearestSampling == false and padding is non-zero~ Megatexture.glsl was correctly clamping to the padded tile dimensions, so there was no issue here
• [x] Clean up duplicate properties, like Megatexture.voxelCountPerTile and VoxelPrimitive._inputDimensions