cesium-unreal
cesium-unreal copied to clipboard
CesiumGS
Allow saving and reloading of mesh paint/vertex paint data
While it is currently possible to vertex paint on Cesium3DTilesets, this paint data is lost whenever the tiles are fully reloaded, like playing the level or closing and reopening the project. We'll need to find a way to save this data in order to enable texture painting on terrain, or painting water masks.
I was speaking about this with @nithinp7, and I'll include a quote from him here: "Until we implement full 3D tiles retexturing, maybe we could save the overwritten textures to disk locally and stream them back in a similar way to how Bao's caching system works. "
Additionally, currently texture painting only applies to a given LOD of a tile - if you move closer or farther, the texture painting for that tile vanishes when a different tile is loaded in.
Maybe we could look into saving volume textures for vertex paint data. If we can re-apply vertex colors based on their worldspace position sampling from a 3D volume texture, I'm guessing it would apply the same vertex color regardless of LOD and we wouldn't even need to save the original texture, we could just have it overlay whatever was already there. Depending on the size of the area in question, though, the volume texture could end up prohibitively big.
We also talked offline about propogate-to-ancestors and propogate-to-children functionality where the textures gets downsampled or upsampled respectively to generate painted texture overlays for the other lods. I think this would be fairly straightforward on terrain but may pose a challenge for 3D tiles in general.
One idea might be to predefine an orthographic projection direction in ECEF for how textures should map to the geometry of parents/children. For the tile that was directly painted, it can probably be kept as-is.
@baothientran What do you think about this idea Bao? I'm not sure what changes Alex made to the parent material to allow this to be possible, but I imagine the painted changes are on a separate texture. When the tile's instanced material gets discarded, the painted changes will be gone the next time it is recreated. So maybe we can create another local database on disk to save the painted textures. So then later when recreating a material instance for a given tile, we can query into the database to see if there are custom painted overlays to drape over the regular imagery for that tile.
Writing into the database can happen in editor-time as someone gets done painting the terrain (probably can happen as a bulk-write after the user is done painting). The database will be read-only when the terrain is not being painted, like during playmode.
I'm not sure actually until we prototype it. But be careful about saving it into local disk though since some tileset can be very large to even fit into local machine (For example, the CWT terrain currently are more than 1 TB currently). But for small one, we may get away with it
Also some tilesets especially photogrammetry can have a lot of tile within a small area (if the log on unreal is right, after I tune down the Max SSE to 3, there are more than 200 tiles that are visited)
That's true, but I think here we would only have to save the texture for tiles that get painted on. After talking to Alex about it, specifically we would need to save a texture reference (maybe some sort of id, but not the full texture) and a vertex-painted opacity mask. If we know we completely want to override a tile's regular raster imagery (issue for that here), we can just set a flag instead of wasting space on an opacity mask. But the idea is, whatever kind of texture we use, dirt, mud, grass, etc. it will be very repetitive so we only need one copy of each used texture for the entire tileset. What we do need to save on per-tile basis in the database (still only for tiles that were painted on) is the opacity mask and possibly the uv coordinates.
This essentially how I modified the RasterOverlay of the material to paint vertex colors on it - though this example just shows one color/texture, that's the core of how you do it. Just putting this here for reference.
And I agree that whatever our solution, we should be careful not to allow people to save massive tilesets accidentally. You can technically flood a mesh with a vertex color, so they could potentially make something massive very quickly.
Hello, any update on this? What about adding to heightmap to get some element above water?
