add_mesh_trimesh ignores transparency from trimesh RGBA colors

[Noietch]

Describe the bug

When a trimesh.Trimesh object is created with RGBA face colors (where the alpha channel defines transparency), and then visualized using server.scene.add_mesh_trimesh, the transparency information is ignored. The resulting mesh in the viser viewer is always rendered as fully opaque.

To Reproduce

This simple script creates three trimesh cylinders. Each is assigned a color with a different alpha value for transparency. When visualized, none of them appear transparent.

#!/usr/bin/env python3
"""
Simple script to test transparency with trimesh objects in viser.
"""
import viser
import trimesh
import numpy as np
import time

def main():
    """Sets up the viser server and adds trimesh objects with alpha values."""
    print("\n=== Testing trimesh RGBA color settings ===")
    server = viser.ViserServer()

    # Create three cylinders
    mesh1 = trimesh.creation.cylinder(radius=0.5, height=2.0, sections=16)
    mesh2 = trimesh.creation.cylinder(radius=0.5, height=2.0, sections=16)
    mesh3 = trimesh.creation.cylinder(radius=0.5, height=2.0, sections=16)

    # Set RGBA colors, where the 4th value is alpha (0-255)
    # 76/255 is ~30% opaque
    mesh1.visual.face_colors = [255, 0, 0, 76]
    # 153/255 is 60% opaque
    mesh2.visual.face_colors = [0, 255, 0, 153]
    # 255/255 is 100% opaque
    mesh3.visual.face_colors = [0, 0, 255, 255]

    # Add meshes to the scene
    server.scene.add_mesh_trimesh(
        "red_transparent",
        mesh1,
        position=(-1.5, 0.0, 0.0)
    )
    server.scene.add_mesh_trimesh(
        "green_transparent",
        mesh2,
        position=(0.0, 0.0, 0.0)
    )
    server.scene.add_mesh_trimesh(
        "blue_opaque",
        mesh3,
        position=(1.5, 0.0, 0.0)
    )

    print("\nViser server started!")
    print("View at http://localhost:8080")
    print("Press Ctrl+C to exit")

    try:
        while True:
            time.sleep(1)
    except KeyboardInterrupt:
        print("\nExiting.")

if __name__ == "__main__":
    main()

[brentyi]

Hi there!

I spent some time looking into this. Here's a quick workaround:

mesh1.visual = mesh1.visual.to_texture()
mesh2.visual = mesh2.visual.to_texture()
mesh3.visual = mesh3.visual.to_texture()

Some notes on the root cause:

• We rely on trimesh's glTF export functionality for sending assets to Viser.
• When mesh.visual is set to an instance of ColorVisuals, trimesh doesn't populate the "materials" field of the glTF output.
• When there's no material available, the three.js glTF loader uses the standard material. This doesn't support transparency.
• Converting to TextureVisuals via .to_texture() fixes the problem.

I filed a trimesh issue: https://github.com/mikedh/trimesh/issues/2436, we can figure out next steps based on the response there.

[Noietch]

Thanks for your reply, it works for me !

[brentyi]

I was revisiting this today. Couldn't reproduce the fix above 🤔

New one:

    mesh1.visual = mesh1.visual.to_texture()
    mesh2.visual = mesh2.visual.to_texture()
    mesh3.visual = mesh3.visual.to_texture()

    mesh1.visual.material = mesh1.visual.material.to_pbr()
    mesh2.visual.material = mesh2.visual.material.to_pbr()
    mesh3.visual.material = mesh3.visual.material.to_pbr()

    mesh1.visual.material.alphaMode = "BLEND"
    mesh2.visual.material.alphaMode = "BLEND"
    mesh3.visual.material.alphaMode = "BLEND"