GaussianSplats3D icon indicating copy to clipboard operation
GaussianSplats3D copied to clipboard

How to get DepthTexture of the GaussianSplats?

Open Pure0319 opened this issue 11 months ago • 5 comments

Thanks for your contribution!

i am trying to render the depth of the Splats to the screen;

i use the postprocess of the threejs like this

class DepthPass extends Pass {
 
constructor( scene, camera ) {

		super();

		this.scene = scene;
		this.camera = camera;


		// render targets

		this.renderTargetDepth = new WebGLRenderTarget( 1, 1, { // will be resized later
			minFilter: NearestFilter,
			magFilter: NearestFilter,
			type: HalfFloatType
		} );

		//this.renderTargetDepth.texture.name = 'BokehPass.depth';
		this.renderTargetDepth.texture.value = '1';

		// depth material

		this.materialDepth = new MeshDepthMaterial();
		this.materialDepth.depthPacking =RGBADepthPacking;
		this.materialDepth.blending = NormalBlending;

		
		this.renderTargetDepth.depthTexture = new DepthTexture();
		this.renderTargetDepth.format = THREE.DepthFormat;
		this.renderTargetDepth.type = THREE.UnsignedShortType;

		// bokeh material

		

        this.uniforms= {
            'tColor': { value: null },
            'tDepth': { value: this.renderTargetDepth.depthTexture },
            'nearClip': { value: 1.0 },
            'farClip': { value: 10.0 },
                }

		this.materialBokeh = new ShaderMaterial( {
			defines: Object.assign( {},  {
                'DEPTH_PACKING': 1,
                'PERSPECTIVE_CAMERA': 1,
            }),
			uniforms:this.uniforms,
			vertexShader: `
        varying vec2 vUv;
        void main() {
            vUv = uv;
            gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1);
        }
        `,
		fragmentShader: `
        #include <packing>
        #include <common>
        uniform vec3 color;
        uniform sampler2D tColor;
        uniform sampler2D tDepth;
        varying vec2 vUv;
        uniform float nearClip;
        uniform float farClip;

        float getDepth( const in vec2 screenPosition ) {
            return unpackRGBAToDepth( texture2D( tDepth, screenPosition ) );
            }

        float readDepth( float depth ) {
                float viewZ = perspectiveDepthToViewZ( depth, nearClip, farClip );
                return viewZ;
            }
        void main() {
            float depthcolor = getDepth( vUv );

            depthcolor=depthcolor;
            vec4 previousPassColor = texture2D(tColor, vUv);
            if(depthcolor==1.0)
			{
				gl_FragColor=vec4(0.0,0.0,0.0,0.0);
				return;
			}
             gl_FragColor =texture2D(tDepth, vUv);
			
            //gl_FragColor = previousPassColor;
            //gl_FragColor = vec4(previousPassColor.x,previousPassColor.y,previousPassColor.z,1.0);
        }  `
		} );

		

		this.fsQuad = new FullScreenQuad( this.materialBokeh );

		this._oldClearColor = new Color();

	}

	render( renderer, writeBuffer, readBuffer/*, deltaTime, maskActive*/ ) {

		// Render depth into texture

		//this.scene.overrideMaterial = this.materialDepth;
		//let target=this.scene.overrideMaterial.getRenderTarget()

		renderer.getClearColor( this._oldClearColor );
		const oldClearAlpha = renderer.getClearAlpha();
		const oldAutoClear = renderer.autoClear;
		renderer.autoClear = false;

		renderer.setClearColor( 0xffffff );
		renderer.setClearAlpha( 1.0 );
		renderer.setRenderTarget( this.renderTargetDepth );
		renderer.clear();
		renderer.render( this.scene, this.camera );

		// Render bokeh composite

		this.uniforms[ 'tColor' ].value = readBuffer.texture;
		this.uniforms[ 'nearClip' ].value = this.camera.near;
		this.uniforms[ 'farClip' ].value = this.camera.far;
		this.uniforms[ 'tDepth' ].value = this.renderTargetDepth.texture;

		if ( this.renderToScreen ) {

			renderer.setRenderTarget( null );
			this.fsQuad.render( renderer );

		} else {

			renderer.setRenderTarget( writeBuffer );
			renderer.clear();
			this.fsQuad.render( renderer );

		}

		this.scene.overrideMaterial = null;
		renderer.setClearColor( this._oldClearColor );
		renderer.setClearAlpha( oldClearAlpha );
		renderer.autoClear = oldAutoClear;

	}

	setSize( width, height ) {

	

		this.renderTargetDepth.setSize( width, height );

	}

	dispose() {

		this.renderTargetDepth.dispose();

		this.materialDepth.dispose();
		this.materialBokeh.dispose();

		this.fsQuad.dispose();

	}

}

export { DepthPass };

and then i add this pass to the composer

    const depthPass = new DepthPass(this.scene, this.camera);
    this.composer.addPass( depthPass ); 
    this.composer.render();

but the result of depth just show the cube i created
image image

how can i get the DepthTexture of the splat? Thanks

Pure0319 avatar Jan 09 '25 02:01 Pure0319

Haven't used composer myself (yet), @jordaneast1 did in #271

seppestaes avatar Jan 09 '25 10:01 seppestaes

Haven't used composer myself (yet), @jordaneast1 did in #271

Thanks for your reply

I do not care the way to get the depthtexture i have tried composer and RenderTarget.DepthTexture
but both of them did not work at all

Have you have any idea about that?

Pure0319 avatar Jan 09 '25 12:01 Pure0319

#372 "the splats do not render to the depth buffer (as is typical with transparent objects)" (@mkkellogg )

seppestaes avatar Jan 09 '25 18:01 seppestaes

#372 "the splats do not render to the depth buffer (as is typical with transparent objects)" (@mkkellogg )

But I think the depth sort works well The Black cube is another threejs Mesh but it shows right occulation relationship with the splat Dose it mean the splats render to the depth buffer? maybe the author add this part in later version? 2)){17H1KZLHT3CY(CL~1~2 Y}LFG 9 ~NQ5XSZV56GN@Q

Pure0319 avatar Jan 13 '25 04:01 Pure0319

"You definitely have to be careful with post-processing effects, especially when they depend on depth buffer data; they're not a one-size-fits-all solution. The issue with combining splats and post-processing results is that the splats are always rendered as transparent objects (no depth-buffer data) and if the results of the post-processing are also transparent, then there will be no way to guarantee proper rendering order (because of the lack of depth buffer data)." #178 (@mkkellogg )

seppestaes avatar Jan 15 '25 11:01 seppestaes