webgpu-samples icon indicating copy to clipboard operation
webgpu-samples copied to clipboard

Texture3d, readPixel example

Open munrocket opened this issue 3 years ago • 8 comments

This examples can be useful.

Some code with readPixel is here, but I don’t understand yet how to do it.

  • https://github.com/tensorflow/tfjs/blob/e295cdb7ca2c2f2d9e7b429606159933d8582d34/tfjs-backend-webgpu/src/backend_webgpu.ts
  • https://github.com/BabylonJS/Babylon.js/commit/ce849b8f47dd5c28ec197301e0ec7b8f95bf31d4

munrocket avatar Jul 22 '21 11:07 munrocket

For readPixel we need to change context usage for example to { usage: GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.COPY_SRC }. Here additional samples with 'toDataURL', 'toBlob', 'imageBitmap' https://github.com/gpuweb/cts/blob/c84e2748ec4ed1daff7cb836c5c7d1a01664be28/src/webgpu/web_platform/canvas/readbackFromWebGPUCanvas.spec.ts

munrocket avatar Aug 27 '21 19:08 munrocket

These first two examples don't appear to be doing a (WebGL 1.0) "readPixels" style operation.

  • https://github.com/tensorflow/tfjs/blob/e295cdb7ca2c2f2d9e7b429606159933d8582d34/tfjs-backend-webgpu/src/backend_webgpu.ts

I don't know what in this file you're pointing to. Is it getBufferData? That is async, using copyBufferToBuffer and mapAsync(READ). It's reading raw buffer data, not texture (pixel) data.

This is also async, using copyTextureToBuffer + mapAsync(READ).

For readPixel we need to change context usage for example to { usage: GPUTextureUsage.RENDER_ATTACHMENT | GPUTextureUsage.COPY_SRC }.

This is actually a Chromium bug that I believe has been fixed. COPY_SRC is not supposed to be needed to read data out of a WebGPU canvas.

Note that reading data out of WebGPU canvas is NOT the normal (or efficient) way of getting data out of WebGPU - that would be mapAsync(READ).

kainino0x avatar Aug 27 '21 20:08 kainino0x

Hmm, interesting. Reading canvas back is useful for screenshots or video capture.

Month ago I am tried to find any example with it. TF.js/Babylon.js was the first projects where I tried to find something :)

I am not tried mapAsync, but simple solution with COPY_SRC from Marcin Ignac solving issue with screenshots.

munrocket avatar Aug 27 '21 20:08 munrocket

Reading from canvas didn't work in Chromium at all until pretty recently. And then we only fixed the COPY_SRC thing after that. Can you try Marcin's example without COPY_SRC and see if it works?

kainino0x avatar Aug 27 '21 20:08 kainino0x

Hmm, interesting. Reading canvas back is useful for screenshots or video capture.

Yes definitely. There are a bunch of usecases where it makes sense - needing to output image data to other apis (like webgl, 2d canvas, video capture) especially if they might be able to avoid reading the data all the way back to JS. And makes sense for screenshots where you already have data in the canvas anyway.

kainino0x avatar Aug 27 '21 20:08 kainino0x

It works for me. But I need to invoke .toBlob() right after device.queue.submit([commandEncoder.finish()]);. For some reason invoking it from another place gives me empty picture.

munrocket avatar Aug 28 '21 04:08 munrocket

It works for me. But I need to invoke .toBlob() right after device.queue.submit([commandEncoder.finish()]);. For some reason invoking it from another place gives me empty picture.

I think this should be expected? If you do .toBlob() before the submit, the texture contents will be empty because submit is what actually writes to the texture.

austinEng avatar Aug 30 '21 15:08 austinEng

It is expected. See here: https://gpuweb.github.io/gpuweb/#abstract-opdef-get-a-copy-of-the-image-contents-of-a-context

.toBlob() will see the current contents of the [[currentTexture]] at the time it is called. If called after the texture has been presented (i.e. after the canvas is presented to the screen), then it will be null.

kainino0x avatar Aug 30 '21 23:08 kainino0x