Not able to read pixels from offscreen render target in filament web

[shubhamgarg21]

I am trying to render a sample triangle to a offscreen render target and trying to use readPixels API to get the pixels buffer on the web. First thing i observed was that there was no readPixels API exposed in jsbindings for filament.js (Not sure if this is a bug or not part of roadmap). Then i tried to compile the c++ sample code directly into wasm and render it from there. I was able to successfully render the triangle on screen but doing the same to a offscreen rendertarget fails with no callback coming into PixelBufferDescriptor.

I am using the below code for offscreen rendering and reading:

using namespace filament; filament::Texture* offscreenColorTexture = Texture::Builder() .width(256) .height(256) .levels(1) .usage(Texture::Usage::COLOR_ATTACHMENT | Texture::Usage::SAMPLEABLE) .format(Texture::InternalFormat::RGBA8) .build(*app.mEngine);

filament::Texture* offscreenDepthTexture = Texture::Builder() .width(256) .height(256) .levels(1) .usage(Texture::Usage::DEPTH_ATTACHMENT) .format(Texture::InternalFormat::DEPTH24) .build(*app.mEngine);

app.mOffscreenRenderTarget = RenderTarget::Builder() .texture(RenderTarget::AttachmentPoint::COLOR, offscreenColorTexture) .texture(RenderTarget::AttachmentPoint::DEPTH, offscreenDepthTexture) .build(*app.mEngine);

const size_t byteCount = 256 * 256 * 4;
uint8_t * buf = new uint8_t[byteCount]; memset(buf, 0, byteCount);

filament::backend::PixelBufferDescriptor buffer( buf, byteCount, filament::backend::PixelBufferDescriptor::PixelDataFormat::RGBA, filament::backend::PixelBufferDescriptor::PixelDataType::UBYTE, [](void* buffer, size_t size, void* user) { // print data here (callback not getting executed) uint8_t * buf = (uint8_t *)buffer; for(int i=0; i<size; i++){ printf("%hhu ", buf[i]); } }, nullptr);

app.theView->setRenderTarget(app.mOffscreenRenderTarget); app.theView->setPostProcessingEnabled(false); app.theRenderer->renderStandaloneView(app.theView); app.theRenderer->readPixels(app.mOffscreenRenderTarget,0, 0, 256, 256, std::move(buffer)); app.theRenderer->getEngine()->execute(); app.mEngine->flush();

Desktop (please complete the following information):

• OS: web
• Backend: [OpenGL]

Additional context Threading is disabled for filament on web. Hence i am calling flush API of engine and not flushAndWait API.

[shubhamgarg21]

@romainguy @pixelflinger Can you please help ?

[shubhamgarg21]

@prideout

[pixelflinger]

@shubhamgarg21 I think you can call Engine::pumpMessageQueues() instead of flush after execute.

[shubhamgarg21]

@pixelflinger using Engine::pumpMessageQueues() also results in same behaviour. Upon some further debug i noticed that the call to glFenceSync is returning GL_TIMEOUT_EXPIRED and hence callback registered with whenGpuCommandsComplete is not getting executed.

[pixelflinger]

Ok I looked at the code and as it stands now you must call beginFrame/endFrame for the previous frames callback to trigger in the single-threaded environnement. I'll try to see if we can provide a better way to do this.