mediapipe
mediapipe copied to clipboard
PoseLandmarker's segmentation is blank when using delegate: GPU
Hello everyone, I am trying to debug the segmentation mask result of PoseLandmarker, but the result texture is blank. I initialize my vision task as the following
const vision = await FilesetResolver.forVisionTasks(
"https://cdn.jsdelivr.net/npm/@mediapipe/tasks-vision@latest/wasm"
);
const poseLandmarker = await PoseLandmarker.createFromOptions(vision, {
baseOptions: {
modelAssetPath: "/pose_landmarker_full.task",
delegate: "GPU",
},
runningMode: "VIDEO",
outputSegmentationMasks: true,
});
Then I am running a renderLoop as such:
function renderLoop() {
let startTimeMs = performance.now();
if (video.currentTime !== lastVideoTime) {
poseLandmarker.detectForVideo(video, startTimeMs, (result) => {
if (result.segmentationMasks.length) {
gl.viewport(0, 0, width, height);
gl.useProgram(program);
gl.activeTexture(gl.TEXTURE0);
gl.bindTexture(
gl.TEXTURE_2D,
result.segmentationMasks[0].getAsWebGLTexture()
);
if (debug) {
console.log(result.segmentationMasks[0].getAsFloat32Array());
}
}
});
lastVideoTime = video.currentTime;
}
requestAnimationFrame(() => {
renderLoop();
});
}
My WebGL canvas which is just rendering the WebGL Texture is not showing anything, so for debugging purposes I console logged result.segmentationMasks[0].getAsFloat32Array() to see if I was getting any values, but it turns out it's a 0 filled Float32Array.
I changed the delegate to CPU to see if I got a different result and, indeed, I got non 0 values inside the float32array, so I was wondering how could I get the segmentation mask with GPU as delegate.
Thank you so much in advance