Can't capture webcam pixel when switch to P3D, everything turns black

[fairwelljack]

Hi, I encountered this in Processing video. I was following an old tutorial about 2018 step by step. The instructor had not encountered the same issue. Everything works fine on their end. Everthing works fine in 2D, size(800,600); while drawing rectangles using webcam pixels to fill the color

when use size(800,600,P3D), to draw boxes, everything turns black, looks like it cannot catch the correct pixel[] in P3D.

here is the code

import processing.video.*;

Capture cam;

void setup(){
  background(255);
  size(800,600,P3D);
  cam = new Capture(this);
  cam.start();
}

void draw(){
  cam.loadPixels();
  int pixelSize = 50; 
  for (int y = 0; y < cam.height; y += pixelSize){
    for( int x = 0; x < cam.width; x += pixelSize){
       int camPixelPos = y * cam.width + x;
       color pixelCol = cam.pixels[camPixelPos];
       fill(pixelCol);
       
       pushMatrix();
       translate(x,y,0);
       box(pixelSize);
       popMatrix();
       
       //rect(x,y,pixelSize,pixelSize);
    }
  }
}

void captureEvent(Capture c){
  c.read();
}

[fairwelljack]

here's some screenshot to explain this bug. I only changed to P3D in size. When it changes to P3D, it only returns -16777216 in the console when i print pixelCol by using .pixel[]

[ajavamind]

Not fixed with video library 2.2.2

[cacheflowe]

I just tested and can also confirm the bug on Windows 11 with the latest Processing & Processing Video lib

[jaegonlee]

try to add this in draw(). image(cam,0,0,0,0);

I tested video library in P2D, P3D mode, there are some problems with getting pixels.(and found workaround)

1. can't get pixel

color c = cam.get(x,y); 
fill(c);
rect(0,0,10,10); // just black only

add image()

color c = cam.get(x,y);
fill(c);
rect(0,0,10,10); 
image(cam,0,0,0,0);

I guess problem is in Capture.java(and Movie.java)

  public synchronized void read() {
    if (firstFrame) {
      super.init(sourceWidth, sourceHeight, RGB, 1);
      firstFrame = false;
    }
    if (useBufferSink) {
      if (bufferSink == null) {
        Object cache = parent.g.getCache(Capture.this); // can't get cache(texture) until call image() or texture()
        if (cache != null) {
          setBufferSink(cache);
          getSinkMethods();

2. slow frame rate when get pixels and display image.

cam.loadPixels();
...
image(cam,0,0); // very slow frame rate

add updatePixels()

cam.loadPixels();
cam.updatePixels();  
...
image(cam,0,0); // ok

Processing 4.1.1 / Video library 2.2.2 / MacBook Pro m1

[ajavamind]

Thanks jaegonlee! Nice work. Your work around fixed the problem for me using Windows 11. I tested it with a modified version of the Video library example "Mirror" with renderers P2D and P3D. Processing 4.1.1 / Video library 2.2.2 / Windows 11 AMD Ryzen 7 PRO 4750G with Radeon Graphics 3.60 GHz Also works with Processing 3.5.4

[jaegonlee]

Update: This does not work with captureEvent() or movieEvent()

~~I found the ways to fix it. In Capture.java(and Movie.java), replace~~ Object cache = parent.g.getCache(Capture.this);

to

import processing.opengl.PGraphicsOpenGL;
...
Object cache = ((PGraphicsOpenGL)(parent.g)).getTexture(Capture.this);

or

Object cache = null;
try {
  Method m = parent.g.getClass().getMethod("getTexture", new Class[] { PImage.class });
  cache = m.invoke(parent.g, new Object[] { Capture.this });
}
catch (Exception e) {
  e.printStackTrace();
}

[Asmatzaile]

Related to #203