How to get byte[] more efficient?

[gordonyfg]

Application: Java client(FrameGrabber)->websocket.sendBinary->php Server(Imagick->readImageBlob)

grabber = FrameGrabber.createDefault(1);
grabber.setFormat("dshow");
grabber.setBitsPerPixel(8);
grabber.setImageWidth(1920);
grabber.setImageHeight(1080);
grabber.start();
System.out.println("Before frameToByteArray Time: " + System.currentTimeMillis());
byte[] imageBytes = frameToByteArray(frameMat);
System.out.println("After frameToByteArray Time: " + System.currentTimeMillis());
// Send image data as binary message
websocket.sendBinary(imageBytes);

public static byte[] frameToByteArray(Frame frame) throws IOException {
    BufferedImage image = Java2DFrameUtils.toBufferedImage(frame);
    ByteArrayOutputStream os = new ByteArrayOutputStream();
    javax.imageio.ImageIO.write(image, "jpg", os);
    return os.toByteArray();
}

Result

Before frameToByteArray Time: 1687021694023
After frameToByteArray Time: 1687021694124

Time needed for such Frame to byte[] operation is about 100ms

Is there more efficient way? Can i get byte[] directly from Frame and php Imagick can decode it?

[saudet]

We can copy from the ByteBuffer in Frame.image[0], that's efficient.

[gordonyfg]

thank you. i tried that in 2 ways.

ByteBuffer byteBuffer = (ByteBuffer) frameMat.image[0];
byte[] bytes = new byte[byteBuffer.remaining()];
byteBuffer.get(bytes, 0, bytes.length);
websocket.sendBinary(bytes);

int width = frameMat.imageWidth;
int height = frameMat.imageHeight;
ByteBuffer buffer = (ByteBuffer) frameMat.image[0]; 
byte[] bytes = new byte[width * height];
buffer.get(bytes); // Copy the buffer contents to the byte array
websocket.sendBinary(bytes);

however, in my php code

$imageSrc = new \Imagick();
$imageSrc->readImageBlob($binaryData);

It prompts "An error has occurred: no decode delegate for this image format `' @ error/blob.c/BlobToImag" at line "readImageBlob". I've check Imagick, the built-in 'delegates' should be able to detect and convert any image format. Also, the frameToByteArray function can generate a byte[] that Imagick can recognize.

Anything i can setup with FrameGrabber to config the image format of ByteBuffer? like setVideoCodec?

[saudet]

We can use FFmpegFrameRecorder to encode the Frames into something else like JPEG or PNG format, if that's what you're looking for.

[Vansonccc]

How can i use recorder to encode frames to JPG format? Literally I just cover to BufferedImage and then use ImageIO to write currently.

We can use FFmpegFrameRecorder to encode the Frames into something else like JPEG or PNG format, if that's what you're looking for.

/

[saudet]

FFmpeg accepts a printf() like format as filename to output to JPEG files, so something like new FFmpegFrameRecorder("output_%03d.jpeg", width, height) should work.

[gordonyfg]

How can i use recorder to encode frames to JPG format? Literally I just cover to BufferedImage and then use ImageIO to write currently.

We can use FFmpegFrameRecorder to encode the Frames into something else like JPEG or PNG format, if that's what you're looking for.

/

I tried bufferedImage and ImageoIO, but still there is around 100ms latency for the conversion (frame to bufferedImage to byteArray). I really want to make it as short as possible.

I look into sending the frame.image[0] again. @saudet Is this ByteBuffer start with something like $JPEG = "\xFF\xD8\xFF" $GIF = "GIF" $PNG = "\x89\x50\x4e\x47\x0d\x0a\x1a\x0a" $BMP = "BM" $PSD = "8BPS" $SWF = "FWS" ? I am trying to decode it in the php server directly. but no luck.