How to show frame after processing?

[ValuableGrbg]

Hello, I'm trying to get frame using addPreviewDataCallBack and function onPreviewData. App crashes when I try to put data that I get from onPreviewData into bitmap. First, I tried to put data into bitmap using decodeByteArray it didn't work. Then I tried to use custom function but it also doesn't work. Here is code example:

camera.addPreviewDataCallBack(object: IPreviewDataCallBack{
                    override fun onPreviewData(
                        data: ByteArray?,
                        width: Int,
                        height: Int,
                        format: IPreviewDataCallBack.DataFormat
                    ) {
                        if (data != null) {
                            mViewBinding.textView.text = format.toString()
                            val b: Bitmap?
                            //b = BitmapFactory.decodeByteArray(data, 0, data.size)
                            b = bitmapFromRgba(width, height, data)
                            mViewBinding.imageView.setImageBitmap(b)
                        }else{
                            mViewBinding.textView.text = "NO DATA"
                        }
                    }
                })

and custom function:

fun bitmapFromRgba(width: Int, height: Int, bytes: ByteArray): Bitmap? {
        val pixels = IntArray(bytes.size / 4)
        var j = 0
        for (i in pixels.indices) {
            val R: Int = (bytes[j++] and 0xff.toByte()).toInt()
            val G: Int = (bytes[j++] and 0xff.toByte()).toInt()
            val B: Int = (bytes[j++] and 0xff.toByte()).toInt()
            val A: Int = (bytes[j++] and 0xff.toByte()).toInt()
            val pixel = A shl 24 or (R shl 16) or (G shl 8) or B
            pixels[i] = pixel
        }
        val bitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888)
        bitmap.setPixels(pixels, 0, width, 0, 0, width, height)
        return bitmap
    }

Can you please explain how can I do this? Also, is there any way I can get frames from both cameras in the same time and put one frame on the top of other (using OpenCV) so it would be mixed frame?

[sheenhill]

yeah.I want to use video frames for AI recognition, which will cause screen blur. I have looked at the source code, and it is possible that rendering the view and transcoding the frames in the same thread have caused performance constraints on my low-end machine.

[zhouqunhuang]

I would like to ask you how to solve this problem at last, and I cannot convert the obtained byre into bitmap for use. Please let me know if it works out. Thank you

[sheenhill]

I would like to ask you how to solve this problem at last, and I cannot convert the obtained byre into bitmap for use. Please let me know if it works out. Thank you

我尝试了几种方法将 byteArray 转成bitmap，都没实现。我的设备也无法支持一直读取 frame，会导致内存不足。所以我使用的方法是，用降低的频率去获取截图，例如1秒截图一次。 I have tried several methods to convert byteArray to bitmap, but none of them have been implemented. My device also cannot support constantly reading frames, which can lead to insufficient memory. So the method I use is to take screenshots at a reduced frequency, such as once every 1 second.

[reganh98]

After digging the source code, I have managed to obtain bitmap. Here is the solution...

Kotlin:

val callback = object : IPreviewDataCallBack {
    override fun onPreviewData(
        data: ByteArray?,
        width: Int,
        height: Int,
        format: IPreviewDataCallBack.DataFormat
    ) {
        if (data == null || format != IPreviewDataCallBack.DataFormat.RGBA) { return }
        val byteBuffer = ByteBuffer.allocateDirect(width * height * 4)
        byteBuffer.order(ByteOrder.LITTLE_ENDIAN)
        GLES20.glReadPixels(
            0,
            0,
            width,
            height,
            GLES20.GL_RGBA,
            GLES20.GL_UNSIGNED_BYTE,
            byteBuffer
        )
        val bitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888)
        bitmap?.copyPixelsFromBuffer(byteBuffer)
    }
}

Java:

new IPreviewDataCallBack() {
    @Override
    public void onPreviewData(@Nullable byte[] data, int width, int height, @NonNull DataFormat format) {
        if (data == null || format != DataFormat.RGBA) {
            return;
        }
        ByteBuffer byteBuffer = ByteBuffer.allocateDirect(width * height * 4);
        byteBuffer.order(ByteOrder.LITTLE_ENDIAN);
        GLES20.glReadPixels(
                0,
                0,
                width,
                height,
                GLES20.GL_RGBA,
                GLES20.GL_UNSIGNED_BYTE,
                byteBuffer
        );
        bitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888);
        bitmap.copyPixelsFromBuffer(byteBuffer);
    }
};

Reference: https://github.com/jiangdongguo/AndroidUSBCamera/blob/master/libausbc/src/main/java/com/jiangdg/ausbc/utils/GLBitmapUtils.kt