javacv icon indicating copy to clipboard operation
javacv copied to clipboard

Is there a more efficient way to write the frames? FFmpegFrameRecorder

Open thisames opened this issue 1 year ago • 5 comments

I'm making an application that records the application's screen and it's working fine, but it uses a lot of memory when it's writing the frames. I don't know if it's because I programmed it the wrong way, or if that would be the only way for my case.

basically I'm saving the sequence views in PNG in the application's storage. (until then, I have no problem with memory). after I have all the images, I can record the video.

below is the code of the moment when I write the frames

public static void fun(Context mContext) {
        try {
            makeDir();
            Bitmap bitmap = getImageByPath(savePath + "/Screen_" + 1 + ".png");
            File dir = new File(savePath);

            FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(
                    savePath + "/video.mp4", testBitmap.getWidth(),
                    testBitmap.getHeight());
            recorder.setFormat("mp4");
            recorder.setFrameRate(20);
            recorder.start();
            int index = 1;
            while (index < dir.listFiles().length - 2) {
                IplImage image = cvLoadImage(savePath + "/Screen_" + index + ".png");

                OpenCVFrameConverter.ToIplImage grabberConverter = new OpenCVFrameConverter.ToIplImage();
                Frame frame = grabberConverter.convert(image);


                recorder.record(frame);
                index++;
            }
            recorder.stop();
            recorder.release();
            new AlertDialog.Builder(mContext).setTitle("info").setMessage("recording complete....").setPositiveButton("sure", null).show();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

every time it passes the while index it spends approximately 30MB

IplImage image = cvLoadImage(savePath + "/Screen_" + index + ".png"); // approximately 10mb
recorder.record(frame); // approximately 20mb

the application ends up reaching 1gb of memory easily in this code

Captura de tela de 2023-07-08 12-01-31

is there a more efficient way to write the frames? am i writing the code in a totally amateur way? thank you very much if you give me a way

thisames avatar Jul 08 '23 15:07 thisames

Well, start by NOT allocating a new OpenCVFrameConverter on each iteration

saudet avatar Jul 08 '23 23:07 saudet

My mistake leaving that line of code there, but it didn't make the slightest difference when I corrected it, thanks for the reply.

thisames avatar Jul 09 '23 16:07 thisames

Ok, let's see, next, since you're working with legacy code, you do understand that we need to deallocate IplImage manually, right?

saudet avatar Jul 09 '23 22:07 saudet

I confess that I do not understand directly. but I did some research and saw that I can use image.release() and image.deallocate() to manage native memory. however, apparently I first need to register a deallocator, and for that I need to use IplImage's create(). apparently cvLoadImage does not register a deallocator.

Is that what you mean by manually deallocating? Or was he talking about something completely different?

thisames avatar Jul 10 '23 16:07 thisames

Right, so please don't use the deprecated C API. Please use the C++ API.

saudet avatar Jul 10 '23 23:07 saudet