use InputStream in FFmpegFrameGrabber cause memory keep fast rising and leak

[FanYaning]

I am using the followed code to convert video to audio. I am using javacv-platform with version 1.5.7 in Ubuntu 16.04.1 LTS (GNU/Linux 4.4.0-31-generic x86_64) the file 2.ts could be dowload from https://dh5.cntv.myalicdn.com/asp/h5e/hls/450/0303000a/3/default/83d9d8cae92a489783bd509a4589014b/2.ts

for (int i = 0; i < 5000; i++)
{
    try (InputStream in = new FileInputStream("2.ts"))
    {
        try (ByteArrayOutputStream outMp3 = new ByteArrayOutputStream(10 * 1024))
        {
            boolean bSuccess = convertToMp3(in, outMp3);
            if (bSuccess)
            {
                byte[] bytes = outMp3.toByteArray();
            }
        }
    }
    System.out.println(i + ":" + Pointer.physicalBytes());
}                                                                                                                                                                                                    

private static boolean convertToMp3(InputStream in, OutputStream outMp3) throws Exception
{
    boolean bSuccess = false;
    try (PointerScope scope = new PointerScope())
    {
        FFmpegLogCallback.setLevel(avutil.AV_LOG_QUIET);
        try (FFmpegFrameGrabber frameGrabber = new FFmpegFrameGrabber(in))
        {
            frameGrabber.setCloseInputStream(true);
            frameGrabber.start();
            if (frameGrabber.hasAudio())
            {
                try (FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(outMp3, 1))
                {
                    recorder.setCloseOutputStream(true);
                    recorder.setFormat("mp3");
                    recorder.setSampleRate(16000);
                    recorder.setTimestamp(frameGrabber.getTimestamp());
                    recorder.setAudioQuality(0);
                    recorder.start();
                    while (true)
                    {
                        try (Frame frame = frameGrabber.grabSamples())
                        {
                            if (frame == null)
                            {
                                break;
                            }
                            if (frame.samples != null)
                            {
                                recorder.recordSamples(frame.sampleRate, frame.audioChannels, frame.samples);
                            }
                        }
                    }
                    bSuccess = true;
                }
            }
        }
    }
    return bSuccess;
}

the used memory keep fast rising and leak, 100:510840832 200:532566016 300:564105216 400:606396416 500:653266944 600:693764096 700:738697216 800:776306688 900:830025728 1000:872636416 1100:911044608 1200:962277376 1300:1003679744 1400:1045749760 1500:1091256320 1600:1127120896 1700:1178357760 1800:1218584576 1900:1258377216 2000:1308553216 2100:1346076672 2200:1396592640 2300:1440198656 2400:1482502144 2500:1515053056 2600:1565724672 2700:1608908800 2800:1648803840 2900:1697247232 3000:1741213696 3100:1775853568 3200:1839468544 3300:1879273472 3400:1919549440 3500:1968091136 3600:2009092096 3700:2052624384 3800:2092404736 3900:2138992640 4000:2186575872 4100:2229788672 4200:2273472512 4300:2314903552 4400:2350354432 4500:2396594176 4600:2445889536 4700:2486898688 4800:2518294528 4900:2570506240

when I change not to use InputStream in try (FFmpegFrameGrabber frameGrabber = new FFmpegFrameGrabber(in)) , as to try (FFmpegFrameGrabber frameGrabber = new FFmpegFrameGrabber("2.ts")), the used memory keep stable,

100: 276180992 200: 398761984 300: 513257472 400: 549097472 500: 556146688 600: 569921536 700: 601571328 800: 618332160 900: 626929664 1000: 638472192 1100: 668401664 1200: 679972864 1300: 679976960 1400: 680005632 1500: 682606592 1600: 683032576 1700: 683036672 1800: 683126784 1900: 684371968 2000: 684371968 2100: 684371968 2200: 684371968 2300: 685002752 2400: 685002752 2500: 685031424 2600: 685056000 2700: 688852992 2800: 688852992 2900: 688852992 3000: 688852992 3100: 690688000 3200: 698650624 3300: 698650624 3400: 698650624 3500: 699326464 3600: 699326464 3700: 699326464 3800: 699326464 3900: 699551744 4000: 699551744 4100: 699551744 4200: 699551744 4300: 699723776 4400: 699723776 4500: 699723776 4600: 699809792 4700: 699838464 4800: 699838464 4900: 699838464

[saudet]

To avoid InputStream from buffering anything, you'll need to set maximumSize to 0.

[FanYaning]

To avoid InputStream from buffering anything, you'll need to set maximumSize to 0.

do not work.

I had done the test that changed code as to try (FFmpegFrameGrabber frameGrabber = new FFmpegFrameGrabber(in, 0)), but the memory still keep fast rising and leak.

100:372957184 200:553082880 300:602619904 400:673243136 500:715407360 600:763486208 700:804704256 800:845275136 900:888684544 1000:931426304 1100:982380544 1200:1014972416 1300:1059172352 1400:1103392768 1500:1152102400 1600:1198166016 1700:1246797824 1800:1292226560 1900:1335730176 2000:1378766848 2100:1421512704 2200:1469149184 2300:1513177088 2400:1556250624 2500:1600413696 2600:1632108544 2700:1683730432 2800:1729318912 2900:1761112064 3000:1795768320 3100:1857126400 3200:1913094144 3300:1956753408 3400:1990672384 3500:2030166016 3600:2085535744 3700:2129141760 3800:2172694528 3900:2215567360 4000:2246791168 4100:2288037888 4200:2331439104 4300:2377129984 4400:2419126272 4500:2464346112 4600:2507677696 4700:2551177216 4800:2594775040 4900:2638446592

[saudet]

It might be the format that doesn't support streams too well, and when it detects it cannot seek, it starts buffering everything...? Have you tried with another input format such as MKV?

[FanYaning]

It might be the format that doesn't support streams too well, and when it detects it cannot seek, it starts buffering everything...? Have you tried with another input format such as MKV?

I just now try flv and mkv format video，find that the memory keep stable in both try (FFmpegFrameGrabber frameGrabber = new FFmpegFrameGrabber(in) and try (FFmpegFrameGrabber frameGrabber = new FFmpegFrameGrabber("141.flv")) situation .

That's right, might be the ts format that doesn't support streams too well.

[shaoneng111]

It might be the format that doesn't support streams too well, and when it detects it cannot seek, it starts buffering everything...? Have you tried with another input format such as MKV?

what kind of input format don‘t buffering everything , or how to avoid that buffering everying。i handle different kind of videos, i m not sure what format they are. but i need to avoid memory leak.

[shaoneng111]

It might be the format that doesn't support streams too well, and when it detects it cannot seek, it starts buffering everything...? Have you tried with another input format such as MKV?

what kind of input format don‘t buffering everything , or how to avoid that buffering everying。i handle different kind of videos, i m not sure what format they are. but i need to avoid memory leak.

@saudet my code

public static VideoComputeInfo computeVideoQuality(String videoContentId, InputStream videoInputStream) { if (videoInputStream == null) { return null; } long startTime = System.currentTimeMillis(); // 直接指定maximumSize FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(videoInputStream, 1000000); grabber.setCloseInputStream(true); try { // 截取jpeg文件 grabber.start(); // 视频帧率 Integer frameRate = (int) grabber.getVideoFrameRate(); // 视频时长 int timeLen = (int) Math.ceil((double) grabber.getLengthInTime() / 1000000); int frameWidth = grabber.getImageWidth(); int frameHeight = grabber.getImageHeight(); // 视频文件大小 int videoSize = videoInputStream.available(); long endTime = System.currentTimeMillis(); VideoComputeInfo videoComputeInfo = VideoComputeInfo.builder() .frameRate(frameRate) .height(frameHeight) .width(frameWidth) .timeLen(timeLen) .videoSize(videoSize) .build(); log.info("FfmpegUtil computeVideoQuality耗时：{}, videoContentId: {}, videoComputeInfo: {}" , endTime - startTime, videoContentId, videoComputeInfo); return videoComputeInfo; } catch (Exception e) { log.error("FfmpegUtil computeVideoQuality异常，异常信息：", e); } finally { try { grabber.close(); } catch (Exception e) { log.error("FfmpegUtil grabber.close()异常，异常信息：", e); } } return null; }