High memory usage when using bitmap subtitles

[umlx5h]

This commit supports bitmap subtitles, but I have observed that the memory usage is temporarily large after enabling bitmap subs. It may possibly be a memory leak.

https://github.com/SuRGeoNix/Flyleaf/commit/9a47e747b8e3e0064ba0b2e4361149d97edbf65e

The following settings seem to temporarily increase memory usage. It appears to be released after a long distance seek.

https://github.com/SuRGeoNix/Flyleaf/blob/9a47e747b8e3e0064ba0b2e4361149d97edbf65e/FlyleafLib/MediaFramework/MediaDemuxer/Demuxer.cs#L511-L512

test video file

• 3.55GB mkv HEVC 2h32min PGS Sub

Below are the results of the memory profiler in JetBrains Rider.

Starting playback of a video with bitmap subtitles enabled consumes a large amount of unmanaged memory usage. ( 2. 750 MB ) If playback is continued without seeking, memory usage does not decrease.

If you use the seek bar and seek large, memory usage is reduced significantly. ( 3. 350MB )

The following result shows the case where probesize and max_analyze_duration were not set.

No particular leakage occurred in this case. Seeking does not result in any particular change.

I checked the mpv and VLC sources and it seems that neither of them specifically set probesize and max_analyze_duration. I would like to know in what situations these are necessary.

[SuRGeoNix]

So does VLC/mpv plays bitmap properly without specifying probesize/max_analyze_duration? From my testing I've noticed that in case of bitmap subs in order for ffmpeg to read the stream's properties you need to increase those. Unfortunately, what FFmpeg does (just a guess) is that it caches even video frames/packets that's why you see an increased memory.

I didn't do any further testing but if you think is memory leak, it should be opened on FFmpeg's trac. If you think is this can be resolve in Flyleaf with another way I'm opened to fix it. (Possible add a config for whether to increase those values in this case or not)

[umlx5h]

Thanks for your reply.

I would like to investigate as well, Does the problem only reproduce itself with certain decoders or container videos? I had no problems with the H264, mkv & mp4 videos I have on hand.

It may be acceptable to increase the setting value only for problematic formats.

[SuRGeoNix]

Didn't test it but it is possible to be affected by HEVC. Generally speaking the FFmpeg's analyze is very problematic.

[umlx5h]

Just putting a seek after avformat_find_stream_info seems to free up memory. Is this measure not sufficient?

$ git diff
diff --git a/FlyleafLib/MediaFramework/MediaDemuxer/Demuxer.cs b/FlyleafLib/MediaFramework/MediaDemuxer/Demuxer.cs
index c414f98..8e5732b 100644
--- a/FlyleafLib/MediaFramework/MediaDemuxer/Demuxer.cs
+++ b/FlyleafLib/MediaFramework/MediaDemuxer/Demuxer.cs
@@ -515,6 +515,11 @@ public unsafe class Demuxer : RunThreadBase
                 ret = avformat_find_stream_info(fmtCtx, null);
                 if (ret == AVERROR_EXIT || Status != Status.Opening || Interrupter.ForceInterrupt == 1) return error = "Cancelled";
                 if (ret < 0) return error = $"[avformat_find_stream_info] {FFmpegEngine.ErrorCodeToMsg(ret)} ({ret})";
+
+                if (requiresMoreAnalyse)
+                {
+                    avformat_seek_file(fmtCtx, -1, 0, 0, 0, 0);
+                }

[SuRGeoNix]

Seeking can cause issues so it should be avoided. Possible flushing the format/pb should work as well and it would be better. Maybe another ffmpeg option can avoid this memory usage too. I will try to look in to it in the future...