obs-studio
obs-studio copied to clipboard
obsproject
[BUG] Multi track audio recording generates empty output file at high FPS
Platform
Operating system and version: Windows 10 1909 OBS Studio version: 25.0.8
Expected Behavior
Recordings at high FPS should function normally.
Current Behavior
The OBS audio_thread is single threaded - it processes and encodes each track sequentially, encoding and sending packets to be interleaved. audio_time gets progressively more and more behind cur_time, the user has no feedback this is occurring and trying to stop the output gets stuck since audio->stop_event is not checked due to the backlog. Meanwhile, the video thread is still outputting packets to be interleaved as the outputs are only stopped in ffmpeg_mux_data once packet->sys_dts_usec >= stream->stop_ts.
High FPS seems needed to reproduce this, the high rate of packets being sent to interleave is likely causing bottlenecks in the audio thread, as each audio packet must iterate the array of interleaved packets once per track.
Despite the audio packets eventually catching up, the output file is completely trashed - in testing I only got a 1.4KB MKV. I've not yet looked into why this happens.
Steps to Reproduce
- Set OBS to 240 FPS, 6 audio tracks in advanced output.
- Record for 30-60 seconds.
- Stop recording.
Additional information
Couple of potential fixes - making audio encoding multi-threaded seems like a good place to start, however the video thread continuing to send packets even after the output has been stopped should also be investigated.
Is this perhaps the same underlying issue that causes these old Mantis issues where not even an output file is generated, or is this a different issue? Talking about any of these:
- https://obsproject.com/mantis/view.php?id=789
- https://obsproject.com/mantis/view.php?id=953 (ignore my comments from then, I was completely wrong)
- https://obsproject.com/mantis/view.php?id=1268
- https://obsproject.com/mantis/view.php?id=1348
- (I'm sure there's more of these, but the titles and description of the Mantis issues is a lacking a lot of info.)
Yes, this issue has been driving me CRAZY! Sometimes it works and sometimes it doesn't. I just can't have that kind of inconsistency and have to guess if it's actually recording or not. It sucks because OBS with a single track at a high frame rate works for me, but I have wanted to start using multiple audio tracks for the sake of keeping game and voice audio separate for use in editing.
What would it take to get this issue looked at and fixed? @WizardCM, I'd put a small bounty down if that's what's needed.
POV I lost a whole month just to discover this was the thing
I know OBS is free but still... Will this be fixed any time soon?
POV I lost a whole month just to discover this was the thing
I know OBS is free but still... Will this be fixed any time soon?
I’m really not sure. The developers seem completely unreceptive to this bug. I brought it up in their Discord and was met with a lot of criticism. It seems like something they absolutely should prioritize given that it’s a bug that ruins recordings. They claim that multiple tracks at high frame rates are not a common use case. It still should be fixed. I guess it will only be fixed when the big streamers and YouTubers complain after YouTube and Twitch switch to 120 FPS. I guess all of us little guys who have wanted to use the feature for years don’t matter.
Just want to clarify a few things here.
Nobody on the team thinks this shouldn't be fixed, let's not make those kinds of assumptions.
OBS currently has only 2 full-time developers, who are already well overbooked in terms of things that they are focusing on.
I understand that this bug might be extremely important to you, but the reality is that high-framerate recordings with multiple tracks are uncommon, and this just doesn't affect that many people, which is why it is not given a higher priority over bugs that affect nearly all users of the program. The bug was initially investigated and reported by one of our core contributors, so we are definitely taking it seriously as far as a bug is concerned, and not dismissing anything. Priorities are evaluated all the time, new issues come up all the time, and shift focus away from lower priority items.
If you'd like to see this fixed faster, by all means, please contract or find someone to do the work and submit a pull request, or submit one yourself. I am not saying that to be dismissive, I am saying that as a course of action you can take, if you want to see this escalated.
@ModernManuh Please avoid comments, such as "I know OBS is free". That does not help give this bug priority, and often puts volunteer contributors off from wanting to work on it as it is incredibly devaluing to the hard work and effort put in by everyone on the team. This is a huge pet peeve of mine, where the perception that something is free must mean it is automatically of lower quality. I would recommend re-evaluating your prejudice on that aspect. It's hurtful to those who spend countless hours working on this project.
@lizardpeter I checked back on the conversation you had in Discord, which was with another standard community member who does not speak for the team, or represent the project in any way. They were not a developer, so please don't make that assertion with no basis in fact. Their behavior is something I would have reprimanded them from if I had seen it when it happened, and is not the kind of interactions we encourage in our community at all. For what it's worth, they don't appear to be a member of our Discord any longer. Nobody from our team of developers or contributors has dismissed you in any way. Those kinds of passive aggressive comments are putting us in an awkward position where if we were to decide to prioritize this issue we'd be encouraging your, frankly, entitled and rude behavior.
Lastly, the proper fix to this issue requires completely rewriting the audio encoding subsystem to be multithreaded, which is a massive amount of time and work. If it was a trivial fix, we'd have already done it, and we haven't forgotten about this.
@Fenrirthviti
Thank you for the in-depth response. I shouldn't have come across in that tone. I was just upset after not really getting a response and after everyone telling me this wasn't a major bug/issue just because it didn't affect them. However, most of these people were simply random Discord members, not core developers. Although, DreadWingKnight did say the issue probably wouldn't be addressed at all. I've never seen so many people claim that something is not a bug or in need of a fix because it revolves around an uncommon use case. I think that's the case for many bugs.
Anyway, I completely understand that there aren't many resources at the disposal of the team and that there are only a couple full-time developers. I am not an expert programmer who is familiar with the OBS code base, so I do not think I would be of any help in actually fixing this issue if the main developers aren't able to do it somewhat easily.
@ModernManuh Please avoid comments, such as "I know OBS is free". That does not help give this bug priority, and often puts volunteer contributors off from wanting to work on it as it is incredibly devaluing to the hard work and effort put in by everyone on the team. This is a huge pet peeve of mine, where the perception that something is free must mean it is automatically of lower quality. I would recommend re-evaluating your prejudice on that aspect. It's hurtful to those who spend countless hours working on this project.
And there it comes language barrier... I'm sorry it looked like I said that OBS and so all the work the team put into is bad just because it free, I didn't mean to say that. I meant that being a free software makes it harder to work on because in top of that, people gotta work to earn money and of course, they gotta live like anyone else.
With that said, I didn't say that you forgot this issue, I just wanted to know if it would've been fixed soon or not but I guess that's not the case due to the work it requires (I'm not a developer but if it was easy you would've done it already, as you said)
Once again, my apologies, I didn't mean to disrespect the team of a software that not only is free, but I use on a daily basis.
And there it comes language barrier...
I completely understand the language barrier here, and thank you for explaining and clarifying your statement. All is forgiven! It's a very sore subject for me personally, as I see how much hard work people put in every day on the project, so I'm particularly sensitive to it.
Has this been fixed yet?
I believe it has not been fixed. I tested with OBS 28 and the issue was still present.
Has this issue been addressed by the devs during the 28.0 development if at all in recent time?
If there are any updates, we will post them here. Please be patient as this is not a simple fix or something that we're even sure what the actual right fix for it is yet.
i have noticed that with rescale output enabled, recording at 1440p 120 fps it seems to work properly with the multi audio tracks. im not sure if there is quality loss with rescale yet but i've consistently been able to record/stop record
i have noticed that with rescale output enabled, recording at 1440p 120 fps it seems to work properly with the multi audio tracks. im not sure if there is quality loss with rescale yet but i've consistently been able to record/stop record
That's very interesting. I will have to try that out on my own machine to see if it works for me too. At least it would be a temporary solution until the developers are able to fix the root cause.
i have noticed that with rescale output enabled, recording at 1440p 120 fps it seems to work properly with the multi audio tracks. im not sure if there is quality loss with rescale yet but i've consistently been able to record/stop record
That's very interesting. I will have to try that out on my own machine to see if it works for me too. At least it would be a temporary solution until the developers are able to fix the root cause.
after more fooling around, it seems to have stopped working again :(
after more fooling around, it seems to have stopped working again :(
That's unfortunate. I wonder why it was working for you before.
Just to add some information as I was able to replicate possibly this or something similar and spent some time looking into what happens on my side:
OBS tries to synchronize audio and video on start at https://github.com/obsproject/obs-studio/blob/master/libobs/obs-output.c#L1743 and in my case this never resolved and the recording output was never started. Because of this there is no resulting file and we cannot stop the recording (we are still waiting for it to start).
This seems related to the fairly large difference in audio and video packet duration at FPS>60. OBS uses AAC and the encoders we use require ~30ms frame sizes for audio packets. If you instead replace AAC with a variable frame size audio codec like Opus, we can set the audio frame duration closer to the video duration (like we have with 60fps video) and successfully start the stream. In my tests I just flipped Opus between 5ms (worked fine) and 20ms (failed similarly to AAC).
Just switching to Opus isnt really a solution here, but I think we can focus on trying to understand why our existing pruning strategy can enter this state where it fails to synchronize. And perhaps prevent the output from assuming it has started before this synchronization point which avoids reporting recording as working when its really stuck.
--edit https://gist.github.com/kkartaltepe/d9bc973b03eda915829d4df10f289439 Contains the packet generation that results in failure to synchronize.
Just to add some information as I was able to replicate possibly this or something similar and spent some time looking into what happens on my side:
OBS tries to synchronize audio and video on start at https://github.com/obsproject/obs-studio/blob/master/libobs/obs-output.c#L1743 and in my case this never resolved and the recording output was never started. Because of this there is no resulting file and we cannot stop the recording (we are still waiting for it to start).
This seems related to the fairly large difference in audio and video packet duration at FPS>60. OBS uses AAC and the encoders we use require ~30ms frame sizes for audio packets. If you instead replace AAC with a variable frame size audio codec like Opus, we can set the audio frame duration closer to the video duration (like we have with 60fps video) and successfully start the stream. In my tests I just flipped Opus between 5ms (worked fine) and 20ms (failed similarly to AAC).
Just switching to Opus isnt really a solution here, but I think we can focus on trying to understand why our existing pruning strategy can enter this state where it fails to synchronize. And perhaps prevent the output from assuming it has started before this synchronization point which avoids reporting recording as working when its really stuck.
--edit https://gist.github.com/kkartaltepe/d9bc973b03eda915829d4df10f289439 Contains the packet generation that results in failure to synchronize.
Wow, thanks for all of that information! Do you know if there is a way to use Opus with the NVIDIA NVENC presets? I know OBS and NVIDIA did work to specifically optimize the NVENC presets (they used to say "new" next to their name, indicating that they had the optimizations), or is this something that is only accessible by using the custom FFMPEG output and forgoing those optimizations?
Wow, thanks for all of that information! Do you know if there is a way to use Opus with the NVIDIA NVENC presets? I know OBS and NVIDIA did work to specifically optimize the NVENC presets (they used to say "new" next to their name, indicating that they had the optimizations), or is this something that is only accessible by using the custom FFMPEG output and forgoing those optimizations?
As mentioned opus is not a solution (for users or for us), I only mention it so other developers could try and replicate my work by using the existing opus audio encoder. my understanding is ffmpeg custom output will use its own interleaving strategy, so whether or not it works is not really interesting to me personally.
This problem also happened to me. I record with CQP rate control but I think it doesn't make much of a difference. I am not a developer and I do not have much knowledge of how OBS works, however, I've done a bit of testing if that helps anyone.
It only gets stuck on stopping recording when the framerate is anything over 220 FPS. I've tried 225 and over but it only worked 1/5 times and it had to be really short clips. I recorded a 12-minute video at 220 FPS and it was fine.
I hope this problem will get noticed by the developers and will be fixed eventually as it is kind of a problem.
Here are my settings if anyone is interested:
