New Mac ProRes Hardware Recorder - Broken Multi-Track Audio in Premier Pro

[FRANKIEonPC]

Operating System Info

Mac

Other OS

No response

OBS Studio Version

29.0.0-beta3

OBS Studio Version (Other)

No response

OBS Studio Log URL

https://obsproject.com/logs/sqOkWLaEm0JlMKkB

OBS Studio Crash Log URL

No response

Expected Behavior

Multi-Track Mac Hardware Recorder ProRes Files to work correctly in Premiere Pro latest versions.

Current Behavior

Only first track of Multi-Track ProRes files play back in Premiere Pro latest versions. This is not user error, has been confirmed with Xaymar, please watch the 3min video here for detailed explanation: https://youtu.be/tSkAkDQWEbE

ProRes Files for Testing: https://www.dropbox.com/sh/wsu6ltkqqdusf85/AADVt3O7B6VezLk5OJ_pIDcSa?dl=0

Steps to Reproduce

1. Record on Mac Hardware Recorder ProRes File with Multiple Tracks
2. Put into any version of Premiere Pro on either Mac or PC past 14.9
3. Only first audio track will play after 20-30s or losing focus of Premiere Pro ...

Anything else we should know?

No response

[gxalpha]

ProRes is only supported on macOS.

[gxalpha]

Hang on - on the report above as well as the log you posted you’re saying that you’re using Windows, in the video there’s some macOS. Could you please clarify which parts of your workflow are on OBS on Windows and which on Mac? Are you able to reproduce this by using only OBS on macOS? Also, please post a log of the macOS session of OBS where you’re recording the files.

[FRANKIEonPC]

Recordings done on Mac, as Prores Hardware Encoder only on there. Files don't work in Premiere Pro on Mac or PC - please go to effort of testing the files above yourself, this is 100% broken.

Watch the video carefully, don't dismiss this issue, ProRes Multi-Track CANNOT be used currently in Premiere Pro on Mac or PC.

The log is immaterial, and was only provided because it is required for submission, if you watch the video this should be clear.

If you have tested the footage above, and still need the Mac log, let me know and will get later today.

[gxalpha]

An OBS log is still absolutely required. There’s a reason the field isn’t optional.

[FRANKIEonPC]

I will provide you with the log later when at home, it cannot help you, it is a fresh install of OBS Beta on Mac with ProRes Hardware selected, nothing more.

What will help you is downloading the files above and putting them in PP to see they are an issue.

Myself and Xaymar (who has been working on ProRes integration since StreamFx started years ago) are likely to know more about ProRes issues than your average user -- we are trying to help you even if you can't see that currently.

As stated ADOBE has an officially licensed version of ProRes that Apple helped them integrate - they have been asked at least 5 times about this by myself and Xaymar and they are pretty certain it is a FLAG issue with OBS exports that stops Premiere Pro parsing them correctly.

As ADOBE have reiterated multiple times (so please don't write "ask Adobe" again, they have been asked): their integration of ProRes was done with Apples Assistance (literally) hence why PP ProRes multi-track exports work perfectly. Maybe other NLEs don't require certain flags in their implementations, but that doesn't mean you shouldn't be striving to make ProRes as compatible as possible.

Would imagine OBS exports require literally a couple of extra lines added to them to work correctly. PP ProRes Exports are also done with Apples ProRes Hardware Encoder, so the only difference between their .mov and your .mov is the metadata / flags / header I assume?

I understand you get 1000 noob questions, or at least seems that looking at the Closed Issues, but this is a major bug with the biggest feature of your new version involving your only interframe multi-track codec (and the most popular professional editing format, on one of the most popular by userbase NLEs) -- it should be important.

[PatTheMav]

Seems like an issue with Premiere - all ProRes files I created worked fine in Final Cut Pro X, all audio channels are present. Same in IIna media player, as well as Quicktime itself, and QuickView - I wasn't able to generate ProRes files from OBS that did not have all expected audio channels (tried all 4 ProRes encoders and MOV as well as MKV containers).

Which ProRes encoder + container did you use?

[alinsavix]

It would probably be really helpful here if someone had the information on what flag was missing (or extra), rather than a handwavey "it's a problem with flags". If Adobe can't say what flag is missing, how do they know it's a problem with the flags?

Also, this description:

Only first track of Multi-Track ProRes files play back in Premiere Pro latest versions.

is incorrect. There's a slightly better description of the problem in the next section of the report, but I definitely had the wrong impression about the problem on first read. Premiere does recognize that the multichannel audio exists, and does play back all channels, but stops being able to play back any but the first channel after some amount of time. (I think @PatTheMav had the wrong idea here, too, based on their recent comment).

Also, I can reproduce this problem with both the files that OP has provided, and my own files, when loaded into Premiere 2023.1.0 on Windows. My mac apparently doesn't support hardware ProRes support, but I recorded multitrack audio using the software ProRes encoder, and still had the problem. I've included the associated OBS log so that we at least have some accurate log to work from, since OP seems outright hostile to the idea.

alinsa_obs_2023-01-11 11-14-34.txt

(tip for OP: The OBS crew are some of the hardest working people I know. Don't be belligerent, and don't presume to know what they do or don't need to be able to work on the problem. Remember that there are human beings on the other end that legitimately want to help with problems. The only thing you accomplish by being hostile is to make people want to look at your problems less, not more)

Interestingly, once Premiere breaks, it stays broken until it is restarted. Opening a new project and such does not appear to fix the problem. As well, there is some output on the Premiere console that I suspect is directly related to the problem:

<134296> <MCCallback> <5> err_printf: Sleep mode is active
<134296> <MainConceptAudioReader> <5> Error Occurred in Reading MainConcept Audio
<134296> <MCCallback> <5> err_printf: Sleep mode is active
<134296> <MainConceptAudioReader> <5> Error Occurred in Reading MainConcept Audio
<134296> <MCCallback> <5> err_printf: Sleep mode is active
<134296> <MainConceptAudioReader> <5> Error Occurred in Reading MainConcept Audio
<134296> <MCCallback> <5> err_printf: Sleep mode is active
<134296> <MainConceptAudioReader> <5> Error Occurred in Reading MainConcept Audio

To be clear, the "sleep mode is active" there is something internal to Premiere, my system did not sleep at all during testing (it's not even enabled on this system).

[PatTheMav]

is incorrect. There's a slightly better description of the problem in the next section of the report, but I definitely had the wrong impression about the problem on first read. Premiere does recognize that the multichannel audio exists, and does play back all channels, but stops being able to play back any but the first channel after some amount of time. (I think @PatTheMav had the wrong idea here, too, based on their recent comment).

That's correct - I went back and checked with longer recordings but still no issues in any software available to me. Same goes for the demo files provided (Final Cut Pro X played both tracks just fine from start to finish).

Given that all native ProRes implementations from Apple themselves have no issues with the generated files (or at least I cannot reproduce them) and neither do other third party tools besides Premiere, I'm veering towards seeing the issue on Adobe's side.

(Do MOV files generated with e.g. HEVC VideoToolbox encoder work?)

[alinsavix]

I was actually just testing this out, trying various combinations of things, and the interesting parts of the results:

• A mov file recorded with VT h.264 hardware encoding works correctly in Premiere without issue (I didn't try HEVC but I'm assuming the same would be true there)
• A mkv file recorded with ProRes software encoding and then remuxed (via ffmpeg) to mov also shows the problem
• If I import a problematic .mov into Premiere but only insert the two audio tracks into the timeline (without video), the problem still occurs
• If I remux the mkv to mov and leave out the video (so it just brings over the two audio tracks) the problem does not occur.
• AFAIK there's no non-mov container format that supports prores + multiple audio tracks + is supported by Premiere, so I couldn't test that combination (but really want to)

It's bizarre to me that it specifically has to be prores for things to break, even though the audio tracks in the .mov file are the same either way -- just straight AAC. I'm not sure why the video codec would matter at all. I'm guessing it's something happening in MainConcept (which is a/the third-party library Premiere uses for at least some of its decoding), but I don't know beyond that. Is weird this depends on the video format and not the container format, though, yeah?

Hey @Xaymar, assuming you look at these anymore, can you be any more specific about your findings on this problem?

[alinsavix]

[from @FRANKIEonPC]

Would imagine OBS exports require literally a couple of extra lines added to them to work correctly.

Just as a note about this -- framing this as "would require just a couple of lines" belies the actual difficulty of the task. The difficulty isn't in typing the lines, the difficulty is in knowing what lines to type. Just because something is a quick fix once it's known how to fix it doesn't mean that it's something that's quick to fix. The actual fixing is usually the easy part of debugging.

[Xaymar]

Participating on request by alinsavix.

Hey @Xaymar, assuming you look at these anymore, can you be any more specific about your findings on this problem?

I do not know how much my input is worth, as I do not use Adobe software myself - my entire workflow is using FFmpeg and DaVinci Resolve. I've also only looked at this at a minimal surface level as it works just fine in other NLEs and tools. From my PoV the issue is with Adobe, but here's my poor guess based on information that is probably way out of date at this point in time:

• In earlier versions of OBS Studio, there were cases of MP4/MOV files being generated with negative audio timestamps. These would randomly cause Adobe Premiere Pro and Adobe After Effects to behave in weird ways, if not outright crash the software.
• It's possible that Adobe requires a certain order of packets in the muxed file, but I haven't looked into that.
• Their demuxers and/or decoders may not like fragmented MP4/MOV as there are multiple "starts" of streams.
• From FRANKIEonPCs MediaInfo, here's a few surface level differences:
  • Adobe tags their MOV files as qt 2005.03, while FFmpeg tags it as qt 0000.02.
  • Adobe tags the writing library as adob0, FFmpeg tags it as Apple.
  • Adobe includes an extra information stream (QuickTime TC).
  • Adobe includes additional userdata tags: ©TIM for starting timecode, ©TSC and ©TSZ for frame rate / time scale.
  • FFmpeg tags all audio streams as "Alternate Group" = 1
  • Can't really compare the rest, as the audio codecs are different in the two files.

As the problem is usually happening over time, it is probably something to do with seeking in the file - at some point it breaks, with no clear indicator why. I don't believe the issue to be the ProRes encoder, but the way the files are muxed.

[FRANKIEonPC]

Current Log File here: https://obsproject.com/logs/sqOkWLaEm0JlMKkB All Log Files here: https://www.dropbox.com/sh/0i3y0m5d74ehitf/AAD4PFZJQfzRwxKZZTaCvhBNa?dl=0

[FRANKIEonPC]

Couple Extra Details:

PP Multitrack ProRes Exports obviously work fine in PP and PP 14.9 ProRes Exports work fine in versions above PP 15.0.

From PP 14.9 (where OBS ProRes Multi-Track works) to PP 15.0 onwards (where the audio bugs out for OBS ProRes Files), Adobe has changed how Audio is Exported when ProRes uncompressed audio is selected from in24 to sowt. https://www.dropbox.com/sh/hmko1na9wlmf7gv/AADZXBfIUkh0ZCMgfOWGfmASa?dl=0

Also in PP 15.0 onwards Adobe changed how the Audio Track Mixer works so you can copy paste effects in it, removed all Legacy Audio Plugins (suggesting something more fundamental with audio engine was changed) and they changed audio routing adding pre/post fader options.

I've asked Adobe about this in past (this has been a very long-standing problem, which we originally thought was due to .mkv), but they again say its just a OBS format issue and the changes were made made to bring the audio engine more in-line with professional formats. They also said that tighter format requirements improve performance significantly, especially when dealing with very high bitrate / resolution and RAW footage. I believe they introduced ProRes RAW support in 14.5, so maybe they updated the audio stuff in 15.0 because it wasn't performing well.

[gxalpha]

@alinsavix @Xaymar thank you very much for the input.

@FRANKIEonPC is any of the communication with Adobe (besides this forum post) public, so that we can read and potentially follow up on it?

[alinsavix]

Thanks for the comments, @Xaymar.

I went through things and tested what I could. Results and comments:

• In earlier versions of OBS Studio, there were cases of MP4/MOV files being generated with negative audio timestamps. These would randomly cause Adobe Premiere Pro and Adobe After Effects to behave in weird ways, if not outright crash the software.

Best I can tell, after checking with ffprobe -show_packets, the audio tracks both start with pts=0 and dts=0, so that doesn't appear to be the issue here (unless I'm looking at the wrong thing)

• It's possible that Adobe requires a certain order of packets in the muxed file, but I haven't looked into that.

Quite possible, not sure how to test it, though. Would almost definitely be a Premiere bug, if so, though (AFAIK the format itself doesn't have any particular requirements here)

• Their demuxers and/or decoders may not like fragmented MP4/MOV as there are multiple "starts" of streams.

This is possible, but even non-fragmented files do the same thing (since they're all not-fragmented without the extra muxer options, I'm pretty sure)

• Adobe tags their MOV files as qt 2005.03, while FFmpeg tags it as qt 0000.02.

Not sure how to test this one -- if ffmpeg is writing the version out as 0000.02 (and that's a valid version, and its output conforms to it) either Premiere should reject it if it doesn't support it, or accept it and work correctly. So with those assumptions, if this is the problem, that's a Premiere problem.

• Adobe tags the writing library as adob0, FFmpeg tags it as Apple.

I can't imagine this possibly makes a difference (but it is something different in the output, yes)

• Adobe includes an extra information stream (QuickTime TC).

This is a timecode track. I could totally see this problem being related to that, but when I add one (via ffmpeg -timecode 00:00:00:00, which appears to add it correctly) the problem remains.

• Adobe includes additional userdata tags: ©TIM for starting timecode, ©TSC and ©TSZ for frame rate / time scale.

I added these, the problem still exists.

• FFmpeg tags all audio streams as "Alternate Group" = 1

Not sure how to manipulate this one, but since it's related to which audio tracks to use by default (and which are alternate content for which) I can't imagine this could be the problem with this particular set of symptoms. If Premiere wasn't importing or playing some tracks at all, perhaps, but not with these symptoms.

As the problem is usually happening over time, it is probably something to do with seeking in the file - at some point it breaks, with no clear indicator why.

Time and seeking don't appear to be actual requirements -- all that's required for it to break is that Premiere puts whatever subsystem into "sleep mode" after it's started playing those tracks -- something that happens, as best I can tell, when you click out of Premiere (which would make sense).

Though it may be doing hidden seeks internally, from the outside world a seek isn't required -- I can reproduce 100% by importing a track, playing any of the track at all, stopping it, tabbing out, tabbing back in, and hitting play again, without ever moving the playhead.

(if I create a sequence from one of these tracks, tab out, and back in, it plays back fine, I'm assuming because the audio hasn't actually been initialized yet. Doubt this information is useful, but there ya go)

I don't believe the issue to be the ProRes encoder, but the way the files are muxed.

I'd generally agree with this. Though given that I can remux problematic files with ffmpeg and still have the problem, I'd think it would have to be an ffmpeg bug, in that case.

I'd also tend to agree that this seems like a Premiere problem and not an OBS problem -- or if it is an OBS problem, it's because the code in Premiere is being so incredibly pedantic that it's breaking on something that it seems like no other decoder on the planet breaks on.

Just for shits and giggles, I dropped a support request to MainConcept to ask if they can provide any insight, since it seems like it's their library where the issue is actually happening. I'm not particularly hopeful (if I was in their position I'd probably not spend any time on someone trying to debug a problem in some random app that uses my library), but it can't hurt to ask. Ticket 1369923470, though I don't think it's accessible to anyone outside that company.

[alinsavix]

Color me surprised, MainConcept responded, and quickly at that -- bravo! I wish more companies were like that.

I made sure they had the right files for testing, and gave them very precise steps for reproduction, and they're going to look into things and see if they can determine if the problem is in their library (and if not, presumably give some insight on where the failure is actually happening). I don't know what the ETA on that will be (and since they're basically doing everyone a favor by even looking at it, I'm not going to push for one), but unless anyone has any fresh ideas for testing/debugging, I suspect waiting for the results there would be the best course of action before anyone invests much more time on this one.

I'll report back!

[FRANKIEonPC]

Any luck with a response? Has been 2 weeks so wouldn't be rude to ask them how they getting on.

[alinsavix]

Vendor reports they're still working on it. And I quote, "it is tricky". If there were further updates, they would have been posted here.

By the way, @FRANKIEonPC, it is not at all cool to hunt down contributors on social media and harass them for updates because you don't like the speed at which updates are happening. Don't do that.

[FRANKIEonPC]

"Harass" is an interesting word for the below after 27 days of no follow up by yourself and I feel that phrasing as such here is you harassing me:

You didn't follow up after 27 days, MainConcepts page states "We will contact you within two business days to let you know the status of your ticket and expected window of resolution." (https://www.mainconcept.com/support).

The issue may not be urgent to yourself but is likely to be urgent to other people, given the breadth of OBS's userbase. If you don't follow up people will assume you have forgotten, and if you don't follow up reasonably with MainConcept (e.g. 2 weeks) they will be unlikely to commit resources to fix it.

Some alternative approaches perhaps:

• Set yourself a reminder and follow up on the issue yourself, am sure you have 5min to do so in 27 days.
• Be on the OBS Discord, as you were originally searched for on there before being contacted on Twitter.
• Don't put your Twitter on your Github profile or people will assume you wish to be contacted by other developers that way.

[jp9000]

I'm really sorry @alinsavix.

@FRANKIEonPC, that's not the kind of conduct we want here. I realize you're frustrated, you're having problems, you want more information, and you want to see these problems resolved. I get it. But someone not responding on a repository does not give automatic invitation or consent to look for and use that person's other social media outlets for this or pretty much any purpose, regardless of whether those social media are public or on their profile or not. The moment you do that, you risk crossing someone's boundaries. If you're in the position where have to ask yourself, "would they be okay with this", and you're not positive about the answer, then you should probably assume the answer is no, or ask them for explicit permission. It's that simple.

We've probably all unintentionally crossed someone else's boundaries at some point in time. While I can understand how such a mistake can happen, when someone tells you that you've breached their boundaries, you should immediately issue and apology to them, back off, take the L, and try to learn from it so you can make yourself a better person. To question someone else's own boundaries after having breached them is a gravely worse thing than simply apologizing. Those boundaries were never yours to set, whether you feel they make sense or not.

I probably wouldn't have been quite as upset had you simply apologized, but responding the way you did is not the sort of thing we want to see here. People have been trying to help you, especially @alinsavix, and yet you do these things and respond to her in this way. How are others supposed to feel about that when you do that to them? Is this really going to help you? Are you really helping yourself here by responding in this way to people who were clearly just trying to help?

[Fenrirthviti]

After review, we've issued a ban on the repo and in our Discord to FRANKIEonPC.

In addition to the above, they also tried asking in our Discord for someone to tell them how to contact alinsavix. We declined to assist with that, and they were told at the time to be patient, and not harass anyone, but decided to ignore that and reach out anyway. This kind of behavior is borderline obsessive, and isn't the first time we've seen this kind of aggressive behavior from them in the past. It isn't welcome in our community.

Unlocking comments here in case there are any updates, but we're still pretty sure this is something on Premiere's side that they would need to fix as they appear to be the only software affected.

[alinsavix]

Ironically, folks like OP are one of the big reasons I'm not on the discord anymore. Thanks for taking care of that.

ANYHOW, after a few weeks of pestering, I did get an update of sorts from MainConcept. The short of the update is "they've reached out to Adobe and they are currently investigating the issue together", no ETA. They also say that so far, they haven't seen any problems from a file format standpoint.

They also gave me their internal ticket number, which is MCC-11514. Don't think we can do anything with the ticket number, but it's here in case it's useful.

So, my personal take, at this point, is that we could probably close this ticket. MC hasn't identified any issues in the files themselves (which doesn't mean they aren't there, mind you), the symptoms really don't -feel- like a file format problem, and we really don't have anything else to go on at this point. And if MC gets back to us with a problem we can fix, or someone who has talked to Adobe about this can be more specific about their findings beyond "it's a flag problem", we can always re-open.

We can keep it open if y'all want to keep actively tracking this, of course, and I will update it, open or closed, if/when I hear more back from MainConcept -- though at this point I'm not really expecting that to happen quickly, given the limited scope of the issue.

[Fenrirthviti]

Agreed with all the above, and on closing. If anyone comes across any new information, or an update comes from MC, we can always reopen this later if there's anything we need to track. Current project policy is not to keep third-party/dep issues open, as we have enough to track as is.

Thanks for the efforts on your end!

[alinsavix]

And a final update: To the surprise of nobody, MainConcept has identified a bug in their library and have fixed it internally. That fix will make its way upstream to Adobe through "their usual delivery process" (ETA unknown). So there's the definitive answer -- nothing to fix in OBS or any of its libraries.