UnityPlugin-AVProMovieCapture icon indicating copy to clipboard operation
UnityPlugin-AVProMovieCapture copied to clipboard
verified publisher icon RenderHeads

App crashes when recording starts when using CaptureFromTexture with UnityRenderStreaming

Open gpiffaretti opened this issue 3 years ago • 20 comments

Describe the bug When the recording starts in iOS, the app crashes. I don't have access to the application logs, but I do have the crash report from App Store Connect. This is attached.

I don't have a clear read on what the problem might be by looking at the logs, but maybe someone else can help with that.

Your Setup (please complete the following information):

  • Unity version: 2020.3.18f1
  • AVPro Movie Capture version: 5.0.0 (July 21st 2022)
  • Operating system version: iOS 14.7.1 / iOS 15.5
  • Capture component used: CaptureFromTexture
  • Capture settings (resolution, frame-rate, codec): realtime capture - 640x400 - 30fps - h264 --- Output goes to persistent data path --- tried with/without microphone audio --- Frame update mode Automatic --- didn't change the default quality and bitrate settings --- we also have AvPro Ultra Edition 2.5.4

To Reproduce Steps to reproduce the behavior:

  1. We launch our application. a script that has reference to CaptureFromTexture.
  2. Set the source texture. This is a Texture2D received from UnityRenderStreaming package.
  3. Call StartCapture()

Logs crashlog.txt

gpiffaretti avatar Jul 22 '22 18:07 gpiffaretti

I'm setting a Texture2D as the source texture BTW

gpiffaretti avatar Jul 22 '22 18:07 gpiffaretti

tried a downgrade to AVPro Movie Capture 4.7.11 and having the same crash

gpiffaretti avatar Jul 22 '22 20:07 gpiffaretti

tried with a RenderTexture and still crashing. Don't know what else to try. Will try to get some more info about the crash, hopefully the stack trace

gpiffaretti avatar Jul 22 '22 21:07 gpiffaretti

Thanks for reporting this issue. If you try the CaptureFromTexture demo scene, does that crash for you too?

Thanks,

AndrewRH avatar Jul 22 '22 23:07 AndrewRH

From the crash log I can see that stop is being called. You say it crashes on start so something is calling stop almost immediately. If you can get log output from Xcode I will see if I can diagnose what is happening.

MorrisRH avatar Jul 23 '22 08:07 MorrisRH

Deeper investigation suggests that the crash is due to Metal API validation being enabled. If you disable this in the iOS specific part of the project inspector you may be able to run. I would be interesting in knowing what the error it's reporting is though.

MorrisRH avatar Jul 23 '22 10:07 MorrisRH

I'll be getting a macbook pro this afternoon to build the demo scene and get some extra logs

gpiffaretti avatar Jul 25 '22 14:07 gpiffaretti

Here's the log for the crash. The permission prompts are being displayed, maybe that is causing the call to stop the capture? Will build the demo next.

SessionRecorder: Start Recording
X.Companion.<StartRecordingCoroutine>d__18:MoveNext()
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
X.Companion.UI.Views.<Start>d__4:MoveNext()
System.Threading.ContextCallback:Invoke(Object)
System.Threading.ExecutionContext:RunInternal(ExecutionContext, ContextCallback, Object, Boolean)
System.Runtime.CompilerServices.MoveNextRunner:Run()
System.Action:Invoke()
System.Threading.ContextCallback:Invoke(Object)
System.Threading.Tasks.AwaitTaskContinuation:RunCallback(ContextCallback, Object, Task&)
System.Threading.Tasks.Task:FinishContinuations()
System.Threading.Tasks.Task`1:TrySetResult(TResult)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:SetResult(TResult)
X.Companion.UI.Views.<CreateTestExecutionDocument>d__6:MoveNext()
System.Threading.ContextCallback:Invoke(Object)
System.Threading.ExecutionContext:RunInternal(ExecutionContext, ContextCallback, Object, Boolean)
System.Runtime.CompilerServices.MoveNextRunner:Run()
System.Action:Invoke()
System.Threading.ContextCallback:Invoke(Object)
System.Threading.Tasks.AwaitTaskContinuation:RunCallback(ContextCallback, Object, Task&)
System.Threading.Tasks.Task:FinishContinuations()
System.Threading.Tasks.Task`1:TrySetResult(TResult)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:SetResult(TResult)
X.Companion.Systems.<CreateTestExecution>d__19:MoveNext()
System.Threading.ContextCallback:Invoke(Object)
System.Threading.ExecutionContext:RunInternal(ExecutionContext, ContextCallback, Object, Boolean)
System.Runtime.CompilerServices.MoveNextRunner:Run()
System.Action:Invoke()
System.Threading.SendOrPostCallback:Invoke(Object)
UnityEngine.WorkRequest:Invoke()
UnityEngine.UnitySynchronizationContext:Exec()

[AVProMovieCapture] Start File Capture: 852x480 @ 30.00fps [RGBA32] vcodec:'H264' audio source:'iPhone Microphone' acodec:'AAC' to file: '/var/mobile/Containers/Data/Application/897D19EF-9BE5-423C-9878-B8A2BCF5E7BA/Documents/recording-8MQyhOb1bUeNobQmQAcW.mp4'
RenderHeads.Media.AVProMovieCapture.CaptureBase:PrepareCapture()
RenderHeads.Media.AVProMovieCapture.CaptureBase:StartCapture()
X.Companion.<StartRecordingCoroutine>d__18:MoveNext()
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
X.Companion.UI.Views.<Start>d__4:MoveNext()
System.Threading.ContextCallback:Invoke(Object)
System.Threading.ExecutionContext:RunInternal(ExecutionContext, ContextCallback, Object, Boolean)
System.Runtime.CompilerServices.MoveNextRunner:Run()
System.Action:Invoke()
System.Threading.ContextCallback:Invoke(Object)
System.Threading.Tasks.AwaitTaskContinuation:RunCallback(ContextCallback, Object, Task&)
System.Threading.Tasks.Task:FinishContinuations()
System.Threading.Tasks.Task`1:TrySetResult(TResult)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:SetResult(TResult)
X.Companion.UI.Views.<CreateTestExecutionDocument>d__6:MoveNext()
System.Threading.ContextCallback:Invoke(Object)
System.Threading.ExecutionContext:RunInternal(ExecutionContext, ContextCallback, Object, Boolean)
System.Runtime.CompilerServices.MoveNextRunner:Run()
System.Action:Invoke()
System.Threading.ContextCallback:Invoke(Object)
System.Threading.Tasks.AwaitTaskContinuation:RunCallback(ContextCallback, Object, Task&)
System.Threading.Tasks.Task:FinishContinuations()
System.Threading.Tasks.Task`1:TrySetResult(TResult)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:SetResult(TResult)
X.Companion.Systems.<CreateTestExecution>d__19:MoveNext()
System.Threading.ContextCallback:Invoke(Object)
System.Threading.ExecutionContext:RunInternal(ExecutionContext, ContextCallback, Object, Boolean)
System.Runtime.CompilerServices.MoveNextRunner:Run()
System.Action:Invoke()
System.Threading.SendOrPostCallback:Invoke(Object)
UnityEngine.WorkRequest:Invoke()
UnityEngine.UnitySynchronizationContext:Exec()

-> applicationWillResignActive()
Application is being paused, stopping capture...
RenderHeads.Media.AVProMovieCapture.CaptureBase:OnApplicationPause(Boolean)

[AVProMovieCapture] Stopping capture 58000
RenderHeads.Media.AVProMovieCapture.CaptureBase:StopCapture(Boolean, Boolean, Boolean)

2022-07-25 15:39:14.309079-0300 X[63109:4806090] [Plugin] 🔹 Stopping capture...
2022-07-25 15:39:14.314001-0300 X[63109:4806471] -[AGXA10FamilyCommandBuffer renderCommandEncoderWithDescriptor:], line 380: error 'A command encoder is already encoding to this command buffer'
-[AGXA10FamilyCommandBuffer renderCommandEncoderWithDescriptor:]:380: failed assertion `A command encoder is already encoding to this command buffer'
dyld4 config: DYLD_INSERT_LIBRARIES=/Developer/usr/lib/libMainThreadChecker.dylib:/Developer/Library/PrivateFrameworks/DTDDISupport.framework/libViewDebuggerSupport.dylib

gpiffaretti avatar Jul 25 '22 18:07 gpiffaretti

Demo scene for TextureCapture is working correctly

gpiffaretti avatar Jul 25 '22 19:07 gpiffaretti

Tried again after accepting permissions. The permission popups were not the cause of the crash:

SessionRecorder: Start Recording
X.Companion.<StartRecordingCoroutine>d__18:MoveNext()
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
X.Companion.UI.Views.<Start>d__4:MoveNext()
System.Threading.ContextCallback:Invoke(Object)
System.Threading.ExecutionContext:RunInternal(ExecutionContext, ContextCallback, Object, Boolean)
System.Runtime.CompilerServices.MoveNextRunner:Run()
System.Action:Invoke()
System.Threading.ContextCallback:Invoke(Object)
System.Threading.Tasks.AwaitTaskContinuation:RunCallback(ContextCallback, Object, Task&)
System.Threading.Tasks.Task:FinishContinuations()
System.Threading.Tasks.Task`1:TrySetResult(TResult)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:SetResult(TResult)
X.Companion.UI.Views.<CreateTestExecutionDocument>d__6:MoveNext()
System.Threading.ContextCallback:Invoke(Object)
System.Threading.ExecutionContext:RunInternal(ExecutionContext, ContextCallback, Object, Boolean)
System.Runtime.CompilerServices.MoveNextRunner:Run()
System.Action:Invoke()
System.Threading.ContextCallback:Invoke(Object)
System.Threading.Tasks.AwaitTaskContinuation:RunCallback(ContextCallback, Object, Task&)
System.Threading.Tasks.Task:FinishContinuations()
System.Threading.Tasks.Task`1:TrySetResult(TResult)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:SetResult(TResult)
X.Companion.Systems.<CreateTestExecution>d__19:MoveNext()
System.Threading.ContextCallback:Invoke(Object)
System.Threading.ExecutionContext:RunInternal(ExecutionContext, ContextCallback, Object, Boolean)
System.Runtime.CompilerServices.MoveNextRunner:Run()
System.Action:Invoke()
System.Threading.SendOrPostCallback:Invoke(Object)
UnityEngine.WorkRequest:Invoke()
UnityEngine.UnitySynchronizationContext:Exec()

[AVProMovieCapture] Start File Capture: 852x480 @ 30.00fps [RGBA32] vcodec:'H264' audio source:'iPhone Microphone' acodec:'AAC' to file: '/var/mobile/Containers/Data/Application/F901A4CC-F21E-4277-AA63-04353C91E56A/Documents/recording-wPteLmECgBO1DGx1StdL.mp4'
RenderHeads.Media.AVProMovieCapture.CaptureBase:PrepareCapture()
RenderHeads.Media.AVProMovieCapture.CaptureBase:StartCapture()
X.Companion.<StartRecordingCoroutine>d__18:MoveNext()
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
X.Companion.UI.Views.<Start>d__4:MoveNext()
System.Threading.ContextCallback:Invoke(Object)
System.Threading.ExecutionContext:RunInternal(ExecutionContext, ContextCallback, Object, Boolean)
System.Runtime.CompilerServices.MoveNextRunner:Run()
System.Action:Invoke()
System.Threading.ContextCallback:Invoke(Object)
System.Threading.Tasks.AwaitTaskContinuation:RunCallback(ContextCallback, Object, Task&)
System.Threading.Tasks.Task:FinishContinuations()
System.Threading.Tasks.Task`1:TrySetResult(TResult)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:SetResult(TResult)
X.Companion.UI.Views.<CreateTestExecutionDocument>d__6:MoveNext()
System.Threading.ContextCallback:Invoke(Object)
System.Threading.ExecutionContext:RunInternal(ExecutionContext, ContextCallback, Object, Boolean)
System.Runtime.CompilerServices.MoveNextRunner:Run()
System.Action:Invoke()
System.Threading.ContextCallback:Invoke(Object)
System.Threading.Tasks.AwaitTaskContinuation:RunCallback(ContextCallback, Object, Task&)
System.Threading.Tasks.Task:FinishContinuations()
System.Threading.Tasks.Task`1:TrySetResult(TResult)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:SetResult(TResult)
X.Companion.Systems.<CreateTestExecution>d__19:MoveNext()
System.Threading.ContextCallback:Invoke(Object)
System.Threading.ExecutionContext:RunInternal(ExecutionContext, ContextCallback, Object, Boolean)
System.Runtime.CompilerServices.MoveNextRunner:Run()
System.Action:Invoke()
System.Threading.SendOrPostCallback:Invoke(Object)
UnityEngine.WorkRequest:Invoke()
UnityEngine.UnitySynchronizationContext:Exec()

2022-07-25 16:52:09.364249-0300 X[63619:4827906] -[AGXA10FamilyCommandBuffer renderCommandEncoderWithDescriptor:], line 380: error 'A command encoder is already encoding to this command buffer'
-[AGXA10FamilyCommandBuffer renderCommandEncoderWithDescriptor:]:380: failed assertion `A command encoder is already encoding to this command buffer'
dyld4 config: DYLD_INSERT_LIBRARIES=/Developer/usr/lib/libMainThreadChecker.dylib:/Developer/Library/PrivateFrameworks/DTDDISupport.framework/libViewDebuggerSupport.dylib

gpiffaretti avatar Jul 25 '22 19:07 gpiffaretti

The root call is done in an async void Start method. This is because we need to await some API calls. Then the StartCoroutine method is called, and the StartCapture method is called inside a coroutine. So it is my understanding that the code will run in the main thread during the coroutines execution in the Unity normal lifecycle. But there are some Task and Threading stuff in the logs. I wonder if this is causing an issue?

gpiffaretti avatar Jul 25 '22 20:07 gpiffaretti

I tried removing any task related code. And switched to a purely coroutine based approach to wait for async stuff (with WaitUntil). Still getting the error:

SessionRecorder: Start Recording
X.Companion.<StartRecordingCoroutine>d__19:MoveNext()
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
X.Companion.UI.Views.<WaitForTestExecutionCreation>d__7:MoveNext()
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)

[AVProMovieCapture] Start File Capture: 852x480 @ 30.00fps [RGBA32] vcodec:'H264' to file: '/var/mobile/Containers/Data/Application/95A482D4-5753-4942-BC24-A9BC30E483E7/Documents/recording-4RBFyxCJpm1plLWZZQpY.mp4'
RenderHeads.Media.AVProMovieCapture.CaptureBase:PrepareCapture()
RenderHeads.Media.AVProMovieCapture.CaptureBase:StartCapture()
X.Companion.<StartRecordingCoroutine>d__19:MoveNext()
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
X.Companion.UI.Views.<WaitForTestExecutionCreation>d__7:MoveNext()
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)

2022-07-25 18:19:15.410108-0300 X[64265:4858723] -[AGXA10FamilyCommandBuffer renderCommandEncoderWithDescriptor:], line 380: error 'A command encoder is already encoding to this command buffer'
-[AGXA10FamilyCommandBuffer renderCommandEncoderWithDescriptor:]:380: failed assertion `A command encoder is already encoding to this command buffer'
dyld4 config: DYLD_INSERT_LIBRARIES=/Developer/usr/lib/libMainThreadChecker.dylib:/Developer/Library/PrivateFrameworks/DTDDISupport.framework/libViewDebuggerSupport.dylib

gpiffaretti avatar Jul 25 '22 21:07 gpiffaretti

does this look suspicious? I'm creating a render texture right before setting it and starting the capture:

TVLog.Debug(Tag, $"Create internal texture for blitting video feed");
internalRenderTexture = new RenderTexture(recordTexture.value.width, recordTexture.value.height, 0);

// set render texture to record
videoCapture.SetSourceTexture(internalRenderTexture);
videoCapture.FilenamePrefix = filename;
videoCapture.OutputFolderPath = Application.persistentDataPath;

TVLog.Debug(Tag, $"Start Recording");
videoCapture.StartCapture();

The previous version of the code was capturing a Texture2D that already existed, but still got the error. I tried this to see if there was a problem with the source. That Texture2D is being written by UnityRenderStreaming, so I created a new texture and I blit the contents from one to the other in the Update methods, so that both systems use separate resources. Just in case this was causing an issue. I'm running out of ideas :/ now I'm kinda trying things randomly to see if I find something

gpiffaretti avatar Jul 25 '22 21:07 gpiffaretti

I added internalRenderTexture.Create and a WaitForSeconds(3f) before calling StartCapture() but still crashing

gpiffaretti avatar Jul 26 '22 03:07 gpiffaretti

I'm not familiar with the UnityRenderStreaming package but I think the problem might be with how it operates with the rest of Unity as you say our Texture Capture Demo works fine.

Our plugin captures frames on Unity's render thread and we do call end on Unity's current command encoder before starting our own encoder which suggests something is interfering with this process.

The only thing I can think of that might potentially help is to have our own command buffer within the plugin, this would not be a trivial change to make though.

MorrisRH avatar Jul 26 '22 10:07 MorrisRH

will turn off UnityRenderStreaming in our scene to confirm and try to isolate the problem

gpiffaretti avatar Jul 26 '22 13:07 gpiffaretti

Ok so turning off UnityRenderStreaming solved the issue. Unfortunately UnityRenderStreaming is very tightly coupled to our core systems, and I am forced to look for another recording solution :( Wish I could add a hotfix myself or something, I was really happy with the package.

Thanks for your help.

gpiffaretti avatar Jul 26 '22 18:07 gpiffaretti

Well good to know that UnityRenderStreaming is the cause. We'll do some investigation to see if there is anything we can do from our end. It might be worth opening an issue with Unity about this though as there is a good chance that other plugins that do rendering will be broken as well.

MorrisRH avatar Jul 27 '22 07:07 MorrisRH

Hello, I used CaptureFromCamera in 2019.3.2 to set the Resolution to Custom, which can also be used for region capture. However, in 2019.4.20, when I set Custom to export videos, it will become full screen compression. What's the difference

1745098830 avatar Dec 28 '22 03:12 1745098830

Hello, I used CaptureFromCamera in 2019.3.2 to set the Resolution to Custom, which can also be used for region capture. However, in 2019.4.20, when I set Custom to export videos, it will become full screen compression. What's the difference

Hey @1745098830, please open a new issue as this issue is specific to using UnityRenderStreaming

MorrisRH avatar Jan 05 '23 09:01 MorrisRH