Flaky test: testTwoBPInstancesWithVideo

[chenxiao0228]

testTwoBPInstancesWithVideo has been very flaky in the checks.

I briefly looked into it and have these findings

• I could not reproduce it on my machine
• tried to change it to 1 simulator, still flaky
• tried to add some debug logging in the code, could not get too much info. Next step could be piping the output of the process to a file and capture it.
• recording videos uses xcrun simctl io, which could be pretty expensive

For now I think we should disable it and try to root cause it in a branch.

[chenxiao0228]

@RainNapper FYI since you worked on this feature.

[RainNapper]

I think disabling it sounds good. I thought it was fixed in XC12, but it looks to have gone back. This has been working in production, so I'm not sure what's different about the test configuration.

On Tue, Jan 19, 2021 at 12:07 PM Xiao Chen [email protected] wrote:

@RainNapper https://github.com/RainNapper FYI since you worked on this feature.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/linkedin/bluepill/issues/469#issuecomment-763096873, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALB5OE4524Y6QW5EKINTXDS2XQ7ZANCNFSM4WJK2PKQ .

[ob]

I wonder if it's just memory on the macOS VMs or some other resource that gets consumed... we could try running ps and vm_stat in a loop in a separate process to see how the machine is doing...