sdl_ios icon indicating copy to clipboard operation
sdl_ios copied to clipboard
verified publisher icon smartdevicelink

Error handling needed in CarWindow for failed frame encoding

Open NicoleYarroch opened this issue 5 years ago • 5 comments

Bug Report

Sometimes the pixelBuffer in SDLCarWindow is invalidated when the app is put in the background due to a VideoToolbox bug. This means that the CarWindow API can not encode the video frames and thus no video frames are sent to the head unit.

  1. This is sometimes fixed by putting the app on the device in the background and then bringing it back to the foreground. When the app is brought back to the foreground the videoEncoder is destroyed and recreated in the SDLStreamingVideoLifecycleManager. I believe this is what fixes the issue. We should try to reset the videoEncoder when the the pixelBuffer is nil.
  2. However if the bug persists after a few seconds then we should stop video streaming and let the developer know that video could not stream because of an error.
Reproduction Steps
  1. Connect navigation app using the CarWindow API to a SYNC 3 head unit. Open the SDL app.
  2. Turn off the ignition and close the doors. When screen goes dark, turn off the head unit by toggling the on/off switch on the back of the head unit. (Don't just switch off the TDK because then the library will sometimes not get a accessory disconnected notification)
  3. Lock the iPhone (This is what causes the VideoToolbox bug).
  4. Turn the TDK back on again and turn on the ignition.
  5. Unlock the iPhone
  6. If necessary, select the SDL app on the HMI.
Expected Behavior

The video streams to the head unit.

Observed Behavior

The CarWindow class does not stream video some of the time due to video encoding failing. This could happen the first time or tenth time the app is backgrounded.

OS & Version Information
  • iOS Version: 13.0
  • SDL iOS Version: 6.5
  • Testing Against: SYNC 3.0

NicoleYarroch avatar Jan 31 '20 19:01 NicoleYarroch

@joeygrover -san @theresalech -san This issue is a top priority for Toyota. Can the reproduction steps described actually reproduce this issue?

E-SAITO-TMC avatar Apr 17 '20 08:04 E-SAITO-TMC

@E-SAITO-TMC The underlying issue causing the pixelBuffer to become nil was fixed, however proper error handling has not yet been added.

joeljfischer avatar Apr 17 '20 12:04 joeljfischer

@joeljfischer -san Thank you, I understand. When will error handling be added?

E-SAITO-TMC avatar Apr 22 '20 00:04 E-SAITO-TMC

@E-SAITO-TMC It will not be added in the release next week, and we have not begun planning for future releases. We will consider it for the July release.

joeljfischer avatar Apr 22 '20 13:04 joeljfischer

@joeljfischer -san I understand. Thank you for your cooperation.

E-SAITO-TMC avatar Apr 23 '20 01:04 E-SAITO-TMC