sentry [Mobile Replay] Skip segments when no UI changes happened

Once we GA mobile replay:

https://github.com/getsentry/sentry/issues/74441

In the event there were no changes in the UI, we skip taking a screenshot and reuse the last one. But we're still making video segments and sending them over the network. The player expect these segments in order to play properly.

An improvement in recording and playback performance is to skip screenshots and making segments altogether. But extend the wire protocol to instruct the player how to continue the playback.

On Android, we know when there were no changes in the UI that triggered a re-render. On iOS, we might be able to use swizzling on viewNeedsLayout and viewNeedsDisplay to set a flag if we need to take the screenshot.

This would affect native iOS/Android as well as any cross platform implementation on top such as React Native, Flutter.

Sep 03 '24 15:09 bruno-garcia

Routing to @getsentry/product-owners-replays for triage ⏲️

Sep 03 '24 15:09 getsantry[bot]

I've tried this out locally, and I think it already mostly works on the player side, there's three potential things to fix:

Fix seek functionality
Indicate "live" replays. Since we're not sending a segment every 5s, it may look like the replay is finished/done
Breadcrumbs appear to be out-of-sync with the playback (maybe needs a fix on the SDK side as well)

Sep 05 '24 07:09 romtsn

Indicate "live" replays. Since we're not sending a segment every 5s, it may look like the replay is finished/done

@jas-kas for thoughts? We should bring this to the team but I was always a fan of the idea of an indication in the UI even if "this session might still be happening". Like the wording having a sense of 'probably' as opposed to certainly. But having a way to tell the user that session is possibly not over yet.

Oct 14 '24 23:10 bruno-garcia

@bruno-garcia I like this idea! It 100% applies to web replays too. Just depends on how accurate we can be with indicating something is 'live'

Oct 18 '24 19:10 jas-kas