SessionReplay causes user-visible frame drop

[dfed]

Platform

iOS

Environment

Production, Develop, TestFlight, Other

Installed

Swift Package Manager

Version

8.26.0

Xcode Version

15.4

Did it work on previous versions?

8.25.0

Steps to Reproduce

Set the following options:

options.experimental.sessionReplay.errorSampleRate = 1.0
options.experimental.sessionReplay.sessionSampleRate = 1.0
options.experimental.sessionReplay.redactAllText = false
options.experimental.sessionReplay.redactAllImages = false

Expected Result

Do not see user-visible frame drops

Actual Result

See user visible frame drops every second

Are you willing to submit a PR?

No response

[dfed]

Probably worth mentioning that the view dropping frames is a Unity view that's been embedded in our iOS app. Prior to 8.26.0 replay did not capture the Unity view.

[brustolin]

Thanks @dfed for reaching out. We will investigate this.

[dfed]

More info! This issue is happening in both 8.25.1 and 8.25.2, in addition to 8.26.0. 8.25.0 does not have this issue.

[brustolin]

I belive I know what cause the change. I dont know whether we can go back. But I will try to find an alternative solution.

[kahest]

related: #4235

[krystofwoldrich]

• Related to https://github.com/getsentry/sentry-react-native/issues/4499

[philprime]

Rough ideas what could be the issue here, needs further investigation:

• DefaultViewRenderer uses drawHierarchy(in:afterScreenUpdates:) which is slower than snapshotview(afterscreenupdates:) according to documentation
• DefaultViewRenderer creates a graphics context, draws the view hierarchy, renders to image. SentryViewPhotographer then creates a graphics context, draws the rendered image into the context, draws masking overlays, renders to an image. Maybe the render passes can be reduced.
• Instead of UIGraphicsImageRenderer it might be faster to directly use CGContext

https://github.com/getsentry/sentry-cocoa/blob/07475f9c7b72f62c9c53e3a3e87c26915cee693e/Sources/Swift/Tools/SentryViewPhotographer.swift#L13-L21

[krystofwoldrich]

Linking what @romtsn and I tried https://github.com/getsentry/sentry-cocoa/compare/kw/test-replay-cgcontext

[philprime]

@krystofwoldrich I understand your approach, most of UIKit's UIView use Core Animation's CALayer in the background, but AFAIK views (especially non-public SwiftUI helper views) can also directly render to the graphics context. That's maybe why you saw partial renders.

[mcguile]

I'm experiencing this issue on 8.45.0 (and was also seeing it on 8.44.0 - my first version of Sentry). I'm adding SentrySwiftUI as the only Sentry dependency.

Frame drop is hugely evident on scrolling views at a rate of 1/s - when taking the photo - and the problem is no longer apparent when disabling Session Replay options.

[krystofwoldrich]

RN User shared an example of visible frames drops when panning in maps.

https://github.com/getsentry/sentry-react-native/issues/4609

[philprime]

Quick update:

I have performed an extensive benchmark and analyzed potential improvement strategies in #4940. We'll discuss how to use the learnings and my proposed changes.

[philprime]

We released two new options with v8.47.0 (and will update the documentation shortly):

• options.sessionReplay.enableExperimentalViewRenderer to enable the experimental view renderer which should reduce frame drops significantly. This is in an experimental state but will become the new default eventually.
• options.sessionReplay.enableFastRenderering to enable even faster but incomplete rendering.

Please test the new renderer by enabling it:

options.sessionReplay.enableExperimentalViewRenderer = true

Looking forward to any feedback if this improves the performance.

[harrisrobin]

We released two new options with v8.47.0 (and will update the documentation shortly):

• options.sessionReplay.enableExperimentalViewRenderer to enable the experimental view renderer which should reduce frame drops significantly. This is in an experimental state but will become the new default eventually.
• options.sessionReplay.enableFastRenderering to enable even faster but incomplete rendering.

Please test the new renderer by enabling it:

options.sessionReplay.enableExperimentalViewRenderer = true Looking forward to any feedback if this improves the performance.

so how do we try this in react-native? Our react-native app's perf just dropped drastically right after we enabled session replay

[philprime]

@harrisrobin the options for React Native are not yet released but you can follow along this PR: https://github.com/getsentry/sentry-react-native/pull/4660

[krystofwoldrich]

The Sentry RN SDK with the performance improvements was released yesterday.

https://github.com/getsentry/sentry-react-native/releases/tag/6.10.0

import * as Sentry from '@sentry/react-native';

Sentry.init({
  integrations: [
    Sentry.mobileReplayIntegration({
      enableExperimentalViewRenderer: true,
    }),
  ],
});

[CMLCNL]

and? i am facing this issue in this package "@sentry/react-native": "^6.10.0".

[CMLCNL]

@brustolin

[kahest]

@CMLCNL did you try enabling the faster experimental view renderer that we shipped in 6.10.0? It would be valuable feedback for us if you can try it and let us know if it changes the experience for you:

Sentry.init({
  integrations: [
    Sentry.mobileReplayIntegration({
      enableExperimentalViewRenderer: true,
    }),
  ],
});

[AlirezaHadjar]

Hey @kahest I just tried out enableExperimentalViewRenderer but it didn't help. I can provide a repro if you want. here is what it looks like with and without session replay for us

https://github.com/user-attachments/assets/10e46df1-6fdd-4e1d-99aa-1afaafdda8a5

https://github.com/user-attachments/assets/af3f2772-627c-43dd-b228-6e53d5de2d0e

EDIT: this only happens on physical iPhone and not on simulator

[philprime]

Hi @AlirezaHadjar, I would highly appreciate the repro! If you do not want to post it publicly, feel free to send it to [email protected] instead.

EDIT: this only happens on physical iPhone and not on simulator

Which model of iPhone are you using?

[AlirezaHadjar]

Hi @AlirezaHadjar, I would highly appreciate the repro! If you do not want to post it publicly, feel free to send it to [email protected] instead.

EDIT: this only happens on physical iPhone and not on simulator

Which model of iPhone are you using?

Cool, I'll create one today. And I'm using iPhone 16 pro max

[AlirezaHadjar]

Hi @AlirezaHadjar, I would highly appreciate the repro! If you do not want to post it publicly, feel free to send it to [email protected] instead.

EDIT: this only happens on physical iPhone and not on simulator

Which model of iPhone are you using?

Cool, I'll create one today. And I'm using iPhone 16 pro max

@philprime here is the repro

[krystofwoldrich]

Thank you for the repro @AlirezaHadjar,

just to confirm that my observations match yours.

Adding the following to the Sentry init improves the stutters from periodical to occasional on my test device (iPhone 15).

  integrations: [
    Sentry.mobileReplayIntegration({
      enableExperimentalViewRenderer: true,
    }),
  ],

Video Replay (no extra flags)

https://github.com/user-attachments/assets/8a23f81c-5c67-47cf-b7aa-6dec9810306c

Video Replay (enableExperimentalViewRenderer: true)

https://github.com/user-attachments/assets/e7f669ce-f572-4d50-bedd-9026026c776b

[AlirezaHadjar]

Hey @krystofwoldrich yes adding enableExperimentalViewRenderer will definitely improve the performance but doesn't fix it completely. in this repro I don't notice the stutter with enableExperimentalViewRenderer but in our app it still stutters but less

[philprime]

@AlirezaHadjar thank you for the reproducible project and thank you @krystofwoldrich for checking it out.

I just created #5155 so we can have a fresh issue to investigate potential improvement strategies.

[krystofwoldrich]

I've also tried running the reproducer app with the following settings, but the output from the FastViewRendering is corrupted (example).

Sentry.mobileReplayIntegration({
  enableExperimentalViewRenderer: true,
  enableFastViewRendering: true,
}),