googleads-mobile-unity icon indicating copy to clipboard operation
googleads-mobile-unity copied to clipboard

Huge number of ANRs in Unity: Input dispatching timed out...

Open bogmix opened this issue 3 years ago • 114 comments

[REQUIRED] Step 1: Describe your environment

  • Unity version: 2020.3.18f1
  • Google Mobile Ads Unity plugin version: v6.1.0, v6.1.1
  • Platform: Android
  • Platform OS version: Android 6 - 10
  • Any specific devices issue occurs on: BQru BQ-4028UP!, Lenovo Tab M7, Nokia 5, ZTE Blade L130, Redmi 9C NFC, HOMTOM S16, Huawei 荣耀9青春版
  • Mediation ad networks used, and their versions: None

[REQUIRED] Step 2: Describe the problem

In past year Admob team introduced a bug into its Unity SDK which causes a huge number of ANRs for many developers. You can find lots of reports on support forum, but the most recent topic is https://groups.google.com/g/google-admob-ads-sdk/c/6LfTaInDGtU.

The issue seems to be a deadlock related to the init of the WebView. Initially Admob team ignored and and attributed the problem to a bug in a specific Android's WebView bug, and as they said issue should have recede as people upgrade to newer WebView. Turns out it's not the case, and we see more and more ANRs in a newer versions of Android. What even more suspicious these ANRs come in waves - few days there is none of them, and then huge wave which pushes apps to a bad behaviour threshold.

This problem is super critical, since i ruins many apps and business, because Google Play severely punishes apps for even slight increase in ANRs rate.

So far Admob team was handling this issue in a disgraceful way: ignoring it initially, and "investigating" it for about 2-3 months already with no result.

There is two types of ANRs related to the same issue:

1. Broadcast of Intent { act=android.intent.action.SCREEN_OFF flg=0x58200010 }

No stack trace

2. Input dispatching timed out (Waiting to send non-key event because the touched window has not finished processing certain input events that were delivered to it over 500.0ms ago. Wait queue length: 9. Wait queue head age: 14852.2ms.)
com.unity3d.player.UnityPlayerActivity

  #00  pc 00000000000706b4  /system/lib64/libc.so (nanosleep+8)
  #00  pc 000000000045b150  /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libil2cpp.so (???)
  #00  pc 0000000000467d58  /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libil2cpp.so (???)
  #00  pc 000000000047a010  /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libil2cpp.so (???)
  #00  pc 000000000043bb3c  /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libil2cpp.so (???)
  #00  pc 00000000002d42c0  /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libunity.so (???)
  #00  pc 000000000017ad04  /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libunity.so (???)
  #00  pc 000000000017ac1c  /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libunity.so (???)
  at com.unity3d.player.ReflectionHelper.nativeProxyInvoke (Native method)
  at com.unity3d.player.ReflectionHelper.a (unavailable)
  at com.unity3d.player.ReflectionHelper$1.invoke (unavailable)
  at java.lang.reflect.Proxy.invoke (Proxy.java:1006)
  at com.google.android.gms.ads.initialization.OnInitializationCompleteListener.onInitializationComplete
  at com.google.android.gms.internal.ads.zzbdx.zzb
  at com.google.android.gms.internal.ads.zzbnp.zzbK
  at com.google.android.gms.internal.ads.zzhr.onTransact
  at android.os.Binder.transact (Binder.java:675)
  at zf.bb (com.google.android.gms.policy_ads_fdr_dynamite@[email protected]:2)
  at com.google.android.gms.ads.internal.initialization.f.e (com.google.android.gms.policy_ads_fdr_dynamite@[email protected])
  at com.google.android.gms.ads.nonagon.initialization.c.run (com.google.android.gms.policy_ads_fdr_dynamite@[email protected]:2)
  at android.os.Handler.handleCallback (Handler.java:907)
  at android.os.Handler.dispatchMessage (Handler.java:105)
  at ato.a (com.google.android.gms.policy_ads_fdr_dynamite@[email protected])
  at com.google.android.gms.ads.internal.util.f.a (com.google.android.gms.policy_ads_fdr_dynamite@[email protected]:1)
  at ato.dispatchMessage (com.google.android.gms.policy_ads_fdr_dynamite@[email protected])
  at android.os.Looper.loop (Looper.java:216)
  at android.app.ActivityThread.main (ActivityThread.java:7625)
  at java.lang.reflect.Method.invoke (Native method)
  at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run (RuntimeInit.java:524)
  at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:987)

bogmix avatar Oct 23 '21 09:10 bogmix

Hi

Asma


From: bogmix @.> Sent: Saturday, October 23, 2021 2:29:34 PM To: googleads/googleads-mobile-unity @.> Cc: Subscribed @.***> Subject: [googleads/googleads-mobile-unity] Huge number of ANRs in Unity: Input dispatching timed out... (Issue #1825)

[REQUIRED] Step 1: Describe your environment

  • Unity version: 2020.3.18f1
  • Google Mobile Ads Unity plugin version: v6.1.0, v6.1.1
  • Platform: Android
  • Platform OS version: Android 6 - 10
  • Any specific devices issue occurs on: BQru BQ-4028UP!, Lenovo Tab M7, Nokia 5, ZTE Blade L130, Redmi 9C NFC, HOMTOM S16, Huawei 荣耀9青春版
  • Mediation ad networks used, and their versions: None

[REQUIRED] Step 2: Describe the problem

In past year Admob team introduced a bug into its Unity SDK which causes a huge number of ANRs for many developers. You can find lots of reports on support forum, but the most recent topic is https://groups.google.com/g/google-admob-ads-sdk/c/6LfTaInDGtU.

The issue seems to be a deadlock related to the init of the WebView. Initially Admob team ignored and and attributed the problem to a bug in a specific Android's WebView bug, and as they said issue should have recede as people upgrade to newer WebView. Turns out it's not the case, and we see more and more ANRs in a newer versions of Android. What even more suspicious these ANRs come in waves - few days there is none of them, and then huge wave which pushes apps to a bad behaviour threshold.

This problem is super critical, since i ruins many apps and business, because Google Play severely punishes apps for even slight increase in ANRs rate.

So far Admob team was handling this issue in a disgraceful way: ignoring it initially, and "investigating" it for about 2-3 months already with no result.

There is two types of ANRs related to the same issue:

  1. Broadcast of Intent { act=android.intent.action.SCREEN_OFF flg=0x58200010 }

No stack trace

  1. Input dispatching timed out (Waiting to send non-key event because the touched window has not finished processing certain input events that were delivered to it over 500.0ms ago. Wait queue length: 9. Wait queue head age: 14852.2ms.)

com.unity3d.player.UnityPlayerActivity

#00 pc 00000000000706b4 /system/lib64/libc.so (nanosleep+8)

#00 pc 000000000045b150 /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libil2cpp.so (???)

#00 pc 0000000000467d58 /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libil2cpp.so (???)

#00 pc 000000000047a010 /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libil2cpp.so (???)

#00 pc 000000000043bb3c /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libil2cpp.so (???)

#00 pc 00000000002d42c0 /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libunity.so (???)

#00 pc 000000000017ad04 /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libunity.so (???)

#00 pc 000000000017ac1c /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libunity.so (???)

at com.unity3d.player.ReflectionHelper.nativeProxyInvoke (Native method)

at com.unity3d.player.ReflectionHelper.a (unavailable)

at com.unity3d.player.ReflectionHelper$1.invoke (unavailable)

at java.lang.reflect.Proxy.invoke (Proxy.java:1006)

at com.google.android.gms.ads.initialization.OnInitializationCompleteListener.onInitializationComplete

at com.google.android.gms.internal.ads.zzbdx.zzb

at com.google.android.gms.internal.ads.zzbnp.zzbK

at com.google.android.gms.internal.ads.zzhr.onTransact

at android.os.Binder.transact (Binder.java:675)

at zf.bb @.***@213806100057.397311113.397311113:2)

at com.google.android.gms.ads.internal.initialization.f.e @.***@213806100057.397311113.397311113)

at com.google.android.gms.ads.nonagon.initialization.c.run @.***@213806100057.397311113.397311113:2)

at android.os.Handler.handleCallback (Handler.java:907)

at android.os.Handler.dispatchMessage (Handler.java:105)

at ato.a @.***@213806100057.397311113.397311113)

at com.google.android.gms.ads.internal.util.f.a @.***@213806100057.397311113.397311113:1)

at ato.dispatchMessage @.***@213806100057.397311113.397311113)

at android.os.Looper.loop (Looper.java:216)

at android.app.ActivityThread.main (ActivityThread.java:7625)

at java.lang.reflect.Method.invoke (Native method)

at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run (RuntimeInit.java:524)

at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:987)

― You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/googleads/googleads-mobile-unity/issues/1825, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJAHSOEIW6OBDI2QVI3YN4DUIJ575ANCNFSM5GSCF2LQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

niceali avatar Oct 25 '21 07:10 niceali

Asma


From: bogmix @.> Sent: Saturday, October 23, 2021 2:29:34 PM To: googleads/googleads-mobile-unity @.> Cc: Subscribed @.***> Subject: [googleads/googleads-mobile-unity] Huge number of ANRs in Unity: Input dispatching timed out... (Issue #1825)

[REQUIRED] Step 1: Describe your environment

  • Unity version: 2020.3.18f1
  • Google Mobile Ads Unity plugin version: v6.1.0, v6.1.1
  • Platform: Android
  • Platform OS version: Android 6 - 10
  • Any specific devices issue occurs on: BQru BQ-4028UP!, Lenovo Tab M7, Nokia 5, ZTE Blade L130, Redmi 9C NFC, HOMTOM S16, Huawei 荣耀9青春版
  • Mediation ad networks used, and their versions: None

[REQUIRED] Step 2: Describe the problem

In past year Admob team introduced a bug into its Unity SDK which causes a huge number of ANRs for many developers. You can find lots of reports on support forum, but the most recent topic is https://groups.google.com/g/google-admob-ads-sdk/c/6LfTaInDGtU.

The issue seems to be a deadlock related to the init of the WebView. Initially Admob team ignored and and attributed the problem to a bug in a specific Android's WebView bug, and as they said issue should have recede as people upgrade to newer WebView. Turns out it's not the case, and we see more and more ANRs in a newer versions of Android. What even more suspicious these ANRs come in waves - few days there is none of them, and then huge wave which pushes apps to a bad behaviour threshold.

This problem is super critical, since i ruins many apps and business, because Google Play severely punishes apps for even slight increase in ANRs rate.

So far Admob team was handling this issue in a disgraceful way: ignoring it initially, and "investigating" it for about 2-3 months already with no result.

There is two types of ANRs related to the same issue:

  1. Broadcast of Intent { act=android.intent.action.SCREEN_OFF flg=0x58200010 }

No stack trace

  1. Input dispatching timed out (Waiting to send non-key event because the touched window has not finished processing certain input events that were delivered to it over 500.0ms ago. Wait queue length: 9. Wait queue head age: 14852.2ms.)

com.unity3d.player.UnityPlayerActivity

#00 pc 00000000000706b4 /system/lib64/libc.so (nanosleep+8)

#00 pc 000000000045b150 /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libil2cpp.so (???)

#00 pc 0000000000467d58 /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libil2cpp.so (???)

#00 pc 000000000047a010 /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libil2cpp.so (???)

#00 pc 000000000043bb3c /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libil2cpp.so (???)

#00 pc 00000000002d42c0 /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libunity.so (???)

#00 pc 000000000017ad04 /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libunity.so (???)

#00 pc 000000000017ac1c /data/app/com.app.id-3lXBPjkSmjrQATT-XgPpFg==/lib/arm64/libunity.so (???)

at com.unity3d.player.ReflectionHelper.nativeProxyInvoke (Native method)

at com.unity3d.player.ReflectionHelper.a (unavailable)

at com.unity3d.player.ReflectionHelper$1.invoke (unavailable)

at java.lang.reflect.Proxy.invoke (Proxy.java:1006)

at com.google.android.gms.ads.initialization.OnInitializationCompleteListener.onInitializationComplete

at com.google.android.gms.internal.ads.zzbdx.zzb

at com.google.android.gms.internal.ads.zzbnp.zzbK

at com.google.android.gms.internal.ads.zzhr.onTransact

at android.os.Binder.transact (Binder.java:675)

at zf.bb @.***@213806100057.397311113.397311113:2)

at com.google.android.gms.ads.internal.initialization.f.e @.***@213806100057.397311113.397311113)

at com.google.android.gms.ads.nonagon.initialization.c.run @.***@213806100057.397311113.397311113:2)

at android.os.Handler.handleCallback (Handler.java:907)

at android.os.Handler.dispatchMessage (Handler.java:105)

at ato.a @.***@213806100057.397311113.397311113)

at com.google.android.gms.ads.internal.util.f.a @.***@213806100057.397311113.397311113:1)

at ato.dispatchMessage @.***@213806100057.397311113.397311113)

at android.os.Looper.loop (Looper.java:216)

at android.app.ActivityThread.main (ActivityThread.java:7625)

at java.lang.reflect.Method.invoke (Native method)

at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run (RuntimeInit.java:524)

at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:987)

― You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/googleads/googleads-mobile-unity/issues/1825, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJAHSOEIW6OBDI2QVI3YN4DUIJ575ANCNFSM5GSCF2LQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

niceali avatar Oct 25 '21 07:10 niceali

Hello

Someone came up with the way to reproduce this bug, read here - https://groups.google.com/g/google-admob-ads-sdk/c/eabu4XtJLzo

bogmix avatar Oct 26 '21 07:10 bogmix

we are facing the same problem with Unity 2018.4.36 and AdMob 6.1.1

Cptweirdo avatar Oct 27 '21 07:10 Cptweirdo

Hi Everyone,

Thanks for raising awareness of this issue. We are looking at this issue, and I will do what I can to escalate the issue.

NVentimiglia avatar Oct 27 '21 18:10 NVentimiglia

Hi Everyone,

Thanks for raising awareness of this issue. We are looking at this issue, and I will do what I can to escalate the issue.

Hello

I'm afraid awareness is not enough here, it should be fixed ASAP, this bug literally kills apps. Do you have any ideas how consequences of this bug can be mitigated on our side?

bogmix avatar Oct 28 '21 07:10 bogmix

Hi Everyone, Thanks for raising awareness of this issue. We are looking at this issue, and I will do what I can to escalate the issue.

Hello

I'm afraid awareness is not enough here, it should be fixed ASAP, this bug literally kills apps. Do you have any ideas how consequences of this bug can be mitigated on our side?

The best mitigation is to switch to IronSource (pretty easy as Admob) or AppLovin. Admob has became unusable since almost a year ago (8 months from last version 5 to first version 6 and then unreliable since then).

NoobOnGames avatar Oct 28 '21 12:10 NoobOnGames

We're facing the same issue and our ranking in Google Play has been lowered as a result of that. This is a critical issue.

Yrlec avatar Oct 29 '21 10:10 Yrlec

Hello

Is there any update on this issue? Any estimates when it can be fixed? We’re seeing growing number of these ANRs and it’s depressing.

bogmix avatar Nov 01 '21 13:11 bogmix

The worst support team and the worst overall team in the world. :)

gagbaghdas avatar Nov 09 '21 10:11 gagbaghdas

The same issue has drastically lowered our rankings. It's ridiculous to get penalized from Google for a problem caused by Google..

KarahanOnarlar avatar Nov 10 '21 18:11 KarahanOnarlar

Hello

@NVentimiglia What is the state of this issue? Why it’s not marked as P1 if so many people report it?

bogmix avatar Nov 18 '21 13:11 bogmix

As soon as we started receiving more of these ANRs without changing anything, we literally started losing tens of thousands of dollars every month due to being penalized by ANR rates. Despite big spend campaings, we're not getting any organic users at all anymore. Our games are very lean, they have absolutely no issues with the exception of AdMob.

Steady 60fps, but as soon as we start loading ads, framerate drops considerably. So far we're loading ads one by one, but availability becomes low, we're losing ARPDAU, and many times this isn't even enough. Especially on low end devices and especially on android 8.1

What are we supposed to do now, @google-admin? Just start losing money because you're too lazy to fix your damn code?

History has never been kind to companies that get to have a monopoly and then mistreat their customers.

alex-blastproofgames avatar Nov 27 '21 12:11 alex-blastproofgames

@NVentimiglia

What is the state of this bug?

romannotroman avatar Nov 29 '21 09:11 romannotroman

Here's a flamegraph of all the stacktraces from ANR Exits we've found in our game. Again, there's many more ANRs (over 1000) - but these are just ANR Exits

If this doesn't help, idk what does :P flamegraph

alex-blastproofgames avatar Nov 29 '21 10:11 alex-blastproofgames

Using this plugin a lot of games use AdMOB, why google is delivering such a low quality plugin? It is also in its interest to provide robust and easy to use plugin to support admob.

adibacco avatar Nov 30 '21 22:11 adibacco

We've switched to IronSource for now. The crashes and ANR went down dramatically.

Cptweirdo avatar Dec 01 '21 14:12 Cptweirdo

@Cptweirdo are you using Admob as ad source in IronSource's mediation? We switched to IS and do use AdMob source, this improved crash and ANR rate but did not fix it completely.

denis-ftc-denisov avatar Dec 01 '21 15:12 denis-ftc-denisov

@denis-ftc-denisov indeed we are mediating AdMob via IS. We still get crashes and ANRs, but it's still a marked improvement.

Cptweirdo avatar Dec 01 '21 15:12 Cptweirdo

Hello there, what is the status of this issue? Just to mention few numbers, by 2020 Goggle Ad Network (Google revenues from advertising outside of its IP's) were $23B. More or less 30%-40% of them are from Admob and probably something like 30% might come from this plugin. So, aproximately $2B revenues were generated from this plugin itself.

Most of the problems are affecting other plugins such as the C++ and/or Java ones, or even other libraries and services such as Firebase. But lets focus on this one, $2B per year, not a big deal for a company such Google but actually it is if we look at it with perspective. The strenght of Advertising is in the capacity to deliver targeted and valuable customers (own ecosystem + external one), if they begin to lose the trust of the advertisers from a publisher perspective they will be getting less data so they will become weaker and the performance of the whole ecosystem will begin to fall. This plugin is targetting mainly Games (Where Admob has a pretty dominant situation but getting weaker due to these problems) where customers spend most of their time when on mobile (+ social networks -> none of them google owned or even showing Goggle Ads). This is a really risky situation for you guys, it has been months of failures and delays.

erpresi avatar Dec 02 '21 10:12 erpresi

I have same problem at : Unity version: 2020.3.13f1 Google Mobile Ads Unity plugin version: v6.0.1 Platform: Android Platform OS version: API level 21 - 30 Mediation ad networks used, and their versions: None

We recently updated the app with the same conditions. Does the problem only happen with newly updated apps?

DevBear89 avatar Dec 06 '21 04:12 DevBear89

Asma

niceali avatar Dec 07 '21 17:12 niceali

Hey @NVentimiglia

What is the state of this issue? Why do you ignore it?

bogmix avatar Dec 08 '21 09:12 bogmix

Is there any workaround? Same problem, almost a year has passed.

genin43 avatar Dec 08 '21 14:12 genin43

Please prioritize this issue

y0rshl avatar Dec 12 '21 19:12 y0rshl

Same problem

nvtuan96 avatar Dec 14 '21 03:12 nvtuan96

We are facing this bug since updating from googlemobileads version 5 to 6 and so on. this bug has bring our games to the floor, im afraid further delay in removing bug gonna destroy our years long progress in playstore!

Rizwan31840 avatar Dec 19 '21 22:12 Rizwan31840

Please, Google, at least a feedback.

On Sun, 19 Dec 2021 at 23:48, Rizwan31840 @.***> wrote:

We are facing this bug since updating from googlemobileads version 5 to 6 and so on. this bug has bring our games to the floor, im afraid further delay in removing bug gonna destroy our years long progress in playstore!

— Reply to this email directly, view it on GitHub https://github.com/googleads/googleads-mobile-unity/issues/1825#issuecomment-997475724, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAURNYLAZYKOMLJA5B62QK3URZOM3ANCNFSM5GSCF2LQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you commented.Message ID: @.***>

adibacco avatar Dec 20 '21 06:12 adibacco

What is the state of this issue?

romannotroman avatar Dec 20 '21 08:12 romannotroman

Hi, We are getting the same ANR's We are looking to downgrade. Do we know exactly which versions are affected and which don't? Is 6.0.0 affected?

Edit: Is version 5+ affected? I'm looking for a safe and tested version to downgrade to. Can anyone enlighten me?

Mecze avatar Dec 20 '21 14:12 Mecze