[AndroidViewSurface] On app resume, a hidden UnityWidget renders on top of other widgets.

[timbotimbo]

Describe the bug This is a bug that occurs when a UnityWidget is kept alive in the widget tree, but hidden by another widget. In the following example the UnityWidget is in a Pageview, where it can be hidden by another page. When you leave the app and return it from the background, the UnityWidget renders on top of the visible page, and does not detect touch input.

To Reproduce Steps to reproduce the behavior:

1. Paste the example code in the main.dart file of the example project.
2. Make sure there is an export in android/unityLibrary
3. Switch between the pageview tabs to get unity to launch.
4. Move the app to the background with the Unity tab not selected.
5. Return to the app, you should see a red page, but Unity will be visible on top of it. Unity does not respond to touch events until you switch to the unity page again.

Expected behavior The Platformview is only shown when the UnityWidget is visible in the widget tree. In the example only on the blue page.

Video

https://user-images.githubusercontent.com/11444698/177542945-42ed8b0b-fbb9-4e28-a21d-a6d8ab6ea1cb.mp4

Code sample Paste the following code in main.dart of the flutter_unity_widget example project.

code to reproduce the issue

import 'package:flutter/material.dart';
import 'package:flutter_unity_widget/flutter_unity_widget.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: MyHomePage(),
    );
  }
}

class MyHomePage extends StatefulWidget {
  MyHomePage({Key key}) : super(key: key);

  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  int _selectedIndex = 0;

  PageController pageController = PageController(initialPage: 0);

  void bottomTapped(int index) {
    setState(() {
      _selectedIndex = index;
      pageController.jumpToPage(_selectedIndex);
    });
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(),
      body: PageView(
        physics: NeverScrollableScrollPhysics(),
        controller: pageController,
        children: <Widget>[
          Red(),
          Unity(),
        ],
      ),
      bottomNavigationBar: BottomNavigationBar(
        currentIndex: _selectedIndex,
        onTap: (index) {
          bottomTapped(index);
        },
        items: [
          BottomNavigationBarItem(
            icon: new Icon(Icons.home),
            label: 'Red',
          ),
          BottomNavigationBarItem(
            icon: new Icon(Icons.gamepad_outlined),
            label: 'unity',
          ),
        ],
      ),
    );
  }
}

class Red extends StatefulWidget {
  @override
  _RedState createState() => _RedState();
}

class _RedState extends State<Red> {
  @override
  Widget build(BuildContext context) {
    return Container(
      color: Colors.red,
    );
  }
}

class Unity extends StatefulWidget {
  @override
  _UnityState createState() => _UnityState();
}

class _UnityState extends State<Unity> with AutomaticKeepAliveClientMixin {
  @override
  bool get wantKeepAlive => true;
  UnityWidgetController _unityWidgetController;

  @override
  void initState() {
    super.initState();
  }

  @override
  void dispose() {
    _unityWidgetController.dispose();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    super.build(context);
    return Container(
      padding: EdgeInsets.all(50),
      color: Colors.blue,
      child: UnityWidget(
        onUnityCreated: _onUnityCreated,
        onUnityMessage: onUnityMessage,
        onUnitySceneLoaded: onUnitySceneLoaded,
        webUrl: 'http://localhost:6080',
        useAndroidViewSurface: true,
      ),
    );
  }

  void onUnityMessage(message) {}

  void onUnitySceneLoaded(SceneLoaded scene) {}

  void _onUnityCreated(controller) {
    controller.resume();
    this._unityWidgetController = controller;
  }
}

Versions

• Unity 2022.1.7f1
• Flutter 3.0.3
• fuw: 2022.1.1+4
• unitypackage: fuw-2022.1.1-v2.unitypackage
• Android 12 (maybe others too)

[timbotimbo]

After testing on fuw 2022.1.0+7 and Flutter 2.10.5, it seems to be surfaceView related. With flutter 2.10.5 the issue only occurs with useAndroidViewSurface: true, setting it to false makes it behave like it should.

Flutter 3 compatible versions require using androidViewSurface, therefore this is a regression in versions 2022.1.1+x.

[dawiddszewczyk]

@juicycleff Can you look on that issue?

[Ahmadre]

Describe the bug This is a bug that occurs when a UnityWidget is kept alive in the widget tree, but hidden by another widget. In the following example the UnityWidget is in a Pageview, where it can be hidden by another page. When you leave the app and return it from the background, the UnityWidget renders on top of the visible page, and does not detect touch input.

To Reproduce Steps to reproduce the behavior:

1. Paste the example code in the main.dart file of the example project.
2. Make sure there is an export in android/unityLibrary
3. Switch between the pageview tabs to get unity to launch.
4. Move the app to the background with the Unity tab not selected.
5. Return to the app, you should see a red page, but Unity will be visible on top of it. Unity does not respond to touch events until you switch to the unity page again.

Expected behavior The Platformview is only shown when the UnityWidget is visible in the widget tree. In the example only on the blue page.

Video

Screen_Recording_20220706-125630.mp4 Code sample Paste the following code in main.dart of the flutter_unity_widget example project.

code to reproduce the issue Versions

• Unity 2022.1.7f1
• Flutter 3.0.3
• fuw: 2022.1.1+4
• unitypackage: fuw-2022.1.1-v2.unitypackage
• Android 12 (maybe others too)

does this also happen on Flutter Stable 3.0.5?

[hidfsobg]

@Ahmadre yes,it also happened on Flutter Stable 3.0.5. I donot kown how to fix it. @juicycleff Can you look on that issue?

[Ahmadre]

@Ahmadre yes,it also happened on Flutter Stable 3.0.5. I donot kown how to fix it. @juicycleff Can you look on that issue?

Did you try:

@override
void didChangeAppLifecycleState(AppLifecycleState state) {
  switch (state) {
    case AppLifecycleState.resumed:
      unityWidgetController.resume();
      break;
    case AppLifecycleState.paused:
      unityWidgetController.pause();
      break;
    default:
  }
}

For more read Flutter's: WidgetsBindingObserver

[juicycleff]

Seems like a Flutter 3.0.0 issue taking a look

[timbotimbo]

It doesn't seem related to Flutter 3.

In Flutter 2.10.5 the bug only happens with useAndroidViewSurface: true, In Flutter 3.0.0 - 3.0.5 it always happens because useAndroidViewSurface is forced.

In Flutter 3.3.0-0.0.pre the bug only happens with useAndroidViewSurface: true. (I uncommented the Flutter 3 AndroidView section in device_method.dart to get the classic platformview back with useAndroidViewSurface: false)

[invisible-defects]

Still able to reproduce the issue. Unity's new input system doesn't seem to work with useAndroidViewSurface: true because of this bug.

[timbotimbo]

Just tried it again in Flutter 3.7.0 on Android 13. It seems to have gotten a little worse.

https://user-images.githubusercontent.com/11444698/214535426-58afcd18-1f84-4865-9434-687d6a6e5aa8.mp4

I found this Hybrid composition issue in the Flutter repo that seems like the same problem. https://github.com/flutter/flutter/issues/89558