[ios][video_player] pip - picture in picture

[p30arena]

Use case

I have a customer asking for PIP when the app is playing a video and the user wants multi-tasking.

For Android I had to edit the plugin, and use this line of code: getActivity().enterPictureInPictureMode(); https://developer.android.com/guide/topics/ui/picture-in-picture#java

But for iOS, it is a different story, iOS does not support PIP in UIWindow or UIView, it is supported only in a Custom Player. We need a custom view for that. https://developer.apple.com/documentation/avkit/adopting_picture_in_picture_in_a_custom_player https://developer.apple.com/documentation/avfoundation/avplayerlayer

The video_player plugin is using Texture to show the video output.

Proposal

I need a way to customize the UIView created by the engine for the texture and add AVPlayerLayer support to it. Any suggestions on how can I contribute this feature to the plugin?

[aytunch]

Sorry if I am pointing the obvious but is there anything wrong with using a Stack with 2 video_players as its children?

[p30arena]

@bparrishMines @xster @gaaclarke @amirh

[gaaclarke]

I need a way to customize the UIView created by the engine for the texture and add AVPlayerLayer support to it. Any suggestions on how can I contribute this feature to the plugin?

You don't need to "customize" the view per-say. Why not just get access to the root view controller (FlutterViewController) and add your own layer to that view, or add another view to it.

Something like:

UIViewController* vc = [[[UIApplication sharedApplication] keyWindow] rootViewController];
MyPipView* pipView = [[MyPipView alloc] init];
[vc.view addSubview:pipView];

[p30arena]

Thank you @gaaclarke I think this doesn't worth a try, PiP is only supported on the iPad.

[StefanLobbenmeier]

@p30arena please reopen, PiP is supported from iOS 14

[p30arena]

@StefanLobbenmeier https://developer.apple.com/documentation/avkit/avplayerviewcontroller

Adopting Picture in Picture Playback Use AVPlayerViewController to provide Picture in Picture (PiP) playback on supported iPad models. PiP playback lets users minimize your video player to a small floating window so they can perform other activities in the primary application or even in another application. This brings a new level of multitasking capabilities to the iPad allowing users to continue playback while performing other activities on their devices.

Use AVPlayerViewController to provide Picture in Picture (PiP) playback on supported iPad models.

[StefanLobbenmeier]

This is the announcement at wwdc: https://youtu.be/GEZhD3J89ZE?t=11m44s . I will check the developer Information for iOS 14 Beta if they also mention it there, but I can confirm on my device (iPhone 7 iOS 14 beta 2) pip works

[StefanLobbenmeier]

The best I could find in the documentation is https://developer.apple.com/documentation/avkit/adopting_picture_in_picture_in_a_custom_player Which mentions neither iPad nor iPhone, but that’s just a subpage of your page so I think o am not looking at the correct place

[alexmarkley]

I'm also affected by this. My users will expect to be able to continue playing videos while the app is backgrounded wherever the platform supports it.

[thanhit93]

Please refer to my github: https://github.com/thanhit93/plugins/tree/master/packages/video_player/video_player

[StefanLobbenmeier]

@thanhit93 thanks for sharing - have you also tested it on iPad? I am a bit unsure about the conditions in your source code / if they only work on iPhone or also iPad

[thanhit93]

Hello @StefanLobbenmeier This plugin I just developed. I only tested on iphone (iOS14), ipad hasn't been tested. I will check on the iPad and fix the error as soon as possible. I also look forward to contributing from anyone to make the plugin even better.

[flymzero]

@thanhit93 This looks great. It would be better if it could be made into a plug-in package. Thank you

[anychhoice]

any updates?

[DesmondFox]

very useful feature! Any updates?

[AnmolSethi]

Please refer to my github: https://github.com/thanhit93/plugins/tree/master/packages/video_player/video_player

@thanhit93 How do I use it in my project?

[thanhit93]

Please refer to my github: https://github.com/thanhit93/plugins/tree/master/packages/video_player/video_player

@thanhit93 How do I use it in my project?

update pubspec.yaml file:

video_player:
   git:
      url: https://github.com/thanhit93/plugins.git
      ref: master
      path: packages/video_player/video_player 

import 'dart:io'; Show PIP:

onPIPTap: () async {
         if (!Platform.isIOS) return;
         await videoController.pause();
         final MediaQueryData data = MediaQuery.of(context);
         EdgeInsets paddingSafeArea = data.padding;
         double widthScreen = data.size.width;
         videoController ?.setPIP(true, left: 0, top: paddingSafeArea.top, width: widthScreen, height: 9 * widthScreen / 16);
         videoController.play();
}

Hide PIP:

if (Platform.isIOS) videoController.setPIP(false);

[IlyaMax]

@thanhit93 why not to contribute with this?

[vanlooverenkoen]

Good news!!! We had to implement Picture in Picture mode for one of our clients. We started from the implementation that @thanhit93 did. The package was so outdated that we had to start over again.

Without any knowledge of objective-c, we started the implementation.

For now everything is working in the example project. (expect swipe to close app without picture in picture manually opened, this is something I am investigating)

When testing the example project you should replace all hosted dependencies of video_player.... to path dependencies.

video_player:
   git:
      url: https://github.com/icapps/plugins.git
      ref: feature/ios-pip
      path: packages/video_player/video_player 

We do want to push this to remote. But it will probably not get merged if we don't have iOS tests. We don't have any experience so we hope we can find somebody that can help us with that. (I had no luck with my previous google_maps pull request 😞 )

https://user-images.githubusercontent.com/21172855/185163421-f4138ddd-e08c-4563-8bbc-174cd996106f.MP4

[vanlooverenkoen]

Thanks to (https://stackoverflow.com/a/67273380/5115254) I found that you still had to set an option if you want your app to automatically go to pip or not. I will add this as well to complete the full PiP implementation.

[AnmolSethi]

@vanlooverenkoen I can help you with iOS tests!

[vanlooverenkoen]

@AnmolSethi 🙏 awesome. Could you maybe already review the pull request and maybe give some suggestions where or what should be tested more, and how I should test them?

[harryandroiddev]

add this feature to the package also.

[harryandroiddev]

Good news!!! We had to implement Picture in Picture mode for one of our clients. We started from the implementation that @thanhit93 did. The package was so outdated that we had to start over again.

Without any knowledge of objective-c, we started the implementation.

For now everything is working in the example project. (expect swipe to close app without picture in picture manually opened, this is something I am investigating)

When testing the example project you should replace all hosted dependencies of video_player.... to path dependencies.

video_player:
   git:
      url: https://github.com/icapps/plugins.git
      ref: feature/ios-pip
      path: packages/video_player/video_player 

We do want to push this to remote. But it will probably not get merged if we don't have iOS tests. We don't have any experience so we hope we can find somebody that can help us with that. (I had no luck with my previous google_maps pull request 😞 )

video-player-pip-demo-ios.MP4

Try correcting the name to the name of an existing method, or defining a method named 'stopPictureInPicture'. await _videoPlayerPlatform.stopPictureInPicture(_textureId); ^^^^^^^^^^^^^^^^^^^^

[sshsato]

add this

dependency_overrides:
  video_player_platform_interface:
    git:
      url: https://github.com/icapps/plugins.git
      ref: feature/ios-pip
      path: packages/video_player/video_player_platform_interface
  video_player_avfoundation:
    git:
      url: https://github.com/icapps/plugins.git
      ref: feature/ios-pip
      path: packages/video_player/video_player_avfoundation
  video_player_android:
    git:
      url: https://github.com/icapps/plugins.git
      ref: feature/ios-pip
      path: packages/video_player/video_player_android

[t0uh33d]

Hey @vanlooverenkoen First of all, i would like to thank you for your contribution towards the PiP mode, It reduced a lot of work since i had to implement this feature in an application. It works well, but the problem is PiP mode is not getting triggered when transitioning to home screen even after setting the canStartPictureInPictureAutomaticallyFromInline to true.

I have been trying various thing, but still there is no luck. Were you able to trigger the PiP mode automatically while the user transitioned to the home screen?

[vanlooverenkoen]

Yes for me that worked perfectly. as you can see here: https://github.com/icapps/plugins/blob/565012be9017a552296e4169537e5c0098b6f288/packages/video_player/video_player/doc/demo_pip_iphone.gif

Please check out my pull request to run the example first and see what is missing in your code.

[t0uh33d]

Thank you the quick response!! I went through the code and found that i had not used the setPictureInPictureOverlayRect method. Once i did that, it worked perfectly for me.

[vanlooverenkoen]

Yes, it is important to use the setPictureInPictureOverlayRect because iOS needs to draw a native view where it should be shown. Otherwise it is not possible to automatically animate the video_player with the pip

[t0uh33d]

I was trying everything but could not make it work, so i created an ios project and implemented the pip mode there, it was working perfectly, but the same code wasn't working in Flutter, i was so confused. But i think i got it now, when we attach the AVPictureInPictureController to a native component i guess it draws the required values automatically and hence it was working.