Ant-Media-Server icon indicating copy to clipboard operation
Ant-Media-Server copied to clipboard

PIP Mode for WebRTC video calls in iOS

Open suneelreddymunagala opened this issue 1 year ago • 1 comments

Discussed in https://github.com/orgs/ant-media/discussions/5934

Originally posted by suneelreddymunagala December 18, 2023 I started working on the PIP functionality for IOS. I finished my rendering with RTCMTLVideoView. Then I realised what I'd done wrong: MTKView/GLKView aren't supported in pip mode. Now I intend to Create frame render to display user's video instead of RTCMTLVideoView/RTCEAGLVideoView . But I can't proceed since I can't convert RTCVideoTrack to RTCVideoFrame. Any code fragments or branches where we can discover the idea/solution would be really appreciated.

/* 1. get videoTrackObject 2. convert RTCVideoTrack to RTCVideoFrame 3. create pixelBuffer using RTCVideoFrame 4. create CMSampleBuffer using CVPixelBuffer 5. update in the localView/remoteview UI using cmSampleBuffer 6. implement PIP related methods. */

I referred following links to achieve the pip for video calls: pip for webrtc video calls in ios adopting_picture_in_picture_for_video_calls accessing_the_camera_while_multitasking_on_ipad

suneelreddymunagala avatar Dec 18 '23 14:12 suneelreddymunagala