Ant-Media-Server
Ant-Media-Server copied to clipboard
PIP Mode for WebRTC video calls in iOS
Discussed in https://github.com/orgs/ant-media/discussions/5934
Originally posted by suneelreddymunagala December 18, 2023 I started working on the PIP functionality for IOS. I finished my rendering with RTCMTLVideoView. Then I realised what I'd done wrong: MTKView/GLKView aren't supported in pip mode. Now I intend to Create frame render to display user's video instead of RTCMTLVideoView/RTCEAGLVideoView . But I can't proceed since I can't convert RTCVideoTrack to RTCVideoFrame. Any code fragments or branches where we can discover the idea/solution would be really appreciated.
/* 1. get videoTrackObject 2. convert RTCVideoTrack to RTCVideoFrame 3. create pixelBuffer using RTCVideoFrame 4. create CMSampleBuffer using CVPixelBuffer 5. update in the localView/remoteview UI using cmSampleBuffer 6. implement PIP related methods. */
I referred following links to achieve the pip for video calls: pip for webrtc video calls in ios adopting_picture_in_picture_for_video_calls accessing_the_camera_while_multitasking_on_ipad