Video call hang Simulator to iPhone

[saroar]

Does anybody have an idea why my video call stuck(simulator) :( after 20/30s but iPhone working fine I am calling from simulator to iPhone 6

[ShaikBaji1506]

hello are you able to send the data and receive the data after connecting to the socket and establishing the connection ?

[saroar]

@ShaikBaji1911 thanks for asking i don't test it but my voice calling working only video is hang :( today i try in production still same bug

[stasel]

Hi @saroar I don't think that video is supported well on a simulator. I suggest testing video calls on real devices

[saroar]

@stasel i think you missed my note i already try in production with 2 real device its does the same here is app if you have time you can check too https://apps.apple.com/us/app/addaa/id1398824560

[stasel]

I tested video between my iPad and iPhone and the video call works great and doesn't hang after 30 seconds. Maybe the issue is related to your stun/turn servers or iOS related code.

Maybe turning on WebRTC logging can help?

[saroar]

@stasel you try my production app yes? about WebRTC logging i will check

[stasel]

No, I used my demo app

[saroar]

can you try my production app please @stasel i am trying to find my turn server logs i cant find anything there :(

[saroar]

here my turn server working good

[saroar]

@ShaikBaji1911 i check today my data still passing :( but video hang after some times and don't know how to debug it :(

here is my all code i use for my webRtc call function https://gist.github.com/saroar/053e387807e42d8b39154739907c22fb https://gist.github.com/saroar/0a9aa624c67cf7667f66e0b844977556 https://gist.github.com/saroar/8fd178f03795c24c04d052bd481afadd

[saroar]

@stasel I think I found an issue CPU use more than 100% that's why it's disconnected but how i can fixed it :(?

[saroar]

@stasel i use your this code yes same problem video is disconnect very quick CPU become more then 100%

[ShaikBaji1506]

@saroar hello dear I have mailed to [email protected] can I have your mail id I had an issue with the same sending data emit and receive when I have used the RTC Example git I have seen your message there seems you were looking for one to many video call right ? please help me I have used the RTCExample demo there I have connected to socket and now I want to emit the data how it is not sending data to server side please help me

[saroar]

@stasel I test your code into my ios 12.4 swift 5 its same error please would you like to check it your self once more time before please delete your driver then build it

[saroar]

@stasel I found a bug in ur code here is it https://github.com/stasel/WebRTC-iOS/blob/master/WebRTC-Demo-App/Sources/Services/WebRTCClient.swift#L104

we can update it like this

    // MARK: Media
    func startCaptureLocalVideo(cameraPositon: AVCaptureDevice.Position, renderer: RTCVideoRenderer) {

        let videoWidth =  640
        let videoHeight = 640*16/9
        var fps = 30
        
        if let capturer = self.videoCapturer as? RTCCameraVideoCapturer {
            
            var targetDevice: AVCaptureDevice?
            var targetFormat: AVCaptureDevice.Format?
            
            // find target device
            let devicies = RTCCameraVideoCapturer.captureDevices()
            devicies.forEach { (device) in
                if device.position ==  cameraPositon {
                    targetDevice = device
                }
            }

            // find target format
            let formats = RTCCameraVideoCapturer.supportedFormats(for: targetDevice!)
            formats.forEach { (format) in
                let fpsR = (format.videoSupportedFrameRateRanges.sorted { return $0.maxFrameRate < $1.maxFrameRate }.last)
                fps = Int(fpsR?.maxFrameRate ?? 30)
                
                debugPrint(#line, fps)
                
                for _ in format.videoSupportedFrameRateRanges {
                    let description = format.formatDescription as CMFormatDescription
                    let dimensions = CMVideoFormatDescriptionGetDimensions(description)
                    
                    if dimensions.width == videoWidth && dimensions.height == videoHeight {
                        targetFormat = format
                    } else if dimensions.width == videoWidth {
                        targetFormat = format
                    }
                }
            }
            //Int(fps.maxFrameRate)
            capturer.startCapture(with: targetDevice!, format: targetFormat!, fps: fps)
            
            self.localVideoTrack?.add(renderer)
            
        } else if let capturer = self.videoCapturer as? RTCFileVideoCapturer {
            print(#line, "setup file video capturer")
            if let _ = Bundle.main.path( forResource: "sample.mp4", ofType: nil ) {
                capturer.startCapturing(fromFileNamed: "sample.mp4") { (err) in
                    print(err)
                }
            } else {
                print("file did not faund")
            }
        }
    }

or you can update or modify my code and push it

[stasel]

Hi @saroar Thanks for investigating the issue yourself. I think that it is expected for WebRTC to have heavy load on the device CPU because it does a lot of things like encoding/decoding video and sending it through the network.

Can you elaborate a bit how does your code different from the one in this repo and how does it solves the issue?

Thanks

[saroar]

@stasel what I know the problem is here https://github.com/stasel/WebRTC-iOS/blob/master/WebRTC-Demo-App/Sources/Services/WebRTCClient.swift#L113 when I remove this code about formating it working fine.

[stasel]

Thanks for the info. I’ll try to test this out. Maybe the very high res may cause performance issues

[stasel]

Hi @saroar Apologies for later response. I tried to reproduce the issue on my devices but it looks like video is working well with 60%-70% CPU load in my demo app. I used iPhone 5s and iPhone X devices. Lowering the resolution helped to reduce memory and CPU load but even with highest res I manage to have video stream without any interruptions

[saroar]

@stasel with same code i video was disconnected on production but after change this code

            let format = (RTCCameraVideoCapturer.supportedFormats(for: frontCamera).sorted { (f1, f2) -> Bool in
                let width1 = CMVideoFormatDescriptionGetDimensions(f1.formatDescription).width
                let width2 = CMVideoFormatDescriptionGetDimensions(f2.formatDescription).width
                return width1 < width2
            }).last

i never had disconnect my video call i dont may be i am done something wrong in my code but working fine my video call now after change that part of code maybe in future somebody will find soemting and help us imporve our code i use choose highest fps and very good quility viedo call happy for that problem was format ing

so you don't change anything? i don't see any new update code

[pixelmax8k]

i am seeing the same disconnect @saroar reported. I am on iPhone XS

After using the code snippet @saroar pasted a few comments above, it's working fine.

[saroar]

@pixelmax8k are you use on your production?

[stasel]

i am seeing the same disconnect @saroar reported. I am on iPhone XS

After using the code snippet @saroar pasted a few comments above, it's working fine.

Was this from this demo app example? Did you use a simulator besides the iPhone XS?

[MussaCharles]

@stasel I found a bug in ur code here is it https://github.com/stasel/WebRTC-iOS/blob/master/WebRTC-Demo-App/Sources/Services/WebRTCClient.swift#L104

we can update it like this

    // MARK: Media
    func startCaptureLocalVideo(cameraPositon: AVCaptureDevice.Position, renderer: RTCVideoRenderer) {

        let videoWidth =  640
        let videoHeight = 640*16/9
        var fps = 30
        
        if let capturer = self.videoCapturer as? RTCCameraVideoCapturer {
            
            var targetDevice: AVCaptureDevice?
            var targetFormat: AVCaptureDevice.Format?
            
            // find target device
            let devicies = RTCCameraVideoCapturer.captureDevices()
            devicies.forEach { (device) in
                if device.position ==  cameraPositon {
                    targetDevice = device
                }
            }

            // find target format
            let formats = RTCCameraVideoCapturer.supportedFormats(for: targetDevice!)
            formats.forEach { (format) in
                let fpsR = (format.videoSupportedFrameRateRanges.sorted { return $0.maxFrameRate < $1.maxFrameRate }.last)
                fps = Int(fpsR?.maxFrameRate ?? 30)
                
                debugPrint(#line, fps)
                
                for _ in format.videoSupportedFrameRateRanges {
                    let description = format.formatDescription as CMFormatDescription
                    let dimensions = CMVideoFormatDescriptionGetDimensions(description)
                    
                    if dimensions.width == videoWidth && dimensions.height == videoHeight {
                        targetFormat = format
                    } else if dimensions.width == videoWidth {
                        targetFormat = format
                    }
                }
            }
            //Int(fps.maxFrameRate)
            capturer.startCapture(with: targetDevice!, format: targetFormat!, fps: fps)
            
            self.localVideoTrack?.add(renderer)
            
        } else if let capturer = self.videoCapturer as? RTCFileVideoCapturer {
            print(#line, "setup file video capturer")
            if let _ = Bundle.main.path( forResource: "sample.mp4", ofType: nil ) {
                capturer.startCapturing(fromFileNamed: "sample.mp4") { (err) in
                    print(err)
                }
            } else {
                print("file did not faund")
            }
        }
    }

or you can update or modify my code and push it

@saroar Did you end up using this? I tried it on my case it fixed the bug too, Thanks :)

[saroar]

@MussaCharles I already use my code which I fixed and share on production you can check my app if want :) my app working good thanks

[MussaCharles]

@saroar I ended up using the code you shared too, thanks. I will have a look on your app :)

[stasel]

@MussaCharles Thanks for the feedback. In your code I see that you are selecting a fixed height/width size. Isn't it going to stream a low quality video?

[saroar]

@MussaCharles I have the best quality video call compare with facebook/viber/whatsup/imo

[MussaCharles]

@MussaCharles Thanks for the feedback. In your code I see that you are selecting a fixed height/width size. Isn't it going to stream a low quality video?

@stasel it's @saroar code, I used it and it fixed my issue and the quality if okay with me but I will dig deep again when I get time to review video call part of our app. Thanks

[kmadiar]

@stasel I found a bug in ur code here is it https://github.com/stasel/WebRTC-iOS/blob/master/WebRTC-Demo-App/Sources/Services/WebRTCClient.swift#L104

we can update it like this

    // MARK: Media
    func startCaptureLocalVideo(cameraPositon: AVCaptureDevice.Position, renderer: RTCVideoRenderer) {

        let videoWidth =  640
        let videoHeight = 640*16/9
        var fps = 30
        
        if let capturer = self.videoCapturer as? RTCCameraVideoCapturer {
            
            var targetDevice: AVCaptureDevice?
            var targetFormat: AVCaptureDevice.Format?
            
            // find target device
            let devicies = RTCCameraVideoCapturer.captureDevices()
            devicies.forEach { (device) in
                if device.position ==  cameraPositon {
                    targetDevice = device
                }
            }

            // find target format
            let formats = RTCCameraVideoCapturer.supportedFormats(for: targetDevice!)
            formats.forEach { (format) in
                let fpsR = (format.videoSupportedFrameRateRanges.sorted { return $0.maxFrameRate < $1.maxFrameRate }.last)
                fps = Int(fpsR?.maxFrameRate ?? 30)
                
                debugPrint(#line, fps)
                
                for _ in format.videoSupportedFrameRateRanges {
                    let description = format.formatDescription as CMFormatDescription
                    let dimensions = CMVideoFormatDescriptionGetDimensions(description)
                    
                    if dimensions.width == videoWidth && dimensions.height == videoHeight {
                        targetFormat = format
                    } else if dimensions.width == videoWidth {
                        targetFormat = format
                    }
                }
            }
            //Int(fps.maxFrameRate)
            capturer.startCapture(with: targetDevice!, format: targetFormat!, fps: fps)
            
            self.localVideoTrack?.add(renderer)
            
        } else if let capturer = self.videoCapturer as? RTCFileVideoCapturer {
            print(#line, "setup file video capturer")
            if let _ = Bundle.main.path( forResource: "sample.mp4", ofType: nil ) {
                capturer.startCapturing(fromFileNamed: "sample.mp4") { (err) in
                    print(err)
                }
            } else {
                print("file did not faund")
            }
        }
    }

or you can update or modify my code and push it

I had the same issue. @saroar Your code works for me, thank you very much👍