VideoConstraint (Dimensions + FPS) setting do not work

[anooshcnayak]

Issue

Unable to change Video Dimension & FPS

I wanted to have custom dimensions & fps setting on this library. Thus, have taken a fork and changed the buildVideoConstraints function. But these changes do not reflect. I have published the getStats output below.

These issues are also in Master Branch:

1. Master branch has set maxFps to 15, But Fps in all my testing was way above 20.
2. Master branch has set dimensions to CIF_VIDEO_DIMENSIONS, But getStats shows VGA_DIMENSIONS

Altered Code:

    private VideoConstraints buildVideoConstraints() {
        VideoDimensions dimensions = new VideoDimensions(30, 30);
        return new VideoConstraints.Builder()
                .minVideoDimensions(dimensions)
                .maxVideoDimensions(dimensions)
                .minFps(1)
                .maxFps(1)
                .build();
    }

getStats Output:

{"2F2479F9be0Ed3bB71c84a4FF2EA109B":{"localVideoTrackStats":[{"roundTripTime":5,"packetsSent":1293,"bytesSent":1401157,"trackSid":"MTc8345c9375cb15a8671f21b3e7631a62","timestamp":1606130226484.107,"ssrc":"157141198","packetsLost":0,"codec":"VP8","frameRate":21,"dimensions":{"width":640,"height":480}}],"localAudioTrackStats":[{"roundTripTime":11,"packetsSent":286,"bytesSent":25410,"trackSid":"MT88a094607f7ecb622f4b4c81a389d8d9","timestamp":1606130226484.107,"ssrc":"2486496672","packetsLost":0,"codec":"opus","jitter":3,"audioLevel":106}],"remoteVideoTrackStats":[{"packetsReceived":1273,"bytesReceived":1406908,"trackSid":"MT6fb6b186e8480933020edd18373d8cb4","timestamp":1606130226484.107,"ssrc":"1105402643","packetsLost":0,"codec":"VP8","frameRate":20,"dimensions":{"width":640,"height":480}}],"remoteAudioTrackStats":[{"packetsReceived":388,"bytesReceived":30231,"trackSid":"MTc786a923bbb986d487aecb12818c870e","timestamp":1606130226484.107,"ssrc":"2657595108","packetsLost":0,"codec":"opus","jitter":2,"audioLevel":19}]}}

Above output shows the following:: "frameRate":21,"dimensions":{"width":640,"height":480}} (the master branch itself has maxFps set to 15 - those configs are all null and void Note: Room type's tested are P2P (peer to peer) & Group

Steps to reproduce

1. Change the buildVideoConstraints as above
2. Invoke getStats and infer the stats

Expected behaviour

DImensions & FPS as given in the config

Actual behaviour

No control on fps & dimension, set either by twilio or some other libraries

Environment

• Node.js version: 13
• React Native version: 0.61.5
• React Native platform + platform version: Android 9 (28)

react-native-twilio-video-webrtc

Version: "master"

[anooshcnayak]

I have also added the captureDimensions field into the getStats api, just to verify whether capture dimensions and sent dimensions are the same. They are the same.

https://twilio.github.io/twilio-video-android/docs/latest/com/twilio/video/LocalVideoTrackStats.html#captureDimensions

[slycoder]

This seems to be happening in the underlying twilio library. I did some digging (https://github.com/twilio/video-quickstart-android/issues?q=is%3Aissue+constraints) and it seems like some devices may not support certain constraints =(.

[anooshcnayak]

Yes! I have been talking with the Support team, The engg. team is going to get back to me. Once I get a clear answer, let me update it here/ Raise approp. PR w.r.t the issue.

[anooshcnayak]

@slycoder I got the answer from Twilio. There is no recommended way in 5.x.SDK. But the 6.0 SDK released on 4th Dec has an easier methodology.

I have made the changes with 6.0, will raise the PR. Currently integrating audio switch & checking any issues with it. The audio focus listener is not used in the repo, trying to make sure it handles phone calls etc & stops the playback appropriately.

[slycoder]

Thank you for your investigation here and looking forward to any PRs you might have.

[umarniz]

@anooshcnayak I have exposed the camera constraint parameters in the JS side (though not with Android 6 SDK yet).

You can help contribute in the PR: https://github.com/blackuy/react-native-twilio-video-webrtc/pull/459