Multiple video frame processors + implementation for ios

[8BallBomBom]

Original pr and commits by wjaykim over here #1331 Decided to just grab everything and make sure things were up to date with the master branch. Saves a lot of the previous mess, my bad.

Should be all good but always best to test. We don't currently have an example for iOS but there is one in the old pr for Android.

[AleanM]

Sorry i have question, can i use methods for frame processing in swift module. If i install webrtc from videoeffects branch or i have to wait until accept this pr?

[AlKor13]

[magiccDev]

Hi @8BallBomBom,

Thank you for this PR!

if you are interested, I would like to sponsor this feature, could you please let me know how I can do that?

[8BallBomBom]

Would need to talk with @saghul about that 🤔 and the original commit was by @wjaykim

[wjaykim]

I'll try to make a example use of the processor when possible. It's been a while after I initially made the pull request(and I'm not working on webrtc related projects now), so I have to realign the current codebases.

[saghul]

Thank you! Nothing has significantly changed, at least ;-)

[anilgupta83]

I think the changes in VideoEffectProcessor.java, specifically onFrameCaptured function has introduced a bug where frames are not sent back to webrtc resulting in black screen. I modified the code to pick just the first processor, used the old logic and it fixed the problem.

        frame.retain();
        VideoFrameProcessor firstProcessor = videoFrameProcessors.get(0);
        VideoFrame outputFrame = firstProcessor.process(frame, textureHelper);
        if(outputFrame==null){
            mSink.onFrame(frame);
            frame.release();
            return;
        }
        mSink.onFrame(outputFrame);
        outputFrame.release();
        frame.release();

also, while looping through the list of processors shouldn't we use the last valid frame returned by a processor? Current code uses the original frame.

[wjaykim]

I added simple video frame processor example here.

[saghul]

@8BallBomBom are you ready with this?

[8BallBomBom]

Should be good to go. Left out changing the capturer to the base class but that could be addressed.

@anilgupta83 I never managed to reproduce the issue you experienced.

[saghul]

@8BallBomBom Thanks! Can you run npm run format so the style is checked and then commit the result and land it?

[8BallBomBom]

Done and done.

[anilgupta83]

@anilgupta83 I never managed to reproduce the issue you experienced.

Probably something wrong with my processor code, it has changed since then. I have tested the changes you have merged and it works as expected. Kudos!