Support for PushAudioFrame and PushVideoFrame

[Saeki-M]

Hello, thank you for the great work! I am trying to implement an application with screen sharing involved. I would like to capture computer screen and audio, apply some machine learning based effects and push it to the other user.

I believe the related methods are PushAudioFrame and PushVideoFrame, but would it be possible to implement these?

[zhangtao1104]

Yes, you can capture the video frame and audio frame by yourself and push it into agora sdk by PushAudioFrame and PushVideoFrame api.

You can also choose the api named startScreenCaptureByxxx api and enableLoopbackRecording.

[YiqingJackieHuang]

Hi @Saeki-M! Our SDK does not support pushAudioFrame and pushVideoFrame at this point. Sorry for the inconvenience! We will discuss this issue within our team and reply to you very soon.

[billychasen]

If pushVideoFrame is not available, how do you send video frames to agora in python?

[Saeki-M]

Currently the only way is to set up a virtual camera device and select that device as the recording device using setDevice function. I use OBS to capture the screen and send it to the virtual camera, but I think you can also do that in python using libraries like this https://github.com/letmaik/pyvirtualcam

[billychasen]

Oh neat, thanks for heads up @Saeki-M

[mybadstudios]

I see the SDK now includes a demo for doing exactly this in Unity. I ran the demo and it works perfectly but when I move it into my real project I get error -2 from the PushVideoFrame function. I cant seem to find any info on what this error actually means or how to fix it. Does anyone have any ideas?

[yoboitel]

Hey @mybadstudios , any link to this demo ? I'd love to study it as i also need to understand the pushVideoFrame api on unity :)