Custom Video Source using AVFoundation

[andreasteich]

I'm looking for a way to add a custom video source like providing the video frames by AVCapturesession with a custom captureOutput implementation. Is that possible using LiveKit?

[ming-chu]

I want to implement the custom CameraCapturer too.

But looks like it was using RTCCameraVideoCapturer from webrtc and also cannot extends VideoCapturer outside the module.

[andreasteich]

From my point of view, it would be enough to know how to feed the frames to webrtc manually. So I could retrieve the frames via captureOutput and hand them over to WebRTC videoCapturer?

[ming-chu]

@andreasteich then I think maybe you want something like this LocalVideoTrack.createBufferTrack().

[andreasteich]

Okay thank you very much! Do you have some more informarion about that? Can't find documentation for that...

[ming-chu]

Hi @andreasteich ,

here is the doc about LocalVideoTrack.createBufferTrack()

here is the doc about BufferCapturer

Here is some sudo code for your reference:

let bufferTrack = LocalVideoTrack.createBufferTrack()
let capturer = bufferTrack.capturer as? BufferCapturer
// var yourBuffer: CMSampleBuffer?
capturer?.capture(yourBuffer)

[andreasteich]

You're amazing thank you very much!

So if I understand it correctly, when I'm receiving CMSampleBuffers inside captureOutput of AVCaptureSession, I can pass those buffers on to the capturer.capture(...) with proper configuration provided for BufferCapturer and LocalVideoTrack on setup?

Can you confirm @ming-chu ?

[ming-chu]

You're amazing thank you very much!

So if I understand it correctly, when I'm receiving CMSampleBuffers inside captureOutput of AVCaptureSession, I can pass those buffers on to the capturer.capture(...) with proper configuration provided for BufferCapturer and LocalVideoTrack on setup?

Can you confirm @ming-chu ?

I think so, but honestly I didn't tested it out. I would like to hear from you about the results 😃

[andreasteich]

So I keep this issue until implementation done. Thank you very much!

[Negi-Rohit]

Hey @andreasteich did you manage to implement customCapturer? I’m also working on something similar, and if you could share any code or guidance for publishing a buffer

[andreasteich]

@Negi-Rohit yes, was able to do so by following @ming-chu instructions. Just Create the BufferTrack and pass it some CMSampleBuffer or CVPixelBuffer objects.

Works incredibly well!

[Negi-Rohit]

@Negi-Rohit yes, was able to do so by following @ming-chu instructions. Just Create the BufferTrack and pass it some CMSampleBuffer or CVPixelBuffer objects.

Works incredibly well!

@ming-chu and @andreasteich Thank you very much

[PiyushSaneparaTPC]

Thanks @ming-chu with this I can send my buffer