[feature] expose raw audio and video data

[davidzhao]

Is your feature request related to a problem? Please describe.

Users would like to record audio and video locally in addition to publishing to the server. Currently audio and video data are not being exposed locally.

Describe the solution you'd like

When bytes are generated, we should expose a callback to receive the streams.

[Parfyonator]

It would be a very cool and useful feature. Really looking forward to use it.

BTW, @cloudwebrtc are you planning to expose methods for tracks from the flutter-webrtc dependency or do something different? Currently captureFrame method of MediaStreamTack and MediaStreamRecorder (for video recording) are producing desaturated images and videos with artefacts on Android platform due to (maybe?) incorrect implementation of some native methods in WebRTC dependency. More details in this issue.

[cloudwebrtc]

@Parfyonator Here's a captureFrame method fix related to color bugs. maybe you can try if solves your issue https://github.com/flutter-webrtc/flutter-webrtc/pull/1258

[shalom-aviv]

Good feature Would be nice if it were possible to set different parameters for streaming and for the saved video. For example locally save a video track in 4k, but that stream was in the lowest quality.

[jtkeyva]

Great feature. Would also be nice to be able to use MLKit etc by adding access to the image stream from Dart.

[etegan]

Is this something we could expect in the near future?

[willsmanley]

@cloudwebrtc do you think its possible to get the stream for at least audio? this would be an amazing thing to have because it is necessary if you have to make an audio visualizer

[cloudwebrtc]

I should try to implement them in the next few months, which is related to native layer audio/video processing.

[jtkeyva]

yeah would be very useful to store the video or portions of it when you hit record locally as well