Integrating with mobile apps

[RahmaYasser]

Hello. I'm trying to use this code in mobile app(flutter). My implementation is as following:

1. Get the voice from mic of the mobile.
2. Convert voice into udp datagram bytes(in the mobile app).
3. Make this package receive input as udp datagram bytes instead of audio files.
4. Process the datagram bytes and send the real time output file to the mobile app. Any thoughts how to change the code here to make just step 3 happen?

[juanmc2005]

Hi @RahmaYasser,

Yes this would be a nice feature to add and possibly related to torchaudio streams (see #27), which is still in beta. I would suggest to implement your own audio source by subclassing diart.sources.AudioSource (see here) and use a network library to receive the udp packages and emit them using the stream property (i.e. self.stream.on_next(udp_audio_samples) like in here).

You should also probably overwrite the is_regular method to return False so that OnlineSpeakerDiarization regularizes the stream to 5s chunks and 500ms shift. In the case you're implementing your own pipeline you can do this by calling diart.operators.regularize_stream(duration, step, sample_rate).

You can take diart.sources.MicrophoneAudioSource (see here) as an example to implement this feature because I believe it should be rather similar.

If you're up for it, I would gladly merge a PR including this new audio source.

[juanmc2005]

Mobile app integration can be implemented with the websocket protocol