How to detect if someone is speaking when muted?

[MathieuLoutre]

Hello! Zoom has this lovely feature that tells you if you're muted when you're speaking. Is there a way to achieve this with Livekit?

Once the local track is muted, it doesn't seem to receive any updates. Would we need to run a separate mic listener while the track is muted (and not broadcasting) or is there a way to do this with Livekit as is?

[lukasIO]

hi @MathieuLoutre, there's currently no helper function in livekit to detect this, but it's an interesting idea! I imagine the implementation on the audio level to detect this, would look similar to how the sdk is detecting silence currently (https://github.com/livekit/client-sdk-js/blob/main/src/room/track/utils.ts#L80-L102), just instead of detecting silence, this would need to detect levels over a certain threshold.

[MathieuLoutre]

Thank you! What I did in the end is run another mic listener that runs when the livekit track is muted and run an audio analyser to check for spikes of activity. It works but would have loved to re-use the mic listener/track from Livekit instead of having another own (though it's fairly painless).