jsmpeg-vnc icon indicating copy to clipboard operation
jsmpeg-vnc copied to clipboard

Published 20 hours ago • verified publisher icon phoboslab

Sound?

Open benjamingr opened this issue 9 years ago • 6 comments

Is sound coming?

benjamingr avatar Jul 27 '15 14:07 benjamingr

Adding sound is not that hard, there will be only one downside and that is when the data is being transmitted. This could create a high latency(especially for low bandwidth users), but i don't think it will.

moughouz avatar Jul 27 '15 14:07 moughouz

Capturing, encoding and transmitting sound is not that difficult, however, decoding and playing Sound in the browser is. Afaik there's currently no native Browser API that can deal with streaming audio, so we'd need to use a JavaScript decoder for that as well.

phoboslab avatar Jul 28 '15 10:07 phoboslab

It's not standard yet, but in Firefox and Chrome you have https://developer.mozilla.org/en-US/docs/Web/API/MediaStream_API

benjamingr avatar Jul 28 '15 10:07 benjamingr

Web Audio API is actually well supported and gives good latencies (around .5ms on desktop, 13ms on mobile).

Additionally, there already is a JS MP3 decoder that sends data to Web Audio API: https://github.com/audiocogs/mp3.js.

WebRTC (MediaStream) would be ideal, both for audio and video, as it is meant for low-latency playback (going as far as using SCTP over UDP instead of TCP). It may require a lot more work, though.

espadrine avatar Jul 29 '15 13:07 espadrine

any API that you can use ?

jochuan avatar Apr 20 '16 15:04 jochuan

In the server side, what will be best api to capture the audio? any samples?

ghost avatar Oct 11 '17 09:10 ghost