speechrtc icon indicating copy to clipboard operation
speechrtc copied to clipboard

Speech recognition using webrtc for FirefoxOS

SpeechRTC

Speech recognition API built on top of webrtc using pocketsphinx to decode.

Demonstration:
http://www.youtube.com/watch?v=pnCRH-Iznrc
http://www.youtube.com/watch?v=cjjFvyH3kdc

Client Implementation

    var speechrtc = new SpeechRTC("en-US");
    speechrtc.gram(["Apple","Oranges","Watermelon"]);
    speechrtc.listen();
    SpeechRTC.onRecognition = function(said)
    {
      console.log( " You said "+ said );
    }

Todo

  • Dictation and continuous recognition (as this http://www.youtube.com/watch?v=3lTtCFaQF2A )
  • Cleanup at onclose
  • Better demo page
  • Stabilize grammar swtich and jsgf write (remove garbage from socket)
  • Multi-lang support
  • Connect Web Speech API at Firefox (https://dvcs.w3.org/hg/speech-api/raw-file/tip/speechapi.html and https://bugzilla.mozilla.org/show_bug.cgi?id=650295)
  • Improve the installation procedure and documentation!

Running the server

1. Pre-requirements

- libogg
- libopus
- pocketsphinx
- nodejs
- libpthread
- g++

Compiling & installing

- Compile and install all dependecies above - Compile voiceserver

Configuring

- Configure and set the paths to models and audio storage on voiceserver

Running

- node server.js - ./voiceserver

Acknowledgments

- Steven Lee (Mozilla) - Shih-Chiang (SC) (Mozilla) - Robert O'Callahan (Mozilla) - Desigan Chinniah (Mozilla) - Randell Jesup (Mozilla) - All Mozilla Community

Author

Andre Natal
[email protected]