See if WebVTT would be a good fit

[westonruter]

Instead of marking up the text with each word, there could be a track file that has words with the time indexes.

http://www.whatwg.org/specs/web-apps/current-work/multipage/the-video-element.html#the-track-element