Possible to infer word by word?

[rumbleFTW]

I was wondering if it was possible to infer coherent speech word by word, without the whole sentence requiring to be passed together. This would greatly decrease the inference time and real-life latency.