james-bowman
Online/streaming LDA?
Is it possible to run LDA (or other processing algorithms) in a streaming/online fashion, such as is done with gensim? It seems that this would not easily support online processing, but I thought I'd bounce the question off of you since you know the internals much better.
Great question. All the algorithms will work in an online setting but the majority require batch training in advance. Some, like the LDA and RI algorithms could be made to work with online training with a small amount of effort. The HashingVectoriser doesn't require training so is particular suited to streaming data. I will take a look and see if I can add the online training and persistence support. In the meantime, Pull Requests are welcome :-)
