g2p-seq2seq
g2p-seq2seq copied to clipboard
cmusphinx
Time to restore saved model?
In the function g2p.py I added a time.time() function around the command
self.model.saver.restore(self.session, os.path.join(self.model_dir, "model"))
to see how long it takes to load a pre-trained model to decode words. With a model trained with 512 nodes I get:
Time to load model: 2.53336596489
with only 64 nodes I don't get much savings:
Time to load model: 2.50763916969
which according to the python time module is output in seconds. That seems really slow. I am using the cpu instead of the gpu, because in the end if we are to include a similar NN model in our software, we don't have any gpu power on our servers. But still, when I compare it with a current openfst implementation of an n-gram model, that one is only 300ms or 0.3s to load in c++.
It may be faster if I can restore the saved file from c++ but I have to see about writing code to allow that.
This is a valid issue, but it does not depend on cpu or gpu or model size. It goes deeper into tensorflow. The architecture of the tensorflow is designed for distributed training/evaluation and uses complex messages across threads to perform operations, so it is quite slow to load the model.
For now we could recommend to avoid frequent model loading by using batch processing and similar things.
Thanks for the explanation. I just wonder how it would be used in a TTS system for instance, where one wants to predict a guessed pronunciation for a word not in the lexicon. I know it is not useful to load in the model every time a lookup is required, but still when a tts instance starts, it should be able to get a prediction for an unknown word in a reasonable amount of time. And since we don't know what the unknown words are ahead of time, we can't batch process in this case.
One could start a separate service/server and query that over local network / other bus for example. In TTS there are other databases which might not be quick to load like unitselection db for example.
