Can we get parse down to 100ms

[kmadathil]

Moving this comment from @akprasad to create a new issue:

Also, I get around 1.8 seconds per verse:


import time
num_trials = 20
start = time.time()
for i, phrase in enumerate([
    'Darmakzetre kurukzetre samavetA yuyutsavaH',
    'mAmakAH pARqavAScEva kimakurvata saMjaya',
    ] * num_trials):
    resp = api_example(phrase, 'slp1')

end = time.time()
print((end - start) / num_trials)

Is there anything we can do to improve performance here? Ideally I'd like around 100ms per verse.

[avinashvarna]

I have my hands full, so I don't know if I can run a lot of experiments, but I'll try to think of ways to improve. It did take some work to get it down to the current timing, though there's probably room for improvement. @kmadathil , could you run a profiler to identify the bottleneck? If possible, could you run it with lexical_lookup=inria vs the default combined to see if the extra entries that result from using the MW dictionary significantly impact the runtime?

[kmadathil]

Will do. I don't think I've aggressively optimized many parts of the parser, so I'll take those up.