identify languages doesn't work with Telugu in v0.9

[goru001]

identify languages function which uses separate model for identifying the languages hasn't been retrained on Telugu in v0.9. Need to retrain it to support Telugu.

[goru001]

@Shubhamjain27 Will you be able to take this up?

[chaitusvk]

Please help me i will train Telugu model .. I can see Language model file in NLP for Telugu ...where is seperate model located I am Telugu Speaking Guy..

[lordzuko]

@goru001 If someone isn't working on this, I can take this up. We can use pycld2, pycld3 , it identifies all the supported language except: oriya, bengali and sanskrit.

I have used the same in my own projects and it's also used by polyglot's language detection. https://github.com/aboSamoor/polyglot/blob/d0d2aa8/polyglot/detect/base.py#L72

What do you think ?

[goru001]

@lordzuko That'll be great! Feel free to raise a PR for this.

[nitkannen]

@goru001 can I take this issue up if it is still unresolved?

[goru001]

@nitkannen Yes sure, this is still unresolved and it'll be great if you can contribute!

[nitkannen]

Sure @goru001

[nitkannen]

@goru001 can you give me some guidance as to from where I can start to retrain the Telugu model. Any notebooks or scripts used for other languages and data can be really helpful