neuraltalk2 icon indicating copy to clipboard operation
neuraltalk2 copied to clipboard

Published 20 hours ago • verified publisher icon karpathy

Could neuraltalk2 caption images by using other language such as Chinese?

Open ploverlake opened this issue 7 years ago • 7 comments

As we know that the grammar is different between English and Chinese, could neuraltalk2 work well if I describe my JPG files by using Chinese?

ploverlake avatar Sep 18 '16 07:09 ploverlake

If you have the training data on Chinese. You can try.

menglin0320 avatar Nov 17 '16 07:11 menglin0320

Generate coco_raw.json with captioning in chinese language. Then train the model based on that. After that you will be able to evaluate your images with captioning in chinese language

andevani avatar Jan 17 '17 11:01 andevani

Anyone tried already for other languages? I need to know if the grammar alignment part needs to be worked or not..

sivakumarkumaravelu avatar Feb 26 '17 16:02 sivakumarkumaravelu

It will caption as per the Chinese sentences you have used to train the model. So if the training model contains correct grammatical sentences then it will caption correct sentences. Please note that I have not tried with the other languages, its my assumption.

andevani avatar Feb 27 '17 05:02 andevani

Is there a successful example?Can you share examples?

j2233ack avatar Mar 26 '17 13:03 j2233ack

I just generated the coco_raw.json and then I modified to my Language. The prepro script cannot import because it cannot read UTF-8 characters. So the solution needs code change and not straight forward. I did change the prepro script to accept UTF-8 encoding. Yet i have been hit with another road block in training. Will keep posted if something works out further.

sivakumarkumaravelu avatar Mar 26 '17 13:03 sivakumarkumaravelu

Please teach me how to generate coco_raw.json. I have recently explored this side

j2233ack avatar Mar 28 '17 05:03 j2233ack