Chinese traineddata does not use ISO 639-3 code

[Blightbuster]

The chinese models chi_sim, chi_sim_vert, chi_tra and chi_tra_vert use chi instead of zho which is the ISO 639-3 standard.

Since all other files are named according to this standard I dont see a reason why this should not be the case for chinese.

[zdenop]

AFAIK chi is (also) valid code for Chinese: https://www.w3.org/WAI/ER/IG/ert/iso639.htm https://www.loc.gov/standards/iso639-2/php/code_list.php https://iso639-3.sil.org/code/chi

Never rely on one source (especially if anybody can change it).

[Blightbuster]

Here is the official site, I should have probably linked to that instead of wikipedia in the first place.

While yes, chi is also a valid code for chinese, it is the ISO 639-2/B code (as can also be seen on the official site which you also linked to). All other languages use the ISO 639-3 codes however. For example Czech is ces.traineddata (ISO 639-3) and not cze.traineddata (ISO 639-2/B).

[stweil]

The Tesseract documentation is not clear about the question whether ISO 639-2 or ISO 639-3 is used:

doc/combine_lang_model.1.asc:   Tesseract uses 3-character ISO 639-2 language codes. (See LANGUAGES)
doc/tesseract.1.asc:  Tesseract uses 3-character ISO 639-2 language codes
include/tesseract/baseapi.h:   * The language is (usually) an ISO 639-3 string or nullptr will default to
src/api/baseapi.cpp: * The language is (usually) an ISO 639-3 string or nullptr will default to eng.

I don't mind fixing this and using ISO 639-3 everywhere, but would like to get more feedback here from people who are affected by a renaming of the models for Chinese.

[Blightbuster]

Thats really unfortunate, that tesseract it self isnt even consistent with the used standard. Note the (usually) 😆

[stweil]

There exist more ISO 639-3 codes for Chinese variants (at least cmn, yue, nan).

[Blightbuster]

Yes those are for variants of the language although Id argue that the model is applicable for the macro language chinese hence zho fits best. Ofcause it would also be possible to have multiple models for individual chinese languages.

[stweil]

So the suggestion is that the OCR models starting with chi_ be renamed to zho_ instead.

Comments from Chinese users of Tesseract are welcome.