tessdata_best icon indicating copy to clipboard operation
tessdata_best copied to clipboard

Tesseract fails to recognize % sign in Hungarian language texts

Open Googulator opened this issue 6 years ago • 8 comments

When the language is set to "hun" (Hungarian), Tesseract is unable to recognize the % sign. This sign is very commonly used in Hungarian to represent percentages, the same way as in English. Tesseract instead sees various letters and digits - most commonly "96", sometimes "9", "69", "0", "S", "Z", or even nothing at all.

Even if I feed a generated image containing a % sign in large black type on a pure white background, I still can't get Tesseract to output the % sign, as long as the language is set to Hungarian.

Both the "fast" and "best" models suffer from this problem.

If I instead set the language to English, % sings are recognized without issue.

Googulator avatar Sep 18 '19 17:09 Googulator

See hun.unicharset which shows all known characters.

The percent sign was not part of the training data set, so Tesseract simply does not know that character with hun.

This can only be solved by new training, either from scratch or by fine tuning the existing incomplete model (which can add new characters).

An alternative would be using the script/Latin model which supports all ~western~ European languages which are based on Latin script.

stweil avatar Sep 18 '19 18:09 stweil

A model "which supports all western Europe languages" is not an option for Hungarian, because of ő and ű, which are not found in any Western European language.

Googulator avatar Sep 20 '19 08:09 Googulator

Have you tried with -l hun+eng?

Shreeshrii avatar Sep 20 '19 08:09 Shreeshrii

Latin.unicharset includes both characters, so I suggest to try it. I updated my previous comment.

stweil avatar Sep 20 '19 09:09 stweil

@stweil the more relevant unicharset will be the lstm-unicharset extracted from the script/Latin traineddata file. Latin.unicharset maybe a superset.

Shreeshrii avatar Sep 20 '19 13:09 Shreeshrii

You are right, it is not identical, but that one also includes both characters.

stweil avatar Sep 23 '19 06:09 stweil

How do I test Latin.unicharset? Do I need to train a new model?

Googulator avatar Oct 24 '19 10:10 Googulator

Just get Latin.traineddata and extract Latin.unicharset using combine_tessdata. Then load Latin.unicharset in your editor and check whether it contains all relevant characters.

stweil avatar Oct 24 '19 11:10 stweil