JMTrans icon indicating copy to clipboard operation
JMTrans copied to clipboard

Program alters the original image when doing text segmentation

Open a-lgil opened this issue 4 years ago • 2 comments

When using SickZil to find supposed text on the image, the supposed text is erased and the image is altered to match the background. Then, the supposed text is processed with OpenCV to detect the real text and translate it.

However, the supposed text detected by SickZil that isn't identified as text by OpenCV is erased from the original image. Most of the time this means that some strokes, random strands of hair or small text are deleted from the original image.

But other times entire objects or facial features are deleted from the image, such as here, where the right eye of the character is missing in the translated version, as well as some details on her hair:

image image

A solution could be changing the aggressiveness of SickZil if that's possible since it seems it's too high right now.

Or you could just reintegrate those parts of the image picked up by SickZil but not recognised as text by OpenCV. That way, in the case of the images I've uploaded, the eye and hair details would be extracted from the image but then reintegrated when OpenCV doesn't recognise them.

a-lgil avatar Dec 28 '20 05:12 a-lgil

Google ocr does not recognise sound effect word image as text Reintegrate method may cause to display all hidden sound word image.

Currently, I have no clue to extract text from sound effect word

Other than that, I am considering to create my own manga text segmentation model using GAN as my next new project(if I have nothing to do )

ttop32 avatar Dec 29 '20 12:12 ttop32

Google ocr does not recognise sound effect word image as text Reintegrate method may cause to display all hidden sound word image.

Currently, I have no clue to extract text from sound effect word

Other than that, I am considering to create my own manga text segmentation model using GAN as my next new project(if I have nothing to do )

Good luck with the text segmentation model then, let's hope everything goes smoothly

a-lgil avatar Dec 29 '20 18:12 a-lgil