Outofmemory error

[ri-sh]

while running swt.py iam facing following memory error

Traceback (most recent call last): File "swt.py", line 375, in final_mask = SWTScrubber.scrub(local_filename) File "swt.py", line 30, in scrub shapes = cls._connect_components(swt) File "swt.py", line 238, in _connect_components layers[common_label] = np.zeros(shape=swt.shape, dtype=np.uint16) MemoryError

i believe that this is because we are creating to many images usingnp.zeros(shape=swt.shape, dtype=np.uint16) which currently filling up ram

could u please suggest an alternative or correction to the connect_components function

[AniketGurav]

I am also having same issue.

[truongtd6285]

This problem due to insufficient memory (RAM) of your computer. You can see that data type uint16 requires 2 bytes of memory. When your image size is 800x600 for example, the memory needed is 800x600x2 = 960.000 bytes for each layer. And if you have 1000 common_label then you need 960.000x1000 = 960.000.000 bytes ~ 937.5000 MB just to cover array > layers[common_label]

I have modified these lines as follow:

try:
                       layer = layers[common_label]
                   except KeyError:
                       layers[common_label] = {}
                       layers[common_label][0] = []
                       layers[common_label][1] = []
                       layer = layers[common_label]
                   layer[0].append(y)
                   layer[1].append(x)

       return layers, swt.shape[1], swt.shape[0]

here only pixels with non-zero value to be stored in layers[common_label] then, in function _find_letters() we restore full_layer as image size by:

#widths.append(width)
fulllayer = np.zeros((shape_w, shape_h), dtype=np.uint16)
for i in xrange(len(layer[0])):
    fulllayer[layer[0][i], layer[1][i]] = 1
images.append(fulllayer)

you can see full modified code as attached (please change extension from .jpg to .py to read)

[elcombato]

@truongtd6285 can you make a pull request with this improvement or share your modification somehow? I cannot open the attached file.

[truongtd6285]

i don't know how to create pull request, so please find attached file here or here swt_up.zip

[chitransh1998]

@truongtd6285 I tried to execute your uploaded code. But it shows a value error as follow: Traceback (most recent call last): File "new.py", line 409, in final_mask = SWTScrubber.scrub(local_filename) File "new.py", line 31, in scrub word_images = cls._find_words(swts, heights, widths, topleft_pts, images) File "new.py", line 326, in _find_words swt_tree = scipy.spatial.KDTree(np.asarray(swts)) File "C:\Users\DELL\Anaconda3\envs\py27\lib\site-packages\scipy\spatial\kdtree.py", line 235, in init self.n, self.m = np.shape(self.data) ValueError: need more than 1 value to unpack

Please help.Urgently required for my project.Thank-you

[truongtd6285]

which version of Python are you using, mine v2.7. Error occurred due to KDTree docstring in scipy.spatial. You can find some hint here https://github.com/scipy/scipy/issues/4527

[chitransh1998]

@truongtd6285 But I am also using v2.7. And the link you provided seems to refer to an issue of Python 3. Can you please help?.

[truongtd6285]

i got the same this error before, but don't remember exactly how i fixed it. It related to input data of scipy.spatial.KDTree(np.asarray(swts)) not properly as required. Let i remember. You can run the code step by step and see what input data you give the function

[chitransh1998]

Thanks I could resolve the error later. But the results are very poor on images.It is skipping some text and also detecting erroneous text in the images. Can you suggest some improvements in the code?

[truongtd6285]

You can set diagnostics = TRUE to write intermediate images created while transforming: if diagnostics: cv2.imwrite('edges.jpg',edges) cv2.imwrite('sobelx64f.jpg', np.absolute(sobelx64f)) cv2.imwrite('sobely64f.jpg', np.absolute(sobely64f)) # amplify theta for visual inspection theta_visible = (theta + np.pi)*255/(2*np.pi) cv2.imwrite('theta.jpg', theta_visible) Quality of transformation depends on your images resolution, you can try with some images to find the best applicable case.

[chitransh1998]

I have tried with different resolutions and images. There seems to be a problem in the implementation of the Stroke width transform. If there is error in the transform then we cant expect the following functions to work properly. If there is a better implementation or something then please do share.