Rotating Angled Images

[ctcheune]

Some of the text I am detecting is on an angle. How should I go about cropping around the text in the bounding box and rotating the image so that it is horizontal?

[jezebleum]

I'm also working on this, so if you do happen to find a solution - i'm very curious to your adjustment. i've gathered the rotation through v2.minAreaRect(pts) however this is not taken into account in any further operations, so only the final cropped image becomes rotated thus leaving out parts of text.

[vneseresearcher]

After reviewing network, I think we can use region score to find text direction but output is too small to do it. :|

[jezebleum]

I've made two images for the rotations, i'm using cv2.minAreaRect to identify whether there's a rotation and if that's the case, i rotate the image both ways to ensure that there's at least 1 rotated image correctly - minAreaRect uses the side that's always between 0 and -90 so you can't have 1 solution only. I'll share my code.

def crop_rect(img, rect):
    center = rect[0]
    size = rect[1]
    angle = rect[2]
    center, size = tuple(map(int, center)), tuple(map(int, size))

    height, width = img.shape[0], img.shape[1]
    M = cv2.getRotationMatrix2D(center, angle, 1)
    img_rot = cv2.warpAffine(img, M, (width, height))

    img_crop = cv2.getRectSubPix(img_rot, size, center)
    img_crop_2 = cv2.rotate(img_crop, cv2.ROTATE_90_COUNTERCLOCKWISE)
    return img_crop, img_crop_2

def crop(pts, image):

    rot = cv2.minAreaRect(pts)
    img_crop2 = []
    if rot[2] != -90:
        dst2, img_crop2 = crop_rect(image, rot)

    else:
        rect = cv2.boundingRect(pts)
        x, y, w, h = rect
        cropped = image[y:y + h, x:x + w].copy()
        pts = pts - pts.min(axis=0)
        mask = np.zeros(cropped.shape[:2], np.uint8)
        cv2.drawContours(mask, [pts], -1, (255, 255, 255), -1, cv2.LINE_AA)
        dst = cv2.bitwise_and(cropped, cropped, mask=mask)
        bg = np.ones_like(cropped, np.uint8) * 255
        cv2.bitwise_not(bg, bg, mask=mask)
        dst2 = bg + dst

    return dst2, img_crop2

Hopefully this was able to help you out on this!