android-demo-app icon indicating copy to clipboard operation
android-demo-app copied to clipboard

I write a letterbox function like detect.py for android object detection, it can make the output probablity higher and more robust.

Open delongwu opened this issue 2 years ago • 1 comments

Just enjoy it , this function almost same as python augmentations.py letterbox function. but has some diff at end of gray border, but it is ok.

you can use letterbox function to replace the

Bitmap resizedBitmap = Bitmap.createScaledBitmap(picBitMap, PrePostProcessor.mInputWidth, PrePostProcessor.mInputHeight, true);

/**
     * Image augmentations: first scale the image and later padding image,  increase the strength of the model.
     * Always this scale and padding will make the image object detection gave more high probability or more robust.
     * <p>
     * Reference:
     * https://github.com/ultralytics/yolov5/blob/db6ec66a602a0b64a7db1711acd064eda5daf2b3/utils/augmentations.py#L91-L122
     * def letterbox(im, new_shape=(640, 640), color=(114, 114, 114), auto=True, scaleFill=False, scaleup=True, stride=32):
     * method
     *
     * @param srcBitmap
     * @param newShape  (640*640)
     * @param color     always gray (114,114,114)
     * @param auto      default:false, no use
     * @param scaleFill default:false,  no use
     * @param scaleUp   default:false
     * @param stride    default:32 , no use
     * @return
     */
    public static Bitmap letterbox(Bitmap srcBitmap, Pair<Integer, Integer> newShape, Triple<Integer, Integer, Integer> color, Boolean auto,
                                   Boolean scaleFill, Boolean scaleUp, int stride) {
        // current shape
        int currentWidth = srcBitmap.getWidth();
        int currentHeight = srcBitmap.getHeight();

        // new shape eg: 640*640
        int newWidth = newShape.first;
        int newHeight = newShape.second;

        // only scale image,no padding,just return scale image
        // I modify this logic something difference with the python code clean & speed.
        if (scaleFill) {
            // filter =  bilinear filtering
            return Bitmap.createScaledBitmap(srcBitmap, newWidth, newHeight, true);
        }

        // Scale ratio (new / old)
        float r = Math.min(newWidth * 1.0f / currentWidth, newHeight * 1.0f / currentHeight);

        //  Only scale down, do not scale up (for better val mAP)
        if (!scaleUp) {
            r = Math.min(r, 1.0f);
        }

        int newUnpadWidth = Math.round(currentWidth * r);
        int newUnpadHeight = Math.round(currentHeight * r);

        //  wh padding
        int dw = newWidth - newUnpadWidth;
        int dh = newHeight - newUnpadHeight;

        // auto always false, no use for android demo
        if (auto) { // # wh padding
            dw = dw % stride;
            dh = dh % stride;
        }

        // resize
        if (!(currentWidth == newUnpadWidth && currentHeight == newUnpadHeight)) {
            srcBitmap = Bitmap.createScaledBitmap(srcBitmap, newUnpadWidth, newUnpadHeight, true);
        }

        // padding with gray color
        Bitmap outBitmap = Bitmap.createBitmap(srcBitmap.getWidth() + dw, srcBitmap.getHeight() + dh, Bitmap.Config.ARGB_8888);
        Canvas can = new Canvas(outBitmap);
        can.drawRGB(color.getFirst(), color.getSecond(), color.getThird()); // gray color
        can.drawBitmap(srcBitmap, dw, dh, null);

        return outBitmap;
    }

delongwu avatar Jul 21 '22 11:07 delongwu

This result is great, have you compared the lost in speed?

zhiqwang avatar Jul 21 '22 13:07 zhiqwang