No detection of objects using C++

[dasari-mohana]

Hello, I am running c++ script with "Opencv==4.6.0" and "gcc==gcc 9.4.0" and "Ubuntu == 20.04.4 LTS" in CPU - 16gb ram syatem

I tried many ways and still could not get detections in the image or video, Even though the code does not show any errors and at the same time no boundary boxes or confidence scores around objects

output image (Inference time printed on it)-->>

This is my code-->>

// Include Libraries.
#include <opencv2/opencv.hpp>
#include <fstream>`

// Namespaces.
using namespace cv;
using namespace std;
using namespace cv::dnn;`

// Constants.
const float INPUT_WIDTH = 640.0;
const float INPUT_HEIGHT = 640.0;
const float SCORE_THRESHOLD = 0.5;
const float NMS_THRESHOLD = 0.45;
const float CONFIDENCE_THRESHOLD = 0.45;

// Text parameters.
const float FONT_SCALE = 1;
const int FONT_FACE = FONT_HERSHEY_SIMPLEX;
const int THICKNESS = 2;

// Colors.
Scalar BLACK = Scalar(0,0,0);
Scalar BLUE = Scalar(255, 178, 50);
Scalar YELLOW = Scalar(0, 255, 255);
Scalar RED = Scalar(0,0,255);

// Draw the predicted bounding box.
void draw_label(Mat& input_image, string label, int left, int top)
{
    // Display the label at the top of the bounding box.
    int baseLine;
    Size label_size = getTextSize(label, FONT_FACE, FONT_SCALE, THICKNESS, &baseLine);
    top = max(top, label_size.height);
    // Top left corner.
    Point tlc = Point(left, top);
    // Bottom right corner.
    Point brc = Point(left + label_size.width, top + label_size.height + baseLine);
    // Draw black rectangle.
    rectangle(input_image, tlc, brc, BLACK, FILLED);
    // Put the label on the black rectangle.
    putText(input_image, label, Point(left, top + label_size.height), FONT_FACE, FONT_SCALE, YELLOW, THICKNESS);
}
vector<Mat> pre_process(Mat &input_image, Net &net)
{
    // Convert to blob.
    Mat blob;
    blobFromImage(input_image, blob, 1./255., Size(INPUT_WIDTH, INPUT_HEIGHT), Scalar(), true, false);
    net.setInput(blob);
    // Forward propagate.
    vector<Mat> outputs;
    net.forward(outputs, net.getUnconnectedOutLayersNames());

    return outputs;
}

Mat post_process(Mat &input_image, vector<Mat> &outputs, const vector<string> &class_name) 
{
    // Initialize vectors to hold respective outputs while unwrapping detections.
    vector<int> class_ids;
    vector<float> confidences;
    vector<Rect> boxes; 

    // Resizing factor.
    float x_factor = input_image.cols / INPUT_WIDTH;
    float y_factor = input_image.rows / INPUT_HEIGHT;

    float *data = (float *)outputs[0].data;

    const int dimensions = 85;
    const int rows = 25200;
    // Iterate through 25200 detections.
    for (int i = 0; i < rows; ++i) 
    {
        float confidence = data[4];
        
        // Discard bad detections and continue.
        if (confidence >= CONFIDENCE_THRESHOLD) 
        {
            float * classes_scores = data + 5;
            // Create a 1x85 Mat and store class scores of 80 classes.
            Mat scores(1, class_name.size(), CV_32FC1, classes_scores);
            // Perform minMaxLoc and acquire index of best class score.
            Point class_id;
            double max_class_score;
            minMaxLoc(scores, 0, &max_class_score, 0, &class_id);
            // Continue if the class score is above the threshold.
            if (max_class_score > SCORE_THRESHOLD) 
            {
                // Store class ID and confidence in the pre-defined respective vectors.

                confidences.push_back(confidence);
                class_ids.push_back(class_id.x);

                // Center.
                float cx = data[0];
                float cy = data[1];
                // Box dimension.
                float w = data[2];
                float h = data[3];
                // Bounding box coordinates.
                int left = int((cx - 0.5 * w) * x_factor);
                int top = int((cy - 0.5 * h) * y_factor);
                int width = int(w * x_factor);
                int height = int(h * y_factor);
                // Store good detections in the boxes vector.
                boxes.push_back(Rect(left, top, width, height));
            }
        }
        // Jump to the next column.
        data += 85;
    }

    // Perform Non Maximum Suppression and draw predictions.
    vector<int> indices;
    NMSBoxes(boxes, confidences, SCORE_THRESHOLD, NMS_THRESHOLD, indices);
    for (int i = 0; i < indices.size(); i++) 
    {
        int idx = indices[i];
        Rect box = boxes[idx];
        int left = box.x;
        int top = box.y;
        int width = box.width;
        int height = box.height;
        // Draw bounding box.
        rectangle(input_image, Point(left, top), Point(left + width, top + height), BLUE, 3*THICKNESS);

        // Get the label for the class name and its confidence.
        string label = format("%.2f", confidences[idx]);
        label = class_name[class_ids[idx]] + ":" + label;
        // Draw class labels.
        draw_label(input_image, label, left, top);
    }
    return input_image;
}
int main()
{
    // Load class list.
    vector<string> class_list;
    ifstream ifs("coco.names");
    string line;
    while (getline(ifs, line))
    {
        class_list.push_back(line);
    }
    // Load image.
    Mat frame;
    frame = imread("sample.jpg");

    // Load model.
    Net net;
    net = readNetFromONNX("models/yolov5s.onnx");
    vector<Mat> detections;
    detections = pre_process(frame, net);
    // cout << detections << endl;
    // Mat img = post_process(frame.clone(), detections, class_list);
    Mat img = post_process(frame, detections, class_list);

    // Put efficiency information.
    // The function getPerfProfile returns the overall time for inference(t) and the timings for each of the layers(in layersTimes)

    vector<double> layersTimes;
    double freq = getTickFrequency() / 1000;
    double t = net.getPerfProfile(layersTimes) / freq;
    string label = format("Inference time : %.2f ms", t);
    putText(img, label, Point(20, 40), FONT_FACE, FONT_SCALE, YELLOW);

    imshow("Output", img);
    imwrite("sample_output.jpg", img);
    waitKey(0);
    return 0;
}

[brmarkus]

Do you use an existing sample&demo from this repo under LearnOpenCV? If yes, which? Have you modified existing code? Have you changed the model "yolov5s.onnx", like retrained, or converted? Is the code you use also available as e.g. Python script, have you tried it, is it working?

[dasari-mohana]

Yes I used the same code available from this repo under LearnOpenCV. I modified the code to fix bugs that are solved in issues. (for example: Instead of --> Mat img = post_process(frame.clone(), detections, class_list); I used --> Mat img = post_process(frame, detections, class_list);)

I used already available weights "yolov5s.onnx", and also used converted weights from "yolov5s.pt" to "yolov5s.onnx", but I got the same issue.

And lastly, It is available in python script, --> https://github.com/spmallick/learnopencv/blob/master/Object-Detection-using-YOLOv5-and-OpenCV-DNN-in-CPP-and-Python/yolov5.py

[shahla-ai]

i faced a similar issue with c++ sample but instead of giving no detection it gave crazy detections that are not False positives #690 note that this issue happened only with my custom model that i converted from .pt to .onnx and doesn't happen with the pretrained model on coco that i converted to .onnx Any Help ?

[dasari-mohana]

Hello, I tried the pre-trained model in ONNX format (yolov5s.onnx) that was readily available online. Even with that weight file I am still not getting any detections

[brmarkus]

As part of the issue "https://github.com/spmallick/learnopencv/issues/674" I was able to run the demo successfully, but needed to make sure to use newer version of OpenCV; with the version "v4.5.5" it was working for me.

[shahla-ai]

@brmarkus i have already installed OpenCV 4.5.5 but still having the same problem mentioned in #690

[brmarkus]

Maybe you have installed multiple different versions of OpenCV now? Can you check that the correct version is used and linked (static or dynamic linking), like calling ldd my_application?

[shahla-ai]

my problem is sloved , check #690