ObjectDetection ESP32cam

Using ESP32-CAM for the Object Detection system

TWITTER · INSTAGRAM · YOUTUBE · WEBSITE

:notebook_with_decorative_cover: Table of Contents

• About the Project
  • Pictures
  • Features
• Getting Started
  • Schematic
  • Installation
• Usage
• Contributing
• License
• Contact

:star2: About the Project

In the this tutorials of the ESP32-CAM series, we saw that using the original code, we will be able to process face image from face recognition to face separation, but in cases where we need to recognize different objects, different models must be introduced to our code. To be able to identify the objects we want with self-learning. But no matter how high the processing power of ESP chips, we can not leave all this complex processing to this small chip, so we will use Tensorflow.JS to combine it with the video sent from ESP32-CAM. Note that in this tutorial, Tensorflow.JS runs in the computer browser and therefore the machine learning model runs inside your browser.

:camera: Pictures

:dart: Features

• Selection of the desired objects in the web server
• Real-time object detection
• No need to run the second code on the computer

:toolbox: Getting Started

In this project, by streaming images using the ESP32-CAM board and receiving and displaying them in the browser, we will also use Tensorflow.JS to process images using the default models applied.

• ESP32-CAM
• USB-To-TTL

:electric_plug: Schematic

To program the ESP-CAM board, we need Arduino-IDE software, and of course, download the relevant board in the software environment, as well as install the required libraries. For information on these cases, you can refer to this tutorial. Make the connections according to the schematic below according to the USB TO TTL used. Note that when programming code, ie after compiling the code, the two pins GPIO 0 and GND are connected to each other, and after successfully compiling the code, you must disconnect this connection to run the project.

Make the connections according to the table and schematic below.

• Complete Schematic

:gear: Installation

In this part of the action code, we will introduce Tensorflow.js to analyze the received images in the browser. In the next line, we will also import COCO-SSD models along with Tensorflow.js.

<script src="https:\/\/ajax.googleapis.com/ajax/libs/jquery/1.8.0/jquery.min.js"></script>
<script src="https:\/\/cdn.jsdelivr.net/npm/@tensorflow/[email protected]/dist/tf.min.js"> </script>
<script src="https:\/\/cdn.jsdelivr.net/npm/@tensorflow-models/[email protected]"> </script>

In the next section, we will load the introduced models so that they can be recognized as a result of processing the received image for our machine.

    function ObjectDetect() {
      result.innerHTML = "Please wait for loading model.";
      cocoSsd.load().then(cocoSsd_Model => {
        Model = cocoSsd_Model;
        result.innerHTML = "";
        getStill.style.display = "block";
      }); 
    }

After our machine detects an object in the image, it should now be visible to the user in the form that squares around the detected image are usually used in image processing, the following code is for this.

 if (Predictions.length>0) {
          result.innerHTML = "";
          for (var i=0;i<Predictions.length;i++) {
            const x = Predictions[i].bbox[0];
            const y = Predictions[i].bbox[1];
            const width = Predictions[i].bbox[2];
            const height = Predictions[i].bbox[3];
            context.lineWidth = Math.round(s/200);
            context.strokeStyle = "#00FFFF";
            context.beginPath();
            context.rect(x, y, width, height);
            context.stroke(); 
            context.lineWidth = "2";
            context.fillStyle = "red";
            context.font = Math.round(s/30) + "px Arial";
            context.fillText(Predictions[i].class, x, y);

:eyes: Usage

After uploading the desired code, I reset the board in the serial ip of the web server will be displayed for us, such as 192.168.1.103, which we will search for in the browser to enter the web server page, we have to wait for a while until Load the related models, then click on the StartDetect icon to display the images and recognize the defined items. Also, if there is a problem with the image processing or other parts, click on the Restart option.

• In the web server, it is possible to count it by specifying the desired item. By selecting the desired object, the number of detected items will be displayed in front of it.

• In this section you adjust the output video settings. For example, it is possible to change the resolution or mirror the image in this section.

• In this section, the contrast, quality and brightness of the received image can be adjusted.

• If the object is detected, its name will be displayed in this section, and also the percentage of probability that our machine analysis is correct, as well as the coordinates and number of detected objects can be displayed in this section.

• Finally, we have the received image, which will inform the user if he detects an object with a square and also enters its name.

:wave: Contributing

:warning: License

Distributed under the MIT License. See LICENSE.txt for more information.

:handshake: Contact

CiferTech - @twitter - [email protected]

Project Link: https://github.com/cifertech/ObjectDetection_ESP32cam