Color Detection & Tracking with ESP32 Camera & OpenCV

jasmine here from how to electronics this project is all about color detection and tracking with esp32 cam module and opencv here we will be detecting any specific colors during live video streaming this method is completely different from the color detection methods as we are not writing the color detection code for microcontroller rather we will use our laptops for python code this method makes the processing faster here we have used the esp32 cam module which is a small camera module with esp32s chip besides the ob2640 camera and several gpios to connect peripherals it also features a micro sd card slot that can be useful to store images taken with the camera the method of color detection used here is hsv or hue saturation value conversion initially we will write the c code for esp32 cam and then install python and the required python libraries later we will go through the python programming for using opencv this is an essential tutorial as you will be able to use any sort of image processing or machine learning on the live video for color detection hence without getting any delay let's get started with this interesting project [Music] this video is sponsored by my favorite pcb manufacturer company called next pcb they offer pcb board and pcb assembly services at the lowest affordable price you can get trial pcb two layer pcb and full air pcb with free pcb assembly shipping services up to a fast hit time of 24 hours all you need to do is visit nexpcb.com and upload the gerber file select pcb quantity color material type and other details then place an order you will get the high quality pcb within 48 hours welcome back again this is an esp32 based camera module developed by ai thinker the controller is based on a 32-bit cpu and has a combined wi-fi plus bluetooth rble chip this is the ob2640 camera module which has the highest camera resolution up to 1600 cross 1200 the board supports an sd card up to 4gb the sd card stores capture images its gpio pins have support like uart spi i square c pwm adc and dac the board can be used in the image processing applications the board does not have an onboard programmer so in order to program this board you can use any type of usb to ttl module please follow this connection diagram for programming and use the programming method and different modes of operations have been explained in earlier videos to make this board portable and stand alone i designed a 3d casing i used solidwork to design the 3d casing and then converted the stl files into a g-code i used my creality 3d printer to print all the parts it took nearly 2 hours to print the case after printing was done i removed the casing from the surface and eliminated the useless extra support part this is a pair of casings but both have a different height the enclosure can be closed using a bottom part that fits perfectly on the casing alright it's time to fit the esp32 cam module there is a hole in the casing one for the camera and the other for the led flash you can simply insert the cam module and tighten it a small battery probably lithium ion or lithium polymer can be installed here inside the box to power on the entire circuit i am not using any battery right now as i will like to power it through the usb cable let's move to the project part now this project uses a code from an esp32 cam library created by your sunny this library supports esp32 cam in ov2640 camera you can download the gif file and add it to the arduino ide using library manager now open your arduino ide and from examples open a file called wi-fi cam in this example you just need to change the wifi ssid and password the code is the same as per the previous projects to upload this code select the esp32 rover module from board list also select the com port and then click on the upload button to upload the code open the serial monitor once code uploading is done remove the certain jumper and press the reset button if everything is fine then you will see the camera ok message on serial monitor and the local ip address of the camera will also be printed now go to your web browser and download the python latest version python is required because the machine learning and image processing part is done by python libraries after the python gets downloaded complete the installation process now we need to install two python libraries on your operating system that is numpy and opencv follow the website article of how to electronics to install the required python libraries copy the following code and paste it on any python editor copy the ip address from the serial monitor of arduino ide and replace this ip address in the python editor code this is a complete color detection code hsv also known as hsv is a conversion that is available in the opencv library it is very widely used to detect specific colors but a limitation to this is that there are some constant values related to detection which vary from different objects and places we will convert the rgb image to an hsv image and then we set a lower bound of all three values that is hue saturation and brightness and also an upper bound thus whatever comes in that range is what we desire to see go to run and then click on the run module option once you hit run immediately three windows will appear with live transmission mode mask mode and rest mode the fourth is a box for setting the tracking option now as in the above frame we wish to track the orange color therefore we adjust the lh ls lv uh us and uv values from the tracking bars in the tracking window by sliding them use your mouse to slide all the arrows add just three bars values such that at the mask window white color appears only at orange colored places as in the image below once the value is adjusted modify the code as shown here set the lb and uv values by representing with the numbers in the code then comment few of these lines to disable the manual operations modify the code again by adding some lines and also assigning some values again hit the run button to run the code so now you can see the orange color is automatically detected congratulations you have detected the orange color similarly by adjusting the tracking bars you can detect any specific colors now our next step is to draw a counter around the colored image and specify the average center coordinates of the detected color to do so save the lh ls lv uh us and uv values in the track bar window from the previous program then modify the code as i did here now run the code so here is the real image the code work like a charm similarly you can detect any specific colors or color by adjusting hsv values and creating as many counters this is how you can use the esp32 cam along with opencb for color detection and tracking this project can be implemented at an advanced level along with the application of machine learning and ai all the documentation libraries details code and written explanation of the project is perfectly documented on how to electronics i hope you like this video so please drop a like comment and thank you so much for watching

Channel: How To Electronics

Views: 29,188

Keywords: Color Detection, esp32 cam color detection, esp32 cam color tracking, opencv color detection python, opencv color detection, esp32 camera color detection, Color Track ESP32 Camera, ESP32 CAM Color Detection Code

Id: Nm_EMey0dkU

Length: 8min 28sec (508 seconds)

Published: Mon Sep 27 2021