AI Robot: Object detection using TensorFlow Lite and Web Monitoring | Raspberry Pi

earth thrower can now perform object detection it can detect multiple objects within an image and draws bounding boxes around them a pre-trained machine learning model along with tensorflow lite python apis are used in this project for camera operations and generating the output window opencv is used the output window shows objects within color-coded bounding boxes and an information bar on the top based on the confidence score the bounding boxes can have green red and blue colors this output window is streamed over the lan using python's lightweight web framework called flask and can be accessed on a web page finally this webpage is embedded in the control panel of rover which already has direction and speed controls a particular object can be selected through control panel for monitoring whenever the selected object is located in the frame this counter is updated and the background turns orange the code can be easily modified to monitor any of the 90 objects available in the machine learning model and actuate robotic motion or sound alarm let's see your demo [Music] so [Music] [Music] do [Music] do [Music] [Music] [Music] the project is implemented in three stages the code for the stages is written in these files you can download the complete source code from the link provided in the description below let's start with how object detection works using opencv the camera is accessed and a frame is captured the frame is then resized into the dimensions required by the model and fed to the interpreter for performing inference through the machine learning model the model returns the results based on the objects present in the frame the results contain these four parameters the location parameter returns the top left and bottom right coordinates of the object these coordinates are used to draw the bounding boxes around the objects names of all the objects present in the frame are obtained through this parameter i have used this information for checking whether a particular object is present in the frame in order to update the object counter the third parameter provides the confidence score with respect to each object i have used this information to color code the bounding boxes the fourth parameter simply returns the number of objects present in the frame in this example it is 4. [Music] the process starts with capturing the camera frame and performing inference in this stage the results are checked for the presence of a fixed object that is person then the frame is populated with overlays and the frame with added information is displayed locally on raspberry pi [Music] the overlays include information bar showing fps processing times object counter and bounding boxes the whole process is repeated continuously interpreter is initialized prior entering the loop [Music] in this stage i modified the code by initializing and running flask in the beginning this part of the code is omitted which displays the output locally and new code is added here to stream the output overland now when we execute the code this message is displayed showing ip address and port to access the output stream in the third stage i created a simple web page that accommodates the output stream generated through flask it also has a mechanism for selecting a particular object for monitoring the motor control section has been reused from original gui in the python code this fixed component is removed and this object name is fetched through a file which is dynamically updated by press of these web ui buttons so during the runtime itself we can tell the robot which object it should monitor next the python code developed for this project is built upon a sample code provided by google coral team this is the location in raspberry pi where the code is placed can compare these three files to see the stage-wise modifications and this folder has php javascript and html files to generate the web controls for this project source code is uploaded on github stay tuned for more such videos on ai robotics internet of things and home automation [Music] thanks for watching

Channel: Jitesh helloworld

Views: 10,097

Keywords: tensorflow object detection, raspberry pi object detection tensorflow, tensorflow 2.0 object detection api, raspberry pi object detection tensorflow lite, object detection camera, tensorflow lite object detection github, real time object detection tensorflow, tensorflow lite raspberry pi example, Google Coral USB Accelerator, jitesh saini hello world, artificial intelligence raspberry pi, machine learning raspberry pi, iot projects

Id: 1pnUkhIL7QA

Length: 7min 0sec (420 seconds)

Published: Mon Sep 21 2020