Estimating position of the objects using Jetson Nano and YOLOv5

in this video we will do position estimation of an objects using yolo v5 since jetson nano computation capability is very limited 416 by 224 image size is used to perform the inference task detected objects are plotted in picture on the right side as the object is moved the point representing the object is also moved understanding relative position of the objects in respect to your robot may be useful in various tasks the first step is to clone the yolo v5 repository go to ultra lytic yolo v5 repository in kappaby the url move to the yolo v5 folder open the requirements text file these are the libraries we need to run yolo v5 in this tutorial we will install these libraries one by one be especially careful not to install opencv using pip since jetson nano has gpu we will install opencv from source with cuda support to maximize performance the second step is to install opencv go to jetsonhack nano build opencv page copy the url and git clone it now open build opencv shell script we need to do several modifications so that building process will not stop due to lack of memory as has been written here we need to modify the non-jobs parameter since we are using an sd card first at line time make j nom jobs change non-jobs to one at the line time sudo make package j nom jobs change nom jobs to one that's all save your modifications and close the file now move to the build open cv folder execute build opencv shell script note that building opencv will take several hours so be patient from now on we are going to install required libraries install python 3 pip upgrade pyoma library pyoma is a full-featured yaml framework for the python programming language next install tqdm tqdm is a python library that allows you to output a smart progress bar by wrapping around any iterable a tqdm progress bar not only shows you how much time has elapsed but also shows the estimated time remaining for the iterable before upgrading nom pi cython has to be installed cython is an optimizing static compiler for both the python programming language and the extended cython programming language it makes writing c extensions for python as easy as python itself now upgrade numpy to version 118.5 before installing matplotlib we need to install some packages the build essentials packages are necessary for compiling software python dev is the package that contains the header files for the python c api install some packages on which matplotlib depends here package versions are specified to make sure that packages with which yolo v5 works will be installed although yolo may work with other versions too now we are ready to install matplotlib before installing scipy some libraries should be installed install g fortran g fortran is the gnu fortran compiler which is part of the gnu compiler collection install lib open blast open glass is an open source implementation of the basic linear algebra sub-programs with many hand-crafted optimizations for specific processor types install a pack lepak is a standard software library for numerical linear algebra now we are ready to install scipy before installing the pillow we need to install the jpeg lib jpeg is a widely used c library for reading and writing jpeg image files install the pillow install typing extensions now we are going to install pi torch pi torch will be installed using pip but for jets and nano we need a pip wheel that was built for arm arch 64 architecture go to this page and download one of the pie torch binaries from the list in this tutorial we will use version 1.10 just copy the command change the library name and execute it in the terminal now install the wheel file we have previously downloaded strangely pi torch version 1.8 is installed but 1.8 will work for us too so we will leave it as it is now we are going to install torch vision basically all you need to do is just to copy and paste these commands firstly install required libraries clone torch vision version 0.9 move to the torch vision directory in this line build version variable is set to 0.9 by executing this command torch vision build will begin install seaborn version 0.11 seaborne is a python data visualization library based on matplotlib we are ready to execute yolo v5 this is how you run yolo v5 script image path model and input image size should be specified we have successfully executed the script move to the run detectix folder as you can see inference is performed successfully now we are going to install librealsense go to this page clone this repository move to install realsense sdk directory before executing build lube real sense shell script we need to do some modifications to avoid building process stopping due to memory shortage go to the install realsense sdk folder open the build lib real sense shell script at line time make j nom proc change nom proc to 1. execute build lib real sense shell script after installation let's check whether the library works properly open a terminal and import pi realsense 2 library module not found error means that something is wrong with the installation directory as you can see the library should be installed here after a little investigation i found out that the library was actually installed here so python path in bashrc file needs to be modified modify this line source bash rc file to apply changes immediately let's try to import pi realsense one more time now we can import it successfully install lib cambered gtk this is the module that takes gui events like pressing a button from gtk plus and plays some configured sound let's see what in the code this code is based on detect pi script here image size format and frame rate are defined pipeline start means that we start pipeline streaming with configuration we set in the above line the alignment utility performs per pixel geometric transformation based on the depth data in this line camera focal distance and optical centers are obtained here wait until a new set of frames becomes available the frame set includes time synchronized frames of each enabled stream in the pipeline here we get aligned frames from rgb and depth camera here we crop the image to do inference task note that image height and width should be multiple of 32. here we calculate the center point of the detected image here the distance from the camera to detected object center is calculated in this part coordinates of the center point are calculated for the theory please watch my estimating position of an object with yolo v3 using realsense tutorial in this line we add object center point coordinates to an array in this part object coordinates are converted to coordinate system of the canvas and plotted when the start button is pushed thread start function is executed inference task by yolo and object mapping on the canvas are done in the separate threads to execute this script open the terminal and execute object mapping pi script push start button to start recognition as you can see recognized objects are plotted on the map you

Channel: robot mania

Views: 12,368

Keywords: ROS, python, deep learning, NVIDIA

Id: oKaLyow7hWU

Length: 17min 50sec (1070 seconds)

Published: Sun Jan 16 2022