How to use DeepStream with Jetson Orin Nano and ROS2

in this tutorial we will use deep stream for car pedestrian and road signs detection deep stream could be useful for iot device development so in this tutorial we will use Jetson or Nano as Edge device after installation of deep stream SDK we will install python bindings to enable development using Python and finally we will use it with raw too let's briefly review what deep stream is deep stream is an integral part of Nvidia Metropolis Nvidia Metropolis features GPU accelerated sdks and developer tools which offer a more costeffective way to build deploy and scale AI enabled video analytics and iot applications deepstream is a platform to build endtoend Solutions for transforming images to valuable information deepstream SDK is a streaming analytics toolkit based on streamer for AI based multisensor processing video audio and image understanding gstreamer is a pipeline based multimedia framework that links together a wide variety of media processing systems to complete complex workflows deep stream can take streaming data from us USB or CSI camera video from file or streams over realtime streaming protocol here are some features of deepstream SDK deepstream SDK has a feature of running inference in Native tensor flow and tensorflow tensor RT using tridon inference server tridon inference server is an open- Source inference serving software that streamlines AI inferencing development can be done using using deepstream python bindings in this tutorial we will use this feature using standard message Brokers like kofka and mqtt or with Azure Edge iut Edge to Cloud integration is possible this means that it is possible to use it even in an environment with unstable network TurnKey deployment of models trained with to toolkit is also possible to stands for training adapting and optimizing Nvidia towel toolkit is a low code AI toolkit built on tensor flow and P torch which simplifies and accelerates model training process now let's install deep stream go to the Nvidia deepstream SDK developer guide page the installation procedure is described in the install Jets and SDK components section under the quick start guide firstly install dependencies these are mostly gstreamer related packages just copy these commands to the terminal and execute them next we are going to install lib Rd kofka Library lib Rd kofka is a high performance C implementation of the Apache kofka client providing a reliable and per formant client for production use clone the LI Rd Cofer repository move to the lib Rd kofka repository git reset command resets the current head to the specified State the hard option means that get resets the index and working tree whereas the soft option does not touch the index file or the working tree at all execute the configure command the configure command is a common way to configure and prepare software source code for compilation on Linux systems then execute the make Command the make Command compiles different program pieces and builds a final executable the purpose of the make Command is to automate file compilation making the process simpler and less timec consuming execute the pseudo make install command this command copies files into some appropriate locations so that they can be accessed finally copy the generated libraries to the deepstream directory now we are going to install deepstream SDK firstly download the tar file note that we should download the file suitable for our architecture in this case Jetson here select the third file from the top it will take a while before the download completes after the file is downloaded move it to the home directory extract the compressed file using the tar command the tar command allows to create and extract tar archives TB Z2 extension means that tar file is compressed using the bzip 2 algorithm move to the Deep stream directory execute the install shell script execute the LD config command the LD config command is used to tell the system about new locations of shared libraries we have successfully installed deepstream SDK now we are going to install deepstream python binding go to this git repository move to the bindings folder here we can find instructions for installation firstly we have to install base dependencies copy this command and execute it note that you have to add pseudo privilege as has been written in the initialization of subm module section we have to clone the Deep stream python apps repository to this directory so copy it and move to this directory now clone this repository here we also need pseudo privilege move to the deepstream python apps repository and execute the subm module update command by running this command we fetch all data from subm modules and by using init option we initialize our local configuration file execute these commands to ensure that we add the new certificates which GST python git server uses now we are going to build and install GSD python move to the gstd python repository execute the autog shell script then execute the make and make install commands now we are going to compile the bindings move to the bindings directory create a build directory and move to it the sample shown here is for x86 architecture but we are using Jetson so we have to execute the cmake command with the PIP platform argument and specify Linux Arch 64 architecture hyphen D sets variables as cache variables which means that even though they aren't set in the configuration files they will be remembered after build files are generated execute the make Command here we run two jobs simultaneously finally we are going to install generated pip wheel copy the directory in which the generated wheel file is located and the file name execute the the PIP 3 install command using join directory and file name now we can execute deep stream python examples move to the Deep stream test one USB cam directory connect your USB camera to Jetson or in Nano execute the deepstream test one USB Pi script we have successfully launched inference example using deep stream now let's use deep stream with RW 2 firstly install the vision messages package this is a message for interfacing with various computer vision pipelines such as object detectors then please download the deepstream raws two-zip file from Google Drive and extract it to your home directory as a base project I have used this repository but package structure and code are modified since the original version works with Ubuntu 18 and raw eloquent whereas we are using Ubuntu 20 and RW foxy now let's see the code move to the scripts directory and open the single stream class P script this code is based on the deepstream test one USB can script we previously executed and the code originally included in the raws 2D extreme repository since in the original code there is already a lot of comments explaining meaning of each operation I will mainly describe uncommented parts of the code here we append the stream Library path to the python path so that our program can find the deepstream library in these lines we are reading configuration files for inference task in this line we are obtaining current frame number of the source in the next line we are obtaining number of object meta elements attached to the current frame in the third line we are obtaining a list of objects that belong to Pi DS and VDS object meta type that are in use for the given frame in this line instance of object hypothesis message is defined this message contains only ID and score information so to get additional information about this ID listeners should perform a lookup in a metadata database in these lines classifier metadata for an object is retrieved in these lines retriev data from the stream is stored in result message which we defined previously now let's execute the code move to deep stream raws 2 directory and execute the culen build command open a new terminal then execute the source command after that run single stream Pi script note that we are specifying video device using raw arguments

Channel: robot mania

Views: 3,930

Keywords: Jetson, ROS, python, deep learning

Id: vDxL2-YJcSY

Length: 14min 58sec (898 seconds)

Published: Sun Nov 12 2023