My First Jetson Nano Object Detection Inference Server

thank you Judson Nano I have a love-hate relationship with that board but there were so many comments asking about tutorial about this particular device that I just couldn't stall any longer note that I hate it or anything but I really think that Nvidia makes semi-optimal job when it comes to making that board easy to use in today's tutorial I will show you how to set up the board we'll talk about the hardware I will show you how to burn the operating system on the SD card how to boot the board for the first time and how to install the most essential libraries that you need to have on that device and to top all of that we'll put the board to the test by running a demo inference server using Docker so backlab because we have so many things to unpack and before we start make sure to like And subscribe it really motivates me to produce long videos like this first thing first let's talk about Hardware this is Jetson Nano 4 gigabyte Heights the two gigabytes version looks very much the same the only difference is the amount of ports that you have at your disposal and speaking of ports let's go from left to right the first one is micro USB that we can use to power the whole device next is one gigabit Ethernet port for USB 3 ports that we'll use to connect webcam mouse and keyboard DisplayPort and HDMI port and the last one from the left is DC Barrel Jack power supply that you can use alternatively I however don't have the required charger so I'll stick with micro USB when it comes to charging it is very important that you use the right amperage and voltage otherwise you may risk damaging your board okay enough of the talking let's set up the board to do it we'll write image with operating system on micro SD card and insert it right over here below the heatsink there is micro SD slot that we can use to insert our card we gently turn the board upside down slide the card in and press until we feel distinctive click to take it out we press once again and the card jumps right out one more thing that I forgot to mention keep in mind to select the card with relatively fast writing and reading speeds it is not worth the save money on that particular item because every time that your application will load or save something on a hard drive that will directly influence the performance to write operating system on SD card we need to connect it to computer if you have SD slot you can use this kind of adapter they very often come together there with micro SD cards if you don't you can use this portable SD card adapter it looks very similar to the flash drive however it has two parts for SD and micro SD cards on the site you just slide the card in plug it into your USB port and you are all set now we can go to Google search type and setup Jetson Nano click the first link and then after the tutorial is Open click Right image to micro SD card we have multiple operating systems to choose from I'm on a Mac so I'll select the middle one and the first thing they ask us to do is to install etcher this is the software that we are going to use to burn Jets on Nano operating system on SD card we download the installer and set it up on our machine in the meantime we also download the operating system image from the link in the original Nvidia tutorial after that is done we decompress as the image and start etcher we select the image from downloads directory select the SD card that we want to use and click next depending on the operating system etcher might require additional permissions however after providing password everything should run smoothly and all we need to do is wait for the image burn to complete [Music] the burn is completed now we need to plug in our cables power the board and install operating system keep in mind that the board might get super hot especially during the inference I guess that's why they suggest to put it on cardboard stand a famously fire resistant material but that's what Nvidia asked us to do so I am not going to argue then we take the card from the adapter slide it into SD slot and connect all necessary cables starting from HDMI then ethernet as I said we'll need mouse and a keyboard but in my case I won't connect any cables single Bluetooth connector is just fine God I love wireless devices cool now we are ready to power up the whole device we take the micro USB cable slide it into the port and we immediately see the green light to let us know that the board is on when we power up the board for the first time we'll need to go for Linux installation process to save us some time I'm speeding up the recording however you can expect the whole installation to take around 10 minutes you can also expect the board to reboot few times during that process so don't get scared by that either the whole installation process is pretty standard as Nvidia is using slightly modified version of Ubuntu we start by accepting the license selecting the language as well as selecting the keyboard layout now the time zone of your choice and we can select our username I'm going for NVIDIA because I'm super creative person now the password I also wanted to go for NVIDIA but I decided okay that's maybe not the best idea let's pick something a bit more secure and we can continue now we need to select the size of our partition I'm keeping the default and the last thing that we need to do before the reboot is to select the CPU configuration once again I'm keeping the default one foreign [Music] will reboot for the last time we can now log in with our credentials pretty much ready to go now every time that I have a fresh device like this to work with I go through a few configuration steps so let's go through them one by one first thing first we need to confirm that we have access to the internet so you can easily do it using ping command and what you can do is basically pass any publicly facing API address however what I like to do usually is just ping Google and confirm that I can send and receive packets from that website next up is Docker I really don't like to use sudo every time that I execute Docker related command however by default only sudo user is allowed to do that that's why when you try to execute I don't know Docker run for example and use your personal user you will most likely end up seeing permission the night in your terminal to fix that we'll create dedicated Docker group add our user into that group and that will automatically allow that user to execute Docker without sudo before we put that to the test we need to activate our changes in the groups and now we can execute Docker run hello world without any restrictions we obviously don't have that image locally so we need to pull it and then run it cool now one of the first things that I always install on freshly booted device is jtop is a tool similar to Linux top however designed specifically for Jetson devices we'll follow steps listed in official installation guide link to that guide will be in the description below and apparently the whole installation is pretty simple we just do peep install at Jetson stats foreign [Music] style is pretty hard when you don't have peep installed so let's go ahead and do that we need to execute app update and that may take a while given the fact that it's freshly installed system and when it's done we can just execute sudo apt install python free pip patience patience that also can take a little bit of time now we can go back to the original jtop installation command and execute sudo peep install Jetson stats this time the installation goes without any problems however we need to reboot manually our board to have the access to our new toys we can do it from terminal using shutdown command with proper Flags after the reboot we log in once again use our credentials and when the operating system loads we open the terminal and we should have access to jtop that tool is really useful because it allows us to measure the usage of GPU and CPU as well as the memory consumption on top of that we can perform changes in the board configuration without going through the hassle of figuring out pretty complicated commands so let's put the board through the first test and use it as the inference server the computer version model will be running on the Jetson however the client will be running on the Mac to do it I first need to figure out what is the IP address of the board in my local network I open the terminal once again and type ifconfig command this command will return a network configuration for my board I scroll up looking for eth0 configuration this one is related to ethernet and in it I find IP address this is the address of the board in my local network now I'm back on my MacBook open up the terminal and use SSH to remotely log in into my board so I use my username and the IP address that I was able to retrieve when I was still locked in into the board provide the password and I'm in now I open up another terminal next to the previous one log into the board once again so I will have two shells open simultaneously locked into the same board and I will use it to run jtop so the idea is that I will use the right terminal to keep track of the usage of the resources on the board and the left terminal to run the server and make requests now I'm going to roboflow inference server documentation and I'm copying the command that will essentially spin up the inference server on the board depending on your internet connection starting that server for the first time may take a little bit of time because you most likely don't have the required Docker image yet on the board in my case I already pre-downloaded that Docker image so after just few seconds I see the information that the inference server is ready to receive traffic now I open yet another shell that I will use to make requests to the server so like I said the first two shells are connected via SSH to the board the third one will work locally I quickly navigate to the directory that contains images of players on the football field and that is because I plan to run my own model that is already trained on football images by the way if you want to learn more about this model and how it was trained make sure to watch another video from this channel you will find it in the description below and in the card in the top right corner in the meantime I logged into the roboflow and I navigated to the projects tab now I select football players detection pick the version of the model that I want to deploy in my case it's number two and at this point I'm pretty much ready to run the inference so I'm going back to roboflow inference server documentation and I call copy the shell of the inference command I paste it in the terminal and before I run it I need to do few changes so first of all I need to point that command to the image that I want to use for inference if you work on Mac you will also need to pass additional flag Dash I to base64 command so that it will know that you are passing the input next up is the IP address of the inference server so I need to change it from localhost to the IP address of the board in my local network localhost would be correct address if I would run the client and the inference server on the same device but because it's not the case I need to pass the IP address to be able to communicate now I'm going back to my roboflow account to copy the ID of the model that I want to deploy on a board I do that and switch back to the terminal to replace the your model part of the request command with my model ID at this point there is only one thing left to do which is copy my API key from my roboflow profile I'm back in browser I go to my profile click settings then roboflow then roboflow API and copy my private API key now I paste it back in the terminal and my inference command is pretty much ready I hit enter my request is reaching the server and we immediately see that it starts to download weights they are loaded into the memory so we immediately see on the right side that the usage of memory is going up for all those reasons the first request is taking a little bit of time to process but after around 30 seconds we see the First Response being returned if we take a look at the response structure we see that every bounding box is described by its X and Y coordinates as well as width and height of the bounding box plus the class and confidence now let's go to J top and switch our view from memory usage to GPU usage and run our Command once again first of all we see that the inference this time took significantly less time second of all we see a noticeable spike in GPU usage and that's all for today that was the first video from the whole series dedicated to Edge devices so you can expect other videos about jet sounds Raspberry Pi's Oaks and all other Edge devices coming to this channel soon if that's something that is interesting to you make sure to like And subscribe this way you will know when the next video from this series will drop my name is Peter and I see you next time bye

Channel: Roboflow

Views: 13,476

Keywords: jetson nano, jetson nano developer kit, jetson nano setup, nvidia jetson nano, jetson nano install, jetson nano docker, jtop, jtop install, jetson nano object detection, jetson nano debug, jetson nano inference, inference server, nvidia, jetson nano tutorial, jetson nano jetpack, jetson ubuntu, jetson xavier, jetson nano boot, edge, edge device, edge inference, jetson nano sd card, jetson nano python, jetson nano demo, jetson nano 4gb, jetson ssh, jetson ip address

Id: X9jt8qb_igo

Length: 14min 23sec (863 seconds)

Published: Thu Feb 23 2023