Review of the new NVIDIA Jetson Xavier NX Single Board Computer

[Applause] [Music] so [Music] so [Music] [Music] hello guys this is paul mcquarter with toptechboy.com and we are here today with what might very well be the most exciting video that we have done since being on youtube yes i have gotten my grimy little mitts on the all-new jetson xavier nx and let me tell you ladies and gentlemen it is one bad boy so what we're going to be doing today is we will be putting the jetson xavier nx through its paces so i will need you to pour yourself a nice big mug of iced coffee and i will need you to get ready to see some pretty cool stuff i'm going to warn you right off the bat this is not going to be your typical review what is your typical review the guy has it in the box he has a little exacto knife he carefully opens it up he brings it out he puts up a chart of the specs then he runs the demo programs that came with it and then he runs some benchmarking you get excited you go out and you buy the board you open it up you boot it you run the demo programs and then you throw it into that huge stack of single board computers on your workbench that you never learned how to use and so to me the question for today's review is not so much how much raw computational power is on the board because no doubt it is immense there is a mind-boggling amount of processing poor power on this board the real question for me though is how much of that processing power can a normal person like me or you actually utilize and what do i mean by a normal person well a normal person i would define as somebody who's done a lot of playing around with the arduino the arduino very comfortable with the arduino platform maybe booted up the raspberry pi and done some python programming there kind of know your way around the linux terminal you know your typical home hobbyist okay and so to me the issue is not how much raw computation power a new board has because it seems like new boards come out once a week and always these great specs but the real question to me is is that what can we do with it and that's what we're going to do today i'm going to be taking this thing through its paces i'm not going to show you anything that i didn't write the programs that i show you are programs that i wrote and if i can do it i can teach you because i taught you the arduino i taught you the raspberry pi i taught you fusion 360 and i'm in the process of teaching you the jetson nano and so you know if i know how to do it i can show you how to do it and so we're gonna see today exactly what can i do with this cool new board you're probably sensing i'm very excited about it already okay so what should we say about this i gave some uh specs there in the opening intro hoped you liked the new intro thought i ought to kind of spiff it up a little bit jazz it up a little bit for the uh for the new board that came out hope you liked it but uh where would i just jump in and describe this board the first thing that you'll see is it pretty much has the form factor of the jetson nano not only does it have the form factor of the jetson nano this new board also runs on jetpack just like the jetson nano does and right off the bat that makes me happy because yeah i'm excited about the new board but i don't want to have to start learning everything from scratch so the good news is because this runs on jetpack all that stuff that we learned on the jetson nano can be put to use on the jetson xavier and so that to me is very exciting i like it also that if you look because of the number of cpus all those gpus cpus and tensor cores that this thing has they went ahead and put the fan on it built it in and you remember on the jetson nano we had to kind of do the fan management we had to find a fan and get it screwed on there get it all hooked up and everything i really like this out of the box it has the fan the other thing that you'll see out of the box is similar to the second version of the jetson nano this actually has two slots for raspberry pi cameras so you can run two raspberry pi cameras my original jetson nano you can only run one raspberry pi camera at a time so i thought it was pretty cool that this one out of the box first version does allow you to run to uh two raspberry pi style cameras the other thing that makes me happy is is that like the jetson nano this has the 40 gpio pins and those gpio pins are what allow us to interact with the outside world so yes you have a super computer but that super computer can interact with its environment it can make sensor readings it can move things it can it can interact with the real world now even better than that what i've found is uh you know most of the work that i'd done on the jetson nano was on jetpack 4.2 and on jetpack 4.2 it was actually a little bit of a chore to interact with these gpio pins but now on jetpack 4.4 they've made it much easier to interact with the gpo pins and what i found is a lot of the standard libraries like the standard adafruit libraries you could just run unmodified on this and therefore you could use a lot of those really cool accessories and devices that are are on adafruit you know sensors and various uh modules those will run very easily on this using the adafruit library so for example i got two servos made a pan tilt platform and i was able to run those things on the jetson xavier by simply plugging them in to the right gpio pins and running the adafruit uh the adafruit libraries and so i thought that was super cool that really has me excited not just that this has way way more processing power but that we're able to more easily interact with the outside world through the gpio pins we can turn it upside down and there is just even more good news what you can see is similar to the jetson nano you have got this slot where you can put your wi-fi and bluetooth module but this unlike the jetson nano sits on top of a little bit of a plastic base and because it sits on a little bit of a plastic base over here they put two nice little slots for the wi-fi antenna so you don't have to do that antenna management like you do on the jetson nano here you've got nice little place for the antennas now perhaps the best news of all that really really made me happy was this pci slot where you can put on in a solid state drive and you can see that today we are running a western western digital wd black 250 uh gigabyte hard drive the reason i really like this i think you still probably i don't know of a way to completely boot from that drive without having the sd card at least at least in there to sort of pivot the boot over to the uh over to the the hard drive but what i like about this is when you're doing development you're installing a lot of things you're really in there doing a whole lot of stuff and i have had that little sd card get corrupted more than one time and so it's really nice to be storing your programs down on this uh on this uh solid state drive now on the jetson nano what i had to do to get external storage was to hook it up through a usb and you know you never have as many usb ports as you really need to begin with and then it's also kind of a kludgy thing to have a hard drive hanging off of your usb and so really really like this so everything i've seen so far i really really like and so now what i want us to do is i want us to go in and i want us to see if we can get this thing booted up and see exactly what we can do because again the analogy to me in getting a board like this it's a little bit like somebody gave you a ferrari okay they give you a ferrari and it's beautiful and it's bright red and you can turn it on and it makes those race car noises and you're all excited and you see the experts and they're out on the track going around the track at 180 miles an hour but the but the problem is you don't know how to shift gears so you get in it and you go around the track at 30 miles an hour because you can't get out of first gear and that's what i r that's what these single board computers are like like just because the gpus are there doesn't mean that you as a normal person can get your code down there executing in it so what i really want to see today is for us normal people of the incredible mind-boggling amount of computation that is on this thing how much can we actually access so that is what we are going to be doing today this will be a little bit of a longer review because i really want to put this thing through through its paces and i want to put it through its paces in programs that i write and then we can kind of see how well that works so i'm going to come over here and i think i will get out of your way and i will switch to a different camera view and we will come over here and yeah i think that looks nice i think this is gonna work okay and so what we're gonna do is we're gonna start plugging things into it and i like to start this is the little dongle for my wireless keyboard okay the little dongle for my wireless keyboard and mouse so i'll start by putting that in and i have found that that does good that kind of does better if you put it in a place where it's got line of sight to the keyboard because i have gotten it kind of buried down in one of those lower further away usbs and the keyboards had a little trouble finding it okay now we are going to plug in our first usb camera and this is the logitech 920 camera and then i will also be plugging in a help camera webcam and i will plug that in up here now like i said earlier this thing has the two slots for the raspberry pi style cameras but it's just cable management and camera management is a little bit easier for the sake of this review to use the uh to use the webcams but i have played around with the raspberry pi cameras and indeed they work great okay now we are putting this thing through its paces today so i do want to give you some demonstrations of working with those gpio pins so i am going to attempt to plug in two servos i did uh some kluji uh scotch tape here hoping that i would be able to then plug all of everything in at the same time and i'm going to kind of be mindful to look at that and i actually believe that i got that in there correctly and now we will put in the hdmi cable and we have already flashed the sd card and guys i've made several videos on how to flash those cards so i'm not going to bore you by repeating that if you haven't seen those videos let me just say it's real easy to flash the cards it's no big deal and so don't worry at all about that but to save time i already flashed it it is already in and so now we are gonna try to boot this thing and because this is our first booth i'm gonna have to ask you to hold your breath and if everyone holds their breath i do believe this thing will boot so let's go one two three yay okay we got the happy little nvidia logo don't you love that logo does it is it just me or does it make you happy when you see the little logo pop up and i don't know why i just always love to see linux uh computers boot there's just all this stuff going by i don't know maybe it's just me but i love watching a linux computer boot okay so it looks like it's at least alive signs of life we got the happy little nvidia logo again and let's see here looks like we are getting really close to seeing something good and boom we have booted up in ubuntu and it just did not take very long at all and so this is exciting i also love the cool background the the uh back screen background that they put on this thing that is just really screams of super computer to me so i really like that okay so let's see if we can kind of get this thing configured and start putting it through its paces now one of the things that i told you is is that a real theme of this video is this video review is going to be how much of that gpu horsepower we are able to access or whether we are just sitting there in first gear our race car in first care gear doing all of our processing on the gpus so what we will do is we're going to come in and we're going to open a terminal and we're going to run a program called jtop and it is run with sudo jtop and let's see if that thing will fire up boom there it is and this is a really cool program even if you guys are on the nano and don't have the xavier yet this is a really cool program because what it shows you here is it shows you that i am on jetpack 4.4 believe it or not i found it extremely difficult to figure out what version of jetpack i'm on like if i have different sd cards floating around this is a cool little program but it shows you that you're on jetpack 4.4 and l4t 32.4.2 it shows you that on the jetson xavier we will be running with six count them six cpu but most exciting down here is this line where it is showing you the usage of your gpu now if we are sitting here in linux doing nothing in particular you can see that nothing is happening on the gpu well every once in a while maybe it seems like when i move my mouse a little bit it uh it hops up and does stuff so it jumps down there every once in a while but mainly things are going on up here in the cpu as you would expect and the cpus the six of them are running at about 10 percent maybe 15 so as we're doing these deep demos what i want you to be mindful of and i'll be mindful of and if i'm not be sure to remind me that we want to be checking out this gpu line and see if we are actually getting our processing down on those cores or not now what i will say is is that probably a lot of times with these single board computers you guys get them and you get them home and you start playing around with them and they're running your programs but they're not actually running them on the you know on the gpus and so we want to be very mindful of that and we want to see are we really using the horsepower that is available to us now the nice thing is is that when nvidia makes these jetpack images they put the versions of software on there that play nicely together and enable them to be on the cpus and so by using their configurations and not going out and upgrading to newer versions of software what i definitely recommend is run the versions of software that come on the jetpack image and then wait for the new version of jetpack and then things will be upgraded in a way that you'll still be able to access those uh those gpus a lot of times people email me oh well this is going wrong that's going wrong well what did they do they went in and they put in this new pack they upgraded this package and this package just started changing things and then all of a sudden they're kind of stuck on the gpus okay i am talking too much let's jump in here you guys know how much i love visual studio code so we will be coding in visual studio code today pop that thing up and with a little luck it will open up here and let me do a little tiny amount of windows management you guys that watch my channel know that we must have our windows perfectly aligned for the universe to be in proper order okay so let's come in and let's run our first program and what i want to do is you guys know that normally on all my videos i write the code along with you right in front of you but i want to do a lot of demos today so i already wrote the code i'll kind of explain it to you a little bit but just rest assured anything that i show you today i wrote and if i wrote it i can teach you to write it okay so you know that i taught you arduino i taught you raspberry pi i taught you all those different things so you know that already so let's look at this program this is a simple python program most of you guys should recognize it we import opencv we then we then tell it what size we want our image to be 800x600 then we create a camera object we do the cam we create the camera object with a simple command cv2.video capture that's an opencv command and then we are going to camera zero and so you can look in your slash dev and you can see what cameras you have we have two video zero video one for this demo we will be operating on video camera zero then after that we tell the camera after we create the camera we tell the camera how wide we want it to be and how high we want the image to be so we're setting parameters on the camera so that we get an image the size that we want and then just to make sure that we get that after setting it we ask it we check make sure we're getting what we wanted and then after we ask it for the size we print it out so really we could skip those five lines of code if we wanted to and just take the default value we then start a while loop while true when is true true true is always true so we've created an infinite loop and we do two very simple things we grab a frame from the camera and then we show a frame from a camera if we sit and just grab show grab show grab show we are making a what we are making a movie which with a little luck it should pop up here boom okay in just a couple of minutes from booting we're already doing something pretty cool we are grabbing a frame and we are showing a frame and this is off of i do believe let's see that is off of the logitech 920 camera so let's go ahead and kill that with the q what this stuff down here is this just lets us kill the program cleanly release the camera and uh destroy all the windows so this is just a little cleanup at the end of the program but we can see really the action is creating the camera and then reading the frame and showing the frame now this is the cool thing with a computer that is this fast you see we read the frame and we show the frame we see we are sitting here now in this magic spot between grabbing a frame and showing the frame and this is where all the magic happens because what i want you to see is i want you to see that if this camera is running at 30 frames a second and you've got like a 30th of a second in here between grab and show in computer time that's like 15 years you've got like all the time in the world to do all types of wonderful or mischievous things and so that is what this is really about is the artificial intelligence is about the magical things that you can do between grabbing a frame and showing a frame now give you one other quick little demo if we can run one camera well why not run two cameras and so instead of just creating the cam we're going to create one camera called cam0 that's going to be on video 0 like before that was the logitech camera and then we're also going to create cam1 which is going to be on video one that's my elp camera let's see if i can show you the help camera yeah it's sitting over there okay that's my up camera so we're going to see if we can run two cameras and with this i also want to show you something else you got to kind of started thinking about an image not as this mysterious data what you got to see is it's simply an array a math array where you have rows and columns and then you have values at each point of intersection of row and column and so those of you who have done any python at all have certainly run into numpy so what i want you to see is i have grabbed a frame from camera zero that's frame zero grabbed a frame from camera one that's frame one and rather than having two windows i'm gonna just treat those as data arrays and i'm gonna horizontally stack them together so i'm going to have one picture that is the two images horizontally stacked together with a little look and then what i will do is i will show just one frame okay i will show just one frame which is or just one window which is the combined frame so let's go ahead and see if we can do that guys notice i don't even notice if the fan has come on yet this is just so trivial for this board that it hasn't even bothered turning the fan on so let's go ahead and run these two cameras and see if we have any luck boom all right camera number one and camera number two uh-huh okay so you see see how easy this is i mean this isn't some big mysterious black box this is nice friendly python and if you look down here particularly like if you look i think that when you start moving it starts doing image compression and then that starts being more computationally intense maybe i'm just imagining it but it seems like the gpus start being used a lot more when there's a lot of motion it seems like they they pop up but you can see that our cpus are running at about 25 and then a lot of the processing is being done down in the gpu and that's what we wanted to see why is that because the guys at nvidia they actually know what they're doing they gave us a jet an operating system image that has a version of opencv that's actually gpu-friendly that's actually trying to push that processing down onto the gpu so that's very good news so let's go ahead and quit this all right and let's go ahead to my next demo now what i was talking about earlier is that magic spot that exists between that intellisense autocomplete i love it and i hate it i love it when i'm trying to figure out what the command but i hate it when i'm trying to do a video and it keeps popping that window up in front of me and covering up what i'm trying to show okay but we're sitting in this magical spot between grabbing the frame from the camera and showing the frame on the display and so what type of things can we do well let me come over here and you can see this program is a little bit longer it's about 65 lines of code but just good old-fashioned python loading in our files but i'm just going to run this and kind of show you talk you through it after we run it so i'm going to run python file in terminal okay look here so what you can see is we've got a picture of me and then floating in front of me is the python logo and so let me kind of show you how that works in that space between the read the frame and show the frame i went out and i grabbed a jpeg image of the python logo i then converted that jpeg image and i am sorry that my cursor is not bigger there but hopefully you can see it i converted that jpeg image to grayscale and then from grayscale i thresholded it to create a mask okay this is the foreground mask and then i take the opposite of the foreground mask to create the background mask and then after creating the background mask what i create is i take the foreground and i end it with this image and then the foreground mask with the python image and then i end up with what the foreground and i'm going to have to move to a different shot here because you can't see all these okay now let's work on the background i run over and without you even noticing i steal a little piece from the picture a little region of interest from the picture okay and then i apply the background mask to that and then what do i end up here i end up with the background now if i add the background to the foreground i end up with the whole image which is the python logo floating over the frame that's coming from the camera now this is the little thing that i created in that space between grab a frame and show a frame and right i stole that little piece of the picture that you didn't even notice now before you even see that it's gone i'm going to go back and put it back over there so this thing that we created here you see that's the logo on top of that little region of interest we're going to run back and put it on the image and then what you see here is this really cool demo where we have a little uh watermark that is floating around the screen and all this is happening in the blink of the eye where you don't even know it and again we look over here and yes we're getting to see more gpu action here i saw a couple of times it was going up to 70 80 percent as it's doing a lot of this image processing so we're doing a cool demo and we are doing it in the gpu which is really what i wanted to see i just think that this is kind of a silly demo but i think it's kind of a fun demo and really this is kind of a neat demo because you kind of got to learn a lot of the opencv to do a simple little program like this so it's a good learning program and i think i did a lesson on it on my jets and nano series so we're going to go back to the larger image of the jetson xavier nx and i apologize if i've ever called it the nano during this thing i do a lot of work on the nano and this review is about the jetson xavier nx so we're going to quit out of this program okay now that was not really doing per se it was not per se doing any artificial intelligence it was really just doing more uh you know more image processing right there was it wasn't looking at anything and learning anything or making any decisions but the next program we're going to kind of just ever so slightly dip our toe into artificial intelligence and why do i say that because what the program does depends on what it sees from the camera so it kind of has to start learning so again not full-blown artificial intelligence but dipping our toe into the topic so what i'm going to do here is see if i can train it to see something see if i can train it to learn something and i hope this works because i hope that it is hooked up to the right camera but if it doesn't i've just got to switch one command because sometimes i'm not sure which is video zero and which is video one but what i want to see is i want to see can i train it to learn this pin okay now the very very simplest way that you can recognize an object is on color and so you notice i picked something with kind of a unique color i picked something with a unique color and then i train on that color so if we kind of just look at the code what you can see is is that i'm setting these limits for hue saturation and value and then i'm creating kind of a threshold where i'm only going to keep the parts of the picture that are within those limits of color and i'm going to throw everything else away and then when i have only the object of interest then i'm going to draw a box around it and then i'm going to go put it back on the main program if things work the way i want you see i got a lot of white space in this code but it's still just 44 lines of code so we could develop something like this in like 15 minutes and again this is the type of thing that i could actually teach you to do so let's come up and let's make sure that we're not already running the program and let's go ahead and let's see if we can run this thing run python file in terminal and nothing seems to be happening do not do that right let's try that again right mouse click and python file in terminal all right boom look at that okay now you see we have a v stack image right we we stacked horizontally on the other one here we stack vertically this is the original image the program looks at the image looks for that color that i defined and then in the bottom image here it throws out everything in the bottom screen it throws out everything that isn't the found image then we do a contour on that we box it and then we put it back up in the up in the upper picture so i just want to make sure you can see that you see that it finds the pin and then it box it and it tracks it okay you see that's pretty cool and if we look at that again we are running on the gpu the cpus are about 25 30 percent and all of this is being done on the gpu another thing that i want you to see is we're really smooth right while we're doing this we're really smooth it is very very smooth i don't later on we're going to get a a frame counter in there so we can look at frames per second but this is certainly running at the 30 frames per second that the camera is capable of so wow i am having fun with this i am really having fun with this okay so i said that we're tracking it we're tracking it as long as it is in the field of view okay we're tracking it as long as it's in the field of view but what if i really wanted to track it what if i really really really wanted to track it let me kill kill this what i am going to do then is i am going to move over to my elk camera which is sitting on top of a pen tilt servo that i built well it's a pan tilt platform that's controlled by two high-tech servos and so the question is can we run those two high-tech servos from the jetson xavier nx kind of out of the box well to make that work what i had to do was i just had to do the whatever sudo apt-get install whatever the adafruit library is the servo library and then let's look at the code am i on the right spot i need to move over to this one okay let's look at this code real quick and see what it takes to run the servo i just from adafruit servo kit i import servo kit i just import the library and then to make the servos move where i want i have to create a couple of servos and so let's see where did i create those servos okay i tell it that it's got 16 channels and then let's see here i guess just right off the bat you're not even you're not even i didn't even have to create them i just have servo zero and then i have servo one okay and to move them all you got to do is just give kit servo which servo are you talking to 0 and then dot angle and then what angle do i want pan which is 90 degrees and then i set tilt to 45 okay so whatever number you put to the right of the equal sign it will move the servo to that point and what i'm doing here is i'm just moving it to kind of the nominal position to get the thing started okay just starting out moving it to the nominal position and then after that what i do is i look for that blue pen and then what i do is i track on the blue pen and so let's see if that will work and again the one thing i have to be very mindful about is killing the old program before i start the new program if you try to go out and use the camera when it's already in use sometimes that actually requires a reboot and that would look very amateurish in the middle of an important review like this so i'm trying to be mindful to make sure i've killed the old programs so let's see here i hope i very much hope uh let's see that blue pen was operating on video zero i think that might need to be video one and it is video one so that hopefully will be operating here i'm gonna go ahead and let you have a look at let's see if i can kind of give you a better view here of my camera and my pan tilt set up like that okay i need everyone to hold their breath no this one i think it takes a while to load the servo library so we won't hold our breath this time but oh i hope this works i hope this works please work please please please work don't die don't die i came to life okay okay here you are look at the pin look at the pin okay now watch this boom tracking tracking in real time do you see this look at that we can go up we can go down it's like it likes the keyboard now stop stop looking at her you're just going to get frustrated keep your eyes on the pin it's like he's got a crush on the keyboard keep your eyes on the pin okay look at that coming over tracking in real time with two servos tilt and pan i told you stop looking at her no i'm not gonna plug her in get your mind off of her think about the pin okay so we're over here and look at that do you guys see that now i would say this is really kind of starting to be artificial intelligence right because this camera is doing things based on what the cam what the the camera is seeing and you're actually recognizing an object and you're trying an object okay yeah we are training it stop looking at her i told you okay stop it okay so i told you that the easiest thing to track on is color so we kind of right we take baby steps before we run and let me see if i can kill this thing now come over here and we will cue out of it all right well if we can track on color we can start tracking on things a little more complicated so what we are going to track on is see if you can guess here and i just got to make sure that i am on camera one because i do believe depending on where i plug them in these things can't change yes i'm on camera one all right so this program it has a lot of blank spaces in it i just want to kind of give you an idea for how how much we actually how much code we actually have this program is about 60 lines of code again you know we could do a lesson on this and learn this in 30 minutes and so this is not terribly hard stuff so we're going to come in and we're going to run this and we're going to see what this one does again it takes a little moment to load the servo library i don't know why the servo library is kind of slow but it takes a little while to load it and let's see boom look at that recognizing me okay it always kind of makes me feel good when i run one of these programs that i am not so odd that it doesn't recognize my face as human okay but you see it it found my face and boxed it and then it found my eyes and boxed them and then i want you to pay attention to the little dots you should see a red dot and a blue dot the red dot is the center of the camera view and the blue dot is the center of my face and this is just kind of a little thinking about control systems well what is the objective you want my nose to be in the center of the camera so the difference between the position of the blue dot and the red dot is the error and what the servo system does or what the control system does is drive the error to zero so if i move over here we got an error and then the system drives that error to zero okay and you see same thing i can tilt i can tilt or i can pan and that thing works really great all right just again like even though this is a single board computer super computer review just can't help saying a little bit about control systems now if you notice if i'm real still the blue dot comes almost on top of the red dot but it doesn't get absolutely on top of the red dot now why is that because this servo i have has only one degree resolution and so when i get within one degree of the target i have to stop because if i tried to drive that tiny error out you see that tiny error if i tried to drive that error out it would jump past it a little bit and then it would see it was past there and it would jump past and passed and passed and so as it's trying to drive that last little bit of error out it's going to be glitchy it's going to kind of go back and forth and i would rather just get if i'm within a degree of the right position just stop and say good enough boom guys do you see this look at that look at that wow i can swing all over and it continues to find me shazam do you see this and again this is a program that we could write like in 30 minutes and that's why i'm so excited about uh this jetsonic xavier and also as i was saying look at this we've got a lot going on down here in the gpus and so this thing is taking advantage of those gpus and so even this program i would say that this program you would have operating like this at this resolution on the raspberry pi so we're really already beyond what you're going to be able to do on the on the raspberry pi and i didn't really optimize this for frame rate i didn't try to go in and really optimize this thing but i think this is just pretty dandy i think it's pretty dandy okay so now we are going to move on to the next demo and you guys uh ask me questions down below if you have questions on any of this stuff i think this is really cool makes me happy makes me excited okay all right so now that was that so now what we did there okay moving right along moving right along and i'll come back and kind of give you your old view back here of the most excellent most exciting most powerful and capable jetson xavier nx i think you guys are starting to see that i love this thing already okay so what did i do there we found a face okay we found a face it didn't specifically find find my face it would have found any face and in fact let me come back over here and python file in terminal let me run that thing one more time let me run that thing one more time hopefully it will run one more time i hate it when things don't go well on reviews but i think it is going to run there it is okay so let's see if it recognize a different face all right yes so now this is the problem here okay so it you see that it's finding two faces so it's kind of uh uh jumping back and forth between them so i didn't really optimize it what i should have done is in the code another couple of lines of code where i could have told it to just watch the biggest face okay and that would have been like the closest one but let me see if i can do that so you see now it is going in and it is putting trump's face in the center of the picture so you see wherever that goes it goes and it puts his picture so the thing the point that i'm kind of making here is the point that i am trying to make okay right there i want to get the servo in the right position before i quit the point that i'm trying to make is is that it's just finding any face it's not finding my face in particular it's finding any face and any face it finds it draws a box around and then you could do something like in your code you could have it box all the faces or you could just have it throw out all the faces but the biggest one you know you could just kind of do different things on how you wanted the program to run but that is finding any face now taking it to the next level is not just finding a face but who's in the box okay whose face did you box and to do that you actually have to train it okay and that's the next thing i have done is i have written a program to not just find the faces but recognize the faces and so i believe that this is the program that i want to run here and i do believe that maybe i want to run that on camera zero maybe do i want to run that on camera zero no i want to run that on camera one just like i had it okay i'm overthinking this thing okay this program again is 65 lines of code we could learn how to do this in one single lesson i could teach you how to do this good old-fashioned python code again good old-fashioned python code nothing fancy loading libraries and simple for loops and knowing a little bit about opencv so we are going to run this thing right mouse click run python file in terminal and i'm trying to think this one might take a second to load that library as well but let's see if this thing can actually not just find people but recognize people so so really when people say face recognition really what they're doing is face detection to recognize it what you really want to do is identify it so you want to be able to say like face identification and i'm getting ah good i was getting scared because it wasn't in there all right look at that boom you see that paul mcquarter giddy up giddy up look at that and so it's finding my face it is keeping the box on my face now i don't have the servos on on this program but i could also like have it track my face in fact i could have it look at all the different people identify everyone in the frame just like it's doing here and then tracking on one person and look at this we are doing this at 14 frames per second try that on the raspberry pi i don't think so i don't think so it's going to work and then also look back here we are really we are putting those gpus now down to like a hundred percent the cpu still look like they're running about 20 25 but the gpu is as i start moving we are really starting to work that thing and if you look at it the fan is really running now so that is pretty exciting now normally i would have my posse in here and show you with live people how it could recognize different people but since we are practicing social distancing and are in the midst of a pandemic i can't bring live people in here to show you how well it works with multiple people so we will uh just do the clue g thing of pointing the camera at the screen i really i really hate that okay it recognizes donald trump that's great uh and you might say well why am i putting donald trump and nancy pelosi pictures i want to be squeaky clean as far as copyright goes and so these are government uh non-copyrighted images and so i am showing those so that we don't get in any copyright problems but you see right off the bat it is recognizing donald trump and we can kind of run the video okay congratulations that's great as part of our commitment to improving okay so you can see that that is working i'm wondering if it can get nancy pelosi as well yeah look at that nancy pelosi that is pretty darn good okay it's uh donald trump okay so that works really well let's go to another video i guess i guess i should i should shut that one off let's go to the corona virus task force and look at that right off the bat we've got my mike pence and sema verma and it is working great let's look she is just partial in it dr burks giddy up this thing is working is working like a charm so we got three people in the picture and it is recognizing everyone dr han uh-huh boom look at that that is really amazing now seriously though the kind of really interesting thing is let me turn the volume down a little bit so he won't be quite so okay the interesting thing is okay it's getting a little confused there on mike pence it every once in a while thinks it's me but one of the things is it does a really good job of overlap okay when the two faces are overlap it's able to see mike pence back behind dr hahn and other than every once in a while thinking that it's me i guess that we both have gray hair and we're both about the same age and so you could understand that it might get a little bit confused there but dr burks is more unique and miss verma is more unique and so that works a little bit better and let's see if anybody else comes up i want to show you one other thing uh okay now you see it's not recognizing faces in the press now that's not because they're non-human life forms it is because the faces are small and dr anthony fauci look at that boom this thing has got them all okay but when you're looking at uh when you were looking at smaller faces i could find smaller faces but to do that it would have to start running a lot slower and so you kind of have this trade-off between how small of a face do you want to find and how smooth do you want your video to be so even with the horsepower that you have here you do reach a point that you've got to kind of make trade-offs between you know even with the mind-boggling amount of power that this board has you can max it out if you're trying to look at a big picture and find lots of little bitty spaces faces you can max this thing out let's also just uh to uh to be completely fair and balanced let's look at the previous president let's see if we can recognize the previous president this thing is working so well okay there he is all right so let's see if we can uh maybe need to be a little closer there he is all right and that was the small face problem okay not not saying obama has a small face saying that the camera was further away from him so i had to scoot it up a little bit to get him so let's see if that works look at that president obama uh-huh this thing is so smart it really works doesn't it it absolutely works perfectly and so i am just really impressed with it against small faces in the press and so it doesn't recognize them okay so we uh we'll go ahead and stop this okay we'll go ahead and stop this but the small faces you can see that that i would have to change some parameters and in fact you probably i don't see it in here but there's a scale factor yeah so i am scaling the images down to 0.35 of their original size and because of that i'm not finding the smaller faces now if i ran it where i was looking at the full frame size then the thing would start really running slow okay so let's see what else we have here uh all right this is a good one this is a good one so the stuff that i've been showing you so far are mainly kind of open cv related libraries but what i've done is i've gone in and i started kind of working with some of the jetson utilities the the utilities that are more specifically designed for the jetson nano or the jetson xavier nx and so yes i wrote this program and the program is 30 lines long so i want to show you what you can do with 30 lines of code let's take some of this nonsense nonsense out here and get it down to 28 lines of code but you can see that in addition to loading opencv i'm loading those jets and libraries okay so i wrote the program but i am using libraries that were developed by nvidia but you see it's like i've got control i can do what i want to with them and so let's run this program and let's see what kind of cool things it will do and i do believe i better think about which camera i am on uh hey let's see i will try video zero but i am not sure if that is the one that we're going to want to use for this okay let's right mouse click let's run oh that was not good that was not good okay i think it'll run this time here we go no errors that's good always encouraging looks like some stuff boom uh-huh look at this i'm a person i'm human look at that and look at that also do you see how it's finding it's kind of seeing the chair back there yeah look at that you see it sees me and it sees the chair and one of the really impressive things about this program is in fact i don't like this program though i do not like that program that was rec 8 i don't think i want the rec 8 alt i'll show you why i was using the default boxing to where here i'm letting that utility find i'm thinks the microphone's a remote control you silly computer okay i'm letting the jetson utility find and identify detect the objects but then i'm using my opencv to draw the boxes around it the way that i want and to do the things the way that i want to do it so you see i'm not just blindly running a demo program i'm using a very powerful library that nvidia provides and then i am writing my own code around it so you can see that we're operating at 20 frames per second i've been boxed with a green box and i am identified as a person so let me get this camera and let's come over here and let's just kind of look oh wow look at that keyboard boom mouse boom bottle keyboard what else cup okay it sees this is a bowl look at that let's see what else we got here let's see what else we got scissors okay and then and then if i look out here look it's finding all the chairs in the room this is incredible look a stack of books it's finding the book and then we come over here and it is recognizing the bottle and then the really amazing thing is we are operating at 20 frames per second 20 frames per second and we're finding all of these different things the other thing let's see let's see how many i can't quite get it to find the cup there's the bottle two keyboards cup cup bowl mouse mouse cup and look at that even looking at my control screen it sees me over there on the control screen it still recognizes me as a person it's just one of those little affirming things where the computer actually does acknowledge that you're a person okay i got the scissors keyboard put it out in the room i'm finding lots of chairs let's look up at my light sees it as an umbrella all right so the thing that amazes me is how it can find so many things at the same time and the other thing is it's really good i was kind of surprised at how good it is at nesting i mean things that are overlapping each other it is able to do very very well and so that is just darn impressive and doing this at doing this at 20 frames a second is absolutely amazing and then again looking at that gpu as we start doing this type of thing you can see that gpu really start maxing out as it's trying to keep track of all of these things that it is finding so that is in fact very very very exciting and there's just all types of things you could do like imagine that you could recognize certain things and like if you oh let's see all right one more thing one more thing uh hopefully one more thing yeah let's just see like i can come over here and just point there cat and dog cat and dog boom so you've got cats and dogs it does a lot of different animals it does a lot of different fruits and vegetables and then there's uh there's different training data sets so this is a particular training data set and there's a dozen or so different training data sets and so that you can play around with different training data sets let's see did it ever see cell phone baseball glove no you silly thing cell phone what you can see here also is there's something that i call sort of like user bias and so like what you tend to do when you're writing programs you put it up and it thinks it's a baseball bat and you start tweaking it until it gives you the right answer and so you got to be careful if you're really testing how accurate your thing is not to try to adjust the view until you get the answer that you want on the screen but i digress okay running 20 frames a second that is pretty darn cool that is just really really really cool okay let's do one more demo or two more demos i am going to kill this kind of a neat demo if you ask me and again this is a simple python program and it's just based on some really cool libraries that some guys put together and then with those libraries you can run a pretty simple you can write a simple program that will kind of interpret what you're doing and so let me show you what i mean we will fire this thing up hopefully let's see there it is boom okay now what i want you to see that it is done is it kind of takes you and it turns you into a stick figure and this is really besides being cool it's really pretty important and so let me see if i can kind of get up here closer the the reason it is important is you can imagine that a person is a very complicated thing and it's very hard for a computer to understand what a person is doing and so what this is basically doing this program is taking a person and turning them into a stick figure okay so you see how it just kind of takes the major joints and turns you into a stick person and so now all of a sudden you have a very very powerful way of interacting a very very powerful way of interacting with the computer because like imagine you could go like this like this and these could be commands to the robot or to the computer to do certain things or get closer closer go further further away so you could start programming your robot to do things based on what you're doing or if it has arms you could have it lift its arms when you're lifting your arms so the robot could begin to track what you're doing this thing actually will do a full stick figure it'll put your legs and your knees and your feet and everything but it's just if i do that it's too far uh i don't have room to get far enough away from the camera so i'll just kind of do the torso version of the demo but boom man this is really really really cool okay guys this is turning out to be a little bit of a long video but you can see that i am in fact very excited about this board and what i will say is two thumbs up i really really really like it i like it for the raw computation power that it has i like it that we are able to go in and access that raw computation power as normal human beings that just enjoy doing little projects at home and then i like that it builds on all the things that we learned from our work on the jets and nano that all that stuff is going to apply right over to this board so all these things i really really really like about it now as far as our ferrari example and being stuck in first gear how did we do where are we okay this is what i would say a ferrari has six gears and i would say today in these demos i've kind of gotten to second gear so i'm not going around the track at 30 miles an hour i'm going around the track at 60 miles an hour and that is pretty darn fun and it really motivates me and it spurs me on to try to learn how to go ahead and shift into third gear but that's going to be a learning experience for me because these programs that i've shown you today primarily are using kind of generic uh software so like opencv is something that a bunch of really wonderful guys are out developing and then that face recognizer program or some things a bunch of guys are out doing but they're not really jetson xavier or jetson nano specific now on within the nvidia website they have some incredibly powerful tools that are more tailored start to finish for the jetson uh for the jetson products and the analogy i would give you is kind of like if you saw with opencv we really are you know we really are using those gpus but but really what you want is one big fat pipe from the start to the end and you want to stay in that big fat pipe the whole time what i kind of feel like that i'm doing here is i've got a big fat pipe and i run to this point and then i stop and i have a little bale that i'm throwing the water from this big pipe into this other big pipe and what i personally want to learn is i want to start learning those frameworks that nvidia has provided so so really like i would really like to learn deep stream and really like to learn their transfer learning thing i would like to start learning those nvidia specific tools but that is a steep learning curve and as i learn it then i can teach you one of the things i'll say is is that it seems like the model that nvidia uses for a lot of their software tools or containers and a container is where you kind of download a sandbox onto your computer and you play in that sandbox so anything you do in there doesn't enter doesn't you know mess up your main operating system and so like if you wanted to try a different version of opencv you could do it inside of a container and not be messing with the uh not not be messing with your main operating system but playing around with it a little bit the containers are still a little bit of a mystery to me i'm not yet at the point that i know how to develop my own uh software in the context of a container that has these super cool libraries in it that would allow me to do all this great stuff so that's the next step for me is to become more well-versed and more expert well not more expert just understand how to develop using this kind of kind of container framework and now is it in is it nvidia's fault that i don't know how to use config containers no it's not their fault that's my fault i gotta learn got to learn how to do that but for me to go from second gear on this thing to third gear and then show you guys how to do it what i've got to do is i've got to move from just falling back on the things i'm familiar with like the opencv and start using some of these really incredible tools that nvidia has put together so bottom line is i really really really love this new board and i'll be honest with you this is the direction that we're going to be going in the future on this channel you know because like i said it seems like that every week there's a new single board computer that comes out and the reviews look great the specs look great but me as a teacher and you as a learner there's only so much capacity that we have so we have to pick and choose kind of carefully right i put a lot of investment in that uh in that arduino training and there's been millions of people that have benefited from that i've put a lot of investment in the raspberry pi and lots of people took that class i'll tell you i kind of got off track over there on the beaglebone black and i spent a summer making uh video tutorials on beagle bone black and then it just didn't go anywhere so so when we're going to invest in training and we're going to invest for you guys in learning we got to kind of pick things carefully and really pick a platform that's got some momentum behind it and and would really be something that we could stick with so i am today committing that i will be moving forward with this now if you're just getting started like let's say you're one of the guys one of the many guys that is watching my uh my arduino or raspberry pi series the next place for you guys to go would be the jetson nano that's your kind of entry board i've got some links down below all the things that i showed today you can do on the jetson nano if you want to start learning on the nano you can get them in the links below and then as you build up your expertise you can then be looking for that point to move to the jetsonic xavier nx the jetson nano i think is still about 99 bucks it's between 99 and 140 depending on whether you get the the one camera board or the two camera board a little bit about prices varying day to day and i believe if i am not mistaken the target price for the jetson xavier is 3.99 that's a little bit pricey worth every cent of it worth every single last cent of it but as you guys are getting started you can start on the nano and start putting your coins away to then buy the xavier as soon as i can find them on amazon i'll put a link to the xavier down in my uh down in in the description below so kind of can give you the opportunity of working on the exact same hardware that i am working on but yeah i do plan to really make a large investment in educational tutorials on this board i am really really impressed with it the guys at nvidia are great guys they put together a really really super duper platform and i am really impressed okay this has been a little bit of a long a little bit of a long video but i think worth it some really exciting stuff we had the chance to to take a look at and hope you guys enjoyed watching it as much as i enjoyed making it so wrapping things up this is paul mcquarter at toptechboy.com i will talk to you guys later

Channel: Paul McWhorter

Views: 34,964

Keywords: Jetson Xavier NX, Review, NVIDIA

Id: 83WEPcDJky4

Length: 68min 37sec (4117 seconds)

Published: Thu May 14 2020