Jetson Orin Nano - 75x Faster Than A Raspberry Pi

hey there my name is Gary Sims and this is Gary explains now Nvidia has launched a new Jetson module and along with a developer kit so this is the Jetson Oren Nano it comes with a 1024 core ampere GPU a hexa core uh processor from arm that's called its a78 eight gigs of memory uh you've got M2 slots for storage you've got USB you've got display port you've got GPA pins there's a whole bunch of stuff this board is great for embedded and Edge type machine learning applications so if you want to find out more please let me explain [Music] okay so let's dive in more into the Jetson or in Nano that's a 1024 GPU cores on a single board computer not something you see every day so this is what the development kit looks like as you can see there is a fan which is on top of a module which slots into this connector here then the rest of the board provides you with the i o so that's HDMI and USB and ethernet and access to the gpio pins and of course power and some camera connectors so this thing offers uh 40 tops using 8-bit integers powered by that 1024 ampere Nvidia design GPU up to 80 times higher performance than the original Jetson Nano and that's because it causes got that 1024 core GPU it's also got a six core arm cortex a78 which of course is an arm V 8.2 64-bit CPU and you've got eight gigabytes of lpddr5 memory it's available for pre-order right now at 500 499 dollars also note that the carrier board can support or in Nano and or in NX modules and we talk more about the other modules uh in just a moment but let's start with something a bit fun this is Doom 3 running on of course an arm processor not an x86 processor running with that Nvidia GPU this is the time demo now of course Doom 3 is a relatively old game now however it's just nice to see this running on this single board computer it's not a PC it's a single ball computer with this Nvidia GPU did the same thing with the other or uh Jetson boards that I have reviewed and when we get to the end here we're going to see the final uh frames per second and this is at 720p and there we have it 50 frames a second at 7 20 P now I also set it up on my 4K TV and I'm running in 1440p here I run the same demo again and I get a speed of 23.7 frames a second again this is a single board computer not a big PC with huge fans and you know a dedicated graphics card with you know this is a single ball computer I think that's pretty impressive okay I think we will talk about the different boards so basically Nvidia are not just into making you know graphics cards for your PC they're not also just into running the latest AI models you know chat GPT and Bing and so on they also make a whole bunch of these Jetson modules which can be in used in industrial and embedded solutions that includes self-driving cars that security systems robots in factories whatever it is you can think of pharmaceuticals whatevers you can think of they can be used and you've got a whole range of different modules that can then be connected up to some kind of project that you you're designing for whatever application and included in that are some development boards it's the Jetson Nano this is the Jetson Orion development board and these are all the different boards in between and the latest board the Jetson orange Nano developer could sit in the middle so originally here on the left you've got the Jets and nalos 149 to buy that kit and everything and then you've still got a GPU on there I'll talk more about that in a minute at the other end you've got an India two thousand dollar development kit and this sits in the middle 499 dollars gives you much more performance than this one less performance than the agx orange sits in the middle and allows you to get going with the orange Nano and the Oren NX modules and so looking at the different orange modules you can see here we've got the Oren Nano which comes in four and eight gigabytes of RAM variations uh 7 to 15 watts 199 or 299 and you get the module and then you can design a carrier board to go with it you've got the NX ones which are offer great data performance 100 tops here a bit more hungry 25 watts there 399 or 599 and then you've got the big ones 275 tops there up to 60 watts uh 1500 1600 there for for a module and so these are the new Jetson Orion modules and as I just said the Jetson or in Nano is the smallest of these three however looking at the overall system this is a quick table I put together so at one end you've got the Jets and Nano 5 to 10 watts 149 for the development kit to 129 for the module quad core uh a57 and 128 core Maxwell GPU absolutely brilliant way to get into sort of uh AI development machine learning development on an embedded device at a very reasonable entry price there and then you go all the way up you've got Pascal GPU you've got voltage GPU you've got the Ampere GPU as as you go across the range here also once you get to the Xavier NX you start to get tensor cores built in which help with the machine learning stuff and also some of them have even got these an Nvidia deep learning uh calls on them as well in fact the really big one here's got two of those plus 66 tensor cores plus only 2 000 GPU cores I mean it's absolute monster and you can see the power changes as you go up as well as the price so it just depends on what application it is that you're building what it is that you're getting into which one of these you would choose and now with the Jetson or in Nano Nvidia are bringing ampere GPU with tensor cores built into it at the lower end of the orange range so there's one of these Oren Nano boards look like well you've got six quarters a70 uh eight cores there as we're just showing here okay and you've got uh then the GPU here in the middle you've got video D code and other supporting bits of Hardware acceleration you've got loads of ports around the site I squared C and gigabit Ethernet and you know USB and PCI because there are of course some M2 slots on the bottom of this and one thing to note here is there's no accelerated video in code so you've got the accelerator video decode but non-accelerated uh encode here and this module can then be used for whatever application it is that you're that you're working at now I've done some of my own testing I am fortunate that I have a Jetson nanojitsun Xavier NX adjustment or in Nano now and an agx orange here because uh Nvidia have sent them to me in the past review so I've got some CPU tests here we'll look about some machine learning tests in a minute this is my thread test tool and as always my tools are available in my GitHub repository Link in the description below or if I put it put it in there just type in Gary explains GitHub and it will be the first thing that turns up on Google so what do we so here my multi-threaded test tool which just looks for prime numbers in multiple threads lower is better so single score core uh single core score here just over one second for the Nano 0.9 for the Xavier X 0.7 for the uh or in Nano and half a second for the agx uh or in very different when you close when you get to multi here we've got four seconds for 16 threads just over two and a half two and a half seconds for the Xavier NX under two seconds for the only Nano under one second and of course this thing here has got you know loads and loads of CPU cores and running very well compared to a quad core here so this follows the path as you expect now this is basically the trend slower too faster as you go across the range and again if we look at my sub lick a compiler test again code inside of GitHub same thing again slow down to fast and you can see that progression as you go across the range as they improve the CPU configuration the memory bandwidth and so on as you go across the different modules and when we go to my software base sha256 hash generation tool again single thread and multitude same idea except for the Xavier NX and the orange Nano get swapped around here so in this case the Xavier NX is actually faster than the Orion Nano and when we go to the open SSL speed test this will be using Hardware acceleration for sha256 we see again these two are swapped around the Xavier NX is actually slightly faster but again still this General progression the Jets and Nano is the slowest the Jetson uh agx orange is the fastest and the ones in the middle uh are in the middle basically and then you'll see here the same thing when we get to my memory test tool this General graph interestingly here though the Xavier enik doesn't do so well this is running 16 threads allocating chunks of 128 megabytes at a time doesn't seem to do so well there but that's an interesting graph again representing the same thing now when it comes to machine learning nvidia's got lots of stuff that they provide out of the box to make life easier for people who are developing machine learning type applications as I said Vision applications or whatever it is that you're doing and one of those is their people net Transformer model and what it does it detects one or more physical objects from within three categories within an image and returns a bounding box around it as well as a label so it can detect people bags and faces the Transformer net people net Transformer model was trained on a proprietary data set of 1.5 million images and more than 39 million objects for the person class the training data it can think of a mix camera height crowd densities and field of views with not just one particular camera setup basically what the idea is you could take this and it's very easy to deploy it in some kind of camera visual setup so you can detect people for example and they have a benchmark that they run that just shows the neural network part of this running and we can see here the Jets and Nano which doesn't have any of those tensor cores running on a much smaller GPU you can do this at two frames a second not bad actually considering you're paying 149 pounds certainly dollars certainly a good way to get into this and then we see that the Xavier NX and the Oren are offering this mid-range performance here and then of course the Ajax orange is running you know really really Mega farty it means you can cope with multiple streams from multiple cameras and still run pretty well on that and I actually have here a demo of this running and this is actually running on the orange Nano so this is a pre-recorded video stream and as you can see it's returning a bounding box saying person person and as they're moving around now this runs at about eight frames a second by time you actually get in the actual video do the decode and do everything else it can run at about eight frames a second which if you don't need real times even for some kind of security camera system for detecting this would be certainly good enough certainly good enough to get you up and running building these kinds of apps and so there you go that is the uh people net Transformer running there on an actual video feed and detecting the different objects okay so there it is the Jetson Nano or in development care I'd love to hear what you think about the board in the comments below okay that's it I'll see the next one

Channel: Gary Explains

Views: 52,044

Keywords: Gary Explains, Tech, Explanation, Tutorial, NVIDIA, NVIDIA Jetson, Jetson Nano, Jetson Orin, Jetson Orin Nano, Machine Learning, ML, AI, Robotics, AI Speech, Smart Cities, NVIDIA JetPack SDK, CUDA, NVIDIA DeepStream SDK, Deep Learning, Jetson, Jetson Linux

Id: abAkHQhLsSI

Length: 11min 45sec (705 seconds)

Published: Wed Mar 29 2023