Self Driving Model Test On my Real Car | Semantic Segmentation Nvidia

hey everyone welcome to my channel in this video we are doing something very special something very different as you can see i am outside and we are going to go for a drive and not any drive we are going to test out nvidia's jets and nano with a simatic segmentation model so let me show you the setup here so i have i have this extension connected all the way to the front and let me show you what's happening in the front [Music] so here you can see we have the jetson nano and then we have a screen this is for me to check what is happening and i have a keyboard and mouse as well so that we can debug any issues that we face now for the main part i have a camera here that will record the main video and then there's a camera there that is going to actually send the feed to our jetson nano and then it will do the simatic segmentation so let's go ahead and try it out so here is our system we have the camera two cameras and then here is the screen and you can see the simatic segmentation model is running here this is just for me to see what is happening and you can see that there you go it is running around 40 frames per second uh 48 was the maximum number that nvidia has shown on their website so it is very close to that and the idea is that if you have the purple side or if you have the purple color then it will be a road if it's blue then it will be a car so we are going to drive around and see how it performs right now it's very sunny and you can see here it's 50 degrees celsius not fahrenheit celsius so and i'm doing this for science because i don't think it will work that well if i do it at night so let's go ahead and try it out so full disclosure this is quite dangerous if you're doing it yourself so don't try it out alone if you are trying to do this don't try it at home or don't try it outside alone so it can be quite dangerous if you're looking at the screen and then you're driving again and again so i'm i'm going to do it in quite an empty spot so that uh there's no issues so let's go ahead and try it out we will go back and as you can see already it's changing a little bit okay so now what you will notice is that the green areas are the trees or the grass the purple area is the one where we can drive and then we have the light purple that is empty but you cannot drive on it and the main part is the blue color which is the cars so if you see any car it will be in blue so let's go ahead and see how it works so we have a little bit of green on the sides so there is a car on the side going in front of us so let's see if it detects that blue yes i can see so on the left and the right all the places you can see there are cars and it is detecting it as blue that's excellent and then we can see most of it is purple it means i can drive through that that is good and on the right you can see there is a little bit of green that is excellent as well and then you have again a little bit of green on the side so again it is good so far and actually it's very good so far surprisingly because this jitsu nano is literally 60 dollars so you can't really expect a lot of performance here but you are getting real time uh what you call semantic segmentation which is amazing if you're not familiar with simatic segmentation it is uh it is classification on a pixel level so every pixel is classified whether it's a car whether it's a road whether it's a tree so this model is trained on cityscapes so this is what it was trained on and as you can see it is performing quite well so i'm going to take a turn now and let's see what do we have in front so there's a car coming there's a bike i can't really see the results properly but i don't think it picked the bike but it did pick the car so so far it is doing quite well better than i expected the car is beeping not because i'm not wearing a seat belt because there's a lot of stuff on my other seat and it thinks that somebody's sitting there so forget that and then we can see on the right there are a little bit of blue spots but it is getting confused but in front you can see there is a car coming and it is detecting it properly that's good and then we have the green areas that's very well and this is an interesting part i can see that there is a light light purple part this area is basically ground so you cannot drive on the ground but it is an empty spot so probably it's telling you that so it's very good that it's able to differentiate and you can see this guy i don't know why he's walking on the road but you can see we can see a person walking on the road that's good uh it showed it in red color okay so there was a little bit of traffic back there so i stopped because as i mentioned before it is dangerous to do this alone so we will go around and we will see now we didn't really test with people that much so i would try to find someone around walking so we can see okay we have someone on this side so let's try that so it might look like i'm following them but that will be weird so i'm going to turn now to see if we can detect them now this is a parking spot so there's a person moving so that's fine and yeah i can see if the person was detected there's a biker there maybe we can go there or let's go a little bit outside maybe we'll find some more interesting stuff but overall i am very very impressed with the performance because i did not expect it to be that good on on the website it said 83 accuracy and this is very good for the amount of performance this chatenado has and the amount of frame rate we are getting out of this it is pretty amazing what i'm looking at here so i am not sure how well it will perform at night but i will be very interested to see what will happen if we run this on the xavier so the xavier on their website they mentioned that it can run up to 480 frames per second with this model so that will be insane and there is another model that works a little bit better than this it has the accuracy of higher 80s so maybe if we try it with that then we will get even better results and we will get faster frame rates but of course the webcam we are using is 60 frames per second so we cannot go beyond that but still it will be amazing to see 60 frames per second so i think that is pretty good what we have so far there's a bike here let's check the bike i'm going to go close to him but there's a lot of what he calls no it's not detecting there's a lot of what he call cars around him so it's not getting detected overall it looks quite good maybe if he was driving probably it would detect him as well so let's try to find something interesting so there's a person on the grass right there i'm not sure oh yeah it is detected so it is giving that red red mark sometimes so it does detect it sometimes that is pretty good and we have another car coming from the side what is this blue line i i i missed it i didn't see what is this blue line i'm not sure what the light blues maybe it's the sky but the sky should not be down it should be up so i'm not sure but anyways so what i can see so far is the green which is the the what you call the trees and the grass whatever you want to call them then the blue which is the cars and then the red is okay like it's not that bad sometimes it detects sometimes it doesn't detect and of course we are using 640 by 480 if we increase it up to uh 1280 by 720 we will have better results so again we are limited a little bit with the power again but if we tried it out with xavier i'm pretty sure we are going to get much better results than this so and we can bump it up to hd you know and probably will get 60 frames per second in hd i mean that will be something to have a look at you know that would be pretty amazing to watch so i think that is pretty much it i'm going to go back and park and i hope this video was fun for you this was something new i tried i thought why not test out the xavier the what do you call the cityscape model and see how it works on the jetson nano so that is it for today i hope you learned something new and i will see you in the next one

Channel: Murtaza's Workshop - Robotics and AI

Views: 20,234

Keywords:

Id: hFQEAUF-WJM

Length: 10min 28sec (628 seconds)

Published: Wed Jul 07 2021