YOLOv4 Object Detection with TensorFlow, TensorFlow Lite and TensorRT Models (images, video, webcam)

[Music] hey everyone and welcome to the video today I'll be showing you how to implement the state of the art object detection system Yolo v4 in the latest version of tension flow - as well as tensorflow light which is perfect for mobile and edge devices such as raspberry PI's and finally I'll be showing you how to implement you'll before with tensor RT a highly optimized inference method for deep learning I will cover how to convert the Yolo v4 standard darknet style weights into saved tensor flow models then we will step through together how to run and save you la vie for protections on images videos and webcam for standard tensorflow TF light and tensor RT model as always I hope you guys enjoy the video and if you do please hit that like button as it helps my channel grow and if you are new to the channel or want to stay up to date with all my future you'll will be for videos and please smash that subscribe button let's jump right into the tutorial [Music] okay we're gonna jump right into it so first things first as always we need to get the code for this tutorial and before we do that I want to give a huge shout out to hung LC 0:07 as he was the one who did the majority of this code and I forked it from his repository and then made the adjustments so kudos to him so to get the code we're gonna go to my github repository the AI guys cooked AI guys code and then you're gonna go to the repository tensorflow - EULA before - TF light and I'll leave a link in the description to the github repository to make it easier for you guys and then you're gonna go ahead and click the green button that says code and you're just gonna click this little icon if you're gonna do a git clone if you want to just download the entire folder and files you can just download the zip but I recommend as you'll see that I do get cloned so you're just gonna run the command git clone and then you're just gonna go ahead and paste that link right in there and that's gonna go ahead and clone the entire repository for you and it's gonna take a little while as it's quite a big repository so I'll come back when it's finished so once it's finished cloning you can just go CD tensorflow the tab and then head into the repository and if you want to look around at the code you just do code dot if you have vs code downloaded which I recommend and you can then go and look through the code as you see fit before starting this tutorial just look around and start to understand things if you want otherwise if you're following along we're just going to go right ahead with it so the github repository has a pretty in-depth read me that I'll be following along for the video if for any reason I go a little too fast or something's not sure and you're stuck refer back to the readme here as it goes through all of it in depth so if I miss anything and you have any errors check it out here so first and foremost you have to get go ahead and get all the required dependencies to run the code off really before in tensorflow there's two different ways of doing this an anaconda way which I recommend and just pip installing just straight Pippins darling and then there's also the configuration for CP if you're just running it on purely on your CPU or if you have a GPU on your laptop or desktop you can run tensorflow GPU and it actually accelerate the speed in which the object detector does the detections so I recommend if you have a GPU to use it and you're gonna see that I'm going to use the Anaconda route because for GPU it actually helps integrate all the CUDA toolkits and stuff in the environment for you so if you're doing the tip install with GPU and you're gonna have to make sure you have all the Nvidia drivers and toolkits properly set up on your machine but if you use the Anaconda route which I recommend you have or if you don't have it already go download anaconda as it'll help a lot with setting up the GPU so we're gonna go ahead and run these commands for GPU once again if you don't have GPU you can run the CPU ones and it'll work everything will work just fine so we're gonna go ahead and copy this first command right here and then go back to our shell and just paste it in this is going to go ahead and within the code it's going to copy all the requirements from whichever yanil file use that's a fly so I'm doing the GPU if you're doing CPU it would just go ahead and download all these requirements so it's pretty straightforward it's gonna take a couple seconds here to set up the environment I'll come back when it's finished once the Anaconda environment is done setting up all the dependencies it'll prompt you like so on screen and then to go ahead and actually activate the Conda environment you just go Conda activate and then the name of the environment which is Yolo v4 - GPU or - CPU if you did CPU so you go ahead and run that and now you have all the dependencies set up and once again if you did it with pip you wouldn't have to do any of those clone to commands so there's the two ways so now we can jump right into it we're gonna want to go ahead and download the pre trained Yolo v4 model this model is trained on over 80 types of classes or objects that it can detect and is really strong out-of-the-box model that's pre trained so why not use it just to see if working correctly so you're gonna go ahead and click this link to download Yolo before weights model file which which we then convert into tensorflow or tensorflow light or tensor RT click the link it'll download I already have it downloaded so we're gonna go to my downloads see you'll be four dot weights and then you should copy it ctrl C and then you're gonna go back into the code of the repository tensorflow Yolo v4 TF light wherever you cloned it and you're gonna paste the weights file into this data folder so go to the data folder and control V and paste it in here so that's what you have to do in order to set up the weights properly and then if you want there's a way to use custom train you'll before weights and tensorflow but I'm gonna save that for a future video but it is here for reference if you want to jump ahead but check back later for how to do that in a in a future video so just using uo v for implementing it in just normal tensorflow dot PB model file is the first step so you're gonna want to go ahead and run this command for saving the model and converting it from the old V for weights into your dot PB model file so you're gonna go back to your shell and run this and just paste it in and it's going to go ahead and run so it's Python save model PI - - weights the weights flag and it points to your weight so if you did save your weight somewhere else you can still run this by pointing to the proper path to your weights and then it outputs this is what you're going to call the model so it's gonna be called you oldie for a - for sixteen because we've said the input size - for sixteen for the model because that's what I trained my model to generically be on or that's what the the tree train goal of before is trained on for 16 by for 16 image size and then - - model flag points - you'll be four so you actually can run all of these commands with Yolo version three weights if you just did - - model you'll be three so this still works for you little version three if you're still using that version so is gonna go ahead and do everything pretty much convert all of the darknet into tensorflow so it's pretty much doing a one-to-one conversion and then it's gonna at the end save it as the tensor flow model file which is pretty sweet just in one command it does all of that and once again you can go to the code and check out what the save model dot pi file is actually doing if you're curious so if the command went as planned it should look something like this you should see the parameters channel params and non-tribal params and if you go into the actual code and you look in the check points folder you should see this saved model dot PB and it's going to be called what we named it you'll will be 4 - 4 16 so it's correct it looks like it has correctly converted from the OL OB for weights into the tensor flow model file so we're gonna go back and you can actually do all of these commands with Yolo before tiny if you're running a you'll be for tiny model and you can download the weights up here as well so I'll just want to point that out before we move on so now we're gonna actually go ahead and run our detector so you're gonna go and grab the command for run you'll before tensorflow mortal just copy it and head back to the command line and paste it in there so detect out there's two different ways you can run you can run detect up PI in the code which runs on images you can also run on video using the detect underscore video PI files so it works for both and detect video also works for webcam which I'll show you but let's go back and actually check out the flags for this this command detect up by so it's Python detect up by the weights pump to the now you point to your tensorflow weights so just the folder it'll be full for 16 size for 16 model and then this is the new command - - images which you can actually do a string comma separated string and have it run multiple which is all at once this image that is pointing to in the images folder let's go check it out in the code so we go to data images and there's two images that you can actually come with the repository that you can try the model on so dog died jpg and type dot jpg and we just did it on kite so it looks like we have successfully ran our model or you'll be for implemented in tensorflow as we can see detection and confidence confidence values on screen so it's properly detected kites a bunch of kites on the beach image and as well all these people within the image so you can see that it looks like it's working you can try it out on any type of image you want just go ahead and upload your own image into that images folder and run it however you want and so one thing to point out after you run the model it will automatically save the detection of you for however many images you run it on in this detection folder that's in the root folder of the repository so just go to detection and detection is one if there was multiple images at once it'd be detections too so show you guys that you can run it on several at once so to run on multiple images in one command like I said it's a comma separated list so you should add quotations around the images flag at the end of the command add a comma and then the path to the second image so that there's also this dog at jpg within the repository so we're gonna go ahead and run it on both of these images and then we will see that they both get saved to the detection folder so it's pretty neat breanne ft and tensorflow is extremely powerful which is a huge perk and as you'll see in a couple minutes we will be able to actually convert into a tensor flow light model and be able to actually hopefully run the model on mobile or edge devices here's the first image there's the second image and if we run back into the code and go to the touch folder we know of two images the one with dog bicycle and truck and then people and the kite so so next up let's run it on video so we're gonna run our car you'll be for implemented in tensorflow on videos we're gonna copy this command right here that says run yellow before on video and it points to that like I said that second script detect video pie so we're gonna go ahead and just paste it in and I'll look over the parameters here with ya so let's expand this and if we go ahead and scroll back up it's almost the exact same command so that's the weights the size and model and now the flag is - - video and you point to the video this video mp4 is in the repository already so this should work for you if you run it and then there's this output flag if you want to save the results of the video you run this output flag if you don't set it then it just won't save the video with the detection is overlaid on top it's perfectly fine so we're gonna want to save it so we're going - output and then you just point to where you want to save the file so in the detection fold it up all the images were saved in and we call it results dot avi so we can see it running here and then you should see if we scroll down in the console window and your your command prompt it's actually going to be outputting the FPS that it is getting on the video so you can see that I'm since I'm running it tend to flow GPU I'm getting fairly decent FPS rates I don't have the strongest of GPUs but I have a moderate one so I'm getting around 15 frames per second whereas live video is usually around 30 frames so it's not bad for a moderate GPU if you're running this on CPU you'll probably get around 4 to 8 frames per second depending on the strength of your CPU so yeah GPU will at least double that framerate so once it's finished like I said if we go into the detections folder we should now see that we have this results dot avi video that will play back in full speed so we can see that it's doing a pretty amazing job of implementing Yolo or tensorflow on this video as we can see it's getting a lot of detections handbag car bicycle and people walking on a busy street so once again it just shows the strength of Yolo before and pretty cool that we can implement it on tensorflow so let's head back now and we're gonna try it on webcam so due to my screen recording having access to my webcam I wasn't actually able to run the command recorded for you guys but I have ran it here and show you that it works correctly so you have the exact same command setting video to zero and it goes ahead and run successfully with a frame rate around the same as the other video around 15 frames per second and if I head here into my detection scolder you can now see that it saved the new video I'll have my webcam and it's seeing my person's detected detects me and it detects my phone so you can see that it does indeed work on webcam if you run that command so I suggest trying it out so that's pretty much it for the tensor flow straight tensor flow TV save models but now let's get on to the tensor flow light version so these are just some demos here and so we can run tensorflow light and what this allows us to do is convert Yolo before into TF light dot TF light file intensive full life if you don't know is much smaller model size than regular tensorflow which allows it to be perfect for integrating with mobile devices or IOT edge devices such as like a Raspberry Pi or Jetson nano so if you're interested in running either an Android application or on a Raspberry Pi this is going to be the route you're gonna want to go so yeah let's get right into it so you're gonna run out this model the save model about pie script right here so we're gonna go ahead and copy that throw it into our command shell and up and then just paste it then and this is actually gonna go so the difference between this one and the save model dot pie for the regular pencil flow is that at the end there's a extra flag - - framework TF light which allows you to it to know that it should be converting to getting it ready for a tensorflow light model and then there's gonna be one extra step here that we're gonna run after this one successfully completes which is convert TF light so this actually takes the model created and converts it down into the smaller size so this saves the model is almost a regular tensor flow and then this is the one that actually converts it into that smaller TF light so once the same model command is finished running you're going to go ahead and paste and be convert TF light and that's just going to go ahead now and as I said convert it into the smaller model size and there's - there's actually multiple different versions of TF light that you can use so there's actually three separate commands here converting to a you'll normal TF light file - the quantize float16 and to the quantize int eight all of these have different strengths and different abilities to be either faster or smaller size so I recommend if you're new just using the regular TF light but if you're an expert in tensorflow and TF light and specific use case you have in mind and I would definitely recommend checking out these two different other types and figuring out which one is best for you or which one will work for you so there are different models that you can run it on so then once it's done converting we're just going to go ahead and run the exact same TF light detect up high script and then we're gonna do it on the kite image again and then it once again all you add is that framework TF light flag to let it know that it is indeed running the TF light version within the code because if you can actually go in and there are some sections of the code that say F framework is TF light save the model as this and run the model in a different way so you have to make sure you have that plug set if you're running it on Kiev light so we're now just going to go ahead and paste in the detect command and let it run so it's gonna be running yellow before and punch it in tensorflow like once again which is perfect for mobile or edge devices so you can see that it ran perfectly exact same as the regular tensorflow but it's a little bit faster and more lightweight so and if we actually head back in this repository that the original owner Hung's LLC 0:07 he actually has some android code in here so if you're looking to run this on android i recommend playing around with that and i will be having a video in the future where I run it in an android application but if you want to jump ahead and get a head start there is code in here so we've now successfully ran our TF light model and you can see that we have the good resulting image just like the previous one so last but not least the strongest of the bunch is actually tensorflow tensor R T so what tensor R T is it's a high-performance inference optimizer meant for these deep learning and AI applications so they're created specifically for AI and deep learning so that they are optimized for doing such commands so tensor RT can all out up to eight times higher performance in the regular tensor flow whereas the TF light is around between four to three to six times three four six times better than the tensor flow regular version but tensor T is up to eight times higher performance which is crazy you actually do need a high end GPU like the Tesla or stuff like a higher version like that to actually run tensor of T so if my GPU on my laptop here isn't actually strong enough to run tensor RT but if you do have a really strong GPU and you've been looking to implement Yolo using tensor RT I would recommend just following these commands as you see fit so one more thing I want to show you guys before we finish off this video is all these command-line arguments so if you're struggling with the flags for all of the commands the save model detect as well as the detect video I've listed them all here so there are a few I actually didn't really show which is iou which you can change the iou threshold or how close things have to be different detection to order to signify them as one single detection so it might detect a person twice very close but they overlap so merges them to the proper one detection as well as this flag for score which allows you to change the confidence threshold that you want to display back on the image so it's set right now to 25% so the model has to be at least 25 percent confident in its detection that it's a person in order to display it but if you just do - - score zero point five zero you get up into fifty sixty seventy so depending on your preference or what you want the model to expect you can change that as you see fit so that's really it for this video I hope you guys enjoyed implementing you'll be four on in tensorflow tents a floor light and ten celerity so you enjoy the video please smash the like button and subscribe to the channel to make sure you catch up on when I released the Android video as well as using the custom model in tensorflow so see you next time [Music] [Applause] [Music]

Channel: The AI Guy

Views: 50,190

Keywords: yolov4, object detection, yolov4 tutorial, tensorflow, tensorflow 2.0, yolo v4, custom yolov4, yolov3, yolov5, custom object detection, tflite, tensorflow lite, tensorflow tensorrt, tensorrt, yolov4 tensorflow, yolov4 tflite, yolov4 tensorrt, tensorflow tutorial, darknet, object detector, object detection classifier, yolov3 tutorial, yolov4 object detection, yolov4 on video, yolov4 webcam, yolov4 on images

Id: iPwepy-SVCQ

Length: 23min 16sec (1396 seconds)

Published: Mon Jul 13 2020

Reddit Comments

[u/thisisabujee]

The video is very helpful. Thanks. I am trying to get TensorRT in working but I am stuck. Will TensorRT work in colab?.

Note: As colab has 18.04 and cuda 10.1, So I downloaded TensorRT5 for these. Plus used tensorflow 1.12, 1.13.1, 1.14, but not working.