Potholes Detection and Segmentation using YOLOv8 (Images & Videos)| Custom Dataset | Complete Guide

hi guys in this video tutorial I will be implementing potholes Direction and segmentation uh using a yellow V8 so in this project we will create our own data set we will uh so to create our own data set I will be using roboflow to so first of all I will go to roboflow and this is the first link so I have created my account so if you haven't created your account you just need to sign up but I have created my account so I will sign in and it will direct me to the side here it might take few minutes so then when you create your account it asks you to create a workspace so I have already created my workspaces I will choose this so let me show you few things so if I go to settings and if I go to this so when you create your work space please choose workspace plan as public plan so if you choose the workspace plan as public plan you can import or export any data set from the robot flow but if you do not choose the public plan uh it you will not be able to export the data set from roboclow into the Google collab so please make sure when you create the workspace please choose the plan as the public plan okay so then I will go to the projects and click on create new project so this is basically instant segmentation and what I am ticketing Port was Project name portal project widget new because I have already created a data set for this so I would in this for the video tutorial I am creating a data set from very start so I will just sync project name as Port wall detection project new and I'm detecting potholes so I'm just written potholes and project type as instance segmentation so create a public project so first of all uh as we are creating our own data set so we will take this video and just copy the link of this video and just paste it over here you can ask also add some images or videos from your local system as well by clicking on select files or select folder you can upload a complete folder of images as well so let's see how many images do we get in this from this video when we split it frame by frame so one frame per second so we have 130 images as this video length is also two minutes two minutes nine seconds is the video length and if we take one frame per second so we have 130 frame in total so here we have 130 frame in total plus uh so this data set will not be enough so let's get go to Robo flow units reverse and see if we have some publicly available data set so we can clone some images from there to our own data set as well so in the meanwhile it's uploading over here and let's go to there as well so it will take a few seconds to redirect over here just click on universe and just write or calls instance segment tests station and just wait for few seconds okay let's continue our c96 images that is also fine so let's consider this data set and let's see the images and let's click on select just click on select all over here and just then I will go to oh let's clone 50 selected image click over here and it will ask you to select the workspace okay no I will not don't want to select this workspace just going back okay okay just go to the images 150 [Music] uh portal detection which is done okay so it is recently created this project I will just select it okay finished cloning and I will not import own images I will also import the annotations as well so I'm just 15.50 finished going to go to clone images in the projects now let's see how much data set do I have over here So currently data set has 50 images and let's see if we can clone the remaining images from here as well okay that's good so these are the 50 images we have just checking just let me see okay why can't see all the images just let me check it I have just refreshed it and I have I can see all the 50 images which I have uh cloned from there so now let's go to again over there and just flow on the remaining 46 images as well so basically uh in the first try we have learned these 50 images and as the data set consists of 96 images we will just go over here and clone this remaining 46 images as well okay so clones 46 selected images and just go below and select okay and I will select the this project and finish cloning 60 images so it might take few minutes but it will be done as well so just let's wait for the few minutes so that it gets cloned into our original data set so it's cloning images go to prone to see this it's done successfully so now let's see these images so data set consists of 46 okay that's fine different way okay if I can we have all the robotics 50 so what I will do generate new version okay the data set consists of 96 that's good so we have currently 51 to 60 so okay so that's fine so we have all the images over here as well that's cool annotation the remaining 46 okay so we have all the remaining 46 images over here as well so that's fine so and this is the video from which we have created basically uh our now we'll have uploaded the video now I will annotate all these images one by one okay so just save and continue over here it might take a minute or so so let's wait it completes and then we will annotate every single image in this although we will not annotate which damages which we don't require but we will annotate all the necessary images uh over here as well so it is a bit time taking process so let's wait for it to complete and then we can move ahead as well so it is a bit okay okay it's done that's cool okay nowaday the files are being uploaded over here so it might take few more minutes okay as I don't have any teammates so I will annotate all those images by myself so I will assign these images to myself okay so now I will skip these images and start annotating from here so as we are doing segmentation so I will choose this and let me show you how we'll connect you will just Connect the Dots here as well and in this way you will do The annotation okay just miss the point okay and just click over here so we will assign the class as object save and enter and here I can also see a part hole so just connecting over here okay that's good and here I can also see a path hole over here okay and then connect it over here and here so we are just doing The annotation over here and now next move towards the next image okay there are so many Pathways over here just do this fragmentation basically this is in the form of uh polygon shape you can see over here okay so that's my mistake I pressed a minute and then connect it okay so now for this just do this one two three four and done and for this as well just connect those key points one by one okay now for this as well okay now for this path hole as well so in this way we annotate all of our images okay that's nice okay so now we'll add cover this path hole as well so in this way we'll annotate all of our images now go to the next image and just annotate it as well like this in this way you will annotate it now annotate this image as well it might take few minutes but it can be easily done and okay this is also a pothole so this is not very you can see that tricky process is a quite easy process and you can do it on your side as well and you can do it for any data set of your requirements of your choice okay okay then let's do for this image as well now I'll just pause the video and I will do for all of the rest of the images and then as I complete the animation then I will get back and then we will move towards that training process foreign guys I have completed The annotation process completely so you can see that uh we have previously 96 images now we have a data set of 202 images now let's go to generate section and we don't need to apply any pre-processing just these two pre-processing Auto Orient and resize these are enough and then select augmentation just skip it and then generate so now I will click on generate so it might take few seconds so one thing I tell you we can train our model using roboflow as well it's a quite simple process to train your model on robofer by clicking on start training you can train your model on roboflow but it's better if we go uh if you train our model using Google collab because there is a thing where we can learn some new things yet there is a no basically Robo flow provides a no core solution so it's involved no coding but by just click using a single click you can train your model uh at on your workflow so just exporting the zipping file it might takes few seconds so just wait until we are able to explore these files so here we have we have the code so using this code we will import the data set this data set into our collab file so okay so first of all let's open the codenab file over here just click over here and just open the Google code app so for the implementation of this project I will be using this wrapper okay so although we are not implementing the tracking but um I will exclude the tracking from uh part from there here and we will use this wrapper so you can I will add this repo link into the description of this video you can check this repo and use this code for this project as I am using it now okay so let's open the file which we can use for example for let's so we will use the instructions provided in this file so just go over here create a new notebook okay and just creating I'm creating a new notebook and just write uh holds segmentation pathway segmentation your low foreign segmentation using the other V8 so let's open this uh for the file so we will just following the instructions from there so the GPU before you run the code okay so I'll just go to runtime change runtime as GPU and that cause it will uh speed up my training process then I will import the required libraries that's right import the the required liabilities so importing all the required libraries over here not as click on this cell to run so that this cell can run then we will turn this GitHub repo so just go over here and click on here and now I will just clone this GitHub repo either just write on the get GitHub rep over here okay so it will take few second until we are able to clone this repo so we have this wrap over here now we will set the directory as this repo okay just copy path and set the for run directory okay so percentage CD is used to set the con then we will install the dependencies or all the dependencies over here just write install on the dependency so to install dependencies just right click install minus is dot Dev I can just missed something over here it should be like this and but it's working fine in that case as well but if you can add column it would be very much good so after this as I am implementing segmentation so I will be redirected to this your low V8 and this segmentation folder just and basically in we are installing the dependencies means we are installing all the required libraries here you can see over here we have installed request pigments Ginger to mark down Alternatives all these libraries here so it is good to before installed on independencies so when we are training our model we don't face any Library issue moving towards the segmentation and directly moving to the segmentation directory so okay so this is my segmentation directory this person just CD now just check our present and Direction present working directory so here I am just checking my present working directory okay so now I will uh import the data set import the data set into the into access from Robo flow just skip this okay import the data set from Robo flow into Google folder into the Google order okay so just go over here just copy this part and okay here and just write down here Okay so so I am just downloading my data set and it will be appearing over here it's not currently appearing but it will appear as it downloads so yeah you see loading roboflow projects downloading over here and you can see the data set over here as well okay that's cool so we have the train test validation folder data dot yml folder okay so here is our data dot ymf folder okay that's fine so now uh basically I will just train my order over here just give me a minute now we will train our models to train our model that's right train in the model training the model Okay so just set up for me for this as okay so this will be your home directory and okay so firstly we can set a data set dot location as well what is our data set dot location let's check it out okay our data set is over here that's cool uh now just said percentage CD as on and just write python3.5 if I am missing anything let me check okay so basically this predict dot file might create an error because it's been adjusted for the tracking as well so let me upload the updated predict.5 which I will have in my your login [Music] okay ultralytics you want to be it just go over here and just in the segment just like this bit I will just upload this product.pi file as well so you can choose it as well so python train dash dot Wi-Fi order is equal to YOLO V8 l dot p e and it's next thing which I need to care of is um let us just check I think I might be missing something just go throughout this GitHub repo and just check that training I think training part is not written out here good training basically just copy this code oh yeah just spot over here okay okay so we can just change it to 120 box now let's train our model so we are just starting okay let me check this error I think it should be a very small error we can fix it now so I have just checked the yellow V8 alternative yellow so I there is an update so we need to update this and I think when we run it this now it will hopefully work fine so just we have made this update and now uh running the script again for 120 epochs and nothing more than that so let's wait until the script runs successfully so it started so it's the the size of pre-trained model is uh 88 MB basically we are fine tuning the model over here data set not for uh okay let me check this issue as well so there was a path issue so I was able to fix this issue now we are ready for the training and the training has started now so let's wait for the training to complete and then we can test our model on some sample images and check out how our model performs so just let's read when our training completes and then we are going to test our model so the training is going on and let's wait for it complete uh mean average position is a bit low at the start but I believe as the model trains it will definitely improve now let's check what mean average Precision we are getting okay so it's increasing now so I the training will be continued so let's meet after a training complete and then we will test our model sure so the training has completed and we are able to get a mean average Precision of 0.532 with IOU of 50 and when the io varies from 50 0.550 to 0.95 then we have a mean average Precision of 0.218 it's not very good but it's okay plus now we are testing our model uh so we are downloading a sample video over here with name is demo.mp4 and now we are testing our model here I have already tested the model on a sample video demo.mp4 and these are my best ways and using trading dot Pi I am testing my model on a sample video and here are the results which we get you can get that model works fine it is able to detect the path holes so the model works fine like it is not then we are testing on a demo video too I'm just downloading demo video to and tested my model on it so as the demo video as the this demo video 2 was quite big so we are unable to display it over here but I can show you the results as well I have already downloaded here the results so we can see that these videos are quite long okay so you can see here the model was able to detect the path holes so our model works fine okay so we can say that the model working is quite fine plus we can also uh display the confusion Matrix over here to explain the confusion Matrix uh like this right display [Music] confusion Matrix okay so just to display the confusion let me just write your hair and just by clicking over here and then going to where we have okay runs okay and now here we have over here segment then okay we have the files okay we have the files over here so here we have the confusion Matrix just copy bar and just write image and just add this copy path over here just copy the path from here and just add it over here now let's see what we get whether we have not okay so just we need to run this cell we have missed the cell so just running this cell and now just running this okay so this is our confusion Matrix so configure metric basically tell us how our data is distributed among different classes you can see here our model is able to detect 50 of the potholes while on fifty percent of time it fails to detect the path hole and it gives nothing he did not only just nothing like this is a blank screen we have nothing is attracted Okay so this is the way plus we can also validate the custom model as well I will take the validation of custom model code from here and just now let's validate the custom model as well so yes [Music] some model okay so we have that I think I should add the best DOT weights path over here which is here and just copy this path and just add it over here I think it should now be working fine let's see what the results do we get over here okay okay I think that's I think the fix over here another issue another day okay okay so what I did it a train dot Pi I will do the same in what I did in predict.pi I think I will do the same over here just copy this and face it over here I think it will now work fine just plus but just check it again if there is an issue we will fix it again but I believe it will for time now but let's see how it works so check in the model and see how does it works okay so it's working I think fine okay in the meanwhile it comes that Addiction on let's display the prediction on the validation that as well Okay so we saw on the when you validate the custom model we are able to get the mean average Precision of 0.538 uh with segmentation we are able to get a mean average position of 0.531 so um it's not very good but it's quite fine you can say that oh yeah now we are printing the prediction on the validation batch so just write image here and just right over here validation batch predictions so now let's display the predictions on the validation batch which we have over here so we can see that results are verifying okay so this we have also done Computing metrics we have we have tested our model on multiple demo videos and our model works very fine so this is all from today's lectures see you in the next video till then bye bye

Channel: Muhammad Moin

Views: 17,594

Keywords: yolo, yolov8, objectdetection, objectsegmentation, computervision, researchanddevelopment, machinelearning, objecttracking, potholes, detection, roadsafety

Id: bAlObHUIErM

Length: 28min 17sec (1697 seconds)

Published: Tue Feb 07 2023