Segment Anything Model (SAM): Build Custom Image Segmentation Model Using YOLOv8 and SAM

hi in today's tutorial we are going to use image segmentation along with object detection so this is our original image we will detect the dog inside the image and then we will use segmentation to grab and extract the the object out of this image sorry for my voice today a little bit sick but we will finish this uh video this is a link for the installation process I've done it in this specific video and this is the file that has all the instructions for installing the cond environment and all the relevant python libraries in order to achieve this process so once again we will do object detection and put a boundy box around our our object and then we will use segmentation process based on segment anything to extract the object out of this image this this segmentation video is part of a a biggest segmentation tutorials playlist H this is the link for the full playlist so uh as always I will leave a link in the video description the link will be uh for the code and for the images as well so you can try it on your local uh machine so let's start coding first of all we will import some python libraries like NPI pytorch and mat mat plot lib we will also um use open CV this would be our original image let's copy the path for this image so this is the the path for our original image so all the playground will be based on this image let's load this image using open CV and let's show the image this is a basic stuff of open CV library and weit key we are waiting for a keystroke for closing the image great so this is our image let's continue now we will use the YOLO version 8 in order to run an object detection so we will need to add to our installation process a very simple pip install P install ultral litics this simple command will install all the relevant python libraries that are required to run the YOLO V8 models very simple command and let's continue now we will import the YOLO object out of the ultral litics and now we will create an instance of uh the YOLO we will use the YOLO Nano which named YOLO 8N this is a very simple and short um model out of the YOLO V8 models are medium small large extra so we will use the Nano which is the smallest one so we are sending the the image to our model and let's see the outcome of the process the the rain value is stored in in object variable and this object variable contains many information it contains all the possibility all the the 8 80 categories that can be detected and also it contains all the detected bounding box so we will run now through the object variable and each obj will contain a detected object so we will We would like to to extract out of this object variable all the coordinates the XY boxes out of the detected H detected variable so uh let's H uh run the the loop through all the detected objects and first let's print all the names that were detected we're expecting to see only one as you can see dog and the model found one object inside our image which is a dog a dog basically is class uh number 16 you can see it on the on the log so we would like to check whether we if if we found a class 16 then we will run our object detection and segmentation process so now we are running the same process but we are asking only for class 16 so now it will detect only dogs and once again we are running through all the object detected objects and all the boxes inside the detected uh object and basically we would like to extract the coordinate of of each of those boxes okay so now when we got our boxes we need to get the classes inside of each of those boxes so this CLS varable get all the classes inside of inside of each of those boxes and we get the class name using the names list and we are sending the the index of the class and we will grab back the class name and now we are checking if we if our index if our class number is a dog we will get the coordinates the coordinates is stored in xyx Yore coordinates and now we're extracting each of the each of the points X1 X1 y1 will be um the left upper corner and X2 Y2 would be the right lower corner so after we got the box coordinates we would like to print them next in order to to use those coordinates and create a bounding box inside our outside our object we need to convert those um points those coordinates from float to integer next we will draw a rectangle rectangle using open CV so this function get our image the integer points which are um pointing our rectangle the color and how thick is the the rectangle and also let let's add text which is the name of our object in our case it would be a dog and these are the position and the font for the text and the color and once again how thick is the text now we're ready to start the segmentation process so in order to do that we will import the same the registry and the same predictor out of segment anything so let's convert our image from BGR to RGB open CV is based on BGR and the segmentation model is based on RGB so we have to compare the channels the color of the channels and now we will load the um the model in order to do that we have to download at Advance the samore vitore H model this is the link for loading and this model I also explain it on the installation process once again I will leave this code as a link in the video description so you can download it while you're watching this video next is the device you should use you Cuda if you have a GPU card and CPU inside the device if you don't have this type of card the model type will be Vore H and now we are creating an instance of the registry object Named Sam we are sending the model type and the checkpoints path now we are creating a predictor based on the same predictor and now we are sending our image our algb image to the same predictor now we have have to send more information to the segment NFA model and is expecting to coordinates where to look for our segmented object so we have the coordinate of the bounding box that we grab using the object detection and we are sending it to our model to the Sam model so this is the input box we are adding Square bra brackets to the to the integer value to to the coordinates this is a request for the for the segment anything model so now we are ready for the prediction we will use the predict function this function can be activated using several arguments in our last tutorial we used the point underscore coordinate now we will use the Box argum argument and we are sending to this function the input underscore box values so these are H these values are the bounding box uh that we grab using the object detection and now basically we would like to display our image along with the object detection and the segmentation so in order to do that we have to copy a three functions out of the let's call it the the demo code of a segment anything so I copy it h to uh to my code once again the code the link for the code will be in the video description and we will use this function in order to display the masks and display the boxes around our object so this function will show the mask and this function will show the boxes around our object and now we are ready to show we will save it on the same path let's save it on on the same path with a different name we will call it out out jpg and we will show the result once again let's run it so this is our original image Let's Wait a few more seconds now he is's doing object detection and also the segmentation process great this is the results you can see the bounding box and the T dog around the the dog and also it's segmented we also grab the exact position of the dog thank you very much I hope you enjoyed this tutorial bye-bye

Channel: Eran Feit

Views: 278

Keywords: sam, segment anything, meta ai, image segmentation, artificial intelligence, ai, YoloV8, computer vision tutorial, computer vision projects, segment anything model, yolov8 tutorial, yolov8, custom image segmentation, image segmentation and object detection, object detection, deep learning, computer vision, Yolo, segment anything for masks, automated mask generation, segment anything with one click, Segment your images without training

Id: 8cir9HkenEY

Length: 12min 21sec (741 seconds)

Published: Thu May 02 2024