Segment Anything Model in Python| SAM | A to Z | Segment Anything Model (SAM)

hello everyone today I'll show you second anything model details which has been recently announced by the Facebook parents company meta and basically segment anything model can detect any object inside the image and it also can generate segmented mass for each of the uh segmented part of the image and today I'll show you how you can extract a specific part of the image from the original image using this segmentation mask generated by the same model and before going to implementation I would like to show you some basic information about the segment anything model so here first question is what is basically same is recently announced by the physics parents company meta on this April and Sam has the ability to identify and the segment different objects within an image that is has not encoded in its training same is the zero shot object segmentation deep learning algorithm and same is prepared by Isa 1B data set and here you can see the link for the essay 1B data set and basically it's consists of 11 million high resolution and private protecting images and 1.1 million high quality segmentation marks and here you can also get some information about the essay 1B dataset here is the number of images and the number of marxes and marks per images is 100 and images and here you can see the image resolution and the second question is how the same can helps developer and the researchers so it significantly reduce the time and the resources required for the data annotation which is often a major bottling for developing a competition model and the resources and developers can build more effective models that can add up to new and unseen object without retaining or additional level data so here if I want to develop a new computerization model that can detect this type of unseen object inside the image so same can help us by annotating the image and by labeling we can develop a new computer vision model that can detect this type of specific object that has not been seen in its training so definitely it will reduce the time and resources required for that annotation and data labeling so here is the basic idea how segment anything model can generate marks for each of the segmented part of the image with the information like the segmentation boundary boxes area predicted IO stability score ProBox and the 0.4 units so if we send this image to the same model and same model will generate a binary marks for each of the significant part with the following information if I show you another image it will be more clear for you here you can see an original image and we manually can identify all segmented part of the image here you can see the segment in part one it is the second part two and it is three and here you can see the small segmentation part maybe this is the fourth and if we send this original image to the same model then same model definitely generate code segmented binary marks for each of the segmented part so here you can see for the segmentation part one this is the binary marks and for segmentation part two this is the binary marks and support segmentation part three this is the binary marks and for segmentation part 4 this is the binary marks generated by the same model with the information like segmentation segmentation box area created IO stability score Chrome box and point coordinates for all of these four marks same model will generate add this information basically same model will return this result for each of the segmented part of the image here I would like to share you uh another things same model basically generate marks using three ways one is automatic marks generator uh using the automatic marks generator if we send any image then same model will generate a mask for each of the segmented part of the image and another way is if we send any point coordinates and the original image to the same model then same model will generate mask considering this point coordinates and it will return definitely this segmentation marks for this point coordinates and another way semolina marks if we send any bounding box information to the same model and the original image then same model will be generate marks for this bounding boxes considering this bounding boxes information and return the following result so if we send this boundary box information and this image then same model will generate only single marks like this before going to implement segment anything model input app we just need to change the runtime from here change current runtime and here you can set the GPU and say and then we just need to execute this first command Nvidia SMI here you can see the Tesla T4 GPU is assigned for me and first here we just get the current working directory using this command OS dot Gates current working directory so this is my current working directory here and we just need to install segment everything model in the current auditing directory by using this link I'll give you all of the links in my video description box hope it will help you and inside the home directory we will create a words directory for keeping all of the signal anything anything model words and we know that there are three types of segment anything model one is bit is for heavy model and PTL for large model and PT for the beta model bit AC is the more accurate but it is very slow but bit B is lightweight and it is feeling faster so basically I will download all of these three model words per segment anything model inside the words directory so we just need to execute this and it will download from this link and keep insert the words directory here you can see the words directory is created if you click onwards directory we can see here three model words is downloaded inside the words directory one is beat is for heavy model with L for large model and bit B is for beta model now we need to move in the home directory and need to install some important packages like Dots Dots Vision supervision and then we need to install dots vision then supervision basically supervision is for the visualization and The annotation and here you can install opencv python opencb is python used for reading the image from the directory and first we need to set the device so this is the command and we just need to execute and will print the device definitely it will return the code 0 because I have already said the runtime to GPU um so the output is good at zero and here we just need to assign the use model type and the model checkpoint path so first I will use uh bit B which is a beta model and it is lighted and the faster so we just need to change the checkpoint path for using this beta model we just need to click here and copy the path and we just need to paste it here now you can check it yes it exists now we need to import same order registry automatic same automatic margin order and send predictor same automated Mouse generator is used to generate mass for ease of the signal part of the image and same predictor basically two way cancel that mask one is if we send any point coordinates and the original image then then same model will generate the mass considering this point coordinates and another way is if we send any boundary boxes information and the original image then same predictor will stand must consider in this bounding box information so basically industry osm model can generate marks so we just need to import all of this and here we can initialize the same model so it's working there's no issue now you need to read image using Siri 2. for reading the image we just need to upload image in this current working directory to upload any image in this current workability we just need to click here will just upload these two images so basically this line will read the image and we will need to convert this VCR image to RGB and we'll keep a copy of BGR image to the original image and resize all of these three images we can check the shape of original image here you can see this test image this one you can see the image here so basically the test image and we will extract this card from this original image by using this segment anything model this origin image size was very high that's why I have resized all of this image to 1024 now we need to import the supervision to plot this image here we just plot this image here we will generate marks using same automatic marks generator for doing this we just need to assign a variable Maps generator and inside same automatic marks generator which is send this model this model same we just need to send it here and we need to send this RGB image to this Max generator then same model will return marks for all of the segmented part of this image so here we will print the result Keys first definitely here you will get the segmentations bounding boxes Chromebox Point coordinates all of the information uh will get inside the result so here you can see the segmentations area bounding boxes predicted I view point coordinates stabilities for under Pro box this information for all of the segmented part of the image we can see number of marks generated by the same automatic marks generator by printing the main length of the result so in result here you can see 73 marks generated with this information from this original image so just now we need to keep uh all of this information like segmentations area bounding box and the point coordinates this four information will keep inside these four variables that's why first I initialize this with blank and inside the loop we will just append all of this result inside this for list for all 73 segmentations inside the list we restore the bounding Box Point coordinates and segmentations and the areas so you can just run it here you can see the 73 bounding boxes information because I printed the bounding box here and now we need to short the segmentations bounding boxes and the importance of considering the areas so here we just short all of this information considering the area here you can see the area is used to short all of this information like second distance and the point coordinates once in the area so inside the segmentation 0 we'll get the largest marks and we can use supervision to plot all of the marks using supervision we can plot all of the masses generated by the same model so here you can see all of the marks so the largest marks in the top the car is the largest marks generated by the same model we can also plot individual marks using here segmentation zero so segmentation 0 is the largest marks denoted by the same model so if we plot this single marks here you can see this is the largest mass generated by the same model and also we can changing the list position we can generate the second largest marks so here you can see the second largest marks is here basically by changing the position of this list we can plot individual marks so here you can see the marks for this for the door of the building so basically in this way we can draw each of the marks from the list you see the mass is the binary image written by the same model if the image size is 1024 by 1024 then generated segmentation marks by same model is also 1024 by 1024. uh this is the segmentation marks for the door so here you can see the segmentation mass is the binary image so same model will generate 73 binary image for this original image and which is basically the segmentation marks and also it will return some other information like the bounding boxes Point coordinates stability score Pro box lots of information also uh same model will generate for this segmentation Marks here we will draw using automatic marks in order for all of the signaled part of the so we just need to execute this your here you can see the original image and the Android image for the maxing quality is not so good here so if we change this beta model to a b model then definitely the masking quality will be improved so for changing the model weight we just need to go here and here you can just replace the B by peace it is and we just need to copy this model path and paste it here and we just need to run this uh we just need to read the image again and if we generate the marks using this bit is heavy model then we'll get more Marks here you can see 111 marks previously we have got 73 marks but here 111 marks for this uh original image so definitely the masking quality will be improved by this beat is or heavy model and we can just execute this and here you just need to short all of this segmented information and if we draw all of the segmented Mass here all of the 111 boxes here and if we draw the largest mass generated by the heat Ace model heavy model they can we can see here the building building is the largest box generated by same heavy model so if we change it here this one uh so second largest uh segmentation part is the car of this image so in this way basically we can upload all of the marks individually by changing this position list position and if we draw all of the mass over the images annotate it by the heavy model then definitely will get the improved Marks here you can see all of the signal part is dated by the we can remove the bounding boxes information by removing this line so here you can see the good quality marks is here heavy model accuracy is very good if we read another image like this color image then what will happen just I'll show you so you just need to change here uh just copy this part and paste it here and also we can change the model to beta model copy path and paste it here so here we read the color image uh here we just draw this new image here you can see the fourth segmentation part of this image you can manually identify this is the first part second third and this is the fourth same automatic marks generator then definitely the length of the result will be four yes that means number of marks denoted by this same automatic Mark is a letter is 4 that we can manually identify easily the four segmentation part of this color image so if it's short all of this information like bounding boxes segmentation Point coordinates and the areas so before shorting users need to keep all of this information inside this list and here we just need to sort all of this information based on the area and then if we'll draw all of the segmented Marks here you can see the four segmentation marks if we draw individual Marks here like the largest one so here you can just need to put 0 and here you can see the largest marks so this is the largest mass of this image we can also manually identify this segmentation part should be the largest one based on this area basically area is here in terms of the P exam so if we draw the second largest marks oh basically this should be one or the second largest mass so this is the second largest marks so now if we put it here 3 then we'll definitely will get the smallest marks uh because total number of marks is uh four so here you can get the segmentation marks for this small portion I think this is clear all the things about the segmentation of mass divided by the same model we would like to show you mark this generation by the same predictor basically same predictor will take input 0.4 units and the bounding box simulation first I'll show you how this sample you to cancel generate marks by using the point coordinates so I have already restore all of the point coordinates inside the points variable here you can see the points segmentations bounding boxes and the areas so here is the four all of these four bounding boxes information here is the four point coordinates of this respected segmentation so and here inside the segmentations and and inside the segmentations all marks is stored so uh I will send a point coordinates inside the same predictor and then definitely sample it will generate the mask for considering this point coordinates here I just standpoint at 0 that means uh the coordinates of the largest segmentation here you can see the marks by points inside the same Predator we just send this point coordinates and the points level and multimax output is false then what will happen for the for this original image one marks is generated by this point coordinates uh we have already know that point zero is the point coordinates for the largest marks so if we draw this largest marks using supervision so here you can see the marks is generated by this largest portion and here is the bounding box information if we change the point coordinates here 3 then the smallest marks will be generated by this same predictor if we draw this here you can see for this smallest portion of the original image uh sample return generated marks and what happen if we send any arbitrary Point like [Music] 500 then what will happen oh we just need to send it inside the list you just need to send in this way that's where getting error so one mass is United for this arbitrary Point coordinates if we draw this here you can see the mask is generated for the segmentation part one uh maybe this original image size is 1024 by 1024 say so 500 500 maybe in the point coordinates is over here that's why mass is generated for this portion so if we send uh like 1000 then maybe uh General marks for this portion for this question if we send it here one thousand one thousand as the image size is 1024 by 1024 so here you can send 1000 1000 then what will happen so here you can see the marks plotted over the largest portion because the original image size is 1024 by 1024 so definitely 1000 by 1000 point will be here so this point uh inside this largest segmentation part and here you can see the mass is generated over this portion now I'll show you generating marks using the bounding box information by same predictor so here we can use the bounding box information for the largest segmentation so box is 0 is the only box interaction for the largest segmentation so here we just send the boundary box information and the point coordinates is none and the generator Max is one if we draw this Mass then here you can see the mass called the largest bounding boxes if I read the another image test image and we'll draw the mass using the point coordinates and the bounding box information for this for this image for the this image I will use same predictor and using the point coordinates and the boundary box information I will drop Mass over this image so we just need to again execute all of this ports here you can see the general mass is 73 by the data model and here you just need to short all of this information based on the area and we don't need to plot all of this say we don't need to plot all of these marks I have already showed you uh so here we can just save the largest marks here you can see the car is the largest marks by Beta model and now I'll show you how you can generate marks using sample predictor for this test image using Point coordinates and the bounding box information so so first I will show you uh using the point coordinates so if we generate the largest mass we just need to put it here zero and run it then definitely the largest mass for this image will get by the same predictor so here you can see the number marks is one and if you plot this Mass here you can see the car is the largest marks for using this beta model for this original image so here you can see the animated image and here we use the point coordinates so now if we use the boundary box information then what will happen so here if we use the Box 0 boxes 0 is the largest bounding box information so if we execute this and just send the bounding box information and the point coordinates is none here so if you plot this here you can see image using annotated image for this Mass now I will show you how you can excited the specific part of the image from this original image and how I can draw it over the white background uh for this we just need to execute this code so here is the basic things of supervision and if we put it here the segmentation 0 7 and 0 is the largest segmentation part and we need to check uh first we need to check and then I will explain all of the things over there so here you can see the original image and this is the annotated image and here the segmentation marks and here is the submitted image basically I extract this second part of the image and draw it over the white background so basically this line is we know that the segmentation is the 2D binary image so we just need to convert this binary image grayscale image and then we need to convert this grayscale image to uh busier image so after that we need to perform a bitos operation with this segmented image and the image becier that means the original image if you plot this if you plot this segmented image one if we remove this line if we remove this line and draw the similar to one image then what will happen [Music] here you can see the uh car is here because we have performed a bitos operation between these image and this image original image and this original image and the similar mask if we perform this video as operation between end operation uh here you can see the video is and operation and this will return basically this image in the last line we just convert all of this black pixel uh using the white pixel so easily we can extract this image for the second part of the image and here you can see the black image is replaced by the white image so this is the basic idea how we can extract uh any segmentation part of the image

Channel: SILICON VISION

Views: 1,880

Keywords: sam github, sam roboflow, sam model, sam model colab, segment anything model, segment anything model github, segment anything colab, segment anything github, segment anything paper, segment anything demo, segment anything model paper, computer vision, opencv python, deep learning, SAM expalnation, extract a part of image using SAM, python, segment anything model from roboflow, segment anything model roboflow, image segmentation, object detection, artificial intelligence

Id: MgWmijAgWHM

Length: 30min 37sec (1837 seconds)

Published: Tue Jul 25 2023