Segment Anything Model (SAM): a new AI model from Meta AI

hello everyone my name is arohi and welcome to my channel so guys in my today's video I'll talk about the latest AI model which was released Yesterday by meta the parent company of Facebook and the model name is segment anything this segment anything model is capable of identifying and extracting the objects from the image or from the video this image segmentation model this segment anything is a image segmentation model right and using this image segmentation model we can you know solve the image segmentation problems very easily and this image segmentation model is trained on a large data set which have around 1 billion mask on 11 million images so this data set is a very huge data set on which they have trained their this image segmentation model the segment anything model okay and guys meta also said that they have evaluated the model performance on various tasks and find out that the zero shot performance is impressive so zero short performance simply means test the model on new or unseen scenarios without additional training right so basically what I'm saying is this segment anything model can identify objects that were not a part of the training also okay so um we can use this Ms segmentation model for this segment anything model for various tasks like suppose you want to cut out the objects from the images or you want to put the mask on the different objects present in the image right and um the guys the important thing is that the demo is available let's see the Demo First after that I will also show you the GitHub code which is available we will implement this segment anything using this python we will Implement that part also so one by one let's see both the things first I'm showing you the demo which is available uh on their website and then we will follow we will create our python environment and then we will run this model and we will perform the image segmentation okay so let's start all right so let's start from here so this is the website here anybody can you know try this demo just visit this site and after that just scroll down you can know about this model more from here the the thing which I want to show you is this demo part okay so let's click on this demo go to demo so over here you will see different images okay you can pick any image from here also or otherwise you can you know upload the image of your choice as well so I'm just picking the image from here so let's say I'm picking this image so once you pick the image okay you'll see the mask is there now the mask is over here now the mask is here right so here there are different things you can perform like cutouts box everything okay let's try the cutout I've clicked on cut out then I came here and see you see the point over here so this model how this model is working we have to put the point on the object on which we want to work so now let's see another example so choose let's work on this image so I've selected this image and now earlier I've shown you how to work on a cutout so this time we are working on this everything let's see what is there when you click on everything what will happen is it will you know detect it will capture all the objects you can see here so it is finding all the objects in the image automatically and if you click on this cut out all objects so all the objects will be cut out now let's see how it works you can see the list of objects over here C all the objects are here right so this is how you can use this cut out okay and then everything is clear now the box so box simply means we can roughly draw the Box around the object on which we want to work so once you select the box from here see this is what I'm doing I have plotted a box here right so now once I plotted a box then we can cut out that object see here you can see that particular object so in the same way you can uh you know use various things so suppose you don't want to you use the image which is already present in their website so you can upload the custom image from here your so this is the image I chose so now this image is uploading and we can work on this image you can see see every flower is getting segmented right so let's say what we want is we have uploaded our image and whatever the task you want to do what do it on it you can do that let's try the cutout okay so first let's see the mask so we've seen the mask over here and then for cut out just click on this and you will get that so this is how you can work on your custom images also you the image of your choice okay so now let me show you the Practical implementation with python for python this is the GitHub repo okay their official GitHub repo Facebook research and then segment anything so this is the code from here you can get the code so guys as I always say that I always work in a separate environment so that's what I'm doing today I have created my separate environment and then um after creating the separate environment I have install the requirements as mentioned here python 3.80 greater firetors version and torch version version okay install these things so in my case my torch torch vision and my GPU all these details are here okay once you've done that you can install this segment anything using this command or you can write these two lines one by one and install it okay so once you install it so guys the this these things these modules you need um later so I'm installing right now only so just write the this command and you will have opencv by Coco tools matplotlib and onx over here on your machine okay so that's it this is what you need to install to run segment anything model on your PC okay now let me show you my Jupiter notebook where I have executed the code so these are the important modules I've imported so so now these three functions so guys I took all these functions from here only Facebook uh research segment anything from this repo I clicked on notebooks and from here uh I think from here one of these notebook have all these three functions okay so I picked these three functions from their uh official um repo and this function is responsible to show the mask and this function is responsible to show the points remember I've told you that whenever you want to use this segment anything you have to put a point on the object on which you want to work on the object which you want to cut out or the object which you want to put a mask on you have to put a point on it okay so for that we have a show Point um a function and this is the Box we put the Box around the object and we can do that using this function okay and then I am reading the image my image is present in image folder we are reading that image and then this is the image you can see okay truck now on this image we want to perform the MS segmentation task so guys so we have to get the off trained weight file right so the trained weight file is here okay so where is that let's go back so when you'll see here scroll down and here see they have written over here that first you have to download the model checkpoint okay so this model checkpoint it is trained on that large data set and that data set is also available if you want to download and you want to see the data set you can you can do that as well here is that data set when you will visit this page and official paper is also here you can check the paper from here so what I was saying is we need this model checkpoint because it is trained on a large data set so from where you can download it just scroll down and under this model checkpoint you can see we have three kind of different models so in my to this class I'm using this default one just click on it and you will have this model in your PC and once you download it put it in your current working directory okay so in my case my current working directory is this you can see images is the folder in which I have all the images this is the I have created a separate environment so all my environment files in it and this is the file okay this is the checkpoint file which I've just downloaded and this is the Jupiter file which I'm showing you all right so once you download the weight we are writing the path of that we you need to give a proper path of that model checkpoint and we are using a Cuda device and this is the default model type okay and after that here you can see from segment anything model we are importing these two things this sum predictor will perform the predictions okay so this is what we are doing this model type our model type is default checkpoint inside checkpoint what we are giving Sam checkpoint what is in that our model checkpoint file okay and then we are applying a predictor on it okay now our predictor is ready now we want to use this predictor on the image or on the video on which we want to perform the you know segmentation or cut out whatever you want to do we need to provide some input to it so that's what we are doing over here predictor okay this predictor which we have created set image and then give our image so this image is a variable which I have used above here so this image variable this image variable have this image okay so on this image I want to work so that's what we are doing we have given this image to the predictor and then it is here this line and this line so now what these lines are now we need to put a point on the truck on the objects right so this is X and this is y so we always give points like this X and Y this is the point um location and point label is this one means foreground class and 0 means background class so this truck is our four gram class so that's why we have written one over here okay and this is X N this is y this is the location of the point where you want to have the um point and the shape of the point is star in this case okay so we are plotting the same image this image and we are also we also want to show the point on it so we are calling the show Point function which I've shown about so this show using this show Point function we are showing the point on the truck okay plotting the image then showing the point over here input point at X and Y location is this and this is a foreground class this is how you can uh plot a point on the object now next thing is to you know segment the area right so for that what we are doing is this is the code mask score and logits predictor will provide you these three things and this important thing to note over here is multi mask output equals to True when's multi mask output is equal to true that means your model will provide you three different masks okay now inside mask we have all the masks and inside scores we will get the corresponding scores value okay now over here I'm printing the shape of the Mask you can see we have three masks and this is the height and width of it okay so why we are getting three masks because multi mask output is equal to true and when it is true we will get three masks okay now let's print all those three masks okay see we are printing the mask and the corresponding scores okay so let's see first mask is this right second mask which is predicted by our model is this and The Third Mask is this and you can see that you know this third prediction Third Mask we are getting uh our object is getting segmented very accurately right so guys this is how we can use this segment anything model so this is just a first lecture on it and in my upcoming videos we will see what all things are there slowly and we will explore the code in more detail we will see the architecture also we will see uh the paper also which they have mentioned here this paper will talk about this paper also the model architecture what is the layout architecture right all those things we will see in my upcoming video I hope this video is helpful thank you for watching

Channel: Code With Aarohi

Views: 10,601

Keywords: SAM, Segment Anything, Meta AI, Image Segmentation

Id: 9MOUY1JqK04

Length: 15min 0sec (900 seconds)

Published: Thu Apr 06 2023