Run Meta's SAM Model in Your Browser to Cut any Object from Image

times back I covered this model segment anything from meta and it blew my mind away segmentation identifying which image pixel belong to an object is a pivotal task in computer vision and is used in a broad array of applications from analyzing scientific imagery to editing photos but creating an accurate segmentation model for specific tasks typically requires highly specialized work by various experts with access to AI training infrastructure and large volumes of carefully annotated indain data but if you look at this project with the help of this model by just click you can segment or cut out any object from any image how good is that so what meta has done here they have democratized segmentation by in producing this Sam project or segment anything model project they have also released a new task data set and model for image segmentation they have released both their model and data set which is 1 billion mask data set for the open source and now you can easily download it also but only for research purposes and the segment anything model is available under a permissive open license which is by to which is awesome now Sam has learned a general notion of what objects are and it can generate masks for any object in any image or any video even including objects and image types that it had not encountered during training Sam is General enough to cover a broad set of use cases and can be used out of the box on New Image domains whether underwater photos or cell microscopy without requiring additional training which is often called as zero shot learning or training I more than sure that in future Sam could be used to help power applications in numerous domains that require finding and segmenting any object in any image so Sam could really become a component in larger a systems for more General multimodal understanding of the world for example you can scrap the web pages with it you can do lot of things I think especially in augmented reality and virtual reality Sam could enable selecting an object based on a user's gaze and then lifting it into 3D just imagine and then there are lot of use cases you can we can think of so there are four main things which Sam can do Sam allows users to segment objects with just a click or by interactively clicking points to include and exclude from object Sam can output multiple valid masks when faced with ambiguity about object being segmented and Sam can automatically find and mask all object in an image Sam can generate a segmentation mask for any prompt in real time now let's look at another new thing where Sam can now run in your browser with eight times faster image en coding how good is that and this is where you can run it on hugging face spaces and I will drop the link to it in video's description you can upload your own image or you can try their examp exle so let me try to upload one of my image maybe I will just quickly go and find out one of the image from my own system so I have just uploaded this image where this is just a backdrop of Australian Outback where there's a tree there are few Birds three Kangaroos and a sun now it is extracting image embeding so let's wait for it to finish extracting and it might uh induce some delay okay so edings are extracted let's wait for few more seconds okay it seems that image is ready to be processed by us so let's let me first hover my mouse over maybe sun you see so it has very very correctly you know identified that this is a different object okay so when you uh left click it's a positive point and when you right click it's a negative point there you go then you can simply put your points in and then you can cut the mask and then uh you can't see it but it has produced another image on my browser which I can save and show you so let me open my another browser window so this is what it has done with that Sun so you can easily segment anything very very quickly now let me over here you see it has just selected these or we can simply select everything I just dragged my mouse over it and you can simply clear the points I just cleared the points here I can clear this again good now maybe I just want to select this one bird wow you see that it has selected one bird amazing stuff and let's select This Cloud how is it it is and then you can just cut this mask let me cut it and then I will show you in my browser it is asking me to save it again so let me save it and show it to you there you go so this is the cloud which we have just cut from here let me now clear the points maybe I'll just go with this kangaroo cut the mask save it here and then let me quickly show you the kangaroo too there you go so now we have segmented the kangaroo out of that image in just one click how good is that and then you can reset this image we can upload another image let me quickly see if it can recognize that text let me upload one of my own image which we I have created for maybe my another YouTube video so let me quickly load it just give me a sec I'm just searching for it there you go so let me try to maybe uh segment the face so let's wait for it to extract the image edings the segment score is this okay that is fine let's wait for it so it is done I'm just going to extract her from the image maybe can I do the here no it's a full image because it's one image that is fine that's done let me cut the mask then let me save it and I'm going to show you that in another browser window and there you go so you have easily extracted that segmentation from this image how good is that now let's see if we can just go with one character yes I just yes we can so you just have to keep it for for little bit longer on that character and it just selects it so I just want to see if I can just go with one eye yes I can okay this is what I was wondering okay so let me see I I'll just clear the I'm just going to go with here no it just considers it one so can I go with maybe finger no it's one it let me select the I there what about teeth yes we can just select the teeth and then can I select this finger or not no because it just takes it whole anyway you have to play around with it so I think amazing model and if you go through this uh blog post which I will drop the link in video's description then you can you'll be mind blown that's it guys I hope that you enjoyed it if you like the content please consider subscribing to the channel and if you're already subscribed then please share it among your network as it helps a lot thanks for watching

Channel: Fahd Mirza

Views: 456

Keywords:

Id: rMxoliT8iFw

Length: 8min 32sec (512 seconds)

Published: Thu May 09 2024