Panoptic Segmentation using Detectron2

hi today let's cover both instant segmentation and semantic segmentation this means panoptic segmentation so this would be our original image and this would be a panoptic segmentation based on this image so first of all the installation will be in on my previous video tutorial so this is the link and also this video is part of a full playlist dealing with semantic segmentation tutorials and this is the link for that as well so let's start coding may I remind you that the decron requires that the working folder will be the de decron 2 as main folder so under the under this folder we will create a new python file let's name it simple panoptic segmentation doy and py and let's start so first of all we will import torch and some more relevant the python libraries like the decatron 2 and NPI and open CV and let's load some more relevant functions and objects from decatron 2 we will import the model Zoo the default predictor engine the config the get CFG function from the config library and also the visual visualizer uh object as well okay and next this is another function next let's H continue with the loading our image so this is the path for our original image this is the image the stretch image and let's copy the path for this image okay next since this is a very large image let's resize it to our needs we will uh use open CV in order to resize it let's resize it by 30% so let's grab the width of the image so this is the code for reducing the WID by the 30% scaling and this is the height now since we have the WID and the hi we we have a full Dimension let's call it dim and we will use the resize function from the open CV library in order to resize our image let's call it my new image okay now we are ready let's show the image let's display it and let's run it so this is only a double check that to see that just to see that everything is okay great so this is our original image let's continue so now it's the interesting part let's do the panoptic segmentation so as I mentioned before panoptic segmentation basically it's both instant segmentation and semantic segmentation so let's use the get CFG function this line obtain a config object for the panoptic segmentation then we will merge from a file basically this line merge the config setting from the specific yaml file to the panoptic segmentation so what would be our Yammer file so as you can see this is the model Zoo documentation and this is the coko p optic segmentation and we will choose the third one we don't have to download and any file just give the right direction so this is the right direction to this specific weights for the panoptic uh segmentation so this line merge the config setting for specific specific yaml file into the config object next we have to deal with the weights so once again we will grab it h using a a model Zoo a function named get checkpoint Ur checkpoint URL and this line sets the weights of the panoptic segmentation model to the pre-trained weights obtained from the model Zoo as always I will leave a link for the code so you can download the code and use it instead of typing the code now if if you have a a GPU you don't need this line this a GPU card with the decatron 2 works well on a Linux machine on a Windows machine and you need to use the CPU so in order to do that we have to add this line next we have to define the predictor this line creates a predictor object using the config panoptic segmentation model and the next line use the predictor to obtain the panoptic segmentation mask and the information about the segment how to segment it from the image now we are H dealing with the visual part so in order to do that we have to grab to grab the the image the the result image from the from the predictor so this line This is a Class by the decatron 2 and that it used to visualize predictors on images and we use this number array inside the myu image to slice in the operation in open CV the images are loaded as a BGR blue green red as a default therefore H this line reverse the order of the color challenge effectively converting the image from BGR to RGB format next H the draw panoptic segment prediction this line draws the panoptic segmentation on the image using our visual object next we need to grab the the image the segmented image out of the the out variable and we are convert converted it back to an BGR format BGR color format in order to display it using open CV and let's add the IM show function in order to display both the original image and the predicted image and let's see the result great as you can see on the left side it's the original image and on the right side it's a panoptic segmentation image so now you can see the predicted objects person car trees very nice I hope you enjoyed this tutorial thank you very much bye-bye

Channel: Eran Feit

Views: 175

Keywords: detectron2, panoptic segmentation detectron2, deep learning panoptic segmentation, panoptic segmentation python, real time panoptic segmentation, panoptic segmentation Python, panoptic segmentation real time, panoptic photo segmentation, ai panoptic segmentation, computer vision panoptic segmentation, deep panoptic segmentation, detectron2 panoptic segmentation, instance and semantic segmentation, awesome panoptic segmentation, best panoptic segmentation model

Id: MuzNooUNZSY

Length: 7min 48sec (468 seconds)

Published: Wed Apr 10 2024