so on today's tutorial we will
be making an object detection web application we will be detecting
tumors on a brain MRI image now let me show you how it works I'm going to
drag and drop an image from my computer so this is the image we have uploaded and if I
click here on detections you can see that we have detected two objects we have detected two tumors
on this image so this is exactly the project in which we will be working today on today's tutorial
we are going to make the entire web application using Python and streamlit and we're gonna detect
objects using an object detector trained with detectron2 so my name is Felipe welcome to
my channel and now let's get started so let's get started with this tutorial and the first thing we
need to do is to create a new pycharm project you can see that this is pycharm and now let me
show you how to create a new project we need to click here on new project I'm going to select the
directory where I'm going to create this project which in my case is here and then I'm going to
select tutorial this is the directory in which I'm going to create this pycharm project and
then I'm going to create a new environment and The Interpreter will be python 3.8 everything else
will be just the default values so I'm going to click on Create and that's pretty much all now the next
step will be to install all the requirements we are going to use today so I'm going to create
a new file which is called requirements.txt I'm going to name this file requirements.txt
I'm going to press enter and then I'm going to paste all the... all the requirements all the
dependencies we need to install in this project which are all of these packages we have over
here so I'm just going to copy and paste these packages over here and that's pretty much all now
I'm going to the terminal and I'm going to press pip install -r requirements and that's
pretty much all I press enter and that's going to take care of installing all the requirements
and you can see that I got an error and basically this error is because we need to install all
of these dependencies first and then we need to install this final dependency... right you can
see that this one is called detectron2 we need to install everything else first and then at
the end we need to install detectron2 so I'm just going to comment this line and then I'm going
to press pip install -r requirements again okay now all the requirements have been installed
and now the only thing we need to do is to install detectron 2 so I'm going to uncomment this line
and I'm going to press pip install -r requirements again and this is going to take care
of installing all the requirements but as we have already installed all these packages the only one
that's going to be installed now is detectron 2 so we need to wait a few minutes okay and that's
pretty much all in order to install detectron2 and now we are all set all of our requirements
have been installed so it's time to continue let me show you how to create a new file let's create
a new python file so we're going to select file new python file and this file will be main.py
so this is the file in which we are going to be coding the entire web application of today's
tutorial and remember in this tutorial we are going to be detecting tumors on brain MRIs
so we definitely need an object detector in order to detect these type of objects right let me show
you the data I used in order to train this object detector this is a dataset I found in roboflow
and I'm going to give you a link to this dataset in the GitHub repository of today's tutorial so you
can just go ahead and take a look at this dataset if you want to, and this is an object detector
I trained using detectron 2. and I'm not going to going to show you the details of how I trained this
object detector because that's something I have already covered in one of my previous videos in
one of my previous videos I showed you how to train an object detector using detectron2 and
I showed you the step-by-step guide I showed you the entire process so if you are curious to know
how exactly I trained this object detector I invite you take a look at the video I'm going to
be posting over there and now let's continue this is the data I used in order to train this object
detector and now let me show you the entire pipeline in which we are going to be working
today let's get back to pycharm and let me show you exactly what are all the steps we are going
to be making in this tutorial the first step will be setting up the title of the web application
so this is the first step in this process then the next step is setting up the header right
the third step will be creating a file widget upload file so the user can upload an image about
a brain MRI so we can detect all these objects on top of this image then the next step is loading
the model right loading the object detector we are going to be using to detect objects then we
are going to load the image the user has uploaded then we are going to detect objects and then the
last step in this process will be to visualize the objects we have detected on top of the original
image and we are just going to display this visualization to the user right so these are the
steps of the entire process the entire pipeline in which we are going to be working today and
I'm going to show you every single step of this process so you can see these are one two three
four five six seven steps in only 7 steps we will have this web application up and running so let's
get started and the first step in this process is importing streamlit as st okay then in order
to set up the title I'm going to call st dot title and the title will be something like brain
MRI tumor detection then in order to set up the header I'm calling st dot header and this
will be something like please upload an image okay then in order to create the file upload
widget I'm going to call st dot file uploader and I'm going to input two parameters the
first one is an empty string and then it's all the types we support in this widget
and I'm going to say something like png jpg and then jpeg okay and that's pretty
much all and in order to move one step at the time let's see if everything executes
just fine I'm going to execute the code as it is so far so I'm going back to the
terminal and I'm going to type streamlit run main.py this is going to open my browser
and we are going to see exactly how our web application looks so far and everything looks just
perfect so we are okay in order to continue so let's get back to pycharm and let's continue with
the next step in this process which is loading the model loading the object detector we are going
to be using today and remember we are going to be using an object detector which I trained using
detectron 2 and remember I already showed you how to use the detectron2 in one of my previous
tutorials so let's go back to my browser and let's see exactly how we can use this model which
I trained with detectron2 let's go to the GitHub repository of this previous tutorial and let's
see exactly how this... training this model or how using this model was all about so I'm going to
this file over here which is predict.py and this is the file we used in order to load the model in
order to make predictions with a model trained with detectron2 so the only thing I'm going
to do in this tutorial is to copy some of the code in this file and I'm just going to paste it in the
main.py file of our ocyharm project right remember that in this tutorial we are not going into the
details of how to use detectron2 so I strongly recommend you to take a look at my previous video
to take a look at this video over here which you are going to find in my YouTube channel so you
can see exactly how this... using this model how using detectron 2 works right because we are
not going into the details in this tutorial right so this is my strong recommendation for
you please take a look at that previous video the only thing we're going to do now is just copy
and paste some of these lines which I'm going to explain super super quickly right you can see
that we are getting a configuration file then we are getting the weights for this model and
we are getting the weights from this model from our local drive so we are specifying a file path,
a location in our... in our local drive and the only thing we're doing is specifying the weights
location then we are creating an object which is our predictor and this is exactly the model we
need in order to continue with this process so this is a very quick explanation regarding this
code we have over here and now let's continue now you can see that we need to make a few Imports
because we are not finding these objects we have over here these functions we have so I'm going
up all the way up and I'm going to say something like from detectron2 dot config import get config
and that should be... should be all for this function we have over here then from detectron2 dot engine
import default predictor and that should fix this issue over here and now we need to import
from detectron2 import model zoo and that should be all in order to fix this issue
over here I'm going to delete these comments and that's pretty much all so everything that's
here is everything we need in order to load this model but obviously we need a model in order to
load right because this is just the default code we had in our GitHub repository so let me show you
exactly where it's my model in my local drive if I go to my file system you can see that I have this
file over here which is model.pth and I have this other file which is labels.txt this is the model we
need model.pth these are the weights of our model and what I'm going to do is to copy this file and
I'm going to paste it in the in the directory of this pycharm project right you can see that this is
the main.py in which we are currently working in this is the requirements.txt file we created a few
minutes ago and this is exactly where I'm going to paste this model and I'm going to do something
else which is creating a new directory which is called model and this is where I'm going to put
the model and everything it's okay and remember I showed you we have another file which is all the
labels we are detecting but in our case this is a very very dummy labels.txt file because we
only have one category we are only detecting one class which is tumor and a very very very
quick note is that remember the dataset I used in order to trained this object detector in this
dataset we had two classes which were negative and positive and this is something like two different
types of tumors... or that's what I think... but what I decided to do when I was training this object
detector was merging these two labels these two categories into only one object and I called
this only one object I called it tumor right so that's exactly why we have only one class over
here given that... although the original dataset I used had two categories so that was it's a very quick
note regarding the model I trained and now let's go back to my file system we are not going to use
this directory anymore I go to model and this is the model we are going to be using this is the
weights... the model weights we are going to be using so remember this is within another directory which
is called Model I go back to pycharm and the only thing I'm going to say is something like model
and then the name is something like model.pth okay and that's pretty much all in my case I'm
going to run this code in my local computer which is using a CPU so this is what I need to
specify if your computer or the computer where you are running this code has a GPU then the
only thing you need to do is to comment this line and everything will run on your gpu but in
my case I'm going to run it locally on my CPU so I'm just going to leave this line as it is, now
let's continue now it's time to load the image we are going to use in order to detect all these
objects so this is what I'm going to do if file I have to make another edit so we are uploading
a file and we are calling the file the user has uploaded we are calling this object file so now
if file so if the user has uploaded something we are going to continue and we are going to call the
image we're going to call this object we're going to call it image and then image will be Image
dot open file and then something like to RGB right an image is an object we are going to
import from pillow right from PIL import image okay that should be all, okay now in order to
move one step at the time let's go back to my browser and let's see if everything executes just fine
I'm going to refresh and everything is just fine and now I'm going to select an image let's see if
everything it's okay the data I'm going to use it's located over here this is train and val I'm
just going to select a random image which is this one and let's see what happens we have an error
because this is not called to but this is called convert if I'm not mistaken let's see now I'm
going to refresh and I'm going to do the same process again I'm going to select the same image and I'm
going to drop it over here and you can see that now we have another error because it's not covert
but it's convert I had another typo okay now let's see what happens I'm going to refresh again
let's hope everything is okay now I'm going to take the image I'm going to drop it here and let's
see what happens now we have to wait a couple of seconds we may be loading the model so this may
take a few seconds... and everything it's okay we are not visualizing the image so if we are not getting
any error that means everything is okay so let's go back to pycharm everything it's okay so far
and now it's time to detect objects we are moving super super quickly we are almost there right we
have almost completed this process this Pipeline and the only thing we need to do now is to detect
objects and in order to detect objects with this model which was trained with detectron2 I am going
back to my browser and to this repository because let's see exactly how we can make this prediction
the only thing I'm going to do is to copy and paste everything that's from here up to here we
don't really need to draw the rectangle but let's just copy everything so I'm going to copy then I'm
going to pycharm and I'm going to paste it here we will need to make a few edits but most of the
code will remain the same right I'm just going to fit this image over here because if I go back
to my GitHub repository you can see that this image is actually a numpy array right we are
reading this image using opencv so the format is a numpy array and we need to input a numpy array
right over here so I'm going to do something like I'm going to Define a variable which is
image array and this will be numpy as array right we will need to import numpy so I'm
going to say something like import numpy as np and that's pretty much all now I'm going
to input image array and that should be it so this is pretty much all, we are going to be
detecting all the objects we have... we are going to be returning all the objects we have detected
with a confidence value greater than 50 percent and other than that everything is just fine
and that's it and we don't really need to draw the rectangle so I'm just going to delete it and
that's pretty much all, so we have loaded the image we have detected all the objects on top of this image
using our model and now it's time to continue with the visualization now we are going to take all the
detections all these objects we have detected and we are going to draw bonding boxes on top of the
image the user has uploaded so this is amazing because we're moving super super quickly and let's
see how we can continue with the visualization, now it's the time in which we are going to draw bounding
boxes on top of our images and in order to do so we are going to use plotly, plotly is an amazing
python Library which I have used many times in my projects it's an amazing Library you can
do some very very crazy visualizations using plotly, some very Dynamic visualizations so this
is a very amazing Library we are going to use now and something that's very important is that in my
tutorials we always focus on the computer vision part of the problem and everything that's
related to the visualization is not really that interesting from a computer vision perspective
so what we are going to do now is just taking the code for the visualization which I have already
prepared over here right this is a function which is called visualize and this is the function we are going
to use in order to visualize the bounding boxes on top of our images so please pay attention please
focus because otherwise you may be lost please take a look at what we are going to be doing
now I'm going to the project I'm going to file new python file and I'm going to create a new python
file which is called util.py then I'm going back to this file I have over here and I'm just going
to copy the entire file I'm going to press Ctrl C and then I'm going to press Ctrl V over here
so this is all the code we need in order to do the visualization remember the visualization is
very very very interesting and very important but it may not be the most interesting thing from a
computer vision perspective and that's why we are not really minding everything that's visualization
everything that's related to how to visualize all these bounding boxes on top of the images we are
just going to use this function and that's pretty much all, I need to do a few Imports otherwise this
is not going to work I'm going to import streamlit as st and that's pretty much all if I'm not
mistaken yeah now let me show you something which is related to all the code I have just
copied you can see that this is the code of two different functions right one of them is called
visualize and this is a function we are going to use now in a few minutes in order to visualize
all the bounding boxes on top of our images and the other function is called set background and
this is another function which is only going to make a very very very small and very aesthetic detail
at the end of this tutorial which is changing the background of the web application right this
is only a detail this is definitely not the most important thing from a computer vision
perspective right this is just changing the background of the web application of the browser
so this is something we are going to do at the end and this is also in the code I have just copy
and pasted into this file but now let's focus on this other function which is visualize you can
see this function receives two parameters one of them is image and the other one is bounding
boxes and you can see that the image is the input image and then the bounding boxes are a
list of all the bounding boxes in the format X1 Y1 X2 Y2 so now let's go back to main because
let's see exactly how we can call this function over here the first thing I'm going to do is from...
from util import visualize right now the function is imported into our main process and now
let's go back here and then this is where we are going to call this function remember we need to
input two parameters one of them is the image we are going to import... the image we are going
to use in order to draw all the bounding boxes and we need to input the image in the pillow
format and then the other variable is bounding boxes right bboxes and please please focus,
please pay attention because we already have a variable which is bboxes but if we go back to
the documentation you can see that this variable is a list of bounding boxes in the format X1
Y1 X2 Y2 so this is not the same as this other variable we have over here please pay attention
because otherwise it may be a little confusing so this is what I'm going to do I'm going to
define a new variable which is bounding boxes underscore this is going to be a list and
what I'm going to do here is just appending the bounding boxes exactly as we need them to
be right so this is what I'm going to do and if I go back to util.py this is exactly what
we need to input okay so we have this object over here and the only thing I'm going to
do is to paste this object over here and I invite you to take a look at this file... I invite you
to take a look at this function visualize so you can see exactly how it works and you are going
to see exactly we are using the plotly library and we are calling some functions and we
are doing some stuff which is related to visualization right I invite you take a look at
this function this is going to be available in the GitHub repository of today's tutorial but now
let's continue and let's see exactly what happens if we refresh this website and if we upload
a new image and let's see exactly what type of visualization we will be getting with this
function so I'm going back to my local computer to my file system I'm going to take a random
image again and I'm going to drop it over here and you can see that now this is what we get
which is exactly the same image I uploaded over here this image over here but now we have
these two buttons one of them is original which is... which means this is the original image we have
uploaded and the other one is called detections and if I press this button you can see that we
are plotting the bounding box exactly on top of the tumor of this brain right... I mean I'm not
a doctor so I have no idea what I'm looking at I have the impression this is a brain and this is
an MRI and based on the colors I have the feeling that this is the issue right this is a tumor so
it looks like we have detected exactly what we should have detected right but this is the data
I used in order to train the model right this is the training data now let's see if we have exactly
the same performance with a data... with an image in our validation set right this is completely
complete and absolutely unseen data for my model so let's see what happens if I just take
a random image like this one I'm going back here this is the image I have just uploaded remember
now we are taking completely unseen data for my model and let's see what happens if I move to the
other tab to the other bottom which is detections and we are detecting successfully detecting the
the bounding box the object we should be detecting in this image so everything is working just fine
and in order to make it more challenging and more fun let's see if we can detect an image with two
objects I know that there are a few... like this one which has two objects so I'm just going to drop
this image here and let's see if we can detect both of these objects both of these issues and
we can see that we can detect both of them so everything seems to be working just fine and this
is pretty much all in order to set up this web application up and running you can see that we
are uploading images and we are just detecting all the issues in this image and we are just plotting
everything exactly as we should the only thing I'm going to do now is to use this other function we have
over here which is set background right the only thing I'm going to do is to change the background
of this web application so we make it a little a little nicer and this is exactly how I'm going to call
this function so I'm just going to main.py I'm going to import... from util import visualize and
then set background and then I'm going back to my file system and this is an image I have prepared
in order to change the background it may not be the perfect background ever but I think it's going
to work we are going to put this background in our web application so let's see what happens I'm
going to copy and paste it over here and now I'm going back to pycharm and I'm just going
to call set background and I'm going to input bg.png and let's see what happens if I refresh
and you can see that now we have a much better looking background so everything looks much much
better now, now let me open a new image I'm just going to select for example this image over here
so we can see how the entire web application looks like with this new background we have to wait a
couple of seconds and now we are getting the image with all the detections on top so this is going
to be pretty much all for this tutorial this is exactly how you can create an object detection
web application using Python and streamlit and this is going to be all for today if you enjoyed
this video I invite you to take a look at other of my previous videos where I show you how to make
an image classification web application and I'm going to be posting a link to this othr tutorial
over there so remember if you enjoyed this video most likely you will enjoy that video too
because it's exactly the same process and it's a very very similar web application
so this is going to be all for today my name is Felipe I'm a computer vision engineer and
these are exactly the type of videos and the type of tutorials I make in this channel this is going
to be all for today and see you on my next video
