Brain Tumor Segmentation | UNET Segmentation in Keras TensorFlow | Medical Image Segmentation

Hello friends welcome to this video in this video we will implement the unit architecture in the tensorflow framework and apply it to the brain tumor segmentation data set so first of all let's have a short introduction about what is unit so here is a short article on what is U unit let's go through the article the unit is an architecture developed by Olaf burger at all or biomedical image segmentation in 2015 at the University of fredberg Germany it is one of the most popularly used approach in any semantic segmentation task it is a fully convolutional neural network So when you say fully convolutional neural network is that we use only convolution layer here we don't use any dense layer or fully connected layer so this is the diagram the original diagram of the unit architecture so fully collusion has a benefit that you give any image size it would output the same size mask so it is independent of the side but still when you train a unit architecture they fix size just test it on fixed size if you change the size the performance effects okay so you can read this article I'll give you the link now we'll directly move on to the implementation path will work on the implementation so here I have imported all the layers first we require is a convolutional layer then we have a bad normalization then activation Max pole 2D cone would 2D transpose concatenate and finally the input layer after that we have imported the model class which would take the input and the output of our unit architecture and build a model out of it so first of all we are going to implement the cone block so let us Define a function called cone block cone block would take an input it would be input feature map then it would take number of filters or you can also call number of feature channels these are the output number of feature channels okay so this cone block basically contain a convolution layer base normalization layer activation layer and again these three layers would be repeated so in short we have six layer so we begin with convert 2D layer first would be number of filters then kernel size so we use a kernel size of three by three then a padding for padding we're going to say same so now we're going to give input to this layer so input would be the input of this cone block or the argument of this function next it is followed by the best normalization so one thing I want to say here is that in the original unit architecture no batch normalization was used because it wasn't invented at that time but now in the 22 we use it because normalization layer is is is basically a normal you will use it in between convolution and an activation function because it stabilize your training it helps to improve your performance so we're also going to include it best normalization and the input to this batch normal normalization layer would be output of this cone X next it is followed by the real activation foreign activation function basically a non-linear activation function so it had known linearity to the network now we have three layer here so we're going to copy them because we need to repeat same set of layers we're just going to change the input from inputs to the output of this activation one so our cone block is done so we need these six layer in the both the encoder part and the decoder part of the network so we have built a function so we just simply need to call this function and we are done next we are going to implement the encoder block to encoder Black Block we have inputs as one argument and number of filters as second argument when tensorflow when I say number of filters so it basically means number of output feature H channels okay so first the encoder block would have this cone block so it would take inputs and the number of filters so we're going to copy them from here and paste them here after that it is followed by a 2x2 Max pooling this is two by two input would be X and we're gonna return two values that is X output of convolution layer or the output of cone block NP the output of pooling layer here x would act as a skip connection and P would act as the output of the next layer so let us go back to the diagram so if you see this gray Arrow this is basically output of convolution layer and it act as skip connection and this downward arrow is basically Max pool 2D it goes to the next in encoder block okay now let us Implement our architecture that is unit so we're going to say build unit so it's gonna take the input shape this is its argument first of all we're gonna Implement an input layer we're going to call it inputs is going to take this argument input shape so we have our input layer then we have four encoder block so we have implemented the encoder block just now so we're going to call it here and it returns to Output first is this skip connection second is this Cooling so in short we're going to say S1 comma P1 equals to encoder block we are done then first input would be input layer then number of filters and in unit number of filter began with 64. and in each decoder block the number of filters doubles so let me if I copy the encoder block okay then this would be S2 and P2 encoder block now the input to the second encoder block would be output of the first encoder block that is P1 then number of filters would double that is 64 into 2.28 now let me just copy this again this time it would be S third and P third input would be P2 and number of filter 256. now we're going to copy it last time we're going to call it S4 P4 encoder block input would be P third and number of filter 512 so now we are done with our encode up okay let's run this part of the code and see if there's any issue here because error would always come error would not leave you first input shape our input shape would be 256 by 256 by Third we're going to call our method or function give it this so now we do not give anything from this function so what do we do we're going to print the shape of let's say first this skip connection S1 dot shape S2 dot shape has three node shape and S4 dot shape okay then we're going to print the shape of all the P1 p2p3 okay so now we're gonna see the shape of S1 S2 St S4 P1 P2 P3 P4 okay so let's run this okay so you can see a lot of messages from the tensorflow now if you go to the last you can see first shape second third fourth then first second third so now the the top one belong to skip connection so first skip connection shape is 256 by 256 so it is exactly same as the input layer then second is 128 by 128 because it passed through pulling layer where it decreases by a factor of two and again 128 by 128 became 64 by 64 and at last it became 32 by 32. same in the pooling you can see it begin with 128 then 64 then 32 and at last we have a feature map of size 16 by 16 with 512 s number of filters now let us again go back to our code now let me comment these out now here we have a bridge in between encoder and the decoder and Bridge would be our phone block so we're going to say B1 block P4 as the input and one zero to four as a number of voltage so now the encoder and the bridge is completed or you can say the bottleneck layer okay now comes the decoder part let's implement the decoder block is also simple note that difficult just a bit of understanding is required equal w so first of all decoder would take the previous input feature map and here it would begin with B1 so let me just give it inputs okay then it need to skip the connection so if we remember from here so let's we are here okay so this goes to there this bridge you can see the up cone after up cone there's a concatenation so we need this skip connection so we're going to call them skip features and after that we require number of filters okay so for now I'll remove these just for some just for explanation because here I need a layer so first of all what we do we'll take the previous feature map and up sample it using transpose with coconolution for that we're gonna use cone 2D transpose we're gonna say number of filters convolution size would be two by two so unit uses a two by two up cone up cone basically means con 2D transpose layer destroyed Sprites would be two because we want to up sample it by a factor of two so we want to convert 16 by 16 to 32 by 32 that is why we are using a stride of two and padding would be same input would be inputs so what I'll do now I'll call this decoder block here okay it requires input B1 for now it regard to input inputs that is previous input feature map and number of filter when decoder the number of filters decreases by a factor of 2. so from 1 0 to 4 it became 500 now I'll print the shape of D1 let's do it inside the decoder only inputs short shape now just see the shape of x it would be increased by factor of 2. we can see we have two shapes here first is this inputs second is X so input has a shape of 16 by 16. and when it goes to transpose convolution it up samples by factor of 2 and became 32 by 32. so let's say you put a stride of 4 here see what happened let's see if it work or not and I can see it is working from 16 by 16 it became 64 by 64. but we're going to put a stride of 2 because that is what is mentioned in unit architecture now next comes the skip connection concatenation now if you see the shape is let's save and run it again the shape is 32 by 32 so we need a feature map from the encoder whose height and width are 32 by 32 so the height and width should match between the output of transpose convolution and the skip connect so now what I'll do I'll uncomment this part so that you can get the zip now we don't require this we just need to check this shape and the ship which matches to each other you can see skip connection shape so we need a skip connection with the size 32 by 32 so this is S1 and S2 s third so you can see S4 matches it has exact same height and width as the output of convolution layer so what we're going to do we're going to say skip features we have add this argument now here we will add S4 oh what we're going to do we're going to Simply say x is equals to concatenate so we're going to concatenate both X output of transpose convolution and Skip feature that is feature from encoder so we're done now let's print the shape of x let us comment down this part run it again see for any error okay so you can see now it became 32 by 32 by 1024 so number of feature Channel now and doubles after that it is followed by the cone of block input would be X the concatenated a feature map and second would be number of filters now it is done we are going to return X so this is the simple decoder block so it returns even let's say we need D2 that is second decoder block output decoder block input would be D1 the output of first decoder block skip connection would be S third node you can calculate the same sprinting appropriately now number of filters 256. then third decoder block just see number of encoder block and number of decoder block would be same that is 4. because we sequentially decrease the height and width of the feature map and then we sequentially increase the heightened width of the feature map and Skip connection as well 62 so this is the decoder block so now the output of D4 should have a height and width same as this input 256 by 250 so let me just print this so if you see 256 by 256 just number of feature channels are 64. because now you know we are working with binary segmentation in binary segmentation the number of class is 1 so now we'll have a simple one by one convolution layer where number of feature channel would be when kernel size would be one padding equals the same inactivation is sigma an input would be D4 now let's print the shape of this outputs let's see the ship now you can see 256 by 256 by 1 and this is the shape of the required mask now we have both the input and the output now we're going to build our model so we're going to say model let's do model first we're going to give the inputs here not the shape the input layer in the output layer let's give it a name and call it unit and return this model let's complete this function it's going to give some model and we're going to say model dot summary now let's run this and check that's the summary of the model so you can see has this much amount of parameter and where this much is trainable and rest 11 000 are on training you can see the entire structure if we go from Top can see the input layer 256 by 256 by Third because we'll input an RGB image the number of parameter here are zero then we have cone 2D layer then best normalization activation then again a convolution as normalization then activation and after that we have a Max pooling where height and width are reduced by factor of 2. so like this all the shapes names and everything is given in this summary so this is about the implementation of the unit architecture I hope you'll have a good understanding of unit architecture now you still have any query any question just write it down okay write that below and I'll try to solve it in any other video or try to put it in next video so if you come to this part of the video this means you are enjoying the content so please make sure you hit the like button and subscribe the channel now we'll move on to the training part of the video now we are in our train node py file here first of all we are going to import all the libraries which we require so we have a lot of libraries here first is OS library and second if you see web statement this OS dot Environ then TF underscore CPP Min log level 2. so if you see have seen we got a lot of message when we run any tensorflow program so this is just to get rid of those messages okay next we have imported the numpy because we're going to play with images and images are going to read as array and numpy is a useful tool to manipulate array work with them and a lot of stuff we can do after that we have CV2 so CV2 refers to opencv so this is the library we're going to use to read resize and write back images next we have Globe the globe we're going to use to extract the image is part in The Mask path very easily then we have this Shuffle so we're going to use to shuffle the list of images and a mask after that we have imported a lot of stuff from tensorflow so first is we have imported the directly tensorflow stf then we have imported some callbacks I'll explain them View after that we have imported the Adam optimizer next we have imported the train test plate function from the sqln library so we're going to use this function to split our complete data set into three different parts that is training validation and test next if you have imported the unit which we have built now okay after that we have imported two matrices from Matrix matrix.com and these are dice coefficient and ISO in this video we're going to use dice laws because it is much better than cross entropy let's now continue with the coding part before that let me just show you the data set so these are the two folders first is images then each of the four luck contain near about 3064 items so that is the size of our data set so you cannot say this is small this is really good data set now let me just show you some samples so you can see this is the image of your brain so we have different view different perspective so this would help us to make the model more robust because it can now deal with different perspective and like every other thing now these are the images now if you see the size these are 512 by 50012 but this data set also has some sample which are of size 256 by 256 so that is why we'll work on a size 256 by 256 so we're going to resize all the images to this size no it's a mask so it's a binary segmentation problem it only has one main class that is this white patch but it's a tumor and the rest black areas the background so these are the masks it has different kind different quality of this is about the data set now let's move back to the coding so first of all first of all let's define some Global parameter and these are heightened so height 256 width so we have initialized them at the top only so that any function any Library can use them easily next we're gonna write a function called create dir so we're going to give it a path so this function is going to help us to make some folders which we require for our task so first of all we're going to give it a path then we're going to check if that path exists or not if not if that path do not exist if that path do not exist then create that path or folder as simple now let's start with our main statement this main statement basically tell us now the execution of program begins from here so first of all we are going to seal the environment seeding is really important for reproducibility because neural network are initialized randomly so you do not want to change your Randomness again and again maybe on my system Randomness is different so I'll get some different performance you get different so it is important to seed your environment so we all have same randomness so that is why I will try to see there is some function in Cuda Library which affect your reproducibility but still seeding is important foreign files so we can say for storing a file so we're going to call this function we're going to say file next let's set some hyper parameter okay before that let me just show you if that works so if you see these are the four files we have now what I'll do I'll execute the code you can see it work without any error so now if you see we do not have those lines which shows us here is the effect of writing that line on the top this line helps to get rid of Polo's unnecessary messages now executed the code now if you see we have a files folder here so this I create dir function help us to create empty folders or directories next we're going to set some hyper parameter first of all let's set the batch size that is 16 learning rate is 1 E minus 4. and number of people let's say 500 then we're gonna have two path first is modal path any CSV bar so we can say model so modal path is basically the place where we will store our fit file we will save our data inside this files folder and the name of the weight file would be model.h5 then CSV path join Ola name is files name of the file would be data.csv or you can even say log dot says log dot CSV it's much better now if you see I have a set a batch size of 16 okay this is up to you and it depend upon your system because I have RTX 1390 I know this is much expensive but not now and it has a GPU memory of 24 GB so with that memory I can set a batch size of 16 and it would work perfectly fine okay but don't use 16 maybe it won't fit on your GPU don't work on you so try to change it okay make it eight or two ones try to see if it work which batch size works for you okay so don't just copy it like that only okay and Hyper parameters are basically things to play with so you can play with these two things like bassides can play with learning rate you can play with okay now let's write a function to load the data set and split them foreign data set so this is the name of the function is going to take the data set path and split size so let's say by default value of 0.2 0.2 means that 20 percent of the data would be used for validation and another 22 20 percent would be used for testing and remaining 60 percent would be useful to training so this means we are doing a split of 60 2020 that is 60 for training 20 for validation and rest 20 percent for testing now let's load images first so I'm going to say images so it would be a list first of all let's set a path what should we path then if I show you the name of the folder is images okay and for mask right images and after images we can write star so star basically means load all the files which are inside this phone so the extension of our images is dot pin so we're going to say start dot pinch so this basically means load all the file which begins with which can begin with anything they should definitely end with DOT pin and that is the case I have done this just because in case there's some text file in any other file because with data set comes explanation why so just by to avoid those things you just want to specifically mention that we want to load all PNG images so this is a path now we're going to use our globe Library this globe would give you a list and we're going to sort that list just make sure you float that list I'll show you what happen if you don't so let us now load all the market instead of images frequency now let's load the data set as you can see data City let's call this function we can see data set path let's go the default values for now I'll paste my data spot and paste it here there so this is the path to the data set now I'll I'll execute it okay now I'll just remove this maybe the program work without sort and maybe not but why take risk okay so what I'll first I'll do is I'll print because it's going to be list I'm going to print that string that is the file path image or mask path at the index position 0. so now if you see it's working fine without floating also it has OneNote PNG and here also it has one node PNG okay so in some case when the naming is a bit different okay or randomly it has one dot PNG and it has two dot PNG or 10 dot pins oh to make sure this thing don't occur we sold the list because when naming is same or nearly same sorting would sort the name in a specific order and that order make sure that you get appropriate Imaging because in most of the data set the naming of images and mask are same just to correspond that this image belong to that mask that mask belong to that image so just make sure you sort them it won't take that much time just just to show to be sure that this would work now we have loaded both the image and mask what we'll do we'll split them first of all we need is test size so we're going to call it split size integer length of images into split so this is split size like how many samples you want in your validation and test set that is split size so now you can easily manipulate this and it would be automatically done so we're going to first split the data set into training and validation so here x refers to images and Y refers to mass just I use this and thing in variable naming we're going to use strain test split first images and test size would be split size and now a random state so this random state is basically seeding because some random operation are going to happen here and it's going to split the image placed into two different lists that is strain X and valid X that's why a random State just make sure you seem a random State always don't specify I think it would randomly split it we run program again and again I'll copy this again now I will I'll introduce X I will say y between X and valid you know train y valid y and instead of images I'm going to see so this way we have splitted the data into training and validation so now train contain 80 percent of the data and 20 percent is assigned to validation okay now we need to make test set so we're going to say train X and Test X I note there are much better way to do this but I'll stick to this only I'm going to copy this from here and now we'll take the strain X which contain 80 percent of data so what we'll do it take this strain X data and assign 20 percent to to this so the train X would became 60 percent now we'll change X to y and inside it framework this would work perfectly fine for us now we're gonna give back those training validation and testings now let us call them here foreign data sets let's put a gap here now we're going to run this program again just to see if everything is working fine you can see eighteen thousand one thousand eight hundred and forty images are used for training and 612 images are used for validation and testing set I think this split is okay for me is 60 of 2020 you can do it like 80 10 10 or up to you or 50 25 25 because the data set is large it's up to you okay so data set processing is done now we'll do we'll make a pipeline for training the first of all we will Define some function here first is reading image read image look it would take a path so first of all we write a statement to decode the path because this function is is going to be used by the tensorflow and it would give you string s encoded so first of all we're going to decode that and then we're going to read the image we're going to read this as RGB image I know the images are grayscale but still I'll read them as RGB you can read them as grayscale and input yeah it would change for one three to one when you call this function here in this strain dot ppy file so let's read it as RGB path then we're going to resize it so X as the array then width by height then we will normalize the image pixels by dividing them with 255 so now till here this the pixel range is between 0 and 255 so this is really huge so what we do we divide the whole pixel values with the maximum pixel value that is 255 so this way it would convert in the range of 0 and 1. it would be like 0.1 0.20.250.68 0.92 something like that good now we'll convert it to floor 32. okay now I'm going to give this value back so this is done now we're gonna move on to the read mask one again we're going to decode the path copy paste and this time we're going to read this image as create scale because it is literally the Master's black and white because the output if you remember the output of our function is 256 by 256 by 1 so it is one channel that is why we're leading it as a grease because grayscale is by default when the channel of okay so now we're going to give path here then we're gonna read it as create scale okay so again we're going to resize it I'm going to Simply copy all these stuff from it because they're literally the same just one step need to be added here so we resize it and we normalize it so in mask normalizing would also be same now you may question let me just give you a mask okay so this is one of our masks the black region is basically pixel value is this is zero and why it is pixel value 255 so we divide that 255 with 2.5 it became one and zero it divided by 255 remains zero so it it normalizes in the range of 0 1 0 1 or 0 and 1. now just a silly simple thing what we need to do we need to expand it at that the dimension here next expand dims x axis okay so what happened is when you read the grayscale image it would be of let's say height by width assistant you resize it it again became some specific height and width here also the dimension remain change here also but here what happen is we added the dimension I hope you get it why we have added this extend expand Dimension line here okay so reading image and read mask function are done now let's build another function called TF foreign basically take a single image path and a single mask okay now inside it we're going to build another function all parts this also take individual image and mask okay here we're going to call both our read image and read mask function read image X Y is equals to read mask okay and we're gonna return both the X and Y now this simple function is done now what this simple function is done you take those path and give us our arrays which we require now if you see we are using the opencv in the numpy function which are outside the tensorflow so you cannot directly include those libraries or functionalities into the tensorflow for that reason we're going to call TF dot numpy function here and we need to call this function so we're going to type this function name then the input that is X and Y and after that we need to specify the output data type so output data type is float32 load 32. so we're gonna say TF Dot float32 now it would return X comma y that is the image and mask tensor I believe then we need to set the ship first is height then width then here we are reading it as RGB image the number of channel would be third three then y dot set chip here height would be same width would be same number of chain would be one and we're going to return X and Y so this is simple function which takes both its first so you can also directly use tensorflow functionality to read resize and do everything but I feel this opencv is like best because I'm using this so I'm got addicted to this way only maybe in some next video I'll try to give another way where it can directly use tensorflow so you need not to go through this step I think okay now we'll have another function EF dot data set I know we have going through a lot of lot of function but just bear with me some more time so here this is our last function in our data set pipeline node entire file this is the data set pipe pipeline here x is a list of image path why is a list of masks so we're going to say data set equals to TF dot data dot data set Dot ROM ancer slices then we're going to give X and Y oh my mistake okay so now what we do we're gonna again say the data set let's do data set you're going to call the map function of it and we're gonna use this TF bars this basically takes this list of and use this TF pass function to read all the individual images and mask then what we do copy this and form a batch we give a batch okay so by default batch is set to okay let's default this only okay then prefetch the previous is basically a function where while the model is training on Epoch not Epoch it it's training a batch it would load some batches in advance now I'm gonna return the data set okay so the data set pipeline is this only it's enough now what we do is simply say train data set and we're going to call TF data set we're gonna give it 3x and train y remember it take list of images path list of masks and then batch batch is equals to the batch size that is 16. and same we're gonna do to valid data set TF data set we're going to call valid X and valid y here we're going to set the same batch size for validation I think you can increase the batch size let's not get into that let's remain symbol so this is still now we work on the data set pattern now just run this one for you for X comma y in data set X and Y are basically batches or of images and masks what I'll do I'll print the ship okay I'll run this for you if you see a batch size of 16 and 256 by 256 by Third and same for the mask it will run run run and after some time it would end let's stop it because it would take some time now we're going to call our model which we have implemented so let's write some comment model now let's call our unit architecture during the same model is equals to build unit is going to take an input that is height you already specified width number of Channel three if you remember we have read reading the images as RGB now we're going to compile the model first we're going to set the lows s die slows I've already told you then optimizer atom would be our Optimizer with a given learning rate that is one e minus 4 then it matters just to test the performance during training as the coefficient now we're going to write all the callbacks so I'm just directly copy paste them so here we have four callbacks which we use first aid model checkpoint so model checkpoint basically save the weight file during training so is it only saves the best weight only that is let's say your loss rate is increasing so when your loss rate increase that's not the good performance so your weight file would not save but when your loss is decreasing especially the validation knows when it decreases it saves those weight file Epoch after Epoch okay let's say in between some it does not decrease it would not save so with this check and this callback that is model checkpoint Vivo save our best weight file okay second is reduce alarm Plateau so this basically checks your value validation laws and let's say for five continuous Epoch your validation loss is not decreasing so decrease the learning rate by a factor of 0.1 next is CSV logo so this all back basically generates a CSV file and save all the training data in this CSV file so that we can check that file after the training is complete and we can build graphs or charts using that and last callback is early stopping so it also monitors your validation laws let's say for a continuous of 20 epoch your performance is the performance of your nose does not decrease okay and it would stop the training so this basically stops your training to make sure that you do not unnecessarily train your network when it's not giving you performance so you can increase it it's up to you you can play with this value you can set it to 50 or 100 it's up to you but as I'm using a home computer I do not want to run it unnecessarily maybe you are using Google Google app which is free we can maybe you make it 50 because our best checkpoint would be saved it does not affect the performance of the save weight file okay so that this is about the Callback now we'll finally call our function model outfit and that is the last function for training at the end we generally just call this so we take a training date data set number of epoch of validation data set and Cooling so I'll just save it and run it for once just to show you I have already trained this model for some epochs and I'll directly copy paste that wait file if you see it does not take that much time just 18 17 second but I need to write for La long time so let's run this book and see the save weight file and every other thing okay so our first Epoch is done let me just stop it here so if you see it's the loss the validation knows if you see it decreases for minifying so initial it is infinite then it came 97. but this is validation die so all these matrices values are save in your CSA file so let me just stop it clean it and just show you those weight file okay so for for now for this scenario this is your CSV file because we have trained it for only one epoch right you have training dice loss learning rate validation dice then validation laws so this is how audio values are saved and you can build different charts graphs using this okay and this is your weight file what I'll do I'll delete those and I'll show you mine one I have already in the model and I'm going to copy those files okay now just see the data.c so this model is strained for about it's too many Park we can see because it begins with zero so which we have trained for 61 Epoch and it is is automatically stopped so if you see validation loss is decreased to 0.18 and validation dice coefficient is increased up to 81 percent and if we see the training dice it is 95 and training loss is 0.4 I believe the training is is enough what are this now now the training is run I will work move on to the testing part here we are in our test.py file so first of all again we're going to import all the library okay so let me just go through the library which are new so first is this tqdm so tqdm is basically a progress bar because here we're going to Loop over the list of test images so we just use this tqdm function and it would give a progress bar that would tell it out that like out of 612 number of samples or Loops this much Loop is done this number of iteration so basically simply progress R just to show you how much stuff is done how much is left next is this custom object scope so this is used uh when you have like an external function let's say here we have used if you remember the dice laws and the dice coefficient which are not built inside the tensorflow so these are external function so in that case we need to use this custom object scope after that we have imported some matrices from SQL that is F1 score Jacquard Precision recall next we have trained a split I think we don't need here next we have nice closed dice cooperation then load data set build unit okay so things are done now let me just copy some more thing because they are repeating it's better to just copy paste them so here we have the height and width and he says earlier for easy use again a function to create a folder now let's go to our main function I'll copied from my code so this is the main function from there the execution start so now in the testing also we're going to see the environment make sure we have same reproducible node same randomness then we can use this function called create dir to create another folder called results and in this folder we're going to save our predicted mask foreign custom object score and inside this you're gonna have a dictionary the name of the function then the function itself okay dice coefficient next dice laws name of the function and the function itself and inside this we're gonna load our mode so you're going to say model equals to TF dot Keras dot models dot load model just path that is OS store path node showing so this is the path for our waiter which is in file after that modals.h so we have load the model so we're gonna simply say model dot summary just to check whether this is working on let's wait okay so you can see the model summary which I have names layer connection everything this means that we are working fine till now there's no error we'll just remove this so now we need to load the data set so let me just copy that part directly from here okay so for now we need just test images okay so now we come under evaluation part evaluation prediction so first of all you're going to predict and then you will let me write prediction because first of all we take that test image we predict the mass then we take the predict mask and the ground truth mask and you evaluate okay okay so let me just have an empty variable a list called score then we're going to Loop over our test images and test mask X comma y so these indicate the individual image path and mask in and here we're going to use our tqdm that is progress bar zip inside we have Test X that is test images in test y that is testimony then for tqdm we need to write a value that is how many time this Loop is gonna run that is the length of test X because both test text and test y are equal so you can say test y also so now I'll just print X and Y just to show you and I'm going to break it you can see the image path is this eight eight three three dot PNG and same name I just want to make sure that they both have same image and mask and if you see this progress bar it just stop at zero that ratio it does not begin from here so it would begin something like this it will be completed let's move back to our code okay one thing from this path we need to extract this name and dimensionals let's work on that so you're going to say name equals to so we're gonna split the X okay so based on some value so if you see this path it has this slash copied maybe if you are on Windows this slash would be different so just make sure to print the path and then do c just try to understand how it work don't directly copy paste it so I just put a slash here now I'll just print this name and you'll see what happens okay let's try to understand the logic which I mean if you see nowadays path breaks based on that slide to slash it disappears for me you see first this less then media then slash if you see then list blank then here's your slash in media then slash then you can so just split based on that so our required thing is in the last so what we simply going to say minus one you can see we have successfully extracted image with Exchange now the name part is done now it's time to read the image let's write it line here also extracting the name next task is reading the image now I'm just directly I'll just directly copy paste it here because this thing is repeating we have done the same thing in the training so first of all we're going to read it as the RGB image we're going to resize it we are going to normalize it and now we are also going to expand its dimension because if you remember the model take the images in the form of batches and now here we have single image okay so this would be a batch of one image only so let's show you here so here it would be something like height by width number of chain like this well let me just copy this here also it would have something like this just heightened with the change here also now here what happen is it would add it would extend the dimension to this so this now would be a batch of one image now we're going to read the mask here we need the mask to evaluate its performance to calculate different matrices so we just read the mask as grayscale and what we do we resize it appropriately now we're going to make the prediction just the first part in this prediction what we're going to do we're going to say y print that is prediction y refers to mask that's how I use this variable name vibrate means a prediction part modal Dot predict access the input and verbose s 0. okay now what I'll do I'll print the shape of Y so if you remember the input is a batch of one image so Mass would also be a batch of one okay but still we gonna run the shape and I'll show you how it works so if you see shape let's say mask okay this is the mass 256 by 256 by 1 and is only one so we're gonna say zero here so we get our required mask so next if you see it has this Dimension we don't need this dimension so what we're going to do we're going to squeeze it on the last axis so let me save just save it and run it for you now you can see the mask is of size 256 by 256. this is the predicted mask now what we do because the values of this mask is in the range of 0 and 1. it can be 0.1.2.3.4 till 0.9 or even 1.00 but we have to click classes that is 0 and 1. so we're going to put a threshold of 0.5 here the value greater than or equal to 0.5 okay r 1 greater than 0.5 less than 0. and now we convert it into integer 32. so this is our prediction now what I'm going to do I'm going to save this prediction and show you okay so what we do we'll take the original image the mask and the predicted mask and we're going to join all the three images into a single image and see so this would give us a better look we can easily compare the predicted masks so let's write a line is saving the prediction so we need a save path so we're going to say save image that would be our fold if you remember we have a folder this results then okay so we're gonna not use this way we're going to use OS store path toward join results the name so we have the path now we're going to write a simple function hold save results first is going to take the input image this image then the mask then the prediction by print and next is going to take the save image so let's take this and build a function okay so now one condition is there because all that three image mask and vibrate at the same height and width that is 256 by 256 just that image has three channels and mask and I think wiper it have only one channel so we need to make sure they all have same number of Channel so what we're gonna do we're going to make sure that both mask and white bread have three channels that is same as to image first of all let me just print the shape of all these three arrays mask dot shape then wipe red dot shape let me just run it for you we can see it has three channels and these are one channel one channel see how the difference would occur first of all we begin with the processing of mask we're gonna expand its Dimension the mask on the last axis for this we have a function called NP dot expand dims mask X is -1 same we need to do for the vibrate let me just copy it by print now just see let me just save it and see that how it changes you can see now we have this channel here one now we need to make sure they have three Channel instead of one for that we're gonna concatenate these masks and prediction three times The Mask equals to NP dot concatenate he should be small you're going to say mask on the last axle this is done now we need to do the same for the upgrade we just change the name here save them and run the program now you can see all the three Aries have same number of Channel okay so just make sure they have same number of same height width and number of Shams so that will be that will make them really easily look concatenate with each other okay now for if you remember the prediction has the value in the range of 0 and 1. so we need to make it in the range of 0 and 255 so that we can see it so we're going to multiply it by 255 so when we multiply 0 with 255 it became 0 that it would remain black and when multi 1 multiplied by 255 it would became White so we can easily see it do we differentiate them as black and white and that is how the ground of mass flow now what we do we're going to say cat images and we're going to need this function and here we need to say images or image mask and wipe it on the axis one and we're going to save them save image path and the array would be cat images so let me just run this an example and we can see so here you can see we have some issue here okay let me just check save result this and and here the error is it's here in the cat images so let's just have a look at it again first we need to give the path then we need to give the array oh my mistake it should be one not minus one minus 1 means the last axis and it would not concatenate on that okay it would concatenate but it will not save as image yeah right so let's say we concatenate it on last axis so it would become 256 by 256 by 9 because 3 channels here three here and three here so that is why if you save an image with 9th channel it won't save this is the reason it's not working here because let's say either channel should be one three or four nine is not like one is for grayscale it is for RGB and 4 is for rgba because this Alpha layer also like in PNG means we have an alpha layer so now I'll show you that one image so this is that one image where we have first the input image ground truth and the prediction but one missing thing here is that we cannot separate them so for that we'll make some changes in the code so what we do we're gonna have a thing called line we're going to make a small white line and we do it once height which should be 10 channels should be same as the rest what I'm going to say a simple line here a simple line so now we're going to run it again now we can easily differentiate okay now if you see input image round truth and prediction now I can easily see the images okay you can make this instead of 10 by 5 it's up to you but now in this way we can easily compare we can easily compare both the ground truth and the predicted mask okay now this uh saving the image part is done now we'll work on the evaluation part because we also need to evaluate the prediction against the ground truth and calculate some matrices now we'll calculate the Matrix values so first of all let's calculate the F1 F1 is also called dice because both F1 and dice have same formula especially for binary segmentation so I'm going to say F1 value then we're going to call the function F1 score first we're going to give it oh one things we have left okay just let me write this code then labels equal to 0 comma 1 and average equals to binary okay and before writing all these we need to make sure we flatten those arrays both for the mask okay and for the prediction so we have already applied the threshold we don't need to write this line we simply ignore this and write mask dot flatten does that it would work so this is for F1 like that we're gonna have three more matrices and these are the card that is mean I you recall and precision so we're going to append these values along with the input emission in the score or we have name F1 a card a call and precision I'll remove the break from here so now we're going to write the code after outside the loop so if you remember for score okay first is the name so what we do we have this small score written here and it contain the matrices after one if you see after one everything so from this that it extract all those values then it calculate the mean score and we print out these four matrices here and we can also make a CSV file of this okay so we're going to save this and I'm going to run this here and we're going to test the score so now you can see the progress okay is the matter okay so it says Target is multi-class but average is binary so I know what that error mean so it's getting a continuous value or binary it should have zero or one okay it is oh yeah it is for mask for mask we also need to apply this thresholding because some things are also there here and there okay so for mask we need to do something like mask divided by 255 okay now we apply thresholding and it should work fine let's run it again and see if any more error is there okay now I can easily see the progress bar so this way we can easily see how much iterations are done out of 612 and how much time is gonna take so it's gonna take 30 more seconds it has taken 15 seconds so this is a much better way than to see the blank screen because it lets it for 10 minutes cannot continuously start for 10 minute blank screen but with this way you can easily see that let's say 50 is done sixty percent is done so you feel more satisfied with with this progress bar and you also know how much time it's going to take so you can just put that time into some other task and come back again so it is done here you can see this course F1 is 76 percent Jacquard is 67 percent recall is 75 percent Precision is 81 percent so these scores are okay but I'm not gonna say these are really good these are just okay for me not really good but with EU unit I'm not expecting very much good because that model came in 2015 and it's 22. I think it's seven years so there's a lot of improvement in those seven years so over the next few videos we're gonna have more architecture and we're going to try to improve the performance by using different architectures and I'll also show you the results which we have saved now we can easily see image mask and the prediction you can easily compare and see which image likes like here you can see this over the segment is so this for this case it's over segmenting if you like this you can see different cases and if you will show you this course.csv file so now even you can see for individual image what is the score what is F1 Jacquard recall precision let's say for some cases the score is okay you see this for this image the score is really low it's like 0.2 F1 and that is not really good let's search for that image and see you can see for this case it's a really bad score so this way you can analyze your score okay and this would help you to further improve your performance so I hope you really enjoyed this video and understand how to use unit then how to train it and test so if you are still here till this end just make sure you like the video and subscribe the channel and please do write down below if you have any query comment anything or even you can write some comment it would help to boost the video because I know this really long gap of six months between this video and the previous video so just for that I'm really sorry but for now I'll try to make at least one video per week so help me to boost my channel but by liking it subscribing share the video and click comment below so for now thank you have a nice day

Channel: Idiot Developer

Views: 15,861

Keywords: idiot developer, brain tumor segmentation in tensorflow, brain tumor segmentation, brain tumor segmentation using deep learning, brain tumor segmentation using cnn, tumor segmentation in tensorflow, unet segmentation in tensorflow, brain tumor segmentation unet, brain tumor segmentation using unet, brain tumor segmentation using unet kaggle, brain tumor segmentation in mri, Segmentation of Brain Tumors from MRI using Deep Learning, unet in keras, unet segmentation in keras

Id: lstBIXVUoSM

Length: 79min 13sec (4753 seconds)

Published: Sun Oct 09 2022