Detecting Skin Cancer (Melanoma) With Deep Learning

hello everyone and welcome to the new video um so in this one i'm going to tackle a problem of early detection of uh melanoma which is skin cancer and melanoma is a very deadly cancer but most of the cases can be cured if we can do an early detection so in this video i'm just going to show you how you can easily build a deep learning model to uh for early detection of melanoma so um we will be using pytorch and uh for for any model we need first we need some data so let's see um recent kaggle competition has some data that we might be able to use so it's called melanoma classification that's sim isic and um it has images of skin lesions so different regions of skin and we will be using that to to kind of build a model to predict whether it has melanoma or not so you can see that the images are provided in dicom format which is fine and they have also provided some images the same images in jpeg format so what we are going to do so this there's a lot of other information about the patient and uh you can probably try to improve on this model by using these information they are quite helpful um assuming so um let's see that there is one csv file called train.csv which has image name patient id approxage okay and some more information and a target column so 32 [Music] 500 zeros and only 584 are once wow so you see that it's quite skewed so the first guess that comes to our mind is using area under the rock curve um as a metric to evaluate models on this um so yeah i mean out of 35 33 000 cases only 580 melanoma and rest they don't and okay so so we have to build a model so if you want to see what these images look like uh i mean yeah it's difficult to look at these images some of these images but i can show you some so let's see okay so this is one of the image and uh yeah so there are different types of images and so you also have some kind of scale here in some images some images you don't so um yeah so these are our images in simple words and each image is associated with a target variable which is binary zero or one and that's what we have to predict we have to predict the probability of an image from the test set having melanoma or not um if we look at the evaluation of this one uh evaluation okay around the rocker so as we guessed it's it's the wise choice and uh now we can try to start building a model on this data so first of all you need to download the data set it's quite huge so 108 gigabytes of data yeah that's a that's a lot of data and the next thing that you want to do is do some kind of uh stratified split now there are only 580 samples in the whole dataset which have melanoma so what you want to do is probably do 10 folds but i'm going to show you only 5 volts so you can pro you can probably try doing 10 folds and maybe that gives you better results but even five folds is quite okay i think so first file that i create is called create folds by you can also uh do it on your own but i'm going to show you nonetheless so uh in this file we read the we read the training.csv so import pandas and then we split the training.csv into uh five different parts using one new column poor pandas as pd and you have from model selection okay let's do this one so from scalar import model selection so i hope the text is visible okay so now i create a main function and inside this i'm going to read the data so i have df period read csv my data is located at a different location so i will say workspace [Music] workspace um i can do like i cannot copy this okay um you can say input path is workspace okay and we can also do import os to join the path so now we pd. read csv read csv takes a part so we do start path.join and then we have input path comma train dot csv so that's that's all and then it's going to read your training data so the next thing that you do is next thing you create a new column df k fold and fill out the value value of minus one it's okay now what i'm doing here is um just i'm just uh shuffling the data set the csv file reset index so i have now i have a new shuffled data frame and then i get the target values so df.values and then i can have model selection stratified k fold so i can do and splits equal to five you can use ten um [Music] y is stratified because we want the ratio of positive to negative samples in every fold to be constant to be the same and now we have now we can do for bold comma train idx test idx or valid idx whatever uh and enumerate and then you have then you define x what is x access your data frame and since you have stratified keyboards you also need y okay so this will be not just enumerate but kf dot um split kf dot split and inside this this enumerate okay now there's this few things that you don't need we don't need this one don't need this one and now you can do df.lock all rows and k fold column goes with fold underscore so we just add a number of fold uh full number to the column and now you can save the new data frame so you can do df dot to csv where do i want to save it so i can do the same input path comma train underscore folds dot csv and we don't need the index to keep it same as the original data frame so now let's run this code and see what happens hopefully it works so i'll go to my terminal and i have create false and boom it works um so now if you go and see the file that has been created i'm not going to do that that file has a new column called k-fold which has values from 0 to we can also create 10 volts let's create 10 volts doesn't matter and just replace the file okay now what we are going to do is train uh create our first model and today we are going to do it a little bit differently so let's see i i hope you like it so let's see what we have here and create a new file called main.pi and inside this file you have you will have your model you will have everything related to training so we might need a few things like we might need os for joining part probably and okay one more thing that i missed is uh you need to resize the data set so images are of really different sizes and some images are too huge and it's going to take a lot of time for you to train the model if you don't resize them to one single size each so you have to resize them uh how to resize them i can show you if you want so let's write a function called uh new python script resize images and what this will do i'm not going to run this one because i've already resized the images i've also published a kernel on kaggle if you want to take a look there i'll probably i can just show you there so let's see um let's see i can just show you here okay i'll let it open so what what we will do is we will use uh pil python library and we will read the image and then we resize it to a given size so now you can choose your size what you want like two for 224 224 512512 uh the bigger the size probably the better your results are going to be so if you do one zero two four cross one zero two four probably you will get a lot uh a much better result but uh um i tried with uh 224 so we are also going to use two to four this gaggle is taking a long time okay um anyways so i will show you when that's ready hopefully soon let's wait for a few minutes okay so so here it is and uh okay so let's see what what i'm doing here very simple you create one function to resize one image you have one image research one image just one function so i take i take the image path then i extract the last bit of the dots a bit but last part of the image path which is something.jpg and which is my base name so i can do that using name so that will extract the file name or the base name then i have a output folder i have output folder and then i create a output path so otherwise it should be output folder uh plus base name so then i read the image using pil and i resize it to a given size and then i save it to the output which is really very simple now why we we do it like this because uh there is one there is one thing that you can do you can parallelize it you can parallelize it and uh then it's going to be super fast so what we do is we have the input folder okay with all the training images we have the output folder uh which is empty for now you have images so i use glob to create a list of all the images inside the training folder uh input folder and then i use job parallel i create a certain number of jobs and then i call this function resize image function with different arguments for each image and now uh in kaggle kernel it's going to take you to time so better use your own local machine to do all this conversion that's going to be super fast took me like 10 minutes and otherwise it's going to take you one and a half hours so yeah you you can do it probably you can probably do it yourself i'm going to show you the uh cool part so i'm going to remove this file and we have we have the main so main function we need we need a few things like we probably need os you probably need charge um you probably need um numpy and pandas okay so you have all these things and now um you can create your first basic model so let's say we will we i mean we are not going not here to reinvent the wheel so we are going to use one of the pre-trained models so which pre-trained model you want to use so let's say we want to use serious next press next uh so i'm gonna click create a class and six fifty fifty thirty two thirty two x forty okay so that's my uh model class and this inherits from and then dodge dot and that module so let's also import that import okay so we got this one awesome and now we have a init function so comma pre-trained equal to image net so we say if it's pre-trained as a magnet and load the pre-trained weights we need to find super and it's going to be same as the name of the class then self and underscore okay so now you can create a base model self dot i will just say self dot model is uh i'm gonna use pre trained models so import pre-trained models and then we have pre-trained models and you can get the res next 50 x 50 32 d 32 x 40 so let's see next uh 50 32 underscore no there's no underscore x4 so i mean i can just go here and see can i okay let's see see res and i need this one okay let me close this one okay so i can just copy this and that's my model okay and i can do trade aha it's copying all the training data for me from somewhere i don't know and i can say pre trained equal to pre-trained so if my pre-trained does none then it doesn't load the weights okay so i'm not going to do anything fancy here um but you just have to create another linear layer so remember that our metric is area under curve so we don't need to do any kind of sigmoids or max kind of thing we just need to rank the images and for that we can just use any kind of linear output so i'm going to self dot out equal to linear and this has 2 zero four right output so you can check that you can just go here take this part and print it i can show you so to import pre-trained models and i have i cannot paste in this terminal so i have to take everything we trained models dot protect and see this next 50 it to x40 i hope this works yeah so you have the last linear layer which has two zero for it inputs right and that's what we have to change now so uh we use two zero four red inputs and one output okay no no no no no no sorry it doesn't work like that so you have yeah sometimes i make mistakes um okay so it doesn't work like that i mean i can obviously just change the last linear layer but to keep some flexibility i'm just going to use dot features from the model so let's see uh what that looks like so i go to serious next here and it has this model and it has a dot features so it's taking it's running it from layer 0 then 1 2 3 4 and then you have the then it returns some features and we are going to use those features we are not going to use the outputs so uh we also need to change this to adaptive pooling uh okay let's see let's see what we have we have the layer four and here you see the fully connected uh from layer four not this one but this one here it returns two zero four eight outputs and that's why we uh in this one we do two zero four eight comma one and then we define a forward function so forward function will take batch of images first of all self and batch of images and then we can calculate the batch size batch size then channel height width is image dot shape and we say okay now we have our first feature which is self dot model.features and then there goes the image batch of images and we have uh then we need adaptive average pooling uh otherwise model size might not be supported so f dot diff torch functional from torch okay and then we have did i import it wrong or what um [Music] adaptive let's see oh yeah from charge dot n n number functional as f and then 2d and then you specify x comma 1 your axis now you need to also reshape it to your patch size so x equal to x dot reshape batch size and -1 um so you have that and you can now pass it through the output layer so output will be uh self dot out and x okay and now you can you can return output okay so now you have you have your model right now all you have to do is create a um training script so you have to create a lot of things training and engine and data set and all that stuff but i've i've done something that can make your life easy so i'm writing the most used functions and classes in a library called uh well that's fantastic machine learning and from here you can just import the engine so the idea is provide the flexibility of pi torch so you don't you don't lose anything that's in pi dodge and just have some helper functions and that's it so helper functions i mean like you have there are some different types of engines different types of loggers different types of data loaders different utils so like uh like that and um then it becomes a little bit easier for us to implement everything uh in one go so if you have not seen my previous videos then go and see them and i have implemented all of this and i will show you how i have implemented the engine and data loader here and which is very nice and it always works so there are only few things that you need to take care of training data path so we specify some kind of path to training data so home workspace and put this part is going to be huge um jpeg train two to four so i have resized them to size of two to four cross two to four you can do that too um okay so that's my training data part so i've created a new function called run or i can probably just call it train and that's my part and now i read the csv file or i can just to check this one read csb train train fault dot csv so this function takes an input argument fold uh and now i can define a few things like we will use gpu so devices cuda and uh we can define a box 50 bucks train batch size 32 valet patch size 16 all right so we have that and now we have to find the fold here so fold is just an integer so when fold is provided uh like when fold is zero your validation for validation k full value uh inside the data frame is zero and um you you sorry you have to take the validation where um k-fold is zero and everything else is training so tf so we can write like dftrain df valid so your train is df df dot k fold not equal to full and we just reset index and drop it okay we got this one and similarly we have df underscore valid but here this is equal to four okay um now what what is the next thing that you do is uh like you have to uh normalize the images so for normalization of these models you have to like take the mean and standard deviation and then using those normalized rgb values so let's do that i specify i can i can just do it here i can specify mean so for this model mean is 3.485 0.406 and standard deviation is 0.229 okay and to normalize you can use albumentation library simple so let me just import that okay so i'm going to create an augmentation and using implementation where it always normalizes i'm going to call it train augmentation because we can have different training and different valid variation augmentations right compose and here i'm going to do normalize uh mean std max pixel value which will be two five five and always apply to true so this is my training augmentation and similarly i have validation augmentation so it's the same for now later it might change okay okay so we have some training augmentation we have some validation augmentation and then we can uh so now we will use this uh well that's fantastic machine learning so because it already has everything it has a data loader that we have built previously so we're just going to use that so we go to data loader images loader dot pi and here we have a classification loader which is which takes two inputs takes the path of the image and it takes the target that's it and you can use it for any kind of uh classification image classification problem so i'm going to do here from dot image import classification loader okay and now classification loader takes part of image so train images we have dftrain dot image name dot values i'll just convert it to a list okay probably it's not required to convert it to ls but let's let's just do that um it's a list of images so train images is my training data path which is defined okay where is it where is it training data part right and comma um i plus i plus dot jpg so whatever the image name plus dot jpg for i and train images so train images if you have not seen the training data okay i can probably show you so it's train data.csv so data.csv has a image name which is the name of the image inside the training folder so not a big deal there image name and target in the end so that's all we are interested in so using the image name we create this list of image paths and then we have train targets which is okay now similarly what we do is we create validation stuff so i will take this one and i will replace everything with valid but we did one replacement which is not required so validation images also come from training data path right so we have training images we have validation images training targets and validation targets so now we can create our data loader object dataset this classification loader and classification loader takes image paths so my image parts is um training parts train images sorry train images and um then it takes targets targets is train underscore targets um then it takes resize which is a tuple but in this case we have already resized the images so i'm going to put it none and augmentations augmentations is trainog boom done so we got classification loader for training dataset and similarly we have for validation dataset let's let's fix it now okay and now we can uh create the data loader so to import torch yeah so now we create the data loader and uh train loader will be torch utils dot data dot data loader and here you have train data set and batch size this train batch size and shuffle will be true num workers i don't know four okay so now you have the valid loader here and you have to remember that you have to keep shuffle to false here because we are using this fantastic machine learning library and it it will uh if you if you provide trouble true then it's not going to return you the target so you have to use your own targets that you have all created earlier to calculate the score any kind of metric loss you can calculate fine that's okay if you're just interested in loss then you don't need to keep shuffle to false but we are not just interested in loss so now we do now we have the model model is serious next pre-trained image net which is fine and then we to device we have the device defined right yes dude goes to the device cuda in our case and now what we do is uh we specify an optimizer so nothing really very complicated here and everything that you've already used adam optimizer and we do model.parameters and learning rate is we set the learning rate to be one e minus four and we can also have some kind of scheduler i'm using reduced learning rate on plateaus when it's that is reduced learning rate so we can use same optimizer we can use a patience of three threshold of 0.01 let's just remove that and mode is max now why is more max because we will be using this scheduler [Music] on auc so which is the metric of the competition and we can also have to make things faster we can do we can use apex now if you don't know about apex just use it for now i will probably make i i'll probably have to make another video to explain epics so from x simple amp it's from nvidia if you don't have it just skip this part amp dot initialize because it's an installation is not a very straightforward it's actually just two lines of code two lines uh to run but yeah i'm not going to explain it in this video so you have model you have optimizer and you can provide some kind of optimization level so let's do o1 and make verbals to zero opposite to zero okay so apex is used for mixed precision training so train a little bit faster and without occupying a lot of memory and then we need uh some more things like we probably need early stopping so this library is amazing it doesn't have a lot of stuff so whenever i find something interesting i put butcher with proper credits and everything so this is one of the functions modification of what you have seen probably at several places uh uh modified it a little bit and i use this early stopping so i can just go here and do from [Music] dot utils import early stopping and it's same as my previous videos so yeah same parameters kind of thing es is early stopping and now we have um we can specify some early stopping criteria so we can we can stop on reduction of flaws we can also stop on uh increase in aoc so whatever if aoc doesn't increase we can stop there so let's do that so i can do like with the patience of five so let's leave some time for scheduler and mode is again max okay so initialize my early stopping too so now for epoch and range of efforts we train and we validate and we early stop and then we are done right but now the main part comes so you have to design a function uh some kind of engine to train the model for one epoch and also to validate the model for one ebook so we go back to our fantastic machine learning library and we see that okay there is also engine and if we go to engine uh there is a very general engine obviously it's not well commented so what i'm going to show you today you can use it uh like that and you can also try to explore it if you have if you have any troubles you find some bugs feel free to send a pull request and so i designed a simple engine which has a data loader a model optimizer and a device so these are the things that we always need and it also has a scheduler probably uh sometimes you need a scheduler inside so which steps after each batch of training right but sometimes you need a scheduler which steps after every epoch of training and you also have accumulation there is also like you can use tpu if you want and uh fp16 using apex so so you have everything here so the only thing that you need to take care of now is uh how is the data going to the device and how is going to the model so if you are using so you can use your own obviously you can use your own custom data loader but if you're using a data loader uh any any custom data loader so let's see the data loader that we used uh i can show you from here no the data loads that we use the classification loader looks something like this it's not very complicated quite easy so now what's happening is it's returning two uh keys in the dictionary and that's how your keys should look like so you have you have these two keys image and targets and that's how it should look like in the forward function of the model so you have image and targets so even if you're not using it just keep it there now i will explain you the most simple and easy part so you have the epochs now you can what what we can do is first let's first import the engine the general engine from uh well that's fantastic machine learning engine import engine hey and now we go here and we say okay my training loss you can see what engine returns i'm in process of writing some documentation about it but this is like very general stuff uh it's it has just been made like this so that it's very simple for you so it returns average loss and so we have training loss equal to train and you have train loader what did we call a train loader or something else oh train loader yeah train loader and you have um model and you can have a optimizer and also you already know the device rest everything you don't need except for fp16 so you can do fp16 is true okay and that's your training bar easy simple and still customizable oh my god i've seen so many libraries where people do a lot of things which is not even customizable so i can now do valid loss predictions because if i go to the engine i see that there is also an implementation for evaluate not only that there is also an implementation of predict okay which just returns you the final prediction so you don't have to calculate the loss so evaluate now where does a loss come from it's also very interesting and uh very simple um so evaluate returns the final predictions which is an array of floats uh and losses and it also like if if you have one class then uh [Music] it's one dimensional it's always two dimensional but you have only one element so it can be used for multi-class classification easily so there is nothing that cannot be used for multi-class less equation it's a trend okay so we got this one and now we do engine dot evaluate and we have the we have the same thing we don't have the fp16 thing do we we don't we don't we don't need that right oh you know we have the tpu thing okay um so we got the predictions we got the valid loss and now we can calculate the uh metric so what we do is we just stack the predictions together predictions is a list of lists right so final predictions so we are just calculating all the predictions from model and appending it to final prediction so final prediction is just going to be a list so what we can do is we can just stack the list so predictions is np.v stack and predictions and we ravel it because we have only one value so we just travel it and now we calculate uc so auc is metrics start rcac score obviously it doesn't predict because i need [Music] let's get learn metric from sklearn import metrics okay and you can do here valid targets that you've already designed that's why shuffle has to be kept false and predictions predictions okay now you got the auc and now you can step the scheduler which is our case it's uh reduce lr on plateau step not static step auc provide the ac value and you can print f buck is epoch and auc is asa all right so now you got this now you say if es dot early stopping right now if it's early stopping you just break but oh okay this one es euc dot model aoc comma model comma path of the model so can just do model path we define it now we will define it now so you can just do print early stopping and then you break done everything done and still very customizable i don't know how people work man classification loader okay so we are done with uh the training part and we have created something using something that we have created before a lot of times um um now we can run it but okay we forgot something model path so where do we store the model so model but so let's store it in the same directory here this one i i don't know if i'll be allowed to copy it no it's going worse or okay um deep learning okay so now we have all the training stuff um let's run it let's run it and run sorry train fold equal to zero so that's how you can train for fold zero so now the idea behind like using the strain function with a given fold is simple uh that you can use arguments and so you don't have to run for all polls in one command you can have multiple commands and you can run it save some memory um but what there is one more thing uh that i forgot to do so the engine from this library returns uh if you look at the code it returns it you see that the model should return something and loss so that something is actually prediction so when you go to evaluate it's using the prediction in the train mode it's not using the predictions so uh your model here in the forward function should have the same name as uh the data loader stuff and now you should it should also return loss so loss here i can just do bc with logic slots and we have out output and we also have um targets so now and i know that this is going to show some error about uh the shape which is fine and it's also going to throw an error about the types not being consistent so you can also fix that by just using just just converting the type inside inside inside here inside uh targets dot reshape um now after targets are reshaped obviously so now we can train the model so just to type as out okay so now it returns an output and it also returns a loss okay so now let's train it python i hope you're on the same one yeah python may not it should run fine and that doesn't happen before obviously okay so i have a problem here bs so bad size the a strong typo but that's okay let's see if it runs now okay ooh training nice pretty excited about it okay so let it drain or no let's not let it train and now we write the predict function uh prediction is also it's also quite easy so you don't have this much stuff that you have in training so prediction function is quite simple and what you can do is like okay take one fold and make predictions for it and use the same augmentation as you used for validation so uh i don't know if i should go there okay let's let's do that let's do prediction part two it's not very difficult and it's fun um i'm gonna copy this whole thing and we have and we create a new function just copy the training function create a new function called predict and instead of training data path now i have no no no too much um so we don't have training data path anymore we have test data about and it changes to test so you have obviously also change the test data i mean um resize test samples and everything here remains the same we can say instead of valid batch size it's now test batch size we don't need this we don't need training augmentations we can call it test augmentation and we don't need oh we didn't read the csv file dfsp dot read csv so it's the same csv for test data without the target column obviously so you have start path dot join and no we don't have the path here right okay no problem i can just copy paste this one for now and just csv okay so we have the test augmentation we have test images so let's change yeah okay let's change this okay so we have test images come from df underscore test and i will also include tf underscore test target new column equal to zero everything is zero it's a fake target everything is fake um so we have test images we have df test and test images and not testing data path but test data path test targets uh test areas of of new use but we need them for classification loader though you don't actually need to create so many data loaders it's very simple and we don't need the train loader we just need the valid loader change the name probably test everything is test shuffle falls number worker 4 we have the model and now we need to load the model um weights so we can do here we can write here we have model path right so model path and uh what where did we save here predictions model path and model parts is a folder oh okay so model part should not be just a folder but path to the model so model path comma model i will just do like model and phone number so f dot bin okay so i can just take this one okay and i can what it is model.2 then we need to load the model static model dot load okay that should work so specify the fold and predict and that's going to make predictions for that okay so we have that we don't need optimizers scheduler this thing we don't need early stopping so we can now just like make predictions so we don't need all that stuff and we have the engine from our fantastic machine learning library predictions is engine.predict and you have test loader model and device as device i it just device it's and then should return and p dot v stack uh predictions dot travel okay so we designed a framework actually okay so now what you can do is uh this is your model the series next 50 and uh you can also like uh you can change the model here uh you can do different kinds of augmentations so like if you if you apply some shift scale rotate here to training augmentation probably also apply some flipping random flipping and if you do that and if you train this model for like five or ten folds then you will be getting a uc of around 0.90 which is quite good quite awesome just remember that you don't try to overflight a lot and uh auc of 0.90 means that given uh one image which is which has melanoma and which the ones which doesn't have melanoma it's 90 that probability we will be able to separate them so uh we have built quite a good model in my opinion and uh there's also if you look at the specific uh competition on kangal uh then there's a lot more that you can do because you have the patient info and we have not used the patient info uh just make sure that you see the prairie function i made it really very fast so you just call here you can call predict fold equal to zero and then you have uh this uh well that's fantastic machine learning library and um i'm just putting some nice stuff i find everywhere uh like trying to modify and trying to fix things that you have trying to implement stuff that i've already done so there's also an example of frcnn you can you can also take a look there and see what's happening so it's quite it's quite good i don't know if tpu works right now on this one but i will uh i'll see if uh um when i have time i can try to see if uh tpu works if it doesn't i'll fix it if it doesn't work you feel free to send up a request and you can also install it using pip so pip install uh wtfml well that's fantastic machine learning and now uh we have built this uh whole thing and it's very customizable you can obviously use your own data loader but you have a simple classification loader and you have early stopping and there is one engine which is already implemented that will work with almost all kinds of almost all kinds of data loaders and it's really very customizable code now and it's giving a very high uh auc value of 0.9 and probably you can improve it much more uh you if you use large image sizes like one zero two four rows one zero two four and some network like uh efficient net probably you can improve it even further but i'm not going to going into details there so this is my model for now the model you have to remember to return loss and it's not like now you will be you might say like uh we cannot use augmentations like mix up no we can we can just implement it here if you want to add mix up add it here you don't have to go to data loader to add anything and uh that's all for today let me know how was it how was the video and if there's something that i missed and if there's something that i can improve let me know in comments and also like subscribe and share if you have not subscribed yet please subscribe now and share my videos and uh see you next time thank you very much goodbye

Channel: Abhishek Thakur

Views: 35,979

Rating: 4.9449539 out of 5

Keywords: machine learning, deep learning, artificial intelligence, kaggle, abhishek thakur, how to build model for skin cancer, skin cancer machine learning, skin cancer deep learning, melanoma deep learning, melanoma classification, medical imaging, deep learning in medical

Id: WaCFd-vL4HA

Length: 66min 45sec (4005 seconds)

Published: Sun May 31 2020