Build a web-app to serve a deep learning model for skin cancer detection

hello everyone and welcome back and in one of the previous videos I showed you how to build a deep learning model for skin cancer detection from skin images using transfer learning we used if I remember correctly we used Sierra's next fifty and we built a model we trained the model we had a documentation said we try to improve on the model results and we generated some predictions but that's all with it so we had everything in our local machine and it generated the predictions in the end but creating the model is one thing and building an application to serve some morale is another thing so what do I mean by that is like building an application is better because then it helps you to showcase your project to others so people who are not familiar with different kinds of technical stuff they can still use your deep learning models so in this video if you have not seen the previous video I would highly recommend you to go take a look at it because here I'm not going to train the same model again obviously so in this video I'm going to use flasks and I'm going to show you how you can build this web application where you can upload an image you can save save the image and then you can generate predictions and return it back to the user so let's see let's see how it's done so we have our this API dot PI here and we have so you can see a bunch of new files we have already seen this main dot pi that we built last time we used we use torch we will use albumin taejun's pre-trained morals and WTF ml so we are not going to change a lot of things and here I mean we just need one function that's it so we have we have this new file called API dot PI and we have two new folders template and static so when you're building a flask app you have to remember that all the HTML files go inside templates and static consists of all the static stuff like CSS JavaScript and all the images and stuff so we have the API so we have to think of a few things what do we want to build first of all we need some kind of way to way to upload image upload image we need a function to make prediction on the image on the image we probably in the middle also need a way to save the image so this step this is kind of optional because if you're uploading something you can do you can do your predictions so on the upload it's tough but this is the easiest way that's why I'm showing you like this and what else do you need you don't need to need anything else to function to predict image and then yeah to show the results show the results so do you need any kind of end point for any of this yes like to upload the image you need some kind of end point right so we will start from there and as a new as I said if I'm a noob I don't know how to upload images but you have seen the flash tutorial in which I build the api's like bird sentiment model and it's also very highly recommended that you go and take a look at that video if you have not done yet so what we are going to do is we are going to search for stuff now what to search for okay so we have to build the end point to upload images and we are using flask upload images using flask feyten okay so I I get a bunch of results mmm uploading files with flask uploading images I think image is very specific so what if I change this to uploading files using flask okay now I get a feature snippet and you you have this link from a flask itself so it's documentation and when you read this it says that you need a form tag with encoding type multi-part form data and an input type of file in that form and it tells you ok you can also use the save method to save the file permanently somewhere on the file system ok that's nice that's all we need actually so when you scroll down a bit you see that there is a function or endpoint to upload the file and there then you see there is some kind of HTML stuff which is a as a string which is returning as a string so what we can do is we can take this HTML string from here and put it in our index dot HTML so let's just put cheer and here we have a title we have h1 we have form method blah blah blah and for method post which is very important so it's a post request and do you need this because you are having an input type of file and then there is some name so let's change the name to something else probably just call it image type will be filed what name will be image and well do something okay I think we can call it just predict now to run this one it's very easy so what we do is now we didn't do anything yet okay so let me close the terminal ok so now we import a few things so are we input from flask and pork flask and from flask import render template and we initialize the flask app apical to flask so this is all same as the bird sentiment API model and then we have and point after trout and we say okay goes to slash so whenever you type the web address and so that's what this endpoint is and then you have methods so we need both get request and was request for this one so get us for like this accessing the website and post is for posting stuff on the forum okay and then you need a function name so let's call it upload predict you can call it anything you want and what should this function do now this function should just show me the index dot HTML so for now it has to do nothing else so I can just say return render template render template will render the templates inside templates folder pretty simple and index dot HTML and then you have the name main and you say you run the app okay so we can run it on localhost so we don't need to change host maybe I will change the port so 12,000 and debug to true in the debug is true so it will show us the errors inside the web browser and now I starter terminal okay and I do peyten API taught by so it has started running the service on localhost 12,000 so I can just open that so let me see follow link yeah so now you can see in that we have something so I'm seeing it on Oh zoomed version of it but you have this title upload new file so you have the title upload new file you have this h1 and then you have the buttons so if you click on the button it it works so you can see like you can access your desktop so you are not seeing the upload the file browser but it works so now what we can do is we can and just keep it here like this and now we can make changes in the code and it will refresh automatically so one more thing about render template is when you render a template you can send it argument any argument so let let's say I'm learning it prediction equal to zero okay and then you go to your HTML and to display this thing with prediction equal to zero what you do is you add a new like let's say I add a new H strict okay so let me see I'm not a web developer and s3 okay and here I say prediction equal to the prediction that I got so the prediction that you got you can use something called ginger templates ginger - or in here - I don't know I don't know how to pronounce it well J I and J a - and here I can just write prediction so this is the prediction variable that I am getting so I can also add a line break here okay so now I now I have this so when I should go back to my application I should see some kind of prediction there let's see what we get okay now we get prediction through which is nice so we have built our something we have built something till now now the main thing the main part starts where we have to get the file itself and save it somewhere so um I I want to make this pretty easy for you right now so what I'm going to do is I'm going to say okay there is a global variable called upload folder okay so we have the upload folder and I say that upload it in static so I'm going to do home work appreciate it just okay so this is my folder here and inside this I have static so this is my upload folder so whenever I get a new image I have to save it here so let's import a few more things import OS for the past or join stuff and from flask you also need one more thing called request so from flask import request okay and now in upload predict you change it a little bit so you what you have to get is the image file image file so new variable image file and image file will be request dot files and the name that you provided here so in file you provided image right so that will be your oops sorry image but this will exist only when it's request method is post so I will say a request dot method is post then I have this then I have to grab this image file now the image file may or may not exist but we are not doing any kind of checks here for now we are just assuming that edge it it exists or you can just do a simple check like if image file and that's it and here you can write a little bit more code so what would could be so what we have to do is user uploads an image so it's a post request and and then clicks on submit then it's a post request and we grab the image file request dot from request dot files and if we have the image file then we save it so we can just do image file dot save and save it in a location so image location image location will be over start part dot join and here we have a upload folder comma upload folder comma image file so the name of the image files is stored in file name I think so if you print image files or file name it will give you the name of the image files or something dot PNG something dot jpg and then you have the save method which will just save it to a given location image underscore location okay so you got this thing and I say ok if this is a success then render me the same index dot HTML with prediction equal to one okay we just want we just want to try this first so we go to a Python API dot pi and let's refresh this okay so what do we have let's select one image and click on predict and it showed me prediction equal to one so everything worked now let's save the image in the static folder so here you can see if the image is saved here now so you if you're hosting this on anything AWS or GCP or Heroku whatever so anybody can upload and you can save the image in your local virtual machine so it saved the image then now the next step let me close this one for a while now the next step is to create predictions on this image now credit predictions is it's fun it's not very difficult right it we already did that in the last video so what we need is we need this model that we had created so I'm copying this model and I'm pasting the model here I will change a few things in this model like I'm changing this model self dot model to base model so that's not a major change and I'm changing out to l0 so that's not a major change either because when I train the model locally I changed a few things I shouldn't have but I did so you got the model and now you have the model so let's say here you say model is none and here you can also set device so for us device will be CUDA okay and now the second thing that you need is the predict function so you have the predict function here let's take everything and we will just remove what we don't need so we have the predict function and we have a chair so what should this predict function do so we had fold so instead of fold we don't want fall we just want a image path just one single image so now we don't have all these things here so we just delete it do we need the device no we don't need the device we don't need test bed size we don't need the box mean and standard division is something that you can also put it globally I'm just keeping it here for now and you have test images so test images is a list of image paths so which will have only one element and test targets is also a targets for test but we we don't have that so I'm just going to delete it and say test 0 now what do we have we have a classification loader which comes from WTF ml and we have test data loader we have number of course is equal to 4 we don't need multi processing here because we have only one image and we can change the batch size to 1 and now we have the model we have the model 2 device and all that stuff so I'm going to delete this and I'm going to put it here so I got the model model dot load state ticked montage dot load this thing and the model name here is model dot bin and it's the same model so I can I have to now change this model to capitals because that's how I defined it earlier and device so now my predict function does not have model in it so I can just add model as an argument so now I should have everything so it has the test augmentations then it has just images just targets the classification loader the torch data loader so classification loader is not actually data loader it's the data set class it should be called classification data set I have to change the name and then you have Engine dot predict and it returns predictions so now what you have here what you can do here is add like predict Fred okay the spread is predict and here you have image location comma because that's where you save the image and model okay and then we can just do a print here print print but we do need a few more things here so let's just grab them we need the imports so there are way too many imports so let me see well grab all of them and paste it here arrange it a little bit not a lot just arrange it a little bit and mmm then we are we should be good to go import numpy and then you have you don't need pandas we need this store Staton and Orton [Music] we don't need epics we don't need metrics we need functional obviously and then we need WTF ml okay we can just add it here and we need to know DF EMA which goes in the and so it's basically alphabetical and yeah if it's OS and torch you import them before everything else and we don't need an early stopping so I guess this should work maybe it will throw an error but we will see so we start the terminal with Python API don't cry I'm also going to do one more thing because since now I'm using GPUs so I'm going to say CUDA visible devices equal to zero so I have two device one of them is busy doing something right now so I want it to be on device zero ok invalid syntax okay no no problem I think we have one extra bracket and let's see it's trying to do something and what's it trying to do by taking time because it loads the model and that's also very fast it's not slow and then you then have browsed for one image and I can click on predict okay so the name device is not defined obviously the device is not defined so it should give me in line 79 predict okay so let's see where we have device so we have device here device here device here and here it should be capital okay so now it will reload everything in there - it's going to restart and let's see what happens let's see okay now we click on browse select one image and click on predict so it shows prediction equal to one but if you look at the console so you can see the prediction here and it has generated reproduction one thing that you must note that in your forward function you did not use you did not use Sigma right here right so what we can do is we can add a sigmoid on on top of this so I think it's just after sigmoid so we add a sigmoid here actually we don't need to calculate the loss right so we will just return loss equal to zero because it's test anyways so let's see we go to Python API dot pi and and then we select the same image click on predict so it was 0.33 now 0.5 it okay so now you see that now it's a probability between 0 and 1 so if it has in cancer then it's 1 so now we got everything and instead of you see this this is a array right so we only need the 0 at element the first element and we don't need this and instead of prediction equal to 1 we can now return thread so now you will have prediction displayed here awesome you build everything that you need but this obviously this doesn't look very good it's like ok whatever you don't even see their image nothing nothing nothing right so to now there are many many different technologies that you can use so you can just build an endpoint here and let it return a prediction and then let your front end front end handle everything else so you can write a front and a back end another back end and then which communicates to this endpoint and sensor image and gets the prediction but obviously I'm not a web developer so I cannot do that what I can do is do some simple things so simple things will be like ok when I'm when I'm building these kind of web apps I use something called bootstrap ok so what is good bootstrap it's the like yeah it is one of the most popular libraries in the world for HTML CSS and yes so you have all these CSS files and JavaScript files which are already there so you don't need to make few changes and then you're done so what you have to do is go to get bootstrap comm and click on download and you download this click on this download button and it gives you a sip file with CSS and GS folded so you take those folders and you copy them in static and that's what I've done here these CSS and J's folder come from bootstrap and now we search for something else so we go back and we go to bootstrap examples so what are the examples of websites being built with bootstrap so you see there are many different kinds of things you have some kind of landing pricing page there is a form there is something else but this one I like the sign-in page so I go to the sign-in page and this one looks good instead of probably instead of filling email address and password I need to have an upload button so what I'm going to do is I'm going to see the source of this page so when you see the source you see that it has a lot of lot of stuff some stuff that we might not need some stuff that we need so I'm going to do is copy everything and I'm going to go back to my index dot HTML and let me close this no let's keep it open and I'm going to paste it here so I got a lot of stuff from this bootstrap example and a lot of things I don't need so I don't need this language thing mmm so probably I need the Cassatt I think this is fine you can have a description and author we don't have that we don't have a small icon thingy and we have a title so let's change it to skin cancer detection we don't need this thing here we need the CSS file so you see that they accesses from just CSS bootstrap mindo CSS but we have already copied it to static so what what you have to do is to change it to slash static slide CSS okay and then it has a sign in dot CSS now if you take a look at your CSS files you won't find sign in dot CSS so it's a custom CSS file I'm not an expert in CSS obviously I always have to ask others but so this when you you have to see this one so you click on sign in CSS and it shows you the CSS file so and what I'm going to do is I'm going to copy this one so it's the sign-in form so you see this class form sign in but I'm going to change this to form upload okay and then I'm going to say it's in slash static / CSS slash form underscore upload dot CSS not from 4 mm okay now I go to static CSS and then I create a new form underscore upload dot CSS and I copy paste everything now it has a lot of things and in the beginning you might not understand all of it but it's okay don't worry about it so mm you see that there is a input type password there is an input type email I don't need all that stuff because I don't have my email and password stuff rest everything I can keep but then there's this name form sign in and I'm going to change this form sign in to form upload okay now go back to your index dot HTML so there is a image that you saw we don't need that image so I'm going to delete that instead of please sign in I can say please upload please upload image and then you have this email address right you don't need that either so I'm going to do this and input type now the type is file as pre we have seen previously I'm going to call it image we're also going to add a name like this like we had before name equal to image class the same placeholder is not required because it doesn't have a text field and then you have required auto focus that's fine it's that we require it and focus goes on this when we open the page we don't need the password we don't need this password thingy either then you have a check box or remember me we don't need that and then we have this button right so all these the custom glasses come from bootstrap so they have done a lot of work to make it easy for you guys and for me also and then I change sign in to predict I don't have a copyright so we did all that stuff and now we try and refresh the page so now you can see that we got something here since everything is pretty messed up and what we are going to do is we also need the prediction part so I'm going to take this predict part from here and add it here okay mmm-maybe I'm just going to use this class maybe I'm I'll just I'll just use this class so it's going to be same kind of style do we need anything from this part now so we got input type filename image and summit values and stuff we don't need anything else from here and then we can save this HTML file and I go back to this one and i refresh it okay now it looks pretty good right and let's see if it still works so I go and browse and select one image and you click on predict then it didn't work okay no problem it didn't work sometimes things don't work okay so let's go to API it tells me that okay yeah it is duplicated I knew that so I should just use torch dot Sigma here what else so it's probably not getting the image so why it's not getting the image now you have to go to the HTML and see why is it not getting the image so let's see did we mess something up okay yeah so one of the very important things that we messed up with this part so we need to have that so form class this that's fine and method is post and you need the encoding type because it's file okay I hope it works now let's try okay so I go select a file click on product I'm getting the prediction but how to make it even better in terms of how it's looking right now is you can also show the user the image right so if you remember you had this image source stuff so let me see here you have this like image class blah blah blah it shows this image so what I'm going to do is I'm going to copy this and then I'm going to add it have a break and you display the prediction and you add the image here but the image is not always going to exist right so if I if I just run this thing what will I get I will get this B image here but this is not my actual image file so I also need to okay so I also need to provide an image inside the API so here I can say image image file is none so when you render template without the post request image file is none but when you render template with the post request an image file or I can say image lock is image location and ok we will use the same image lock okay with ginger too you can also write if statements inside the HTML okay so what we do is we have this thing curly brackets and you write percent and if image lock so if image lock is not none and you can end it by writing and if it's not very difficult so percent and if so now you have this image and other stuff and I can put it here so anything that you put inside if else you can put HTML inside this if-else thing so if I have image lock then it will show me the prediction otherwise it won't and I go back here refresh it's not running for some reason okay let me run it again okay and then we refresh the page so let's refresh and then I select an image so okay one thing you saw now it doesn't say a prediction equal to zero because we don't have the image and now we select the image click on project and it tells me prediction is this and it shows the image which is not there right now so where is the image the image isn't static because that's where we saved it and you already have the image in this variable called me clock right no it shouldn't be image lock it should be image name because it's we are saving it in static but we don't need the image location we just need image file dot file name so let me fix that and now you have I think you have everything that you need you fix the width in height to let's say 256 256 and go back to your app refresh it okay so now you're getting the image so I select another image click on predict and you're also getting the prediction so that's basically it for your first good looking web app and you can so you can see this list looks pretty good so you have you have a upload button or browse button to browse images and then you have a break button and it's showing the prediction so it's a little bit kind of I would say responsible people are going to hate me for that but this this is a simple way in which you can create your own web application and if you use bootstrap you can combine to tear flask endpoints and ginger to there's so much that you can do and you can build really complicated apps using this but I would prefer using this only for some kind of prototypes and then move to other web development technologies and like ask your web girlfriends what you can use and I'm not a developer so I'm not very much familiar with everything there so that's basically it for today's video and I hope you like it and if you have any questions let me know in the comments if you have not subscribed to the channel please consider doing it and like the video if you like this tutorial and let your friends know so share it and if you have ideas on what should I create tutorials on so if you have if you want to give me some ideas there feel free to do so in the comments section or you can just write me a message on Twitter and yeah let me know how was it and I'm trying I'm planning to build something really cool around this and the upcoming videos so stay tuned and thank you very much goodbye

Channel: Abhishek Thakur

Views: 40,021

Keywords: machine learning, deep learning, artificial intelligence, kaggle, abhishek thakur, how to use flask for web application, how to serve deep learning models in python, pytorch in production, using bootstrap with python, flask and bootstrap, skin cancer detection with deep learning, building an app for skin cancer detection, melanoma deep learning

Id: BUh76-xD5qU

Length: 42min 32sec (2552 seconds)

Published: Sat Jun 13 2020