Tutorial 9-End To End ML Project-Deployment In AWS Cloud Using CICD Pipelines

hello guys my name is krushnayak and welcome to my YouTube channel so guys finally we have developed our end-to-end project now what we are going to do is that we are going to deploy our project and recently we don't deploy it trust me in the interviews right when you talk about the project when you talk about CI CD pipelines when you talk about what are code pipelines how what all deployment techniques you have specifically used so what I'm actually going to do is that going forward we will be discussing about different different deployment techniques in this video we are going to see how we can probably deploy our entire application into AWS Beanstalk elastic bin stock and over there also I'll be using some code pipelines also and again first we'll go with the easiest one and then we'll probably catch up the most difficult one you know and I'm also going to make sure that I use multiple Cloud platforms so that you get extensive knowledge regarding this okay and yes if you're happy with this particular series please make sure that you subscribe the channel share with all the friends share these videos with all the friends trust me Guys these are something very important that can actually help you crack interviews right so uh let me go ahead and let me just see over here first of all what all configuration changes we really need to make so already we have discussed about creating our prediction pipeline everything is working fine that deployed the code now here if I probably talk about two important configuration we need to set up whenever we are working with uh elastic Beanstalk you know which is a kind of instance that will be provided to you where you can probably deploy your entire application so that extension is that dot EB extension so I have created this over here and inside this I have created one python config file the python config file is mainly to tell to the elastic bin stock instance saying that what is the entry point of your application right that is the most important thing that what is the entry point of your application now over here by default when I was searching in the elastic bean stock documentation page they had given this kind of configuration for that right so it was option setting AWS elastic bin stock container that we are going to use is python this is specifically for container as python if you are using Dockers for that there is a different process right now we are not into Docker Dockers will discuss as we go ahead right now here and I'm this is the most easiest technique of deployment right so over here you have wsgi path with application now this application should be your app.py name okay the the entry point of the file and inside this that is the reason I updated the code to application right and it is compulsory that inside this python.config you give in this format only that is application colon application but the problem over here is that I don't have an application.py file right so what I will do the same application.py code I will copy it and I will create a file over here just for my deployment purpose which is application dot py right and I will paste it over here same thing okay so this is what we are doing and I'll also be removing this debug is equal to true because while we are deploying we have to always make sure that we remove that now these are just the two settings that is probably required to do the deployment okay now how do you go ahead so first of all we'll go ahead and get commit this entire code so let me quickly see this and write git commit dot ok so two things have been added manage.eb extension and whatever file is present inside that and one is application.py so after this what I will write I will write git commit okay minus M and this will be deployment configuration that we are basically writing okay so deployment configuration okay so here it is ah so your branch is up to date committing is done get commit file on track files okay first of all did I add it or not get commit okay so let me just clear the screen I think I have to I have used git commit so I'll add git add dot okay and now if I probably see my get status so these are the new files that are basically getting added now we will go ahead and do the git commit so here I will write get commit minus M and this will basically be my deployment configuration right so this is the deployment configuration for AWS done so this is done now the next thing that what I'm going to do clear the screen okay CLS clear the screen and one final thing that I really want to do about is that I will try to push this code minus U ah minus U from origin to main so once I push this code you'll be seeing that my entire code will go over here uh if I probably try to see my repositories okay and this is the ml project repositories now you'll be able to see this extension file has got created okay now this is just the requirement and inside this I think python config has also been created so yes it has been completed now the next step what we will do we'll try to deploy this now please make sure that you follow this step by step now first of all you need to have an AWS uh Amazon account AWS console account so that and obviously for this credit card is required the first thing that you will be doing is that just go ahead and search for elastic bin stock okay so here you can see elastic bin stock it is nothing but it is run and manage web apps so inside this I will just go and click on applications and these are the application that has been created uh what I will do is that I will go ahead and create the application over here so I will just go back and I'll create the application now let me just give you one example what exactly we are going to do okay with respect to the deployment and I hope nobody has actually taught you with respect to writing also on the deployment part okay so with respect to this deployment you know as I said we will be using elastic bean stock now this is my elastic bin stock okay elastic bin stock now what exactly this is this is just like a server or a cloud environment of some instance right instance basically means this can be a Linux machine right and this will specifically be a Linux machine now here you can probably create an environment you can deploy your code okay and this is basically present in the AWS okay the next thing that is there is basically your GitHub repository right now my code is basically there in the GitHub repository right the only thing that is this elastic bin stock requires is the configuration that is there right so configuration you have actually set it up with that extension file and you have created that python.config file right so this too has been created over here now with respect to uh the deployment part uh over here you already have the GitHub repository in the GitHub repository you have the code right and this configuration has already been made over here in the GitHub repository now if I want to do the deployment see over a requirement.txt everything will be there if I want to probably do the deployment from here to here right I have to definitely create something right my code should be able to go with respect to this any changes whenever in GitHub repository it is happening right so this specific thing we basically talk about uh we in order to create this pipeline right we can use an amazing thing which is present in AWS itself which is called as code pipeline now what this code pipeline is going to do is that whatever code is over here it is going to basically commit or deploy it automatically as soon as you click a button inside this elastic bin stock which is just like a Linux machine right uh so in the GitHub repository you have the code Whenever there is any code changes automatically one button will get created do you want to deploy this code so once you probably click it you will probably deploy the code in the AWS bean stock elastic bin stock and then that deployment will get completed so this pipeline that you see is basically uh it's a kind of a pipeline and this pipeline we say it has continuous delivery pipeline okay continuous delivery pipeline right now this is the pipeline that we are also going to create it over here so two thing first step we will go ahead and create my elastic Beanstalk instance which can be a Linux machine here we'll also create an environment right and do the setup next thing that we are going to do is that we are going to basically create our code pipeline now through this code pipeline will integrate with our GitHub repository and then we will continue this deployment over here into the elastic bin stock so this is the process that we are basically going to follow now over here first of all we will go ahead and create the application application name can be anything I can basically write student performance right student performance done oops I'll destroy it student performance so this is the application name that I'm going to create over here by default you'll be getting a platform don't don't put any values in key value pair so in the platform I will go ahead and select uh Python and here by default python 3.8 is basically there and obviously I've I've actually developed my entire application in Python 3.8 okay now here you have two options sample application one is uploaded code in upload your code you can probably upload a zip file from your local or if there is a S3 bucket you can probably put it okay but right now I'll just go and go ahead with sample application because I need to integrate with my GitHub repository that is present over here right so I will go ahead and create the application now as soon as I create the application it will uh automatically create one environment also see creating an environment by default environment is basically getting created and it will probably take some time okay so till this setup is done see now we have done this particular part right it is a Linux machine we got and this particular machine is getting set up and environment will also get created now we have to go and focus on creating a code Pipeline and integrating with the GitHub repository so for that I will just open AWS again okay and here I have to just search for something called as code pipeline now code pipeline is nothing but really software using continuous delivery as I said it is for continuous delivery I'm still not focusing on continuous integration that CI CD pipeline we say right we are just right now focusing on CD pipelines later on we'll go and see different techniques with respect to how to create a CI CI pipelines the CD pipelines will automatically create a cicci pipeline also okay so uh I will go ahead and click on code pipelines so code pipelines once it opens it looks something like this okay and I will go and probably create a pipeline okay create a new pipeline now let me go ahead and probably write the pipeline name so the pipeline name over here I can basically give student performance okay same thing and then over here you have advanced settings don't do anything for the advanced settings just go ahead and click new here you have a source provider now as I said we want to integrate code Pipeline with GitHub repository instead of this GitHub some companies may be using gitlab they may be using some other repositories like TFS and all right so here we need to integrate with that right it can be Amazon S3 also but here I'm going to just write GitHub version one okay now it will ask me to connect to the GitHub so my local GitHub whatever it is I'll allow it to authorize so I'll just go ahead and confirm it okay and this is confirmed now after this I will go and select my repository Now what is my repository name over here if you see ml project right so I will go ahead and write ml project so this is just to connect with my GitHub right and Branch will be mean now uh get this GitHub web hooks will be already there we don't have to change this to code pipeline uh later on I'll also show you how we can do with AWS code pipeline But Here by default AWS code pipeline is only going on okay next we click on next over here there is something called as build provider let's say you want to build your project before uh for creating artifacts and all right so for doing that any steps if you want to do that you can probably use build provider but over here uh here do you have two options one is code build and Jenkins right now I'm going to skip this stage later on we'll also try to keep this okay so here it says that your pipeline will not include a build stage it's fine skip it now the deploy provider is super important now what is this deploy product where I want to deploy from my code pipelines right I want to deploy from my code pipelines to AWS bin stock right now this is the step right I'm basically doing the configuration now this code pipeline needs to deploy in the elastic bin stock right so that is the reason we have selected this what is the application name here we have created student performance what is the environment with respect to the student performance environment and the next uh we'll just go and see whether this environment has got created or not you need to wait till till this entire environment is basically created okay now in just to understand in the elastic bean stock if you've probably seen the application your student performance is ready and that is the reason now we can go ahead and deploy it so once I click on next now this is your entire pipeline setting this diagram that I had drawn right it is basically talking about that same thing okay over here you have your pipeline settings you have your action provider action provider basically means where is your Source right adding The Source this is a pipeline setting with respect to all the default information about your elastic bin stock then when sorry not elastic bin stock your pipeline name and all and then finally here we have skipped this build action but finally we are going to deploy this in AWS bin stock that is what we are able to see now as soon as I create click on create pipeline this will probably take some time to get created and once the pipeline is basically getting created right now now you'll be able to see that step by step everything as I said my GitHub will get connected to code pipeline so that is where this is getting connected now my code Pipeline and GitHub will get connected and then we will try to deploy that in the AWS bean stock so that is what this diagram actually says right you can probably see this diagram right and this is a very important diagram altogether now it shows that the deployment has failed let me go ahead and check what is the problem with the deployment okay so guys we were getting some error but now we have I fixed that you know again I saw the documentation page and all that app.py file was there right I have deleted that file and just by deleting that file right AWS Cloud they were actually able to deploy this okay in the bin stock and I think the main reason is that see in the configuration right we have written application dot py but if they see app app.py probably they find some release problem so please make sure that you delete the app.py and instead just keep application.py which has the same code okay now you can see the deployment is done I will just go to probably the link so just click on this particular link and here you go to the environment and this is the URL that you will be able to see so once I click this URL now you are in the home page you can see that I'm home my home page is running and finally I can just write predict data to just see that how everything is working so guys finally we have deployed our application into the AWS uh Cloud that is an elastic bean stock uh again you can probably do the prediction you can probably use it this is just one way of doing the deployment still there are a lot of ways because we also need to discuss about the CI pipeline we have completed about cacd pipeline right the code pipeline itself now probably if you go and see in my GitHub right this entire build what is basically happening everything will be you will be able to see it okay so if I probably go and see in the repositories right if I go into the ml project the same repository that I have given you okay so here are all my files okay and uh pull request is also there you can see okay this pull request is fine issues is fine action if I see with respect to action ah okay nothing has happened over here because since we have not again created the CI pipeline we have just created the CD pipeline based on this configuration now as soon as we create a CI pipeline what will happen you'll be able to see everything in this actions itself here right as soon as I push it and we'll be able to do the deployment now let's say that I just make one simple changes right let's say in this requirement.txt I will just go ahead and add one simple Library the library can be or I'm just added one extra space right and I'm committing it okay now automatically when soon as I commit it now you see over here it'll ask for the release change now see again that update it is being able to understand right the source with respect to the GitHub now if I go ahead and do the release chain the deployment will automatically happen and over here the from The Source then again the deployment will happen with respect to all the new code that has happened now obviously I have not added such so huge changes the way I've just added one extra blank so uh with respect to that that changes only I'm deploying it again now it is in the deployment phase and it will it will get automatically deployed so I hope you understood how to do the deployment with respect to AWS bean stock so yes this was it uh I will see you all in the next video where we'll mostly discuss about CI CD pipelines and all right so yes this was about the end-to-end project I hope you like this particular please make sure that you subscribe the channel share with all the friends I'll see you in the next video thank you take care have a great day

Channel: Krish Naik

Views: 54,021

Keywords: yt:cc=on, ml application deployment in elastic bean stalk, data science with project with deployment, deply ml application using cic d pipelines, krish naik end to end machine learning projects

Id: gbJn2Ls2QsI

Length: 17min 56sec (1076 seconds)

Published: Sat Mar 18 2023