Deploying Machine Learning Models with mlflow and Amazon SageMaker

hi everybody this is Julian from AWS in this video I would like to show you how to use ml flow to deploy machine learning models on Amazon sage maker and my flow is an open source library that lets you track and organize your machine learning projects and you can also deploy to different locations ml flow supports all kinds of frameworks all kinds of algos so that's pretty cool and I'm going to use XJ boost here let's get to work first I need a data set of course and I'm going to use data that I've used before for direct marketing so it's a very simple data set that you can easily download from from the web and it has about 40 1,000 samples 20 features and the label telling you yes or no if a certain customer has accepted a marketing or honor so it's an easy one and and we're going to use XJ boost to build a binary classification model so here's my code to to load it basically just grab the file from the web reading as a panda's data frame do very basic processing here just one heart encode categorical variables and then basically split the data frame into features and labels and then split the data set for training and testing right what about those lines well there are ml flow api's that lets you log all kind of information you know pretty much anything you want on on your experiment so we'll see how this is useful when we're training the model okay but as you can see i'm logging the path to the data set and its shape and some of the splitting parameters and i just flag the fact that i used one of encoding on categorical variables okay so that's just my loading function here nothing nothing weird now what about training so like I said I'm gonna use extra boost and the first thing I'm gonna do is create an experiment in ml flow and an experiment is just you know a project pretty much where we're going to store information on as many training jobs as we want right before I do that I need to start ml flow so we can easily do this start the UI okay and then I can just open that window here okay all right so that's that's a local UI but you can also use a remote server if you want to okay so now let's look at the code well the code is very straightforward so start a run inside that experiment load the data set okay that's the piece of code we just saw and then basically build an XJ bus classifier using the AUC metric training scoring printing out the metric logging that metric 2mm flow and then logging the model to ml flow and we'll see how this is useful basically saving the Train model so that we can deploy it later and then end the run okay so let's just run this code okay creating a new experiment loading the data set and training it should change for just a few seconds okay and we see a you see is 91.7 now if we go to the UI here and reload we can see our training job here I can see the parameters that I stored okay when loading the data set I see my metric okay and that's you know it's not really graphing here because I loved it only once but imagine you were logging a metric after each epoch or something you would you would see graphs right okay and very simple job but I can see this and if I run it again and I'm going to see different run and I could try different hyper parameters and I could keep track of all that stuff inside the UI so that's pretty neat okay okay now we have a model so let's see how we can deploy it we have actually two options one option is to deploy locally and that's one of the cool features in ml flow or we could deploy to Sage maker okay so let's do both of course let's start with local deployments if we take a look at the ml flow documentation we can see of course we have sage maker api's and we have API is for all operations including local deployment okay but there's also a CLI that we can see here okay so you can use the one the one you like best I'm gonna use the CLI and I could check that CLI ml flow sage maker help okay and that's the run local command that I want here so what it needs is basically I prepared it it needs the model path and the local port okay so I defined some environment variables to make our life easier here okay so I'm gonna use that's the path to the model I just trained I'm gonna deploy locally to port 8888 and then I just need to run this command here okay okay and we see that the model is being deployed locally and it's downloading the required packages according to my training script all right so let's just pause the video for a second and I'll be backing okay so the model has been deployed locally so now the only thing I have to do is just load my test set grab the first ten samples and use the request library to HTTP POST to that local model okay then this is really standard Python courier just posting in JSON format and printing out results okay so let's run this okay and I can see the ten predictions for this model okay and of course these are probabilities between zero and one because we built a binary classification model okay so local deployment now what about deploying to Sage maker endpoint well as you know we need a container for this ok we need to build a prediction container that gets pushed to Amazon ECR the docker registry service and this is what stage maker will use to create the endpoint and load the model and building that container is not always easy right fortunately ml flow provides a super simple way to do this so again you can use an API or you can use the command line I'm gonna use the command line so the first thing we need to do is to build a container because as you know stage maker is based on docker containers and so we need to have a container that gets deployed to an endpoint on a fully managed instance and inside this container sage maker is going to load the model ok so how do we build that container well it's actually super simple okay the only thing you need to do is to run this command okay ml flow sage maker built and push container and what this does as the name implies it's going to build a Python environment inside a container it's going to push it to your one of your ECR repositories and and you can just use that one to deploy to at the end point okay nothing nothing more complicated so I've run this before so this is going to be super fast if you run this for the first time it's going to take a few minutes and you only need to do it once okay you don't need to do this every time because it's it's a vanilla container where your specific libraries and requirements will be installed as you deploy the endpoint okay so don't run this all the time it's not needed okay and I should be able to see this repo so let's just describe it okay using the describe images and I can see ml flow created an ml flow Python Creepo and it pushed this image inside of it okay so again you just need to do this one okay and then you're good to go so now we can move on to deploying and deploying again is a one line thing okay you'll only need to do this ml flow H McCree deploy the app name which is really the endpoint name the model path the role the I am role so you can just use your VAR n or your sage maker role and of course the region you want to deploy okay so I can just run this now okay and here we go and this is going to take a few minutes so let's just pause the video and I'll be back after a few minutes the endpoint is up so let's check the sage maker console yep so I can see the endpoint is here seems to be okay well not much happening for now I can see by default this is deploying to m4 excel I didn't specify an instance type when I deployed the endpoint of course you can you can past extra parameters instance type instance count etc etc okay and if we look at the log well we see we see packages being installed right just like I said this is a generic container and it will automatically install using count the packages that are required by the trading script so that's pretty cool you don't need to mess with that and then we see Guney corn start ending and we see the health check etc except it's all good okay so now let's predict and in order to do this well we're I guess we're back to say to make your business as usual grab a bottle three client described the endpoint check it's fine load the test set once again grab the first ten samples and then send them to the endpoint in a single request and print results so let's run this bit of code and then we should see predictions yep so we can see the endpoint in service and once again I get my 10 predictions okay so this is really the exact same code you would use on sage maker because it is a sage maker endpoint so there you go this is how you this is how you deploy models on ml flow and what I like about it is you can work locally as you saw I used PyCharm and docker and I was strictly working on my laptop here debugging my script checking at the different experiments in the ml flow UI running and testing locally and then when I'm happy with the model I can just deploy that to manage infrastructure on Sage maker with the minimal fuss the container is taken care of automatically no matter what I use I will end up with a proper container on Sage maker creating the endpoint is super simple so I think that's an interesting that's an interesting combination and of course you can do hundred percent of this using api's I use the mix of api's and CLI but if you want to automate completely those workflows then stick to the Python API and you can automate that stuff very very very easily well that's it for today I hope you liked it and I'll see you soon with another video bye bye

Channel: Julien Simon

Views: 16,655

Keywords: aws, amazon web services, databricks, machine learning, automation

Id: jpZSp9O8_ew

Length: 13min 18sec (798 seconds)

Published: Fri Feb 14 2020