Deploy a Deep Learning model on AWS Lambda using Serverless framework

what's up guys how are you doing today in the previous video we looked at cartoon gun the model behind this application we understood how this model works how it's been trained and then we developed a small python script using python and by torch to test this model on some local images today we deploy this model on aws using serverless architecture without further ado let's do this [Music] let's now have a look at the whole architecture of our project on one side we will have a react client and we'll look at this in the next video on the other side we will have a backend that is deployed on aws these two services will communicate with each other through post requests and for example if you send an image this image will be sent through a post request then this image will be received via the api gateway and once the api gateway receives this request it will trigger a lumbar function and basically a lambda function is a runtime that will receive the image from the client it will then load the models directly from s3 execute the style transformation and then sending the result back to the client through the api gateway this architecture is quite simple and it's also serverless serverless means that you don't have to create and manage servers yourself this is automatically done for you by aws and it has a lot of benefits a serverless architecture is first of all cost efficient if you don't use it you don't have to pay for it the second benefit is scalability for example if your application starts having a lot of users aws will manage the load for you it will allocate more resources so that your quality of service remains the same now how can we deploy such an architecture when i started using serverless i had to go through the aws console i had to i had to select each individual service individually and manage the connection between all of them it was painful and it was also prone to many errors hopefully i won't be doing this again in this project instead i will be using a very powerful tool called serverless framework the serverless framework is a tool that will allow us to write our code directly our infrastructure directly as code by modifying a yaml file before deploying it and you see this is very simple and you can also have more control on the interaction of your individual services this is very nice and you'll see how to do this before using the serverless framework you'll have to make sure that you have created an aws account and also an s3 bucket where you have to put the pre-streamed models and then you'll have to configure serverless by creating an im user so let's do this very quickly you'll have to go through the imconsole you have to add the user like this i've created a user but i will do this just to show you how it's done so i will name it test i will allocate programmatic access i will then give it administrator access then i won't be using any tags and once the user is created you will have some credentials copy these credentials and paste them directly in this command and then once you run it you will be able to configure your serverless account okay now when this is done you'll have to create a boilerplate code for our project so run this command directly on the terminal so once this command is done we'll have a folder called calculify and inside this folder we'll have two files handler.buy is the lambda function and serverless.yml is the file that holds our architecture so let's use it okay let's fire up ps code so as you can see here we have lots of boilerplate codes so we won't be using any of that so i'm gonna just erase it i will only keep the service name and we'll modify it to make and deploy our application before using serverless we'll have to install two plugins you'll have to know that serverless has a lot of plugins by the community and we'll be using two of them the first one is called serverless python requirement it will allow us to manage the dependencies of our lambda deployment package and the second one is serverless plugin warm-up it will allow us to keep our lambda function always available to any request so this will basically prevent the cold start effect so let's install them okay now let's install the second package and we're done now we can start modifying our lambda our serverless file so i will just erase all of this like i did earlier and then try starts to modify now to define our infrastructure as code we'll have to modify five section inside the serverless.yaml the first section is the provider section and basically we will specify here the cloud provider the runtime will be using we'll set also the profile that you have created the region the timeout and also specifies some role statements we will in our case allow the get object action to allow and read directly the models from the s3 bucket and will also allow the lambda function to access to this packet okay so copy paste this first then the second section is the custom section we will be where we'll specify the requirement for our plugins basically we'll have two plugins the first one is the python serverless python requirement we will tell it to slim zip all the dependencies and also to not deploy some other useless packages in order to make our deployment package very light at the end the warm-up will be also customized and we'll tell it to run the warm-up effect on our lambda function every five minutes so that it gets it remains available okay yeah okay the third section is the package section and basically we will tell it to handle the production environment from our lambda by excluding files and the necessary files okay now the fourth section is the function where we define the runtime of our lambda function basically we will first specify the handler it's the lambda it's a function that is contained inside the file okay and then we'll have to allocate the max memory to our lambda specify the timeout and also specify the event that will trigger our lambda in this case it will be a post request directly on the path transform and then we'll have to specify warm up to true because we want our lambda to be warmed up okay okay this is the fourth section and finally the last one is the plugin section this will indicate which plugins we will be using okay now we're done so let's now deploy our infrastructure oh right we can do this because we have to specify our lambda function first and also the requirement of our packages so yeah let's do this we'll have to create a requirement file requirements dot txt inside this requirement we will have three packages torch torch vision and pillow we'll have to make sure that the python version of each package matches the runtime that you'll be using we'll have 3.7 so it's all right and also you have to make sure that the packages are based on a leaks based environment because it's the same os as the lambda function runtime now we'll have to modify the handler so i'm going to just erase it basically the handler is inspired from the function that we have defined in the previous video so i will first start by importing the packages so as you can see we have torch dependencies we have some native dependencies from the os and we have also these dependencies that depends on the lambda runtime environment i will have also to add the folder network inside this folder i will add the architecture of our generator model this architecture you can grab it directly from the github account so don't worry about that and we're done now inside our handler function we'll have to specify first two functions the first one is image to base64 so it gets an image from the client to the pillow image and then this image will be it's a binary image and then this image will be transformed to a string representation and then the second function is load models it will get an s3 object and a packet name it will be used to load the models directly from s3 create a dictionary so something like what we saw earlier in the previous video inside our notebook so i will specify some parameters inside the main thread of our lambda so gpu minus one because we won't be using any s3 so it's the object that we use to call our models directly from s3 and the bucket name i have also specified the mapping id to style and we will see why how we will be using it in the in this script i start by loading the models from s3 and then finally i will define the lambda handler so the lambda handler looks like the function that we defined earlier in the previous video it will get an event in context by default so ignore this for now once the lambda handler gets triggered by a post request it will receive a body object and inside this body will have a key that will have and hold our image in the text format so we'll have to decode this image and convert it to a pillow image and then we'll have also to load the metadata directly from our client basically we will have to load the model ide and the load size so basically the model id will tell us which style we'll be using thanks to this mapping and then once we have these two information we can load the appropriate model given the style and then we can also resize our image given the load size that you have extracted then it's the same code as we saw in the next step so i'm gonna just skip it and finally once we have the output result we can send it back inside json object and that's where we use the image to base 64 representation to re to return the image as a string format so we turn return it inside a body and then we can access to it by the output key all right so now we have our handler we have our serverless file and also we have our requirements now we'll have to deploy our lambda function so before doing this you'll have to make sure that you have docker running because it will be used to create the dependencies of our lambda so run it if you haven't already and then go to the terminal and run the following command sls deploy and once you run it and if it's successful you will be given a new url that will be used to execute the lambda function so if you have done this you can later test it directly from the jupiter notebook so let's now deploy this lambda okay now our lambda is done deploying as you can see we have a url that we can test and directly play with our model before deploying this lambda you'll have also to make sure that the handler.pi is inside src it was not the case when i first deployed this lambda function and it failed so i had to rearrange the structure of these files and now hopefully it will work again so let's test this lambda on our jupyter notebook i will load here an image from a path and then i will convert it into a base64 format in the string and then i will build a payload data composed of the image the model id and the load size i will start with something small so let's say 200 then i will create a post request on this url with this payload data and then we can see the response time so we have one second it's very good for 200 pixels and we can see and visualize the content of this response as you can see we have an output and inside the output we have the json the string representation of our image it looks good now we can read this json convert the output to a base64 image and then convert it back to a pillow object and you can see the result now if we increase the size a little bit you can see that it takes a little bit more time maybe four seconds we have the same output structure and as you can see we have a better nice looking result alright so thank you guys for watching we've seen how to create an api with our model now in the next video we'll see how to interact with this api by building a nice and custom react interface thank you guys for watching and and see you in the next video

Channel: Ahmed Besbes

Views: 2,367

Rating: 5 out of 5

Keywords: serverless framework tutorial, deploy machine learning model to production, deploy machine learning model on aws, aws lambda python, aws lambda, aws lambda tutorial, serverless architecture aws, scalable machine learning, deploy machine learning model on aws lambda, serverless, serverless framework

Id: palz3TLB6TE

Length: 15min 29sec (929 seconds)

Published: Mon Nov 16 2020