Create a Cloud Hosted Python API Server - Using Python Chalice and AWS Lambda

hey what's up everyone this is brad from devops journey and today we're going to have a look at chalice which is a way to create python applications and push it to the cloud in a serverless fashion so with chalice we're able to create simple python code and push it up to an aws lambda function and it's actually quite simple to do i'll take you through the documentation and then we'll deploy our own api server so let's go ahead and have a look at the quick start guide here so you can see the quick start guide first goes over the installation chalice is just a python module so we'll do a pip install chalice and that'll get us there after that we have to set up our aws credential files i'm not going to go through this in this video but i'm going to leave some documentation in the description below so if you need to set that up then you can set it up basically we just need an aws user and you just get an api key for that user you set your access key and then you set a secret key and then set your region just make sure to know what this region is when you hop into the aws console you might be looking for your lambda function and you can't see it it's probably because you deployed it to a different region and then you're viewing on so just be aware of the region that you're on other than that i think the permissions for lambda functions i think it's just like you need access to s3 buckets or something like that again i'll throw some documentation in the description below about that so after you have your credential file set up you go and create your project and we just use the chalice command for that so you just go chalice new project and you give it a name and this sets up a directory for us with all the chalice files that you need so you have this chalice hidden directory we won't really be touching that then we have this app.py file this is going to be the main file that we're configuring i'll show you how to customize this file and add functionality to it the example they give you here is pretty bare bones so i'll give you a nice template to work with and then the requirements.txt file if you've worked with python before you're probably familiar with this basically if your application needs any external modules just make sure to have them in the requirements.txt file so when we push it up to the cloud aws will know which modules to download other than that we'll probably be creating another directory that isn't listed here because i want to have a secrets file to store my passwords in uh for one of the integrations that we're going to be using so i'll show you how to do that and yeah after that we deploy to the cloud and then it gives you a url that you can get to this is basically going to be your api server so you can see they say go here and it says hello world and that's what's returned there so we're going to go through all this i'm going to give a more advanced example that you can use and customize to your needs so that's going to be good i'll also hop into aws and show you what you can do to troubleshoot if you're running into any issues so let's go ahead and get started okay so i have a terminal open and i want to start my chalice project again you need to do a pip install chalice if you don't already have chalice installed i already have it installed on this machine so i'm going to create a new project so i'll go chalice new project hello chalice you can name this whatever you want and it says your project has been created so let's hop into that directory we'll go cd hello chalice and then if we have a look you can see that we have the requirements file and the app.py and if i do an ls-l you can see that i see the same thing i actually wanted to do an ls-la you can see that it gives us a chalice directory and a git ignore file okay so let's go ahead and open this up in visual studio code all right so i have it open in visual studio code and you can see that it is a very simple example basically it's just importing in chalice uh it's creating an app object with the app name hello chalice this is just the project name we gave it and then it gives us an index server so one thing that we should always do when we're working with chalice is run a local development server and just test things out before we push anything out to the cloud so to do that just hop into your directory then go chalice local and i actually add these extra parameters where i go dash dash host and i go 0.0.0.0 this is to bind the chalice server on all the ip addresses so i can access it across the network because i'm actually running my chalice environment on a server different than my environment so it tells you you can get to it on any ip address and then port 8000 so for you guys if you're running this locally you just go http 127.0.0.1 colon 8000 but i'm running this on my remote lab server so i actually need to specify the exact ip address so this is the ip of my remote server so i'll just hit that and you can see that it returns hello world all right so we know our chalice app is working we can go there and we get some data back let's go ahead and start customizing it so i'm going to add quite a bit of functionality to this default application let's take out these comments and i'm just going to copy and paste this here this route up here is basically going to be the url that you go to so let's call this one time let's rename this function to get time and then we will go here and go now equals date time dot now and then let me paste in some code here so basically it's grabbing the time it is now setting the current time so it'll display in this format and then let's return to the user the current time now we're also going to need to import the date time module so let's do this up here let's go import date time the chalice server in development mode should restart automatically and we can see that yeah it's been restarting automatically so let's go ahead and try to get to it now let's go slash time this is the url that we set and we can see an error and let's hop on over to our development server and see why it says module date time has no attribute now so this is a great example of when your code doesn't work so we can fix this let's hop back into visual studio code and uh i think i want to go from a date time import date time and this should fix it so the chalice server should restart automatically let's go here and refresh and you can see that we get that time returned now we can customize this output one thing that i like to do is sort of wrap it in an f string so i just go like this f and you need to use these curly brackets the time is and then i'll return this so let's go and refresh here and there you go so pretty good so far that's just a basic get request so we're going to the server we specify the url and it's going to return us some information now let's have a look at what we could do if we send a post request and this is similar to if we were to send like a web hook to our api server basically a web hook is just an http post request that has a payload of information so i'm going to copy and paste some code that i have ready here and i called this one echo and let's say echo back for the function name and the main difference here is this is only going to accept the method post so if i were to actually go here and say echo you're going to see that we get the error method not allowed and that's because when you access it through chrome it's doing a get request we actually want to send a post request so to do this you can do it through python through the request module and sort of customize a post request or you can use a tool like postman or insomnia so insomnia is a great tool that i use when i'm working with restful apis so let's go ahead and have a look at that and this tool is free to use just google for it and download and you should be able to find it so i'm going to actually create a new dashboard here these were just some things i was working on before so i just go create and request collection and the collection name i'll just say i'll say chalice app and this brings us into our chalice app and this is where we can build out our http get and post requests so to add a new one just go new request and for the request name i'll just go my request is fine and we actually want the method to be post so i'll go create now with a post message you're supposed to add something in the body so let's go here and let's send a json payload and in that payload we'll just go message hello world name devops journey okay perfect oh and we need a comma here so this should be good and what we need to do is add our url here so let's go ahead and grab this so i'll take this and put it in and now let's go ahead and send and i'm getting an error bad request error parsing jason so i'm missing a colon here so let's go ahead and send this again and there you go it's basically sending a message and it's getting it back so let's have a look at our python code and sort of explain what's happening here so it's called echo back and basically these two lines right here are taking in the payload of the original request that was made and then sending it back so this is a good bare bones example of sort of like a web hook you can have a post request come here it can take in the payload you could sort of modify or do something based on that payload and then you could return some data back so i want to keep this example pretty simple but if you have any questions please leave them in the comments below i'm going to have a look at one more example here which takes our project a little more in depth and then we're going to deploy this to aws so i got some more code that i'll copy and paste here so this one returns the price of bitcoin litecoin all that stuff uh i'll explain how it works here so you can see the url is not a static url it actually has a variable built in and it's basically going to take a symbol we're going to be able to send btc ltc or whatever cryptocurrency you want and it's going to take that and then it's going to send an api request to coin market cap ask for the price of that coin and then return it back to our user so this is sort of like a full example of how powerful these apis can be one problem i see here is this needs to actually be in here so basically it's saying symbol take symbol as input this line of code right here is just making sure that it is always uppercase so it doesn't matter if our user types in lowercase btc or uppercase btc it's going to convert it then it's going to do a request to the coin market cap api and it's going to use that symbol and this is the actual request that it's going to make a couple things to say here first we need to import the request module so let's do that and since this is an external module we need to make sure that we include this in our requirements.txt file so we'll type in requests now if you forget to do this when you deploy this to the cloud aws isn't going to know to install requests and basically your app is just going to run into an error so make sure that you add the correct things to your requirements.txt file let's hop back into our app and this next thing is the api key so for your code you really shouldn't be putting in sensitive information into the logic so i want to use this api key so what i usually do is create a secrets file something like this and then i just go api underscore key equals asdf123 or whatever your api key is then i would head on over to app and go from secrets import api key and this would work except for with chalice it doesn't like including any files other than your app.pi your requirements your get ignore and your chalice folder so if you need to include any external files with your chalice app you actually need to create another directory named chalice lib so we'll go new folder chalice lib and i'm going to create a file here just called init.pi and then i'm going to throw the api key in there and then i'm going to hop into this app and instead of saying from secrets i'm just going to say from chalice lib import api key and that's a pretty simple way to do it for chalice so let's just keep it like that i'm gonna go ahead and put my real api key in here so we can test out our app okay so i put in my real api key now let's just quickly go over the rest of this basically we get this request object back so this request object goes over to coinmarketcap and basically pulls the price of whatever symbol that you gave it and then this is just formatting the data into simple variables all three of these lines are just simple formatting and then the return that we give to our user is the symbol that they gave and then the price of name of the symbol is price and again this is using f strings so we can include variables in it so let's go ahead and test this out on our local server and if that works out we'll deploy it to aws all right so let's go here and uh the url was price and then we can include any symbol that we want so btc i'll just make it bigger you can see it gives the price of bitcoin i'll try litecoin i guess ethereum is a coin and i don't keep up with crypto markets too much so i don't really know any other coins other than that but i'm sure if you just type in the symbol there's probably a crypto coin for it i guess bb is in a coin maybe bnb finance coin is 266 so i guess finance is an exchange and they have a coin themselves so so it looks like our api is working we have those three different examples that we gave the basic get request which is just a static url and just basically returns some static information we have the post method where we can send a payload to our api server where it does something and then returns a message back to the user and then we have one that takes in a variable from the url and then it does some logic and then returns some information to the user so pretty powerful stuff let's go ahead and have a look at how we deploy this so it's super simple i'm just going to control c out of our server here clear the screen and we'll just go chalice deploy and you can see it's creating the deployment package and that's basically it it's going to go out to aws and deploy our chalice to a lambda function and it's going to return back the url for our api and if you run into any errors it's probably because your aws config file hasn't been set make sure to check in the description below for instructions on how to do that so it looks like we got some output returned here and this is our url so let's go here just control click and let's make this a little bigger you can see we get hello world uh if we go and go time we can get the time back and then if we go price btc you can see it returns the price of bitcoin so how awesome is that so all we really needed to do was download chalice basically two chalice commands and uh just some customization of python script and we have a serverless function in the cloud and one of the main benefits of doing serverless is you don't have to pay for this server constantly running basically i'm only going to get charged when someone does a request to this page and i think for the pricing tier for lambda functions it's actually really affordable so this is going to be a super cheap api server to run but anyways let's have a look under the hood on the aws side because i know a few of you are probably interested in that okay so under the aws management console it's going to be under lambda so click lambda and then under lambda you can see that i have two serverless functions here this was when i was testing it out yesterday and then this is the one that we just deployed if you're looking in the aws console and you don't see your function go on up here and make sure that you have the right region set this region needs to match the one that was in your aws config file so let's hop in here and you can see that it actually does show our code i've seen in the past when i've had larger applications where it says it can't show the code but i guess it does show it here you can see this is our exact function and you can actually even just modify the code in the cloud and apply changes but i don't really advise that you may as well just do it on your own workstation and if you need to redeploy just go chalice deploy and it automatically updates everything one other thing i would like to go over is how to actually test from the aws console i know i had to do a lot of troubleshooting when i was building this actual example so i went over to test and you can actually invoke the function there and you just go invoke and then it gives you detailed logs so if i were to run into an error the error would be here and it would be exactly like that example that we ran into earlier in our development environment you know the one where it said we didn't have the right module imported so that's a good way to troubleshoot if you need to but anyways that's all i have for this video if you found it helpful at all please go ahead and hit this like button so we can get this information out to other devops and python engineers and if you want to learn more about devops python or just want to learn more about how to start your career in the it field go ahead and subscribe to my channel thanks again for watching and i hope to see you all in the next video

Channel: DevOps Journey

Views: 1,539

Rating: 5 out of 5

Keywords: aws lambda, api gateway, lambda function, aws api gateway, python serverless, python aws, python aws lambda, python api server, chalice, python chalice, aws chalice, aws chalice tutorial, aws chalice python, aws chalice vs flask

Id: r60-90Stb2o

Length: 21min 11sec (1271 seconds)

Published: Mon Mar 22 2021

Reddit Comments

[u/DevOps-Journey]

Hey Everyone,

Just found out about the Python Module Chalice and fell in love with it.

It's similar to Flask, but it allows you to deploy your webapp directly to AWS in one command. So if you use AWS and need a simple webframework try it out for your next project. It's also super cheap to run AWS Lambda(Serverless) functions, with a super good free tier.

https://aws.amazon.com/lambda/pricing/

In the video, I created a simple API server that can return you the date/time and the price of Bitcoin etc. Code is here:

https://github.com/bradmorg/chalice-example