AWS Lambda + MySQL + Docker using sam CLI | CloudFormation

hello and welcome back to the channel my name is Mahesh karya you guys are watching me on YouTube live and today I'm gonna take you through how to deploy your MySQL and AWS Lambda function two AWS Cloud using Sam CLI and using cloud formation templates so this is a super easy tutorial because I have seen lots of my colleague colleagues was uh doing it manually by going into AWS console and doing creating MySQL RDS cluster manually which is a kind of uh really not a good way and not productive way of doing things instead of that we are going to use a cloud formation and Sam CLI so without wasting any time let's steep dive into it so I have vs code open already so I'm going to open my terminal and I have my AWS or CLI already installed inside my system so if you don't have it you just need to search it uh for AWS CLI and window you will get this very first link over here you just need to download it and configure it once you have it go here and simply search for uh AWS configure hit enter you just need to add your access key and secret key here that you will get from your IM user that I am assuming you already have this access and after that you need to install Sam CLI so simply search for sand CLI and hit enter so this is uh also you need to install this is uh serverless function that you can do it manually here okay and that's it nothing more than that after that you just need to hit this very first command that is Sam in it and let me increase my font size so it should be visible to you so you can select this very first quick start template and after that you can select this hello ad example because this is the simplest uh yaml file that you will get with this so you can select the other options as well according to your choice for now I'll I'll put this the first one uh after that this Python and zip combination is very popular but we are not going to use this because I wanted to use a dockerized container that's why I'm going to select no for this and these are the runtimes are available that you can choose from so of course you are a Java developer node or python or Ruby developer I the rust whatever you can like familiar with you can choose like dotnet 5 is also there so I'm going to go with python 3.9 so this one 13 I'm going to select after that I'm going to select this image file because there are lots of dependencies I will be using and dockerizes something uh use ECR behind the scene just like your micro services so which is easy to do and write your own configuration in Docker file that's why I like to go with this because for the production I always prefer to go with image I it after that even you can add this extra tracing of course in the production it would be helpful I don't need it for now so it depends we'll go with the yes for now after that my project is about connecting python with mySQL without going to your AWS console not with the UI just with the yaml file and with the CLI only okay let's see how it goes so I'll name it by MySQL this is all about just connectivity rest of the things are really very easy so it's gonna get uh from this URL you can click here and you can see all the templates are there that this CLI is working uh online and this these are the templates it's getting so yeah this is it this is this is it we got it already and once you have it you can open your Pi MySQL here you can see your hello world code which is look like this the simplest code that you can see and nothing but just a request and response that you will get in in the form of this event you will get events and this is it okay so I'm not going to deploy this one but the more uh familiar thing is about this Docker file if you guys are familiar with Docker file you can I'll quickly go through it like this is a Public Image you can go to ECR these are the open images of AWS okay so it's getting it then it's copying your app.pin requirement.txt to here instead of you can use dot slash as well to grab all the things but we will be having many of things like EnV folder and much more so you can name it all the files or folder uh so it will a copy from here to that container and after that it will run this command so you you can create here uh your custom commands to create a virtual Envy if you required also it all up to you how you utilize this so for now this is the simplest things that you should know events is something used to uh kind of Pass events and based on that you can test the Lambda functions if you uh like the API Gateway a pattern to directly hit the HTTP APS not a rest apis there is a difference between a difference between HTTP API and rest API that I'll tell you later for now our test cases also if you guys are familiar with it definitely look after it and you can write your own things I can prefer to delete this for now but let it be the redmi file I don't need this redmi file but template is something where the magic happens okay and that's that's what you have to uh go through it let me close this up so I'll go through it one by one this is a pattern of versioning okay so don't worry about this instead of numbers they prefer to use this date format this is not today's date this is the date where it wasn't launch okay and we need to follow that so don't worry about this after that transform is also will be a stay there and description you can manipulate about your project you can write here globals are the function with the timeout of three you can increase with the 10 seconds or on and on okay tracing that we have provided this is kind of tracing uh like it's like Matrix where you can check logs with tracing okay what's going on and uh just for the production things and testing debugging and lots of uh this is this is I'll I'll show you later so for now I'll I'll say this tracing enable Okay and after that resources is something if you guys are familiar with the docker compose file this is same as your services where you D you provide the services these are my services okay let me remove all The Unwanted comments that are not needed for now so I need one hello word function so here is my hello ad function okay it's a type of serverless function then it is a package type is image here you can Define your AMD or two types of processors as there x86 or AMD processor you can choose off now I'm not going to create each route for every uh function instead of I'll be using a single function to map all the routes inside that using switch cases all the methods using switch cases we can route for that I need to make a little change to handle the proxy like this and instead of get method we can use any to grab all the endpoints with any methods okay and once you have it you can like this is Docker context you can name it according to your choice for now this is the folder name okay and this this here is your python version so for now this is this is all you have to know output is something something that you can write or don't write it's all up to you uh once this cloud formation will be generated this kind of links will be generated uh for your API Gateway this will be link will be generated okay so I'll remove this hello I'll um this is my staging environment like prod QA and that kind of staging that you can add here you can add multiple endpoints here by grabbing this uh many times and here as well for the function like it's gonna show the Arn in the output format once it is getting created and for now I need a MySQL for that what I'm going to do I'm going to uh grab one uh [Music] for that I'll I'll put this as of my MySQL DB so this is my MySQL DB with type of RDS you might have Familiar of RDS relational DB services that AWS provides and and where you can allocate your storage like up to five gigs or 15 or on and on you can provide 200 okay uh T2 small is a type of instance of your ec2 so you might have heard of T2 micro which is a free tire eligible and after that this will be a kind of paid so this is my MySQL username and MySQL password with database name and this policy allows you to open your inbound rule 3306 to publicly accessible so that's why I'm using this to true for production we need to Define our custom groups and uh Ingress okay so for will go the simple way so just write this and once you write this it will automatically create your MySQL container on AWS using RDS and which will be publicly available to the internet that's it so this is it and after that we'll jump to app.js and I'll grab one code uh which is a simplest instead of this I'll be using a normal connection code of python and you might have familiar with this I'll just hit the query of create users okay and nothing more than that here I'll put my uh connection later once this this will get generated I can use manual or dynamically as well that I'll show you later in the form of passing environments so for now what what I'll do is I'll first use Sam build okay before that as well I need to install MySQL because I don't have MySQL here so let me grab this so inside Pi MySQL I need to change it and inside that I need to check where is my Hello World I'll go to the hello world and inside that I can create my own uh python virtual environment so Python 3 hyphen m v e n v space EnV is uh Envy folder will get created here so it got created so I can now use source to uh go inside EnV slash Bean slash of course activate to activate virtual environment and after that I'll hit the command to install my Pi MySQL okay and once this is done you can do this as well if you want to get the latest uh pip so let's wait for it okay this is done after that I'll use uh pip install again sorry I need to generate of I need to generate my SQL oh sorry requirements.txt okay I forgot the command so I'll grab this command let me grab this quickly I used to work on node.js and other platforms so I'm not much familiar with this so let me grab this quickly okay here like this so you can see this is added here now what I can do is use Sam build in order to build uh make sure you have to have Docker available because okay I need to go to root directory so I already have Docker running you can see here so it's gonna take time to pull for the very first time if you don't have it and and once you have it it won't take much and for the first time you can directly use this command to to uh deploy using guided mode so it will help you to set up the stack uh so Sam config terminal file so I'll use my name as Pi MySQL because that's the agenda of project and episode 1 is my region so region is something very important to choose from so if uh generally we use Us East one which is a kind of monster we can say uh in because most of the guys they they don't care about these things and directly deploy the things here which is not ideal way of doing things because uh according to your region it will pick the things easily and it it will be load faster so that's why we deploy in the region where we are located at so this is my region that's why I have selected episode one once you have it after that it will ask that you need to confirm changes before deploy yes it will ask yes or no at last when uh before deploying it so I'll go with yes and after that it it needs to create I am role so provide permission yes a disable rollback is allows you to help to not roll back so so uh you can get your errors messages okay you can see what's going on and what went wrong that's why uh if you are not uh sure then you can use yes that I need to check the logs so don't roll back everything so I can find and debug the things after that this may not be authorization defined because nowadays all the API functions that you need to authorization you should provide now for the production uh support uh for the production Radio application so I don't have it so it it is asking that okay this is this is fine so I'll say yes this is fine uh and now the last thing is it's going to say that all the arguments that we have just selected will generate the Sam dot doml file and save this configuration there so I'll say yes so name it will ask you can provide another name like something else so I'll go with the default name and configuration environment that will be default this is the profile that is it is getting selected when you guys are remember like I did AWS configure so that was profile got created inside my system by the name of default that's why it is asking and we can create multiple profiles for QA and prod so uh for now I'll go with the default so last uh it will ask again because it needs ECR to get created and ECR is something elastic container registry so I'll I'll search for ECR okay so this container registry is a is we used to uh deploy your images here so see this image got created here and no it it's my previous image so somehow we can see the timing 155 so now it is 323 so it's it's two hours before I was testing so let's go back so this is elastic container registry share and deploy your container software publicly and privately so we are deploying it here okay so it is asking to create and manage ECR repository functions I'll say yes because we are using dockerize containers which is more flexible and you can utilize your own languages there as well like very different languages that we rarely use like a lang Alexa R language this kind of language we do so rollback complete okay this is a showing one of the exception terminal failure let's see what we can do there and if we can have uh things we can build first let me build again and let me try to deploy things so I don't have that Sam or config dot normal yet here it means I need to uh do this thing once again we might need some something wrong so let me do this again by MySQL test maybe okay I'll say yes yes yes yes yes again yes for everything as usual and it will ask a bucket can be set okay it's fine let's create and let's see so this time it should work my Internet connected I have checked that as well so now let's go there and yeah now now seems like it's getting created uh sometime it happens uh might be because of internet not sure by might be because of CLI it was showing so for the first time it will take time so till then I'll walk you through what we are going to create the first thing we are going to create is uh we have a Lambda Lambda function which is a serverless function okay and that Lambda function is going to connect with your MySQL which is a RDS which can be anything like postgres or MySQL or any other okay and now we have cloud formation here so here I'm going to say this is my cloud formation cloud formation is just template which is which is not real okay and what is real is a container which is getting created here okay so this Lambda function will be inside here so I'll say this is my ECR so it's going to deploy inside easier and my network because this thing is is publicly accessible because of my inbound rule okay that we have we said here uh inside my template I have wrote this public accessible true that's why this is this is accessible true and I'll say this is part of now I'll write like this so this is more convenient way of saying things okay now let me grab this inner one here so now it's it's looking good to me okay and here we go you guys are might uh might getting confused but this is how it works The Logical layers I'm I'm trying to show you guys okay this is the simplest way and I what I'm doing is I'm using this external world to provide all the apis to me okay all the apis for single container using this proxy keyword that I have used here this proxy is self enough and these two things are self capable to to get all the things here okay so because of this thing it is it is super easy to get all the things at once okay let me grab this here and my RDS cluster I can create manually as well most of the developers they like to go to RDS and they do things manually okay the last one is remaining so here is the RDS relational database service okay I've seen many developers what they do they create a database manually here manually they can choose the database like Oracle and all the things they can select the version and they can but uh the thing is these are the things are are for a beginners but when when you do the professional devops things you should use the CLI or yaml cloud formation file so it will take care about roll back changes and all the things that we don't have to worry about okay and the next thing is Lambda function so Lambda function and API Gateway are two different things the major problem that developers are facing when they create a function from here they need to manually connect a API Gateway okay that API gateways they need to create manually okay that have I have created the separate video for API Gateway so there are three types of four types of API gateways like HTTP which is better than rest okay they they have a build low latency and cost effective than rest apis okay there is a slight difference that I've told you that I'll say later so these are the HTTP or Advanced apis that we generally use these are the two types of rest apis private and public so you can choose this one and websocket we rarely use but lots of support of or2 and core support they provide nowadays in build so you don't have to worry about and here you can provide integration and then choose your Lambda function and select Lambda and on and on you can do manually for each API gate API post API each method you need to provide core support and all the things you you do need to do manually that we did in just two lines here okay and we have created a API Gateway in just a single line okay this this API Gateway have connected so yeah the this this is the simplest way of doing things okay now the last thing is it is asking to deploy the change set I'll say yes because you guys are remember I I the very first time when it asked uh to to check at last that everything is correct so that was that was the option that we got selected and we got this final uh approval that it is asking and this thing you can check on AWS console as well if you go there on AWS console we search for cloud formation again so here if you choose everything is getting created okay here we go something went wrong here see so you can check here what went wrong so here as well you can check and let me go to the stack once again so this is getting created okay you can see the events what's happening so these are the things you can monitor from here as well and here as well oh excuse me so it will take a while because it's going to get the things and deployed there so I'm not going to pause this video so it should be as it is if you want like you can skip or you can move forward or use 2x speed which would be preferable uh for now because this is a time consuming process okay because uh but yeah once this is done the rest of the things will be very easy so now the next process we'll go to the escalator now what's the problem are uh what's the wrong things are lots of developers are doing okay they do uh big giant applications like let's suppose next yes okay so I'll say next yes or I'll say then spring boot okay then I'll I'll grab this if I can two times here so please do not use next yes okay why because it's very bulky and it is not why it uh the serverless functions are created for they are they have very small memory and small Footprints so don't use your spring boot you can use actually but I I won't prefer to use spring boot or any framework of let's suppose RoR uh if you are from RoR or if you are from uh Express family node Express family so please do not use this okay just use the normal request response pattern and use on the core things okay because uh springboard plus GPA is super slow and if you use the combination of spring boot plus uh I'd say bring boot Plus no Java Plus uh your jdbc then it is a 10 times faster than this combination okay that's why we we don't use this this is not meant for Lambda please do not use this okay this is this is not in don't use any framework Lambda within Lambda you can create a small things now the next thing is how you can connect the two Lambda functions that's that's the most important part okay let's suppose if you have a function of user okay I'll say user with name email password let's suppose Okay so I'll say my user table which have name use this and then I'll use name email let's suppose but you need to have lots of tables in order to create a meaningful uh big application right so I'll create another one here which will be address okay with the field of pin code City okay so if you have a single form of sign up and you wanted to insert the data here okay this is this is my signup form and if you have to insert the data into the two Lambda functions what you will do you have to connect this two things using sqs and that sqs is a simple queue service or Kafka okay let's see let's say Kafka or any any Pub sub or anything but we are using a single of AWS environment so what I'll prefer to go with sqs so if you guys are not familiar with sqs till then let me see what's going on here okay it's going on so search for sqs let me search here so simple queue service so this service allows you to share the data in the format form of queue okay so let's suppose this function got the data okay then it will automatically and of course uh the normal micro Services these are the kind of micro Services you can assume okay so these are the services smaller services and it it so we should have a single his on it it's on database might be dynamodb mongodb or any SQL database and once you get hit here event will get triggered by sqs and it will call this another service and this service will return back the response this is how it works okay sorry so this is the actual format so don't get confused and don't use these Frameworks to do entirely in the single uh small or serverless function okay it's a function it's not a complete uh bulky container okay behind this in its it's a container but not for these things okay so let's go back to CLI so as I told you guys this is time consuming but everything went well and we haven't got any errors so now I'll go to cloud formation again okay Here and Now you can see you can click here thank you and you can see the outputs these are the three outputs that we have wrote here here and I forgot to add one more thing actually here I just need to add one more thing which is my output of database connection so that database connection I'll add it later no worries so let's go back to here and inside stack info events okay this many things let's go to the RDS ones I RDS got created so here our instance is I have one instances available so let's try to connect this thing first here's my end point okay and I am using this extension okay mySQL database extension and if let me add my previous connection here I am using a host name this is my host name and you can get your name email password from here not an email password but root name password and database name from here and try to connect this so everything went well it got connected but we don't have any tables the agenda is to uh run this application and let's see if we have the end points okay so I'll go to my function I'll name it serverless function Lambda functions so if you go there the Lambda function got created if I go here I have this one API endpoint API Gateway so you can test and monitor the things okay x-ray tracing that is enabled so you can use this traces so for now we don't have any logs but once we do test or once we hit this API we will start getting logs okay so we have this internal server error because guys I have told you once I get this API endpoint then only like I'll start doing things so for now I have this end point so what I'll do is I'll add this endpoint to here in my code of here actually so now let me show you how to redeploy or update the stack so I'll say Sam build again so Sam build it's going to build once again so now I have this new endpoint so it's doing the things again and now I'll be using Sam deploy so this time it's not going to take much as I told you the rest of the layers are already pushed this new layers got pushed and now it's going to take small chunk of time to to reflect the changes that's why it is super easy and if you are doing these things manually then it will take a while actually the manual process I I really don't like to go with or I I will not suggest you to do in production uh things that's why like I have spent uh many hours to make this simple video to show you all the possible ways of doing things if you if you go through this video if you understand it properly the rest of the things will be very easy for you guys now the last step I need to show you like how to run this thing locally so for that we use a Sam local start API so this will start API locally that you can see here you can simply hit this uh and of course you need any API slash hello or slash whatever so you got this server error and here now you can see what kind of server error that you are getting okay this is your error and this is your event got called so users is already exist so it got created so I'll name it something else let me see if I have user so yeah great user got created actually let me let me delete this user if I can or drop table from here yes I'll say yes drop table Let me refresh this and let me call this no you can uh we are getting error but it's fine okay we haven't called any uh anything there but now you can start watching or monitoring logs if something goes wrong you can check the things here okay this is getting started and this log stream is will be resumable so it's a live stream that we are getting anything goes wrong you will clearly see here is already exist once again we got this because table is getting created okay great so that was the agenda the MySQL is getting connected and this is getting created now if you if you wanted to create a restful apis for create read update delete don't forget to use the chat GPT or any air tools that I'll say but the agenda of this video is uh is completed okay so I'll stop here I want to waste much time of year guys so thank you so much all of you guys for being with me supporting me all the time it's been pleasure to be here please let me know inside comment box what you want from my side definitely I'll make it for you and don't forget to subscribe my channel if you haven't already and smash that like button if you enjoyed see you soon in next video till then tata bye bye

Channel: Mahesh Kariya

Views: 1,239

Keywords:

Id: 54T0Yu9oPRA

Length: 39min 42sec (2382 seconds)

Published: Sun Aug 06 2023