Amazon API Gateway | S3 Bucket | AWS Lambda | API Endpoint for File Upload to Amazon S3 Bucket | AWS

a hi in this video I'm gonna show you how to create an https endpoint for file upload to Amazon S3 bucket with using Amazon API Gateway and AWS Lambda as you see in this PPT this PPT depicts My Demo here we're gonna construct a solution which will create an https endpoint which will help us to upload the file with using the secure https end port so the solution will be constructed with using three services of AWS that is Amazon API Gateway which is backed by AWS Lambda which in turn receives the payload from the mission API Gateway converts that payload into an image and finally uploads into Amazon S3 bucket right and here we can use this Amazon API Gateway from Postman or from using the any kind of programmatic way for example say within python we can use the request module and I know and invoke this Amazon API you know Gateway yeah all right so basically this is what I'm gonna show you in this demo so without delay let me take you to the my AWS account so this is my AWS account um currently I'm in Oregon reason likewise I have opened the another tabs another two tabs one is mean for uh you know the AWS Lambda purpose so I have a AWS Lambda been pre-created already and then I have an Amazon S3 bucket which I also created here so this is the Amazon Ester bucket which I have created okay so don't get confused all the three tabs belongs to my AWS account and belongs to reason Oregon right so here I have created an API and the name of the API is something like this that is face recognition API I just given for just for the demo purpose in that one I have created a resource called upload because this demo deals with the an operation with AWS S3 bucket just to upload a file so here in this one we have you know created a method called so basically if I go to the method that is uh you know we have created a post method in that one so the post method definition sum looks something like this first one let me introduce to the integration request configuration that you need to maintain for proper you know prop for working of this particular solution so here integration type would be AWS Lambda and before you construct the Amazon so here AWS API Gateway yeah you must have your Lambda you know created so that becomes a prerequisite so here I have already constructed or created AWS Lambda like this and then I have started to create an API Gateway right so here you choose the integration type equal to AWS Lambda and then you're gonna choose the region where you are Lambda is sitting so my Lambda is sitting in US West 2 that is Oregon region I selected the Lambda and then execute role so you can keep it all default and then use the default time mode yeah all right so with that you need to leave the other options that is URL path URL query and just HTTP sender leave it default go to the mapping template so the mapping template is something which is very important configuration which you need to maintain it a proper way so here in the request body pass through you know you're going to select the where there is no template so select this one in the content type you add a content type so in the sense here in my case I'm gonna show you how to upload a PNG file into the uh S3 packet so likewise if you want to upload the PDF file jpj file or another format you can create those kind of content types here so I created a content type called application for slash PNG and then in this one so I have left the the generate default option so in this one we're gonna choose like you know the method request pass through and then click on a save button right so with that configurations you are done with the method execution configurations are indeed the integration request then let's go to the method request configuration in the method request configurations there is an option called API key is required R so basically when you create a method so you must select API Keys required because since you are exposing this HTTP endpoints to the public you know you should have some authentication to prevent you know the the you know unauthorized usage so for that case here we have used uh you know the the API K required equal to true yeah all right in the request body so just keep all other options default so if you select if I select my configurations just follow me so basically you can keep other configurations as expected in the sense just keep it default yeah so once the resource is created you can go ahead and and deploy the API yeah so once you deploy the API it will ask you the give the stages and also here I have created a stage name called version one and then so if I go click on this one so this is the my stage I have created and then this is the post method which is actually required yeah all right so this is or the you know configurations which we do to configure the apis there's one more things which you need to keep the configuration in places you know go to the settings in the settings you know so you have to go to the uh binary media type and here you need to you know copy paste your media which we have configured earlier in the integration request that is application for slash PNG because we are using the PNG app and here in the API Source cannot choose in the API key Source choose headers all right so this is the configurations again you need to maintain the stages so remember that you know you are first you created a API then you created the resources then you stage the resources in the sense so you have created a stage of the resources here it is it is a version so our basically production something like that yeah all right and then we go to the API keys so here I have created an API key because here you know we are actually you know authorizing our https endpoint with using API key so we should create an API set to create an API key just click on a create API key give the name and API key just say Auto generate and save it so you will so you will have a API key you can just click on a show you will have the API key all right and then let's go to the API usage plan so you need to create a API usage plan so click on a create AP usage plan give the name uh give the configurations of throttling quota yeah so in my case if I open and show you so these are all the details I have added in the sense API keys so these are this is the API key I have added yeah and then so if you open so it will take you to the same APK which we have added so in the usage if you go to the users plan I have created SS plan something like that and attach it to the API key which we have created yeah and in the details you know so basically what you are doing is you will need to maintain these configurations at API Gateway site all right so once this has been done I'm gonna take you to the um the AWS Lambda site in the AWS Lambda cell what you need to what are all you need to do the configurations here you just create a AWS Lambda give the right permissions to the AWS Lambda so in my case you know um basically have a im rule which has access to the AWS S3 bucket and has the trust the that role has to be trusted by the uh basically the Amazon you know the um AWS Lambda principle okay so basically that's kind of configuration you can maintain if I can show you that I am role I can show you here go to the permissions here and click on the imbro that you have attached so here this is the iron role which I have attached to this particular role yeah and then um while that role is getting opened I'm gonna go to the general configurations these are all the general configuration I have maintained in my Lambda and in the trigger so it does not has any kind of triggers you can just uh if you see here that it will be automatically added these triggers because you know because we have we have integrated the you know the uh API Gateway right that's the reason this will be automatically to worry about it all right okay so now um I will go to the IM rule I was talking more on this one in the trust relationship you know you need to maintain that it will be trusted by I know AWS Lambda yeah and then let's go to the code side in the code side so this is the code which I have constructed uh you know so that it will convert the binary data into a image file and then finally it will upload so this code will be shared in the video's description I have I will be uploading this file to my GitHub and the GitHub repo link will be uploaded in this video's description you can find it from there so here it does nothing but just it creates a S3 bucket client reads the event nothing but it just reads the you know the basically uh the binary content of the um of the image yeah and then it will decode that binary content into base64 and then um so then you know then finally it will upload into the so it will upload into the S3 bucket with using put object API yeah so here we I'm using the random key generator so that I can randomly create a file name and upload into S3 bucket and finally if everything is gone good it will return a status thing like you know https 200 all right so that's the configuration which I maintained on the Lambda side all right so right now I have created S3 bucket if I go back to the my code so the bucket name I have given is this is the bucket name yeah and that is where my bucket name is here all right so with this configuration we are actually set with the the solution here so that is we constructed Amazon API Gateway we written a Lambda we written the Amazon S3 bucket and they all these resources are connected each other right now time has come to test the solution so what I do is I'm gonna go to the API Gateway go to the API Gateway click on the your API so if you go to the apis you might have multiple apis choose your API which you have constructed for this solution and go to the stages click on version you will see a URL here so copy this URL all right and go to the postman so first let me show you how to run this solution with using you know Postman so here this is your base API URL then forward slash the resources if you open this there is a resource called upload so in that one we are using method post so remember and use the same thing here so I have constructed my complete API URL something like this chosen the method post yeah all right so with that now let's go to the the other parameters that we need to pass when you hit the https both post method right first one in the header side we need to add these headers one is key equal to content type and it should be application for slash whatever you have given in the configuration here is the PNG file which I wanted to convert yeah or which I want to upload into S3 bucket then X hyphen API key which is nothing but the API key that you have created the uh you know for the authentication of this API right so if you go to the API key and click on a show so this is the API key don't worry I'm going to delete this API key after the demo so you don't need to try it from your side all right and then here we have kept the API so these are all the configurations you need to keep in a header side now on the body side so in the body side what you can do is you can go ahead and select a file something like this so in my case I'm just uploading this file yeah so that is a very beautiful uh you know nature pick if I go to this file all right so if I go to this file and show you from the vs code so this is very nice file which we're going to upload into the S3 bucket and we see that you know this image image gets uploaded into the S3 bucket so here if I go to the S3 bucket right now it does not has any object I am refreshing it again and I'm also making this demo very partial you know very transparent there is no other so instance basically the configurations meet here are to be real and you know to the point all right so right now if this bucket does not has any object for your information all right now what I do is you know we're gonna hit the end point from the uh from The Postman so for that case what I did is I have uploaded that image file selected it from the my local laptop and all set so in the sense basically I all set with my configurations click on send you should be receiving response status equal to 200 which means that you know file is is actually uploaded you see the internal error so basically looks like something is error here if I go to the so basically looks to be something has gone bad so let me try it again yeah so something is error here so if I go to the um uh PNG yeah let me convert let me try to run the same code from the uh my python file and let me see if this works so basically it is not working from the uh you know so from the um from the postman so it says request is too long so basically white is request is too long because if I go to this file and this file size is you know so basically uh you know um it's the file size is basically more than 5 MB right so if I go to the so this is very good learning for us so if I go to the images um and the image the image size if you see it is 5 mg and that's the reason it is not able to send the image okay so for now what I do is I will just uh choose the you know the another example my thumbnails let me use this file okay so if I go to the images so this is very good learning because it is not able to send the 5mg file through this API Gateway remember that all right so if I go ahead and keep the another file so this time I'm going to upload this my thumbnail which I have created for the my video purpose so what I do is if I show you the size of this file uh by going to the properties so properties and see the size of the file so this is only like 120 KB so it should be uploaded so in that case what we do is we're gonna change the file name here uh first let me show it from the uh from the you know the postman then I can show you how you can do it in a programmatic way in the python so I'm going to upload the re-upload the file so this time it is 0 1 PNG file so I'm gonna hit upload here we go so file is uploaded here if I go to the S3 bucket and refresh the console we see that you know the file is uploaded to make you confirm you can download this and check it so if I download this I'll click on download and just open the file here you go the file is uploaded into the S3 bucket so remember that a file if the file size is too big it is basically not and also basically it is not passing through the API Gateway which needs an extra configuration which you need which you can take it help from the AWS all right so now um we have shown you how to invoke the API from the postman site now let's do a bit more that is how to invoke this same thing from the python side click on that course nippet button and then choose the your favorite language here so my favorite language is Python and in that one request method so what you do is you're going to copy this yeah so this is the this is the same code which I have copied and constructed uh you know uh I constructed a pi you know so basically uh example.pfl which I'm going to share this file as well along with the Lambda file in my GitHub repoint the link of the GitHub repo is shared in this video's description you can find it from there all right so here the code is I'm gonna I'm gonna use the request method uh i o method then I'm gonna use the pillow um you know so basically you need to install um so you need to install the you know the pillow module of the Python that is PIP install pillow module click on this one so basically which will install pil module in your system right so it's already satisfied because I have installed and then only you can realize I then only can use this module yeah and then then the natural loan that is import OS which is default library of AWS then this is My URL and I have added the dummy print I'm going to remove that and then comes y so here I have imported pillow module why did why did I do that because I want to convert the image this is the Hardcore image in the sense uh yeah this is um this is not the right image because that is very too big image I want to I want to go to the 120 MB image that is 0 1 10 yeah all right so here what we are doing is we are reading the image converting into the byte stream here and uh and and saving in the format of PNG and getting the binary value of the image so basically every image or every image file present in your system could be converted into a binary you know binary values and that is what I'm doing and that value is passed as input to the API payload so here my header is I'm constructing the header with the content type equal to application PNG remember that you know it should be equivalent to whatever you are given in the API Gateway and then it comes that this is the APK you need to give it if you don't give it you're gonna have the you know the unauthorized error and then we are invoking the a request method with a method called post URL headers and the binary value of the image that's all so the basically this is what you know I have constructed the python file and this file could be used for automation purpose so I'm gonna invoke this it we're going to upload the duplicate copy of the same thing so if I go to the S3 bucket again and refresh the file we should have the two files now so this is the file which I just now got uploaded you can download it I can see it so I download it again and if I open it you see this is the file you know which is the you know uploaded so basically what I have shown you use so here I have showed you how to construct an https endpoint for uploading the file into Amazon S3 bucket with using API Gateway and Lambda all right so this kind of API Gateway really helps us to build your automations at your applications at your any kind of business workflows could be automated with using this Method All right so with that note I have shown you the things need to be shown in this video finally I can't request please do subscribe my channel that would really encourage me a lot so with that note thank you thanks a lot and see the next video

Channel: Cloud Quick Labs

Views: 4,608

Keywords: #cloudquicklabs

Id: ed0QCU-KboY

Length: 17min 59sec (1079 seconds)

Published: Fri Sep 01 2023