Get Images from S3 Bucket with API Gateway Demos | Two Options | With Lambda | API Gateway S3 Proxy

hi guys and girls raj here back with another video in this video we are going to learn how to retrieve images from s3 buckets so images are little trickier than regular files because image requires binary media support and i was searching the internet to do this for one of my projects and i couldn't find any good step-by-step walkthrough and after i did the project i decided to share my approaches using this youtube video so in this video we are going to retrieve images from s3 using two different ways one is using api gateway with a lambda integration and the other way is using api gateway but without any lambda we are going to do direct s3 integration we are going to go over designs of both these approaches as well as step by step demos that you can follow i have given the link to my code in the description and the timestamps are included as always for your viewing convenience all right with that let's start with option one so in this option uh we are going to retrieve image from s3 using a lambda so the user invokes the api and the api invokes the lambda and the lambda fetches the image from s3 bucket and then returns it let's jump into the console and see this in action so i have this s3 bucket created the name of the s3 bucket is cats versus dogs and i have the picture of my dog shadow and my cat max in this s3 bucket so all right so now let's create the lambda function and then we are going to go create the api okay i am in aws lambda console click create function and you can give a function name i named mine get image from s3 runtime select python 3.8 and under the role you can create a create new role but we have to go update it so that it can go and access s3 all right click create function all right now double click this lambda underscore function.py so this is just a straightforward hello world lambda now go to my github link that i have given in the description we have this code get picture from s3 dot py so you can either download or just copy from the code and then go back to the lambda remove the hello world code and just paste the code so what this code is doing is it is getting the name of the s3 bucket in a path parameter field and we are also getting the name of the file or the object within that bucket as a query string parameter so when we set up the api in api gateway i will go over how to set up this path parameter and query string parameter uh it's just creating a s3 bottom three connection and then we are getting the object from the bucket and the important part is we are getting the file content by reading the object and when we are sending it the content type is set as application slash jpg and it will be sent as a attachment so whoever calls the api will have an option to save the file and the body this is the important part is base64 encoded so whatever was read using this file obj dot read is converted to base64 encode that's how you transmit images because that's how it is different than regular file right you cannot just send the file as is it is not gonna go you have to uh transform it to a binary encoded format finally that is base64 and credit is set to true all right let's deploy this changes deployed but this is not gonna work because the role attached to the lambda does not have the permission to go to s3 so we're gonna change that so click configuration so this is like a new lambda console i think it just got released a couple days back so click permissions on the left and you can see this has access to do amazon cloud watch logs so we need to update it all right so click this role name under execution role click this link it should open up the iam console with the role loaded okay then attach policies and then search s3 for this demo i'm going to give amazon s3 full access click attach policy and we should be good if we go back to the lambda screen let's refresh this okay now you can see from this drop down under resource summary now this lambda roll has access to s3 buckets okay so the lambda is all set now let's go to api gateway to create the api i'm in api gateway console click create api scroll down rest api click build give a name of the api i gave the name get image from s3 with lambda very verbose name click create api now click actions and click create resource this is where you generally give the name of the resource but now we are going to create a path parameter to create a path parameter instead of just giving the name of the resource in the resource path you include it within curly braces so i give the name as bucket within curly braces under resource name just put bucket without any braces it's gonna give error if you try to put braces in the resource name click create resource and you can see the bucket within curly braces appear under the resource section and the curly braces means this is a path parameter so path parameter is basically a variable that you can pass to the back end so this is very similar to query string parameter but things will become more clear when you deploy it and run it using postman okay click actions and then click create method and we want to fetch the pictures so it will be get click the tick mark in the lambda function give the name of the lambda function we created get image from s3 and this is a lambda proxy integration so api gateway doesn't have to touch the header or anything everything will be dictated by the back end click save click ok all right now click method request remember we need two parameters to pass to the lambda one is the name of the bucket which is gonna be input into this bucket path parameter so if you click request path you can see the bucket is already here because it grabs it from the resource now we also have to define a query string parameter and the name of the query string parameter we are expecting if we go back to the code as file so i'm going to go back to api gateway click url query string parameters click add query string add file click this check mark and i'm also gonna make it a required field so api gateway is gonna check and to enforce the check under request validator click the pencil and then click validate query string parameters and headers click the check mark now one more thing we need to do remember this image require binary media support so now go on to the settings under this api i don't go under this settings under the whole api gateway we need to go to the settings of the specific api so click settings scroll down and the binary media types click plus and just give asterisk slash asterix so it's gonna support any binary media type click save changes go back to the resources click actions and then click deploy api click new stage give the name as version 1 click deploy okay our api is deployed so click this v1 click get so you can see the invoke url also has this curly braces and that's how you know et is expecting the name of the bucket in this field and it is going to pass this field to the back end all right so let's try to invoke this i'm gonna copy the link address open my postman in the postman now paste the link okay the curly braces come as this percentage seven so i'm gonna delete this so whatever i put here will be assigned to the path parameter bucket so i'm gonna give the name of my s3 bucket cats versus dogs and then i have to pass my query string parameter which is file equal to let's get picture of my dog first or shadow.jpg all right let's click send and this should work okay we got the binary media stream and you can see save response we can save this to a file how about shadow return click save and then let me open the file there you go this is a picture of my two-year-old german shepherd so all right so this is how you retrieve the image using api and lambda but remember if you were just fetching an image from s3 bucket since s3 is a aws service an api gateway is the native api management service you don't need a lambda in between api gateway can directly retrieve images from s3 or save images from this bucket this is also faster cheaper and involves less code all right so let's do this in step by step demo as well so to do this we are gonna upload a swagger file so aws gives these access binary files in s3 through an api gateway api it is little screwy though because it doesn't say what to do with couple of the things for example the role and also there is a section with this amazon aws.com so i'm going to show you guys and girls so with this link and i'm going to give this link in the description copy the whole thing and then open it up in your favorite ide so for me i'm going to open it up in visual studio code save this as json all right so we have to change couple of places so if we scroll down so see there is a role here so this is the role that the api assumes so this role you need to create and you have to give s3 access so let's do that let's go to iam click roles on the left click create role click api gateway click permissions so you don't have the option to add more policies here so you have to attach inline policies but for now click next review give a role name api s3 image role click create role now we have to create a policy i'm going to click this policy open in the new tab click create policy click json and you can just paste in this or type in this json so this action for this demo i'm giving it to all the services which is not advisable but you can specifically give s3 dot star give the bucket name all that stuff click tags click review give api get to a admin policy yeah i'm being a little lazy here because i plan to do more demos on api gateway so that i don't have to go change the policy every time okay click create policy okay go back to the role and then get the api s3 image role that we just created now click attach policies and then type in a part of the name of the policy we created we give api gateway admin policy so select this click attach policy okay our role is created with proper policies now this role arn copy this role go back to the json and replace this with the role scroll down and then you have to change this api gateway integration as well just past this okay scroll down a bit more and the servers you don't need to keep this you don't need to have this gateway domain and all that stuff for this demo so you can just remove this okay so now we are going to upload this to our api gateway api so copy this whole json then go back to api gateway all right go back to apis click create api rest api and then click import and here you are going to paste the open api 3 file so this is a really great way to export your apis or import apis without going through console so this is going to create the api components so click import okay you can see it created the resources one for get and one for put and if you go to the settings go down you see put two different binary media types jpeg image type and the octet stream uh go up i wanted to show you one more thing so how is this api getting the name of the bucket and the object if you go to method request click url query string parameter so it is getting everything in the key right so it's not getting in two separate parameters like the last example we saw bucket name is going to the path parameter bucket and the object name was going to query string parameter file in this case everything is coming to the key all right so click actions and then click deploy api to create a new stage v1 click deploy all right our api is deployed let's get copy this link go back to our postman now the query string parameter so this time we are going to pass the name of the bucket and the name of the object that we want to fetch separated by front slash so key equals to cats vs dogs slash max.jpg click send and this time it returned the picture of my cat max all right so now let's try the put part right all this time we have been retrieving image from s3 so now let's try to post a picture of a animal that my daughter thinks is better than cats and dogs so i'm going to copy this open another tab change this get to put paste the url and this time in the query string parameter key you have to pass the name of the bucket that you want to save the image in as well as the name of the object so we are passing the same bucket cards versus dog and it should be saved with better than cats and dogs.jpg and now go to body select binary and then you need to select the file that will be uploaded so for this i select better than catsanddocs.jpg from my local desktop so if i go to s3 real quick before i execute this api so currently there are only two objects so let's execute our api click send status is 200 okay go back to s3 bucket refresh so now you can see we got another object better than cats and dogs dot jpg so now let's retrieve this let's go back to the get and instead of sending max.jpg we want to say better than cats and dogs.jpg click send there we go this is a unicorn my daughter's favorite animal she thinks it's better than any other animal all right this is how you save or retrieve images from s3 bucket with api gateway in two different options one with lambda another without lambda if you like this video smash that like button click subscribe all the likes and subscribe really help this channel grow we are almost at 10 000 subscribers can't wait till we cross that milestone all right with that guys and girls i end this video i'll see you guys and girls in the next video bye

Channel: Cloud With Raj

Views: 26,592

Keywords:

Id: kc9XqcBLstw

Length: 18min 54sec (1134 seconds)

Published: Mon Mar 22 2021