Use Presigned PUT URLs to Easily Upload Files to AWS S3

I saw this tweet from Theo the other day about using pre-signed URLs to upload image and video files to S3 from your next or trpc application and I thought this is a great excuse to farm more views from Theo I'm kidding this is genuinely a good way to upload your image and video files to S3 the problem that we're trying to solve here is we want to have a private bucket but we want to allow some users to be able to upload videos and images so what we're going to do is we're going to make a server-side request to get a pre-signed URL and that pre-signed URL is going to last for a finite amount of time say 60 seconds then we're going to use that URL to make a port request to S3 from the client to upload our files let's have a look at this little diagram here so we have our user interface and then we're going to make a get request to our Lambda or API route in our case we're just going to use a next.js API route this could be a Lambda it could also be a trpc Handler then once we get our route back we're going to use that URL and we're going to make a put request to it with our resource and then that's going to go to S3 this demonstration is going to cover everything you need to have a very simple file upload to S3 using pre-signed URLs but I'm not going to do any styling and I'm only going to use axios as a library the reason I'm going to do this is because the requirements for this sort of implementation vary so much so for example somebody might just want to upload one file somebody might want to upload many files there might be some authentication in there and all sorts of stuff and so once you have this basis you're going to be able to take that and then convert this into what you need for your application so I'm going to start here in the terminal and I'm just going to bootstrap a next.js application so yarn create next app and I'm going to call this pre-signed S3 so once my application has finished bootstrapping I'm going to CD into pre-signed S3 and I'm going to open this up in vs code okay so now that we have our application set up I'm going to create a new file here and I'm going to call this dot end and in Dot N we need four things we need our access t we need our secrets key we need our bucket name and we also need our region so let's go over to AWS and get all four of these things here so I'm signed into AWS if you don't have an AWS account now is a good time to pause the video and go sign up but I'm going to go into S3 once I'm in S3 I'm going to create a new bucket here and I'm just going to call this free signed URL demo and I'm just going to use this us East one region here and I'm going to leave all of these default here I'm going to block all public access and then I'm just going to create the bucket so my bucket has been created we have the first requirement here and that is our pre-signed URL demo so I'm going to fill my bucket name in here then I'm going to go into the bucket and we just need to add some cordless properties here so I'm going to go up to permissions then I'm going to go down to cross origin resource sharing and I'm going to edit this inside of here I'm going to put this array and so if you have a look in the description below you'll see a link to a GitHub repository where you can find this demo and there'll be a notes file in there inside of that notes file you'll find this array so this is going to set some course policies for us the two that we need to have a look at are the allowed Origins and we're just going to use a star here if you want this to be extra secure then you should put in your domain you can create several different policies so we have an array of objects here and I could put in my localhost domain and then I could put in my staging domain and then I could also put in my production domain the other property is the allowed methods so for now we only need put but if you were using pre-signed post URLs or pre-signed get URLs then you would also need to include those methods I'm going to save these changes okay so this is my bucket setup now I need to get some credentials so I'm going to come over to my user and then I'm going to click security credentials next I'm going to come over to access keys and I'm going to create new access key I'm just going to show my access key here I'm going to copy the access key ID then I'm also going to copy my access key Secret we also need my region here so if we come back to the bucket you can see that my region is Us East one so I'm going to paste that in there okay so the next thing we want to do is to create our API route for getting our pre-signed put URL so I'm going to come into pages and API and I'm just going to reuse this hollow route here and I'm going to rename this to Media okay so we can remove this data type here so I want to create a new instance of S3 so firstly we need to install some dependencies so I'm going to yarn add AWS SDK and I also want to use axios instead of the fetch API so I want to import S3 from AWS SDK slash clients slash S3 next I want to say const S3 is equal to new S3 and I want to pass in the API version and I want this version to be 2006. 03 zero one I want to set my access key ID and I want this to be equal to process.m Dot and if we come back to our n file here it's going to be access key then I want to set my secrets access key and it's going to be process.m Dot secret key next I want to set my region and this is going to be process.n.region now the last property is very important this is going to be signature version and I want to use version 4. if you don't include this you're going to get some funky Behavior so I'm going to remove data from here so the first thing I want to do is generate a key and the key is going to be the file name so you could get the file name from the file that the user uploads or you could generate this yourself I do recommend generating this yourself using uuid or some sort of random character generator the reason I recommend this is because if you upload files with the same name you'll get a collision and the new file will overwrite the old file to avoid that you should generate some key that you know is going to be unique so I'm going to say const key and I'm going to use a capital letter here it's equal to random uuid and this just comes from the crypto package so we don't need to install entity dependencies for that and then I want to add a DOT and I want to get the extension of the file that the user is uploading so I'm going to say const EX for extension is equal to request Dot query dot file name and we're going to add the file name to the request query and I'm going to cast this as a string then I'm going to call Dot split slash and I'm just going to get the first parameter so this is actually going to be file type the reason we're going to get the first parameter is because the file type is going to be something like image slash jpg and we want to get jpg so we can add it here so it's going to be random uuid Dot jpeg okay so now we need to Define some parameters for S3 so I'm going to say const S3 params is equal to an object the bucket that I want to put this in is going to be process.m dot bucket name then I want to set my key then this is going to be expires and I'm going to set this to 60 so this is going to be the link expiring after 60 Seconds so these are single use links now I want to set the content type 2 image slash and I just want to add the extension here so EX and we'll make these strings so when the user on the client goes to use this pre-signed URL it needs to match this signature so if they try to upload a DOT exe file for example but here we have a JPEG file the signature won't match and the upload will fail okay so now we need to get the upload URL so I'm going to say const upload URL is equal to a weight S3 Dot get signed URL and now we need to define the type of signed URL we need so I'm going to say put object and then I want to pass in the S3 params and we're getting an error here so I assume my Handler needs to be an async function okay so I want to return with a status code of 200 and I want to return with the upload URL and I also want to return the key so I'm just going to say key with lowercase k and then key okay so this looks good for our API Handler let's go to our index page and I'm just going to remove everything here and say function upload and then export defaults upload and I want to return and I'm just going to return a fragment and I'm going to have a P tag that says please select file to upload and I'm going to have a form inside of the form I'm going to have an input element the type is going to be file we're going to accept and I want to accept image slash jpg and image slash PNG and let's change this to jpeg and then I want to name this field and I just want you to name it file next I want to have a button and the buttons type is going to be submit and then I'm going to give it a label of upload the next thing we want to do is to create a submit Handler for this form so I'm going to say async function handle submit and this is going to take the event and then the type of this event is going to be change event and then we need to pass in HTML form element into the generic the first thing we want to do in here is called event Dot prevents default and then this is going to stop the page from refreshing when we submit the event so we can use this Handler so in form I can say on submit is equal to handle submit now I want to create another function for handling the upload to S3 so I'm going to say async function upload to S3 and I'm just going to pass in my event again so we can call this function so we don't forget so I'm going to say const key this function is going to return our key is equal to a weight upload to S3 and we can just pass in our event so the first thing I want to do is to create a new form data so I'm going to say const form data is equal to new form data and I want to pass in my event dot Target then I can get my file so I can say const file is equal to form data dot get and now I can get a property by its name and I want to get file and the reason I can do that is because this input element has a name of file next I just want to say if not file then I want to return null so if the user has clicked submit but hasn't attached a file we're just going to return null the next thing I want to do is to get the file type I'm going to say const file type the reason we need this is because in our pre-signed URL we're going to attach our extension to our content type and when we use our put request it needs to map the signature for this pre-signed URL so our file type is equal to encode URI component and then this is going to be file DOT type and for some reason type does not exist on the form entry value but if your console log this it does exist so I'm just going to have to ignore this which is a little bit unfortunate okay so the next thing I want to do is to get our pre-signed URL so I'm saying cons and I just want to get a data property it's equal to a weight axios.get you can obviously just use the fetch API if you like I'm just used to using axios slash API slash media and I want to add a query string and that query string is going to be our file type so I'm going to say file type is equal to out file type then out of data I can say const it's equal to data and now out of here I can get an upload URL and our key now that we have our upload URL we can use this to make a put request so I can say await axios dot put and I want to make a put request to our upload URL I want to attach our file and now we can just return our key let's go over to the browser and try this out so I have the network tab open here I'm going to select a file so I'm going to select this grapefruit click upload and you can see we're making this get request here to our localhost API media and then this is going to respond with an upload URL then we're going to use that URL to make a post request and now we can go over to S3 and we can see in our bucket here that we have uploaded a file so let's open this up and I'm just going to click open and we need to save this and you can see that we have our grapefruit uploaded let's try this again with a different file so I'm going to upload my pair you can see again we're making the get request then we're making the put request you can go back to S3 we have two files now I'm going to get the latest file here click open and we can save this and this is our pair so that is how to upload images and videos to S3 using pre-signed put requests if you enjoyed this video please make sure you leave a thumbs up and subscribe so you get notified whenever I release new videos thank you for watching and I'll see you next time

Channel: TomDoesTech

Views: 23,021

Keywords:

Id: wbNyipJw9rI

Length: 16min 51sec (1011 seconds)

Published: Wed Aug 31 2022