Upload Images with React & Node JS to AWS S3

hey what's going on guys welcome back to another video today we'll be building an app that allows you to upload images and they'll be stored in S3 and then you could vetch those images and display them in the browser so if I go here and upload a new image you'll see that it loads and once it's stored it refreshes the posts and you'll see it here this is also fully responsive so it'll look nice on mobile and I'll be using Chalker UI to style the front end and you also don't need to know anything about AWS S3 I'll be going over the basics but it's basically just a file storage service which will allow us to store our images so without further Ado let's get into it [Music] all right so I have the starter files open we just have a button here that doesn't do anything and a section of posts that is empty so if I open up the code in the app we just have the Chalker provider and a container for our app and then this profile component which contains this vertical stack which is this card here and then the post component and this is a component that we're going to be changing so we have the upload button and just posts here then we also have an axios client and this basically creates an axios instance with this base URL which comes from our environment so you should Define your API endpoint right now I have it pointing to localhost 4000 and then whenever we use this axios client we're always going to be using that base URL and then we add these headers to every request and this mimics like an authentication token you would never want to do this in production but I just want to keep the video simple and so that's pretty much it for the front end and for the back end it's just a simple Express app with one endpoint defined and we'll be adding on to this as we need to so let's go back to the front end and start working on this upload button so the way we're going to do this is we're going to have a hidden input and when the user clicks on the button it's going to trigger that input and they'll be able to upload an image file so let's go ahead and create that input so import that from Chakra and then we want to add some properties to it so we want to First give it an ID and that will be image input and then we also want to give it a type and that'll be of type file we want to make sure it's hidden so that it's not displayed to the user and then we want to give it an on change Handler and let's come up here and just Define that really quick and this is just going to log the event then we'll use that in here and then for the button since we want it to be hooked up to the input we can use the html4 prop let's actually give it the as prop and we're going to tell them that we want it to display as a label and then it'll allow us to do the html4 prop and we want to use the ID for the input so let's just copy that over so now this is basically saying that we want to display a button but we want it to be a label and we want to hook up the label to this input by this ID and so if we come back to our app and we open up the console zoom in for it and then we hit this upload button it'll trigger that file input type and then we could upload an image and then we see the event here so we'll be able to grab the file off of it so let's go back here in our Handler and let's grab the file we can get the file that was passed by accessing the event so we can do cons file equals event dot Target dot files and then this will be an array of files that the user passed in in our case it'll just be one item so we want to grab that first one and now we have our file and so now if I just log the file so the console will be able to see that so come back here clear the console and upload the image again I think I just need to refresh and I'll try to re-upload that and now we see the file and this file object just has some properties about it like the name and the size and image type now in our Handler we want to do some basic validation so let's create an array up here of valid file types so we could do const valid file types it's equal to an array and then I'm just going to paste in some file types so we want to only accept these files so down here we can come and check if that file type is in this array so we'll do if and then we want to look at the file the valid file types and we're going to try to find the file type that was passed in and so we're going to call the find method on the valid file types and for each type we want to check if that type is equal to the file here that was passed in so we'll do file DOT type because that's the property down here and we want to check if it's not there so if it doesn't find the file type of the file that was passed in in this array then we want to return an error so let's create some error State first we'll just call it error and it'll be an empty string to start with and then we also need to import that from react and down here we want to call it a set error and then we'll just pass in a message like file must be jpeg or PNG format and then we want to return out of the function so that we don't continue on and now we also want to display that error message down here so we'll go right below the upload button and do a check so we'll check if error exists and if it does then we want to Output some text and that's going to come from Chakra and then we just want to Output the error and I'm going to add some props to this text so I'll just copy those over so now if we hit save and we try to upload a file that's not one of those valid file types like a Json file then you'll see that the error message is displayed because the file type is Json so that validation is working now the next thing we want to do is take that file and post it to the API so the way we do that is we need to create a HTML form data object then we could append the file that was passed to the form data and then we just post the form data to the back end we can just start by creating that form data object and this is built in so you don't need to import anything and then once we have this form data object we can call the append method and we need to give it a key so we'll just call it image and then the value and in our case it's the file that was passed in so once we have this all we need to do is post this form to our API I'm going to create a custom hook for posting data so let's create a folder called Hooks and then in here we'll create a file called use mutation so A mutation is usually when you're posting or changing data and here we want to create a react component so we'll do const use mutation it's equal to a function and this function is going to take some parameters and we want the URL and the method and if the user doesn't pass in a method we want to default it to post so in here we want to create some State and I'm just going to copy that over and we also need to import that from react so our state is just going to be an object with an is loading and error properties and then we first want to return that state so that when we use the component we'll have access to it and now the way that this Hook is going to work is we're going to define a function and return that function here and then when the user calls that function in the other components it's going to take care of making the API request and updating this state and all this is going to be abstracted away in this use mutation hook so the front end would not have to deal with it so let's create that function so we could do const FN you could name it whatever you want and it's going to be async and it's going to take some data and this is the data that you're going to want to post so then within this function we first want to set the state to tell the user that it's loading so we want to get the previous state and return a new object with the previous state spreaded and then we want to set is loading to true so now that we have that loading State we can make the API request so we want to import our axios client and I think I just need to open it up and then it'll Auto Import there we go and then in here we want to pass in the URL and this is just the config for axios so as the URL property the method and then the data these two properties will be the ones passed into the use mutation and then the function we return is going to take a new parameter this data and that will get passed to the axios client and then we want to just call the then catch so we can catch any errors and in the then block this will be called when it's a successful request and so in here we could just all set State and then we want to set is loading to false and error to an empty string if it was previously an error and then if there was an error we want to do the same thing except set the error property so we'll just copy that is voting will be false and then the error will be equal to the error that was caught here okay so now that we've defined this function we need to return it we'll just call it mutate and then pass in the function and then we want to spread the state so now anytime that we call this use mutation hook we're going to pass in some parameters and then it's going to return this mutate function and then we're going to take that mutate function and whenever we want to post data it'll run this chunk of code here and it'll set the loading State make the API request and then when it's either successful or error response we'll set that state in here so now in the other components we could just use this mutate and have access to the state let's go ahead and Export use mutation and now we can come back in our posts here and follow it and we want to destructure the properties from there so we'll call use mutation and then in here we want to pass those parameters so the URL and the method and I'm going to create a constant for the URL here because we're going to reuse that and it'll be slash images and we'll make this endpoint in the API eventually and so URL will be equal to URL and then we're not going to pass the method because we'll just use the default of post so now we can destructure the mutate function and we'll just rename it to upload image and then the other properties so is loading and we'll rename that to uploading and then we want to get the error and we'll just rename that to upload error now we have access to the function and the state all handled by this use mutation hook so now all we need to do is call this upload image at the end here once we've created the form so we'll just do await upload image and then the data we want to pass in is that form and we need to turn this into an async function and now when we call upload image we're passing in that form and so that's basically calling this function with this data and so our data will be that form so if I go back here to the front end and open up the network tab let me just refresh when I go to upload an image and I select a valid image type you'll see in the network you'll see that request and you see it was a post request it's a slash images and if I look at the payload you'll see that it's form data which is good it's exactly what we wanted but it returns 404 because we haven't defined that endpoint here before we make that API endpoint I want to go back to the post and add the loading and error States for the upload so we'll take uploading and then on the button we want to pass the is loading prop and whenever this is voting prop is true it'll display a loading state so we want to make it equal to uploading so when uploading is true the button will be displayed in a loading State and then we also want to add the error text but instead of just copy and pasting this I want to create an error text component which is just going to be more reusable so I'll just come up here and create that component here so the error text component is just going to take the children and any other props and output this text component with the font size and color and just output the children because this size and color is going to be reused so down here instead of displaying text we'll display error text and then we just want to copy this and handle the upload error we'll just paste that in there and now we can get rid of these props and they should all look the same and we actually need to put this as the upload error and then in our use mutation we need to set this error to the error.message because we just want that text string not the entire error object itself so now if we hit save and we refresh you'll see that when you try to upload an image you'll see the error text here so that's all working good and now let's create that API endpoint so let's come in here and change this to a post request and it'll be to slash images and then we're going to be using molter to parse the images from the request and it makes it really easy to to grab it and in order to use it we need to create a constant up here and we'll call it upload and then it's going to be equal to multer and we're going to invoke it and then in here we can pass some options one of them is storage so let's create that storage variable up here so do con storage and we're going to call another method that comes from multer and it's called memory storage and we're going to invoke that and this basically allows you to store the image that was uploaded in memory so we can then upload it to S3 uh we could also pass in a destination if we wanted to save the file somewhere in our local directory but we don't want to do that we're going to store it on S3 now that we have this we can call this upload middleware and so we just put it in front of our request Handler and we want to call the single method this basically will parse a single file from the request if we have multiple files then we need to call the other methods and then in here as a parameter we need to pass the name for the file that we're passing in so in our case if we look at our front end on the form we're calling the key image and so that's what we have to pass in here so molter is going to look at the form data for this key in order to try to parse the file and if it finds it then it'll add the file to the request object here so that in our Handler we can then get access to the file that was passed so we can come here and just log the file and I want to grab a couple of other things so in here I'm grabbing the file off of the request object and then I'm also grabbing the user ID from the headers and that's where we're setting this in the axios client and then I'm also just doing a small check like if there's no file or user ID then return a 400 which is bad request and then I also want to log the file just to show you guys what that looks like so now let's open up and query the console now if we upload a file we should see that on the back end and there it is so you can see the field name which is image the original name image 2 and here's the buffer this buffer is what we're actually going to send to S3 and then S3 will store this image you can also set some file validation on multir and that would be a good practice but just for the sake of time I'm not going to do it but you could pass it in here file filter and then pass in a function that will filter based on the file type that was passed in but we're not going to do that for now but you would want to do that in production so now we need to upload that file to S3 and to get started you should log into your AWS console and if you don't have an account create one and then head over to S3 you're going to want to click create bucket give the bucket a name so I'm just going to paste this one in because that's what in my EnV file here and then you could basically leave all these default values and just click create bucket and now that we have the bucket created we're basically going to store each image under the prefix of the user ID and then the image name in the bucket we're going to have files with like user123 and then the file name create our S3 client and we come in here and do S3 and the way that the S3 SDK works is you create an S3 client and then you could call the send method passing in different commands so let's start by grabbing the bucket and creating the client so we can import that from the SDK and then let's create a function called upload to S3 and in here we're going to take some parameters and that'll be the file and the user ID and then here we want to define the object that we want to upload so first let's give it a key so we'll do cons key equals and then I'm just going to generate a random uuid but first we want to give that user ID prefix so we're going to use template strings here and do the user ID slash and then we need to generate the uuid so I'll just call that then we're just going to import that from the uuid package okay so now that we have this key it's going to be a user ID and some random generated ID and now we need to create the command for the S3 client the command we want is the put object command because we're putting or adding an object to our bucket so we'll do cons command equals we're going to call new put object command so that comes from the S3 SDK and then in here we want to pass in an object the bucket is required so we need to do bucket and we'll set it equal to the bucket in our environment so up here we also need to give it a key so we'll do T we'll set that equal to the key that was generated up here and then we also want to pass in a body and this will be the file bot right so we'll do body we want that to be equal to the file dot buffer because we want this stream of binary data we want that to be the actual content of the file that gets stored um and then lastly we could give it a content type the content type will be the file Dot and then we want the mime type property because that has the file type okay so now that we have this command all we need to do is call the send method on the S3 client and then that will send the request so in here we could put the command and we want to wrap this in a try catch because it'll throw an error if something goes wrong and so we want to catch that error uh we want to also log it to the console and then we'll just return an object with that error property set and then we'll use that in the Handler to check for errors so we'll try to await this if this was successful then we just want to return the key and we'll eventually return that back to the front end so now let's export this and use it in our Handler so after doing the validation we'll call that method or function and we want to import that and then we need to pass in the necessary thing so the file and the user ID and we're already checking that they exist so we don't need to do that in here and then this function returns either an error or the key and we want to check if there was an error and we'll just return a 500 if there was and I'm just going to paste that in real quick so if there was an error return a 500 with the error message if there was no error we just want to return the key of the object that was created and we're not actually using this but it may be helpful in the future now that we have all this we can remove that log and if we go back to the front end and upload an image we should be able to see that in S3 this will do image one upload it and there was an error so let's see okay so looks like we're getting an import error uh this has to be module yeah I didn't do the correct file type okay so now this should all work and there's actually one more thing we need to do in order for the S3 SDK to work you need to tell it which credentials to use so if you've never set up AWS credentials then you should look up how to do that and I'll leave a link in the description but it basically configures different credentials for your different AWS accounts on your machine at this directory so your root and then in the AWS folder so if you open this up you'll see all your profiles in here so you could see that there's a default profile and then there's this named profile and these keys are fake but in your case they would be your actual access keys and secrets and so in order to tell the SDK which profile to use you can go in the EnV and either set this AWS profile variable and the SDK automatically checks for this and then you could pass in the profile name so you could pass in this or if you just leave this out then the SDK will just use the default on your machine but if you have no default profile defined and you didn't specify another profile then the SDK will throw an error saying that it couldn't find credentials so you want to make sure that you set up your AWS credentials and you either use the profile here or just use the default which is what I'm going to do you want to make sure you restart the server once you've added those and the SDK should work all right so let's start this back up let's close this out and now when we upload the image it should get stored in S3 so let's do image one and see the loading State and then if we refresh our bucket you'll see one two three that's the user ID and then this was the randomly generated ID and if we open this up we see that photo that was uploaded so now that everything's uploading successfully we should let the user know that the upload was successful because there is nothing displayed to them so let's go back into the use mutation and here on the API request when it's successful we want to add a toast to let them know that everything went well so let's actually come up here and create the toast so we'll do cons toast equals used host and that comes from Chakra this returns a function that we can invoke and configure the toast with so let's come up here and after we set the state we want to invoke the toast and then in here we could pass all these options I'm just going to paste in the options so we want the title to be successfully added image we want the status to be success so it's green and we want it for two seconds and the position will be at the top we save this and we come back here and we try to upload an image I'll do image 2. we should see successfully added image and then that again is stored here now we need to work on fetching those posts let's go back to the front end and create another hook and we'll call it use Query so let's define it here and again it's going to be a function and it's going to take a URL as a parameter in here we want to Define some State again so I'm just going to copy and paste that in and then import that at the top and then we're going to want to return that state inside of this use Query we want to run a use effect and so that's going to run right when the component renders let's call use effect and in here we're going to pass a function and then let's create the dependency array so this will only run once when the component renders and then whenever the URL changes and that's what we pass in here and so inside the use effect we want to Define an async function we'll just do fetch equals an async function after we Define it we're just going to want to call it here so when the component renders it creates this fetch function and then just calls it and this fetch function is where we're going to actually make the API call let's bring in the axios client and then we're going to want to call the get method on the client and then we just need to pass in the URL and since it to get request it doesn't need any more parameters so then again let's tack on the then catch blocks and in here we're going to get the axios result and on that result is going to be a data property so let's just destructure the data and then we want to call set state we're going to set the data and that'll be equal to the data that we destructured and then set is loading to false and error to an empty string so very similar to the other hook I mean here we're going to want to catch the error if there was any just call set State again this time data will be null because there was an error is voting will be false and then the error we want to be the error.message okay so now that we have this call here we'll be able to access the data and the loading State and error state from other components so let's just export this and now we'll come back to our post component and let's use it up here so we'll do const we'll call use Query okay now in here we need to pass in the URL and we'll get access to the different properties of returns so again we're going to have data and this is going to be equal to the image URLs that'll make more sense later and we're just going to default it to an empty array and then we want to get the loading State and this will be renamed to images loading and then the error and that will be called Fetch error we should also display the error down here so we're going to check if there is a fetch error and if there was we're going to use the error text and then we don't need these properties but we want to be text align left and you remember we're spreading the properties up here where we should be okay so now it'll be text align weft and there's our error message now we also need to add the loading State let's say images loading and then we'll check here if the images are loading we want to Output circular progress and then I'm going to paste in some properties so just setting the colors and size and all that if we just set this to True you'll be able to see that loading spinner there so that's all working as expected so we'll set this back to images loading okay now let's deal with this error and create that API request so let's go back to the API and before we Define the new route I want to explain how this is all going to work so when you upload an image to an S3 bucket you can view that image by just clicking on it and then clicking open here and you can see here that there's this URL here and this is an AWS signed URL so it has a bunch of query parameters like the Amazon signature and the ID and a couple other things so you can take this URL and you can view it but this assigned URL expires and the default is 15 minutes if you remember this bucket we turned off the public access to it so if you come here block Public Access is on but here we are viewing the image publicly and that's because of this pre-signed URL so pre-signed URLs allow a user or a client to temporarily have access to an object in a bucket and one that they normally would not have access to and it's only for a short amount of time and so what we need to do is create pre-signed URLs for all of the images in this bucket and then return those URLs to the front end and that front end will then create images with the sources being those pre-signed URLs with all that in mind let's come back and create a Handler so we'll do app.get and it'll be to slash images we want it to be async and then we don't need the file anymore we do want the user ID and we just remove all that stuff well we could actually check for the user ID okay so now let's uh create a function that will retrieve all those pre-signed urls so let's come back to the S3 file here and the first thing that we need to do is get all of the keys for the objects that we need to create pre-signed URLs for so the first thing we need to do is create a function that will get all the keys that we need in order to generate the pre-signed URLs so we're only going to generate URLs for the keys that we find and we want to get all of the keys under this user ID one two three so let's create a function here called get image keys by user and this is going to be an async function it's going to take in a user ID and then the AWS SDK gives us the list objects B2 command and that basically allows us to list all the objects by a specific prefix in a bucket so I'm just going to copy that command and it works the same way where we call new list objects V2 and we need to import this at the top and so it's going to take a bucket and I'm going to pass in the bucket and the prefix will be the user ID so it's going to tell the SDK that we want to get all the objects under the user ID prefix so it'll be under one two three in our case so it'll get all of these objects we need to call the send method make it equal to client or S3 dot send and then we want to pass in that command that we just created and so this list objects B2 command returns an array with some Properties or an object and the ones we want are the contents and this is going to be an array of all the objects that were found so if there's no objects it's going to be null so we want to just default it to an empty array and now that we have these contents we want to map through them and get all the keys from them so we'll just call return we're going to take the contents and map through each one for each image we're going to want to return the image dot e so on the image that's returned you get all these properties like the key of the image the size and all these things so we want the key so now at the end of this call we're basically just going to get an array of all the keys that we need to generate pre-signed URLs for so let's export a new function called get user pre-signed urls this is also going to be async and it'll take a user ID and now in here we want to wrap everything in a try catch so that we could catch any AWS errors and the first thing we want to do is call this function we'll do const keys or image keys and we want it to be equal to get image keys by user and then we want to pass in that user ID so now image keys will be an array of all the keys if there was an error we want to log it and then also just return that object with the error set on it so now that we have these image Keys we want to generate a pre-signed URL for each key and the way we do that is by using the get signed URL function and that comes from the S3 request pre-signer SDK so that should be Auto imported and you can see as the parameters we pass in the client so that'll be S3 and then a command here so we'll just leave that for now and then lastly in options object this has different properties like expires in and this is the number of seconds the default is 900 so if you guys want to increase it or decrease it's up to you I'll just leave it at the default now we need to create the command so this command basically tells this get signed URL function what it needs to run in order to get the object that you want to pre-sign I'm just going to copy what we did above and we'll do a get object command we pass in the bucket and the key that's going to get that object and then it's based basically going to generate the pre-signed URL so we need to do this Logic for every single key in the array let's take the image keys and we're going to map through them and then for each key we want to do this logic and now this key will be passed in for the current key in the array we want to return this now this image Keys map function returns an array so let's just capture that and we'll do pre-signed URLs make it equal to the output of the map function and so the map function basically Loops through each item in an array and returns a new array with the output of this function that's passed in and so we take every key we create a command to get that key from the bucket and then we generate a pre-signed URL using that command so what eventually gets returned is the output of this function and the output of this function is a string which would be the pre-signed URL in order to get all the pre-signed URLs we could just wrap everything in a promise.all and you can see here that the promise at all creates a promise that is resolved with an array of results when all the provided promises resolve or reject so we'll just copy this into the promise.all now this is going to be an array of promises and there's some syntax that I'm missing a parenthesis so now these pre-signed URLs it's going to eventually resolve to an array of strings here I hope that makes sense now that we have these pre-signed URLs we can just return them so we'll do a return so we can come back to our Handler and we can destructure the properties that it returns so let's call it here and we need to pass in the user ID again and now in here we're either going to get an error or the pre-signed urls and then we want to check if there was an error and if there was then we'll just do a 400 and then we want to return the pre-signed urls so now this endpoint will should return an array of strings and those strings are going to be the pre-signed urls let's go back to our browser and just refresh if we open up the network tab we should see a get request to images and I think we missed in a way somewhere so oh yeah we need to await that and then in here we also need to await these promises okay so now we come back here and refresh we should see those images and here they are it's an array with those two links and now if we just copy the value here and we open up the image in a browser we can see it here so now that we get this array returned we can output images using these links so let's go back to the posts here we have our image URLs here so let's come down and we're going to use a simple grid to display them and here we can Define some columns and in Chakra in order to pass responsive values in you could pass in an array and then each value will correspond to the screen size so on the smallest screen size we want one column meaning it'll take up the entire container and then on small or medium we want two everything above that we want three columns we also want to give it some spacing so we'll do spacing equal to four and now in here we want to Output all the images let's take the image URLs and we want to map through each one for each URL we want to Output an image so we'll use the image from Chakra and in here for the source we're going to set that equal to the URL the key will also just be the URL because that's unique and then we'll just give it an ALT of image okay so now if we hit save we should see those images pop up here and there they are and the app is crashing for some reason and I think we just need a question mark here so checks if image URLs are defined and if they are then mapped through all of them so now if we refresh we will see the images and there are no errors we also want to handle when a user doesn't have any images I'm just going to copy this in so we're going to check if there's no error and the image URL is a length is zero then we're going to just output no images found and this isn't going to be error text this will just be grayed out in here we actually want to check if the image URL away is greater than zero then we'll map through and we don't need this question mark anymore so now it all works and then I also want to add a border radius to the images just to style them a bit so now if we upload an image let's do image three it was successfully added but it doesn't display here until we actually refresh so let's trigger a refetch when someone uploads an image the way we're going to do that is we need to add something to the use Query hook we need to add another dependency to this use effect call so we'll take in a second parameter here we'll call it refetch and then we'll pass that to the dependency array so now anytime that the URL or refetch changes it's going to rerun the use effect and then rerun this fetch which will get the data and set it in the state so now that we're passing that we can go back to the posts and we need to set some State for the refetch because we need to know when we want to refetch let's just create some State here we'll call it refetch and then it's going to start at zero we're also going to pass this refetch into the use Query as the second parameter and now anytime we want to refetch the query down here we just need to change this variable so right here in our handle upload we can call set refetch and then we'll take the state and return State plus one so it'll be zero and then when this uploads we set the refetch to one higher so then when it comes to the use Query hook one is obviously different than zero so it'll trigger the refetch the other thing to note with S3 when we upload the image there is still some time where the image isn't fully accessible and so when you upload it if we trigger the refetch right away we won't actually get that image in the results so we need to actually set like a timeout let's add that foreign so in this timeout we're just going to run a function that calls set refetch after one second so once we've uploaded the image we're going to wait a second and then call set refetch which will then trigger refetching we should see that being displayed here with all the new images so we have three images now I'll upload image four so it waits a second and then it'll refetch but you can see here image 4 was just added here instead of at the top and the reason for that is by default when we get all the images these contents are not sorted or they're sorted rather by the key not by the date and so we need to manually sort them because S3 doesn't provide a way to fetch them sorting by the date so let's just call the sort method in here I'm just gonna paste in a handy uh sort function that will sort everything to have the latest object be first so if I save that and we refresh we should see that this image was the most recent one and that appears first if I go and upload image 5 you should see that it gets uploaded and then it displays first that's it for this video thank you guys so much for watching I hope you enjoyed leave a like if you did and subscribe if you haven't already it would help me out a ton and let me know in the comments if you have any questions or what you guys want to see next so thanks again and I'll see you in the next one foreign

Channel: Onelight Web Dev

Views: 10,918

Keywords: javascript, react, node, aws, aws s3, node api, chakra, chakra ui, react chakra, react ui, react tutorial, aws tutorial, node tutorial, node images, node image uploads, react and node, fullstack app, react fullstack

Id: vVBqEYNXxy8

Length: 42min 17sec (2537 seconds)

Published: Fri Oct 21 2022