Storing S3 Bucket Image URLs in MongoDB using Multer, NodeJS

hey fellow developers i hope you all are doing fine it has been almost two months since i last uploaded and i'm really very very sorry for that so without any delay let's continue on with the final video of our profile image upload to amazon s3 series so in the last two videos we made the front end of our application which you can see is this so here we can basically select a photo and then we are able to crop and also zoom that photo so now the only thing left is to upload this editor image to an amazon s3 bucket and the url that amazon s3 will return will be then stored in mongodb so this is a tax which is left and we're going to do that same exact talks in this video so yeah let's get started so now let's go to vs code and here you will find some files i will put the link to this files in the description and i recommend you download it so that you and i are both on the same code base for more clarity on the code and also the folder structures you can watch my very last video because as i already said this video is a direct continuation of my last video now i will go to the server folder and then controllers and then this user controller file so here i have written a very basic function called set profile pic so all our code related to our profile picture upload will be inside of this function so here i have concerned or locked the request.file subject so now let's try to upload this image okay i've already selected one so now let's click on this upload button and you can see that this is the cropped image that we have just sent from our front end so now our main job is to upload this image to amazon s3 so for that i will install some packages so those packages are malta and then monitor s3 and also aws sdk so the packages have been installed and i am gonna run npm audit fix to fix almost 6000 vulnerabilities so while it's getting fixed let's go to amazon s3 so i am already logged into my aws account which you can also do and then i will go to services and then s3 now let's create a new bucket which i will name as let's say profile picture upload youtube and then everything is fine and now just click on this create bucket button so our bucket has been created so now let's go to iam so from here we will be generating our api keys to access our s3 bucket now let's go to user groups and i'm just going to create a new user group so just click on this create group button and then i will name the group as let's say use s3 so this is the name of the group and below here we will have to attach some policies to this group that we are creating so the policy that i'm gonna search is s3 and here if you will see then there are a bunch of different policies so the one that i'm going to give to this group will be amazon s3 full access so now just click on this create group button and yes so this is our new group that we just create now let's go to users and here i'm going to add a user let's name this user as s3 user and we're going to specify this programmatic access right and then next permissions and then we will be adding this user to the group that we just created so we created a group called use s3 so now i'm gonna add this user to this group use s3 so now let's do next just you can just skip it it's not required so just review your settings before you create your user as you can see our group is use s3 this is the username s3 user okay so everything is fine so as you can see this is our access key id which i'm gonna copy and then i'm gonna paste it inside of our environment variables so here i'm gonna create a new variable called let's say uh s3 access key and paste it and also i'm gonna copy this secret access key which i'm going to name as s3 secret access so now we will be using this keys to access our s3 bucket so okay it's fine to close it now and we will also need the region in which our s3 packet is so i will again go to s3 and then i will click on here my s3 bucket is in this ap south 1 region so i'm going to copy this and i'm going to create a new variable called s3 bucket region and paste the string that i just copied so this variables are fine now we can carry on with our melter setup now let's import the three packages that we just installed so first i'm gonna import aws and then let's import malta and at the last i'm gonna import molten s3 so now in order to connect to the s3 bucket we have to do new aws dot s3 and here we have to pass an object that accepts few options so the first one will be xs key id which if you see then it is this s3 access key so we are going to pass this string here as the property value so that will be process dot nv dot as three access key and then the second option will be secret access key which will be this s3 secret access key and the last option will be the region in which our s3 packet is so that will be process dot env dot and the bucket region and we will store the return value in a variable let's say called s3 now let's use monitor to upload the files so i'm going to go inside of this set profile pic function and call monitor function so this accepts an object and here we can pass different options so for this project i'm just gonna pass one option which will be the storage option and here i'm gonna pass monitor s3 again this is a function which accepts an object and here also we can pass different options related to our monitor s3 so here what it accepts is s3 and then i will pass s3 so this s3 is basically this so we are storing the return value in a variable called s3 and we are passing that return value to this s3 right so now the other option will be bucket the bucket is basically the name of the bucket so if we remember then the name of the bucket is profile picture upload youtube right so this is the name of our bucket so i'm gonna just copy and paste it here as a string so so this is a bucket name right and then the third option will be metadata so this is not that important so i'm just gonna paste some code over here and the last option will be key this is a bit important so here let's write the function it will be function then request file and then cb so if you're wondering what is this cb then cb is just a callback function then inside of this function i'm gonna do cb null and the name of the file so this callback function accepts two parameters one is one i've passed as null and the second one will be the file name so let's say i'm gonna put filename as image dot jpg right so this will be the name of the file which will get stored in our s3 bucket so now let's store the return value in a variable called let's say upload hence from now onwards all operations will be done on this variable upload and also what i'm seeing is that the key and the value is same so in javascript if the key and the value is same we can just put a single value like s3 so this will work exactly same as before and also to make this piece of code reusable we can move this inside of a function let's say i'm going to call or write a function called upload and this will be an arrow function so let's cut and paste the code over here so i've created a function called upload right so to make this piece of code more reusable what i can do is that instead of putting a static bucket name i can do bucket name and then pass the bucket name over here so now this bucket name becomes a parameter right so whenever we call this function we can just put the bucket name and that bucket name will then come here and it will accept or we can say this can be used for different bucket names right now let's use this upload function inside of our set profile pic function so upload as you can see we have to pass our bucket name over here i'm going to pass this and in fact this will return another function which is called single so let's chain it dot single and it accepts a string which is which acts like a key so let's put the key as let's say image upload i'm going to show you what this exactly does or where this can be used just hang on a second okay and then we can store its return value inside of yet another variable called upload single now let's call this upload single function it will accept uh some parameters so the first one will be request and then response and the last one will be a callback function which in turn will accept a parameter called error and this will be an arrow function to be specific right so here i will basically run a check which basically says that if there is any error then we return it with a status code of let's say 400 and a response saying that success is false and a message with error dot message but if everything is fine then we can console. log our request dot file subject right and also i'm going to send this file subject back to the client so let's cut and paste this line over here so before we upload the image i'm gonna check one thing so for that we have to go to our front end code which is inside of this src folder components and then cropper.jsx so here if you see then i'm doing form data dot append cropped image right so here in our back end this string that we pass inside of this single function has to be exactly same as the one we are using here right so if this is cropped image then this has to be cropped image or if this is image upload then this has to be image update so let's do cropped image so i'm going to change this image upload string to clocked image and also inside of our index.js file which is present inside of this server folder you will see that we have used express file upload to parse the form data but this is not required since malta already does that so i'm just gonna remove it from here and let's also remove the import so now we are finally ready to test whether our file upload is actually happening or not so let's go to our front end and i'm going to click on this upload button what i'm seeing is undefined so request dot files is undefined but let's check whether something has been uploaded here or not so i'm going to refresh and yes as you can see we have successfully uploaded the file but here we are not getting anything so this is because our request.files is not present it will be request.file so the s will not be there so now if i check it again and yes we are getting this object so request.files dot file is this object right and if you see at the last we have a property called location so this is the location of the image which is present on our s3 bucket so we are going to use this image a link and upload it onto our mongodb database so before doing that let's go over some of these properties so the first one is field name so this comes from here so we have passed cropped image inside of the single function which ultimately becomes the field name and the original name comes from the front end because here as you can see we have uploaded our cropped image to the backend using this name using crop image.jpg which ultimately becomes the original name and here we have encoding mime type and the size of the uploaded image the bucket name and the key so this key is this right so we have used image.jpg as the key which you can see is here and also the name of the file is exactly same as the key so if you want different files i mean different file names you can do some operations over here to produce those dynamic file names right but let's not do that here right now and also one thing that you haven't noticed is if we try to upload multiple times we still are getting the same file name because we are using the same key right so whenever we upload it it just overwrites the previous file because the name or you can say the file name is exactly the same so that is the thing that you have to remember you cannot upload two files that have the same file name because then it will overwrite it please keep this thing in mind before like uploading files let's say we want to keep the files every time we upload so to do that we just have to make sure that whatever name we are passing it here should be unique so to make this string unique i'm going to convert it to a template string and add a timestamp to this name so to add a timestamp we just have to do date dot now so now this will give us a unique timestamp every time we upload because timestamps are something which is always unique so let's check it out i'm gonna upload this image couple of times so one two so i have uploaded this image twice so let's check how many files we now have here so in total we should have three files so let's do refresh and yes so now we have three files these two files are the files that i just uploaded and as you can see there's a number after this image string and this number is the timestamp in which this file was uploaded so this is how you can create unique file names now let's create our mongodb schema so i will just go to the models folder and i will create a file called user model and i'm gonna import mongoose to create a schema we have to do new mongoose dot schema and here we have to pass the field names so here i will just give one fill name which will be let's say photo url and this will be a string and then i will store its written value in a variable called let's say user schema and then we have to do mongoose dot model and the collection name will be user and i will pass the user schema now let's export but before that i have to store the return value in a variable let's say call user and then i will export this user variable so we have created our mongodb schema for the user collection so now let's go back to our user controller file and here i am going to import that user model and the location of the model was models and then user model now i'm gonna have to make this arrow function an async function because we will be using the await keyword so now let's do await user dot create and the field name was photo url so i'm gonna do photo url and if you remember then reg.file gave us the file object and the s3 url was present on a property called location so now we are basically storing the s3 url in this photo url field which will in turn become a document in our user collection so now let's test it out i'm gonna go back to a front end and let's say i'm gonna upload this picture and let's check our s3 bucket of how many images do we have now and yes we have four images now let's check our mongodb database and here i'm going to connect to our mongodb database by using the mongodb ui which i can find inside of our environment variable and this is the uri which we can use to connect to our database so i'm just going to paste the uri over here and click on the connect button so my database name was my first database and inside of this database as you can see we have a collection called users and this user's collection has one document and yes that document has a field called photo url which in turn is storing our url of the last uploaded image which we just did few seconds back so now i'm just gonna copy this url and go to our browser and let's paste it here so as you can see now we have an error which is saying access deny so by default all s3 buckets are private so in order to read or you can say view the image we have to make our s3 bucket public so to do that we just have to go to our bucket and then permissions and here as you can see we have blocked all public access enabled we have to first disable it so let's uncheck it and then save changes now we have to write confirm and then confirm so this has made our bucket public but still our objects or you can say the images are not public yet so to make that public we will have to write a bucket policy so the bucket policy is just some json data and i'm going to paste a policy so this is a policy which will make our objects or images public but before that i just have to replace this piece of string with our bucket name so bucket name is profile picture upload youtube so by writing this policy we have made our objects public so now let's save our changes and now if i refresh as you can see now i'm able to download the image right so let's download it and check so yes this is the image that we just downloaded so yes our profile picture upload is working fine right so now let's go back to our back end and here i'm gonna send instead of rec dot file object i'm gonna send the s3 url which is present inside of the location property so now let's go back to our front end and as you can see we are console logging the response that we are getting from our backend so let's go back to our website and i'm gonna upload this image again and here as you can see this is the response that we are getting from a back end and this is the url of the image that we just uploaded so now our main job is to replace our avatar with this image but there is a catch now going back to our app.jsx file uh before that let's close these files so in our app.jsx file we can see that we are using a render avatar functional component which is this so we are using this render avatar functional component which is this and here we are using the cropper component below right so this cropper component that we have here is being used inside of this avatar component and our avatar image is here right but we are getting the response inside of the proper component which is this so what we have to do is now i will just create a state inside of this avatar file let's say avatar and then set avatar and then i will pass this avatar here right and then we can pass this set avatar function to our render cropper component and then we can extract it over here and below when we get the response we can set our avatar to rest to dot data so this will change our avatar whenever we upload or you can say whenever we change our profile picture so let's test it out i'm going to select a new photo but now i'm going to select this one and then upload now let's clear it and as you can see here is the photo or you can say the uploaded photo right so everything is almost working fine so there are some small things that we have to fix and those things are like whenever i click on this upload button there should be a spinner over here right so to do that we just have to go back to crop.jsx file and import our spinner context which we can get not spinner sorry backdrop context which we can get from here so here i am exporting this backdrop context so i'm gonna do import backdrop context from backdrop and then backdrop so to use this backdrop context we have to first call react dot use context and pass the backdrop context that we just imported so this will give us if we see we have provided close backdrop and show backdrop as the value to this provider right so now we are going to extract these two functions from our backdrop context and now i'm gonna use the show backdrop function just before we actually make the api call right so this will start the spinner before the api call is made and then at last we can after the response has been passed we can then close the backdrop we will also have to close the backdrop in case the in case if there is any error right so let's test it out i'm gonna do again okay we have to select an image let's select this image now and as you can see we are getting the backdrop right so yes our image has been changed again so profile picture image upload is completed there are many things that you can improve on like we are now just storing the images right we have no idea which user has uploaded which image but that requires some added information like authentication we have to do many more other things right so that is totally outside the scope of this video so here my main aim was to show you how you can store the returned s3 url in a database like mongodb which we have achieved so with this we have come to the end of this video i hope you all like it and if you haven't subscribed my channel i recommend you do subscribe it because i make videos related to web development see you in the next video stay safe stay healthy bye bye

Channel: FrontendInterviewPro

Views: 26,638

Keywords: upload images in react, react easy crop, how to upload image to s3 bucket, uploading files in react, uploading images to s3 bucket using nodejs, uploading files to s3 bucket using nodejs, uploading files to s3 bucket using multer, crop zoom and upload images in react, how to crop images in react, crop image in react, how to make an image cropper in react, profile picture upload using react

Id: XC5rdgxyioY

Length: 27min 57sec (1677 seconds)

Published: Tue Jun 01 2021