Upload | Download | Delete | Files to Amazon S3 bucket using NodeJS | ADITYA JOSHI |

hey guys in this video we will be going to learn how we can upload file to the s3 bucket how we can download the file from the s3 bucket and how we can delete the file from the s3 bucket so for this uh let's get started so let me open my vs code here so i'm in my vs code now and let me just start by creating a folder where i'm going to keep all of my source code so let me create one folder here i will call it as s3 nodejs and let's just open this folder in vs code so here our folder is open now let's create a node.js project so i am going to do npm init hyphen y to initialize the project and our packaged audition is created now let's create one another file which is app.js and now uh we can install the required dependencies so i'm going to install express aws sdk multi and motor s3 so let's do npm install express aws sdk then multi and then multi h3 so we will going to use multiple for storing up the files to the aws bucket so let's install the dependencies okay so i think there is some typo so let me just fix this this should be multi multi s3 let's just download it okay so that dependencies are downloaded now let me just clear this and let's start uh so in order to interact with the s3 bucket we will be needing uh the aws credential and s3 bucket that is already created in our sd account so let's open the browser and create those credentials and s3 bucket so let me open a browser here here so i'm already logged into my s3 console you can see i'm already logged in and now let me just search for s3 so let's just open this s3 bucket and here let's create a bucket first so i'm going to click on the script bucket and let's give a name to our bucket so i'm going to give this the name as s3 node.js one two three four five six and uh also remember this region so this region we will be needing when we will be configuring our s3 client so let's uh click untick this block all public access because we want to have the public access to our s3 files so that we can download those files that have that we have uploaded to s3 bucket and let's check these two uh in order to make sure that uh security is implemented properly now let's just acknowledge this and click on create bucket so now our bucket is created now let's create the aws credentials so in order to create the credentials click on this there should be account information pay panel and here click on this my security credentials now here you can see this option access key access key id and access i mean secret access key so click on this and here you have to click on this button which says create new access key so now let's create the access keys so click on this and our access keys are created now you can just click on this show access key and we can just use them so let's uh open our vs code and put all these key there and to store all these keys i am going to create one env file so let me create one in my emv file where i am going to put all the credentials so the env file has been created and now let's put the put our key here so i am going to create those variables here access underscore key then access underscore secret then we need the region as well and then we need the bucket name let's call it as bucket now let's grab all these value so here let me just copy this key first let's put it here now secret key let's put it here and we need the region as well as bucket so that we can get from the s3 settings so click on s3 here just type s3 and open this link okay let's just quickly copy this copy the name and let's copy the region as well so the region is ap southwind so now we are done with the credential file now uh let's uh write the implementation so first i am going to create an express server so fonts express equal to require express and instance of the express is equal to this and then let's start the server on uh port 3001 so this is also done and now uh let's do one thing let's install one package which is uh dot env so that we can load those credentials from the env file so let me just install that package which is dot enb and now we can load that this amd file to this ngs file so we can just use require dot enb then config so now let's quickly require all the dependencies that we have installed so let me require first aws dependency so it const either is equal to require aws sdk then let's require the monitor as well and let's require the multi s3 as well so our dependencies are loaded here and now we can just quickly configure this aws instance with the environment variable with those access keys and region and bucket name so let's do this here let's call this aws.config dot update and let's put all those configuration here so here we need the secret key secret access key and let's load that key from our environment variable which is process dot env dot c ms uh oppose that it was access secret so let's just get that and in the similar way let's get the access id as well access id and let's do same thing process dot emv dot axis this should be key let's verify this this just this access key and in the similar way let's do this for the region as well so region same process dot env dot region so our s3 client is configured now so now let's create the configuration for the multi so so let's first create the s3 instance aws s3 is equal to aws.s3 this one and now let's do the configuration for the multi as well so const upload equal to monitor and let's here pass in that configuration so first we will going to define the storage and our storage is going to be the this one the storage which is multi s3 so multi s3 and here also we can define the functions so let's do them so let's define the bucket so let's create the bucket variable const bucket and let's load the value from the environment variable let's pass in the bucket name here so bucket and then let's define the s3 instance so s3 we have already initialized and now we can define some acs which is access control list basically here we define the permissions on the file so i want the file to have public read access so there so that they can be accessed publicly and now let's define the key as well so key basically means the file name and here let's uh do one thing so here we will be going to define one custom name to our file we have request here file and then we have the callback here and here let's call that callback function let's pass the error as null and then here we can pass the file name that we want to have for our uh files that are going to be stored into the uh s3 bucket so for now let's for the simplicity let me call i mean let me keep the name as is so let's do file dot original name file.original name and now so our this object is also created the upload object and now let's quickly start creating the endpoints so let me first format the code here and now let's create the endpoint so the solo so the very first endpoint that we are going to create is for uploading of the files so let's create that endpoint quickly app.post it is going to be a post request and i am calling the endpoint as upload then here request response and we have to put our middleware as well that we have defined here so let's put it here upload dot single so i want only thing one single file to be uploaded one at a time so here i can define the field name through which i will be receiving the file so let me call it as file only and now uh so now let's uh just return the statement because this this middleware has done its task of uploading the file so let's just simply return uh do the res dot send and return the response like this and if let's do console here as well so that we can see like what is coming inside this file object so we can do console.log request dot files and this this object basically contains a bunch of options and one of the property that this object contains the location which returns the location of the uh basically it returns the url of the file that we have uploaded so let's let's create another endpoint which is for listing up all the files that are available inside the s3 bucket so let's do app.get and let's call the endpoint as list then here let's define the request response and here we have to call the function from this s3 object so we're going to call sc dot list all but all objects v2 and here we have to pass the bucket name so let's pass the bucket name so market is our bucket name and let's let's promisify this so we can call promise on this and i can use await here and let's make this function as async because we are using the await keyword now let's store the response in some variable let's call it as r and this r variable contains a lot of properties but we are in all interested in the file names so we will going to filter out the file names so i'm going to res dot contents and here i can do map operation on the array and here i can just return the file name so it is present inside this item dot so key is basically the file name and let's store this value into some variable let's call it as x and let's return the response so i'm going to do less dot send x this endpoint is also created now let's create one other another endpoint to download the file so i am going to create one endpoint and it is going to be of get let's name the import as download and we need the file name as well so let's take the file name in the form of path variable file name request we need to pass and response as well and then we can download this file from the s3 bucket so let's do this using s3 dot get object and here we have to pass the file name as well as the bucket name so bucket we can give using this property bucket and then we have to get the key as well so key basically is the file name so here we can get grab this file name into one variable let's grab that const file name equals to request dot parents dot file name and i can put it here like this file name let's just use promise here so we can use promise and i have to use a weight keyword if i'm using promise and then here i have to use async as well let's store the response in some variable let's call it as x and i can just simply return the response so i can do res dot send x dot body so this function is also implemented now let's create another endpoint for for deleting a file from the sd bar so let's create one another endpoint app dot and this is going to be type of delete so i've got delete and let's call it as delete and then we have to pass the file name as the path variable then here again request response and let's just read that so we can use s3 dot delete delete object and here we have to pass the bucket name as well as the file name so bucket name we can respond directly because we already have that inside the variable and we have to pass the key as well so key basically means the file name and we can get the file name from this property so let me just grab the above snippet this thing and we can put it here file name so let's just promise this promisify this and let's use await keyword here and let's just return a static response so i can do res dot send file edited successfully so where all the four end points are created now it's time to test those endpoints so let me just start my application node application and let me do node app.js okay so we are missing something here okay we forgot to pass the async keyword here now let's clear up this and run yes so our application is running now i can open the postman and let's test these apis that we have created so let me open postman and let's just first try to upload a file so that is going to get a post request and let's put in the http colon localhost colon double slash localhost and it is running on port 3000 and the endpoint is slasher and we have to pass the file in the form of body in this form data and here we have to pass the keys so if you if you just take that look at the snippet here we have chosen the field name as file so we have to put this field name here so let's put it here file and let's upload one file here let's choose the type as file and let's select the file here so let me just use this file results csv let me just send the request now and if everything goes well we should be getting the success response right so you can see and the file has been applied successfully and here we have returned the response uh that is having the string i mean the url of the file and if we go to the vs code to show you here we have done this console statement console.request.file and here if you see we got a bunch of options bunch of properties are there like the field name the file name this is the original file name encoding mime type the size the bucket to which this file is uploaded the key basically the file name the acl is publicly the content type everything you can see here and let's to verify this let's open browser and see if the file is there or not so here if you if i just open this this bucket here you can see we have the file and now uh let's do one thing let's uh so if you look here carefully we have passed the file name as is but here if you want to put some unique identifier to this file as well then you can do it here so let's do let's append the date here so that every time we get a unique file name so let me do date dot now it is going to return us a numeric string and now let's try to run this application again node.js and let's try to upload the file so my postman is here let me choose with one different file this time so let me choose this pdf this time just click them click on send and here you can see our file got applied uploaded successfully and if you just and if you just go to the s3 console here you should see the file and if you carefully you look we got some additional parameters here so basically the date dot now has created the timestamp and it has appended it here here if you look at this this part we have basically appended the current timestamp to this uh file name so let me just remove this as of now because uh we are fine with that as well so now let me just clear this and let's uh try other endpoints so the second input that we are going to try is the list endpoint which is going to return the list of all the files that are available inside the bucket so let's open the postman let me type in line point here local host 2000 slash list and it is going to be a get request let's click on send so here you can see we got the name of the those files that are available inside the s3 and if you need more and if and if you need more properties what you can do is you can go here and right now i'm just returning up the key here but if you need more properties you can return them as is so let's just uh leave this as if as of now and let's try to download the file so let's move to the postman and let's download the file so this is the endpoint and here we have to pass the file name basically at the end so if you look this carefully here we have passed the file name as the path variable so we have to put that and let's just click on send so let me verify the file name as well so here the file name is this or let's use this this file only and let me just click on send so there you can see we got the file and if i just open the same link on the browser as well i should be getting the file so if i just put it here the file should be downloaded here you can see and you can see it is trying to open so the file has been opened so now let's try the other endpoint which is to delete the file so let's open the postman here let's type in the end point which is to delete the file localhost colon 3000 slash delete and here we have to pass the file name so let's grab the file name so let's say i want to delete this file let's just quickly copy the file name and let's put it here click on send okay so it should be of type delete so let's choose the delete type and then click on send so we got the message as file it successfully let's verify this here so if i try to refresh there you can see i have only one file so this endpoint is working perfectly so that's all for this video if you have any doubts or any queries then put them into the comment box and i will definitely try to answer them and if you enjoyed this video then please give a thumbs up and subscribe to our channel and stay tuned for the more updates

Channel: Aditya Joshi

Views: 33,042

Keywords: Apache Kafka, Apache Druid, Redash, Kafka Streaming, Report Visualization, Realtime Kafka, docker, blockchain, hyperledger, hyperledger fabric, kubernetes, big data, confluent kafka, javascript, typescript, docker compose, hyperledger explorer, etherium, besu, aditya joshi, blockhain world, coding, chain, block, java, python, server, client, crypto, Cryptocurrency, github, git

Id: hpO_3GwmBAk

Length: 24min 43sec (1483 seconds)

Published: Fri Oct 22 2021