Upload Image to Google Storage with NodeJS + HTML Form

hey what's up guys today i'm going to be showing you how to upload a file to google storage using the google storage api essentially what we're going to be doing is submitting a image file through a html form sending it to our express server and our express server will upload it to google storage by the google storage node.js client libraries so that's something you're interested in strap in and we'll get into it alright guys i've just opened up a new empty folder in the visual studio code i'm gonna open up a new terminal here and just say npm in it i'll just continue through these all right so from here what we're gonna want is we would need to get express so to say npm i express um we're also going to use malta today because i want to use that for handling the the file upload so i'm just going to say npm i express malta and we'll also need to get the um the google storage uh packaging as well or npm packages in as well um so i'll just say npm i at google cloud storage okay so i'll take a second to download in the meantime i'm going to create now our index.js which is where our server is going to leave our express server is going to live and we'll just require express here so say const express equals require express and we'll define our port here com support equals 8080 and then we can turn this on by saying app.listen then i'll just turn this on and just say node index and there we go server started on port 8080. all right cool so from here uh basically what i want to do now is to serve our index.html when somebody comes to the root of our application so i'm just going to say app.get root and we'll say um rec res request response i'm just going to say res.send file uh index.html which we haven't we haven't built yet but we'll do that right now so index.html all right so i just want to make sure this works now so if i go to a browser and i come here and we'll go to localhost 8080 at the root and we cannot get that why can't we get that ah because we haven't restarted our server which reminds me actually i'm going to install nodemon so we don't keep restarting and refreshing it all the time it'll just do it itself so mpi npm install nodemon great and now we can just say up here i will create a new script in our package.json dev and we'll say nodemon index and now we can say npm run dev now no mom's just going to keep watching our server file for any changes um okay so from here i think we can go back now to our browser which i get this path must supposed to be absolute oh yes okay so we do need to send the full path it doesn't know so to do that uh we can just say their name plus index or html i think that should work go back to here there we go all right so now that we're getting our index.html file i basically want to build the format so that we can sort of just have a user upload something from here and then we'll we'll post that to our endpoint so in our index.html i'm just going to have an input here and it's going to accept the file and we're going to call the name image file all right so now i'm just gonna say it can accept images and only images of jpeg or jpg so if i just save this now go back to our browser refresh that um we should see here that i can only accept jpg files right and there's a handsome picture of me right there um so we've created our input so now we just need a button uh which will submit uh the image that we upload here so i'm just going to say button and we'll say submit all right so now we've got our input ready and the submit button there we just need some javascript now to handle the submit button and post it to the endpoint on our express server which we haven't created yet but uh we'll do this side the client side first so if we create now a new uh javascript file we'll just say submit dot js first of what we need to do is actually get a reference to this this submit button so i should have given this an id sorry so let's say submit button and now we can add a event listener to our submit button so if we say document.getelementbyid submit button and then add event listener click and it's going to take in the function and it can be anonymous because we don't really need a defined function for this one i do want to just copy in some code here which i've got on the side and i'm going to have to get github repository so you can actually just copy this yourself if you want but basically what this creator is doing is just giving us a unique id for our image for when we uploaded typical storage all right now so building on our submit function uh basically now we have to build the form data which we're going to post to our endpoint or our express server um so to do this i'm just going to say let post id equal uidv4 and we're going to use this in the image name itself so if i say let input lm equal document i'll get element by id image file and i don't think i created the id for this yet so i'll just do this right now before i forget there we got the name we need the id are the ids over here image file so i'll save that so input elem equals document.elementbyid image file and then we want the actual file in the input so and now because i want to change the file name i don't want to have the original name so i can keep track of um ids and stuff like that we need to actually turn our file into like a blob and then recreate the file with our own file name and so to do this we can just say uh we'll say let blob equal file dot slice 0 file.size and we'll say it's an image of jpeg and now we can say new file equals new file taking in the blob and now we want to give it the name so we'll say post dot id or post id and we'll say post and dot jpg and then we just need to find the type here as well again so now the type is going to be an image of jpeg all right so this is basically going to get the file for my input element we're going to basically recreate the file and give it this name so it's going to be the post id underscore post and jpeg all right so now we want to go let form data equal new oops form data and we just need to append our file to our form data so i can say form data dot append and now we want to call this image file and we'll say new file just like that cool all right so now we've got everything we need to send with the post we actually have to make the request itself so i'll just say fetch and the endpoint is going to be i will say upload and we're going to give it an object and we'll say the method is post and the body is going to be our form data all right let's just save that and we want to do some dot then so we'll say response and we'll just say res.txt so i'll just send back a text response and then then we'll just console log the output all the response back from our server so just say that now okay so that is everything we need to do on the client side basically so we'll go back to our inde uh so we'll go back to our express server and just build out the endpoint there for the upload all right so now just back on our express server index.js file uh we need to bring in some of the dependencies that we installed earlier and then we can go ahead and build out the posting uh the slash upload route so i'm just gonna say const storage oops uh eco is require and we want google cloud storage just like that and then to instantiate our google storage client on our express server we'll say com storage equals new storage and we can pass in a project id which is an expected parameter and also the key file name which is also another expected parameter and we haven't got those yet but we will but i'll just define these up here so i'll say let project id equal blank for now and let key file name equal blank for now now these two variables are going to come from google cloud itself so the project id is the project that the id the project that you're working in and the key file name is actually a key that we need to get from google cloud when we create some credentials and stuff so we'll do that soon but i just want to instantiate the client build out the route and then we're going to uh bring in our start from google cloud that we need to at the end all right so now we also need to bring in malta so we'll say require malta and we can say down here con smelter i just instantiate alter and we'll say storage we want to say storage and multi memory storage and what we're doing this is because uh i don't want to save the files on our express server so if we're chewing out resources and storage and space like that i just want to be able to send the memory stream to express server and then upload it directly to google cloud so we don't have to download the file and then send it we can just sort of uh pipe it to google storage right we wanted to find some limits and now it's going to be about file size and you can change this as you want but if we say 5 times 24 times 24 oh sorry 5 times 10 24 x 1024 this is going to have no files larger than five megabytes but you can change this as you need and then we just need one more variable as well for our google storage which is going to be the bucket definition to which we're going to be uploading into so i'll just say khan's bucket equals storage stop bucket and we need to define this still so i'm just going to leave it blank so i'll just say here to be defined so we're going to come back to these three variables which are all going to be in uh provided to us in google cloud or once we go there so we'll just continue on now with the building of our posting route so for anyone that posts to our slash upload route we're going to use malta to upload a single file so we can say multa single and this takes in a string which is actually the name of the file that we sent in the post request so if i go back to my submit.js not the name of the file rather so but the name of the value so if i look at this form data.append we can see here the name the name value here so our name is called imagefile which is why we're going to use imagefile in here so i'm just going to wrap this in a try catch as well and then if there's any errors we're just going to do res.send error and also just here have a status as well okay so when we try to upload the file basically what i want to do is say if there's a file so uh there's a file on the request object uh we want to basically now create a new blob equals bucket file dot file and we want to pass in the request to file the original name which is going to be our post id that we created in our submit.js and now we just need to open up a stream and write to it so if i say cons blob stream equals blob dot create oops uh right stream is what we want and now we can say blob stream dot on and when it finishes it's going to do the callback that we list here so if i just do that so i'll say red status 200 and send success right and down here i'm also just going to say blob stream dot end rect.file.buffer all right so i think we can save this off now and let's see what we have here a bucket name is needed to use cloud storage okay all right so now we have to go to google cloud and get some of the pieces that we need so the storage bucket that we're going to be using and also the project id and the key file name so let's go to google cloud now and get all these bits and pieces so we can actually put it into our application and then start running this right so going back to your browser if you go to your google cloud project you can get your project id from the main dashboard here so for me this is that one there so i will just go copy this and put it in there like that and the key file name is something we need to actually download so if you go to your google cloud dashboard and you type in credentials so from the credential screen uh you need to have a service account so we can download the json key file or the pen key file uh so as you can see here i've got two service accounts but if you don't have a service account you can just go create credentials servers account right and then once you've got your service account you just want to go to within it and then we're going to go i think there's a key file here we can download or something so we go keys add key create a new key and then you're going to create that so i'll just create it now as well and it downloaded that right so now from here i'm just going to literally drag and drop it into our application folder just like that all right sweet so now we've got our key in our directory uh we basically need to just reference this key and we can just literally do that by saying i'll just rename this actually so my key and i'll just say here my key just like that so now we've got our key file and we've also got the project id now the only thing that's missing is our storage bucket so let's go back to google cloud and we're going to create a new storage bucket so i'm going to go to our dashboard and you should see it down here on the side if not just type it into the top so storage and we can create a new bucket here and i'm going to say google storage tutorial my bucket name i can't start with google that's great so i'll say story tutorial just like that that bucket name is taken okay how about dan's storage do that all right that one's free so i'm just gonna leave the region in the us i'm not too fast and continue through all these and we do want uniform access control so this basically means that uh the access is defined at a bucket level and not at a file level if you want fine grained you can kind of define each the permissions on each individual file but i just want the permissions to remain the same across the whole bucket so i'm going to keep it as uniform and we can create that now all right so once you've got your bucket created i'm going to actually just copy the name here and paste it into our storage.bucket so i think this is all looking pretty good now so let's actually try to upload something uh from our index.html form so i'm just going to save all this stuff and actually lastly as well i just want to have a console log here just to make sure i can see that we're coming in here so i was saying made it to slash upload and in here console.log file found trying to upload just like that all right so let's go back to our front end and just refresh this make sure we're getting the latest file choose a file and we'll upload this image of free guy submit it let's see what happens here so i'm not getting any errors so it looks like nothing's happening there at the moment so let's see where that is i know why it is because we haven't even referenced our client-side javascript here so let's say script source equals what do we call it submit js right cool all right let's try this again refresh this choose a file and submit still getting nothing ah sorry i know what the reason is that's because we're still not actually serving this through our express application so we need to actually serve static files from our express application of course so if i go back to our server here and i just want to say at the top say something like app.use express dot static and we're also going to need path now as well um const path equals require path i'm just going to say const source equals path.join their name and we'll call our public folder which we're going to serve views right so in here express.static passing in source now we need to create a new folder called views and i was going to put the index.html in there and also the submit.js in there now when anybody comes to the root of our application uh where we have it down here i just want to say uh it's going to be sourced now instead so source plus index.html so just make sure this all works now come back to the top refresh this all right we can still get our web page which is great close this off choose the file and we've got the file there and it's going to try send it now all right so now what's happened so it's tried to get the my key but i couldn't find my key and i think i know why because i think i didn't say my key.json so this has to be my key to jason let's try that again submit it all right so it's made it to the upload file found trying to upload so it's in here somewhere but it hasn't fully sent it yet so all right so i'm just going to refresh this and also just want to say here as well maybe on our if statement uh just in case the rec.file isn't there or something so i'll say else throw arrow with image save the self just try this again one more time all right so it's going into here file found trying to upload and i'm going to guess it's probably stuck in here somewhere um so let's just check the console actually and what we have uh oh wait a second we've got success so i think that's actually all good i think um ah because i didn't even write here so we'll say console.log success so so we can actually see something on the server so let's just go back here and we'll resubmit this so it's file found trying to upload success it's great and we get a recessive success response back as well all right so that's looking pretty good so if we go to our storage now a bucket give this a refresh we should see now three items in here i think with three unique post ids and there we go we can see three unique posts all with our image uh there we go there he is all right so now lastly just to wrap everything up uh we want to just be able to get the images that we posted to our google storage and to do this we're going to need to change some of the permissions on the bucket itself so as you can see here if i created permissions for this bucket uh there's no public access um and so what we want to do is just basically change this so that everybody can read from this bucket right so if i add principles or add permissions sorry and i go all users and we can select a role and we'll say store legacy object reader all right and i'll just also add bucket reader as well just like that all right allow public access now you'll notice if we go back to our objects in our bucket that we should now have a public url which we can copy go here and you can see we can we can view our image from here now as well so basically what we need to do is do a get request on our bucket get all the post ids and then just append it um to this url basically over here right so that's what we're going to do so we'll just go back to our client side first and we basically want to do a get request now to our express server which will return a bunch of post ids and then we can sort of look through and create some image elements on the front end so basically here i'm just going to say so basically now i just want to chain our upload to basically call a function which will get all the all the posts after we've uploaded so from here i'm just going to say function um and now here we're going to do another fetch url and we can actually leave uh we don't need to change any of these methods because we're just doing a get so and this is actually going to be upload not url because we already know the url so um i could say then uh res again res.json this time and then then with console.log sorry we'll just console.log this again all right so now with that in place uh let's actually go back to our server and define the slash upload path for a get request okay so let's go to index.js and we'll just go above here app.get slash upload and we can say rec request response as it comes files equals await bucket dot get files just like that and we can say now res does send files ah and we get away is only valid in asic function so i'm just gonna say up here async so save this off now if i go back to our front end all right so now when we refresh the page just make sure that we get the new uh submit.js code that we put in we're going to select the file and i'll put this one in here so the city now if i go submit i'll just put this up we get a success pack and now we're also getting the array back as well so in here we've got a couple of things that we've uploaded now already and looking what we can see now in this array which is basically each object in the bucket primarily what we want is the id right because we already know the base url which is actually provided to us here but we're really after the id so if we go now to our client side javascript we can say in here we can now do a for loop on uh this response y equal to zero y is less than x dot length and it will say y plus plus and in here we're basically going to for each element in the array create a new html image element and we're going to set the source attribute to the url that is provided to us so if we just say document i will say const new image equals document.create element image oops image and we'll say new image dot set attribute source and this is going to be equal to uh x x y is going to be equal to uh the element that we're iterating through uh storage dot base id plus i think a slash plus x y dot bucket dot id okay we're basically iterating through each element and in there we need the storage dot base url and we're concatenating that with the forward slash and then also the dot id which is this here so i think that's pretty much it i get the naming wrong there is it base id or base url base url so that's what we want there just copy that and then we're going to add this element uh into uh into a div on our index.html which we haven't defined yet so i'll just say images just like that and now in here dot append child new image just like that all right so let's give this another shot refresh the page and we'll get me again why not submit advance what do we get cannot read properties of undefined reading based url all right so let's see what went wrong here i might just console.log x again while we're here but that still uploaded it right okay so let's try again refresh this uh we haven't got a file picked so pick another one submit it all right let's see what we're getting here so we've got a bunch of things here i'm actually going to move this function just to be called uh on page load so we have to keep uploading stuff all the time so i'll just do this like that and we'll say function load post and here we can do this to get rid of those two brackets all right so do that and then we'll say here load posts right all right and then i think in here we'll just go unload equals load posts all right so nowadays we don't have to keep submitting stuff every time we uh want to check this out all right let's see what we get got a bunch of items here and it's having trouble reading it's array ah my bad okay so it looks like it's a nested array so i think now if we do like this x0 this should work i think so refresh this and now you can see we're getting each individual thing there but i haven't put it down here so it's not going to work yet and that is what we want just like that all right let's refresh the page and there we go we're getting something now all right so we'll just um go to our google storage over here let's get this to refresh i've got a bunch of things in here now this is probably just the easiest solution i think so if we just go to the public url here because i know that we've got this part here i don't think we need that bit actually so let's try this down instead go back here refresh the page so it looks like it's pulling this bucket id from somewhere else maybe we just do id so okay that looks like it's working now um it's gotten at least one of them but i think we want to set this attribute so that it's not so big maybe all right so we're getting one of our things but we're still not getting all of them that's because it's only going through this one here all right let's refresh that i think we should now get everything hey there we go all right cool all right guys there you have it so it took us a little bit to get towards the end there um but got full functionality of submitting something to google storage and also retrieving uh anything from google storage as well so i hope you guys learned something today if you want the code for this all the code is going to be uh on my github which i'll leave the link to the repo in description below so if you want you can just sort of you know copy it into your own uh your own editor and stuff and mess around with it there in upcoming videos i'm going to be releasing a basically a series which is kind of covering uh full stack development from the database uh to the front end and to the api layer as well so that's coming shortly in the next coming weeks um if you don't want to miss out on content just like this and those future videos don't forget to subscribe all right thanks guys [Music]

Channel: Daniel Otto

Views: 15,363

Keywords: google storage, google api, nodejs, web development, google cloud

Id: pfKR0eBkpTY

Length: 32min 13sec (1933 seconds)

Published: Wed Apr 06 2022