How to Upload Files to S3 Using Django Storages

hey everyone in today's video i want to show you how to use django storages to store user uploaded files to amazon s3 so typically when you have user uploaded files the default way of storing them is just on the file system that the app is running on but this can be problematic in a couple of reasons one it can be a security issue because you're allowing users to just upload something to your server so you may not want that and the other issue is sometimes your method of deployment doesn't allow for uploaded files so for example if you use something like heroku you won't be able to upload files to heroku after you've done the initial upload of your entire app because of the way the heroku file system works you just can't do it so you need to host your uploaded files somewhere else so in this video i'll show you how to do that i'll show you how to set up everything it's not too bad it's just a bunch of configuration really and i'll go through all the steps for that so right now i have a project set up i have storage's example as that project and i have a single app in there called examples so for this video i'm going to create a model that allows me to upload some things and i'll just create it here and this will be a model called let's say cats or just cat i have pictures of cats on my computer for testing so i'll do models.model and then i'll put a name of the cat so models.tar field max length we'll say 30 and then i'll actually need the picture of the cat so picture and i'll say models dot file field and then for the upload location upload two i'll put media okay so i'll go ahead and make the migrations for this make migrations and then i'll migrate and then finally i'll add it to admin so i can see it so admin dot site and i'll need the cat model cat and then i'll import that okay so now if i start the server i should be able to see it on my admin dashboard i'll go over there and i'll go to admin and just log in okay so i have my model here cat and if i go to add cat i could put in a name and browse for a particular file so before i do that i want to set where i'm going to upload this so right now by default it will create a directory on my computer every time i upload a file and put the cat image in there but i want to use aws for this so i need to set up an s3 account and i need to install django storage so i'll start by installing django storages first so pip install django storages so let's do that down here and django storages is the library that allows me to do this easily save the media to amazon s3 without having to do much so in addition to django storages i also have to install something called boto3 this library allows you to connect with services on aws and you can upload images through it to s3 so i have those now i need to go to my settings because there are a few settings that are important for both three so if i go down to the bottom i can just do it down here there are four settings that i need for aws and then there's one that basically tells django storage is which backend i want to upload to so in addition to s3 i can upload to other places so for example if i click digitalocean they have configuration for digitalocean ftp dropbox and so on but i want amazon s3 so here the first configuration that i need is the default file storage this will actually tell django storages where i want to upload things so i'll put that here so storages.backend.s storage and then i need some configuration for aws so let me go back and there are four so i need the access key id the access key the secret access key the storage bucket name and then the auth query string it's optional but i'll put that as well so let me go ahead and put these three so access key and i just copy the same one twice secret access key i have a storage bucket name and then i'll use the query string off uh just because i don't want to have my access key in the url i'm going to make my bucket public so anyone can view the images so i'll just put this to false i can put this to false now because it's a true or false value okay so i need these other three so let me go over to aws and i'm logged in already and i once you go to s3 so i can go to services here and click on storage or s3 under storage and then i'll create a bucket and this is where the files get stored so i'll say pretty printed cat pictures right that's the name of the bucket and then for the public access what i'll do is i'll uncheck the block all public access so when it comes to security settings on aws it can get pretty complicated so i'm going to use the simplest security setting here just to allow anyone to view the file but if you want to make it different then you'll have to go through the documentation for the security settings and do it yourself and it can get quite complicated and this isn't the only thing and i'll show you the other thing but uh i'll just make it public for demonstration purposes and then i'll click create bucket and i think i need to check that create bucket okay so pretty printed cat pictures i can go ahead and copy that and then put it in the storage bucket name and now i need to add permissions to it so i already checked the block public access thing i turned it off and in the bucket policy i need to create it so i'll click the edit here and then i'll go to policy generator and this little tool allows you to generate a policy and it's just json and you'll see that in a moment but for the type of policy i'll use s3 bucket policy uh the effect will be allowed so everyone will be allowed star for everything in the bucket and then the service is s3 the action i want to go down to get object so i'm looking for that get object and then the resource name is back here we see this thing so i'll just copy that and i'll paste it in here add statement and then once that's added i can generate the policy and then i can go ahead and copy that and paste it in here so this will allow anyone to view the images that were uploaded if they have the url so i'll save the changes there and i have an error let's see action doesn't oh so in addition to the resource here i need slash star so pretty printed cat picture star and then i'll hit save changes and that should work okay so i have my bucket set up so the files are going to be uploaded there and now i need the credentials so once again the way you get credentials can be complicated you can set up certain users that only have access to certain things but i'm just going to use my root account to generate access keys and of course i'm going to delete this so no one can use my access keys after this video so i'll go and add these to my configuration so the access key id and then the secret access key i'll add those to the strings and then i have them there of course if you were actually doing this you might want to put these in environment variables and then load them into the settings.pi instead of just putting them directly here especially if you're uploading your project to github in a public repo so just keep that in mind if you have configuration values like this do not upload it to a public repo until you create environment variables and then substitute the environment variable here or you just have a private repo or you just don't upload to a repo but i suggest using environment variables for sensitive credentials like this because what happens is there are automated tools out there that are looking for access keys on things like github and as soon as they find them they use them in some way and then it runs up the cost in your account and you're gonna have to deal with amazon and explain to them why your account has so many charges on it when it wasn't you doing it so just make sure these these values are never public so i'll close this and if you didn't see where i went just go to your name and then security credentials and then you'll have this page here you can click access keys and then generate the access key so i believe that is everything that i need to run this so i'll go ahead and run the server and i'll try uploading some pictures so i'll go back to my admin dashboard i'll refresh this and then i'll just add some cats so fluffy i have a picture called cat4 i'll save that so this means there's something wrong with either my access key or the secret and if i go here i see there's a space at the beginning so for some reason it added a space so i'll go ahead and save that and try this again so i'll go back and try to re-add fluffy so cat4 and then save okay so we see it was added successfully so if i go back to aws and i'll go to s3 again i should now be able to look in my bucket and see the file so it's cat4 i click on that and i see the media folder so the media folder is the one that was created for me because i have that upload too and i see cat4.pink and if i click this you'll see in the url i have an s3 url instead of one for my app so this is exactly what i want and of course if i upload another one like cat 3 and save this go to cat object 2 click on the link once again i see an amazon s3 url instead of a url from my app and if i refresh the bucket here then i see the two cat pictures so that's all that's necessary for allowing for uploads to amazon s3 like i said if you're using something like heroku this is really helpful and really for any purpose it's helpful because you won't allow users to upload things directly to your server it goes off to amazon server which presumably is more secure they can handle static files better so you don't have to worry about any issues there so if you have any questions about this process feel free to leave a comment down below if you like this video please give me a thumbs up and if you have subscribed to my channel already please subscribe so thank you for watching and i will talk to you next time

Channel: Pretty Printed

Views: 4,824

Rating: 4.9802957 out of 5

Keywords: django storages, amazon s3, aws s3, django file upload to s3, django, tutorial

Id: nzLMA9WZqMM

Length: 11min 50sec (710 seconds)

Published: Mon May 10 2021