Django with AWS - S3 Buckets and CloudFront Distributions for Media Files

in this video we're going to learn how to use Amazon S3 in order to store our user uploaded media files in the cloud and we're also going to learn how to integrate Django with cloudfront which is a CDN offered by Amazon and that can help you get your assets to users throughout the world much more quickly and efficiently now specifically in this video we're going to learn how to use a package called Django storages in order to actually upload the assets to the cloud we're then going to learn how to store user uploaded images on S3 from our Django applications we're going to learn how to set expiry times and sign the URLs so that they're only available for a short period of time and that can help prevent them being shared and distributed across the web after that we're going to learn how to set up a cloudfront distribution to serve our media files from An Origin S3 bucket using Django and finally we'll also learn how to create signed URLs in cloudfront for those media files so there's quite a lot to cover here let's get started now here we have an application that we built in the previous video it's a simple system that allows users to upload media files and it stores them at the moment or on the file system and the server now we want to move beyond the file system and actually store these Assets in Amazon on the cloud using this S3 service now if we go to the documentation for Amazon S3 you can see that it's an object storage service and you can see some big numbers here such as this one it's got 99.119 data durability so very unlikely that you're going to lose any data that's stored on S3 and that's because your data is replicated to different availability zones on AWS so it's very unlikely that anything will be lost and this is a very scalable solution because it offers you basically unlimited amounts of storage so it can be used in your Django applications to store media files or to retrieve any kind of asset that you want to use in your application and you can see at the top right here that AWS offers a free tier and it offers five gigabytes of storage on S3 for 12 months and you can get that for free as your data grows you might have to pay a little bit of money but it's actually very cheap to store Assets in S3 so let's get started with the application and what we're going to do is install this package here it's called Django storages and it offers a collection of custom storage backends for Django and you can see that we've got S3 here as well as things like Azure storage Dropbox digital ocean and also Google cloud storage and a few others as well what we're going to do in this video is use Amazon S3 if you want to see how this works with other providers you can just let me know when I might make a video on that now what we're going to do to start with is copy this pep install command and we're going to go to vs code here I've got a virtual environment set up with the previous videos code and we're going to copy this pep install command into the environment once that's installed you can go back to the documentation and we're going to click this Amazon S3 link and that'll take us to the page that's specifically for integration with AWS now this Library offers a back end called S3 bottle 3 storage now bottle 3 is a python package that's used to integrate with aws's services so this package is actually using that under the hood what we need to do is we need to actually install all bottle 3 as well so let's go back to vs code and we're actually going to put the pep install command in here again and it's the boto3 package that's going to install that in our environment and then we can actually use that package to connect to AWS and perform actions such as uploading user files to s3d once that's installed we can go back to the Django storages documentation and we need to add this setting here it's called default file storage and we're going to set that equal to that bottle 3 storage that's provided by Django storages so let's go to our settings.pi file and I'm going to go to the very bottom of this file and we're going to paste in this setting Now the default file storage setting it tells Django as it says here what is the default file storage class that's going to be used for any file related operations and this storage class defaults to this one here it's called fail system storage and that will basically store any files on the file system and then it knows how to retrieve those files how to store files and save files and so on what we're going to do is basically replace that because we're not going to work with the file system what we're going to do is work with Amazon S3 and we need to set this to the S3 bottle free storage in order to do that so what I'm going to do right now is just run the Django development server and we're going to go to the application that we built in the previous video it's this one here and it allows us to upload a simple file and store that in the media directory as it was specified in the last video we're now going to see that this doesn't work anymore so let's add a dog and we're going to call this dog Bruno and then we're going to select an image now we have these two dogs here I'm just going to select one of these and we can submit this and we're going to see that this causes a problem we have an error here a value error and it says that the required parameter name is not set and you can see if you look at the stack Trace if we scroll down here you can see that this error is actually coming from the S3 bottle 3 dot Pi file which comes from Django storages now we need to set some settings here using Django storages so let's go back to their documentation and what we're going to do is scroll down here and you can see that there are several different methods for specifying AWS credentials that you can then use to create the client now this is the problem in a nutshell we are telling Django that we want to store our assets on S3 on Amazon but we have not told Django anything about which AWS account to use and what the credentials or the keys are for that account so we need to set a couple of values here in our Django settings.pi file and what we need to say are these settings here it's the AWS access key ID and also the secret access key but to do that we actually need a user in AWS that can actually have these keys so what we're going to do now is go to AWS itself this is the AWS console here and we're going to go to a service called IAM and that stands for identity and access management and that service allows you to create users and groups and also assign permissions to those individuals and assign permissions to the grips as well so what we're going to do is exactly that we're going to start by creating a group using this user groups tab at the left hand side at the top right we can click create group and then that will take us to this page here so we'll give the group a name of Django S3 grip and then if we scroll down this is an important part here it's the permission policies for the group and basically what this means is we can attach permissions for any services on Amazon and once those permissions are attached to the group any user that belongs to the group will then have those permissions by default so what we're going to do is find a permission here I'm going to search for S3 full access and you can see that that gives us this permission here so if we select the box here we can then add that to the group once you've selected that box we can then hit the create group button and that will then create this group in the IEM service we can then go on the left hand side to the users Tab and currently I have a user called Bug baits we're going to add another user here and this is going to be the user that actually has the access and secret keys that we're going to use in this application so let's give this user a username of Django S3 and then we can hit next here to move on to the permissions Tab and we want this user to be part of this new group that we just created so we're going to tick that box here and go to next and this is finally the review page we can just create the user and once we've done that we can see that we have this new user called Django S3 so we can click on that user and this is a tab here called security credentials we can click that as well and if we scroll down we have this section here for Access Keys now this is what we need if we go back to Django storages we need an access key and a secret access key and in order to generate those we need to create them here in this page for the given user that we're using for this application so let's create an access key and this access key is strictly going to be limited to local code we don't want to give anyone access to the CLI or to the AWS console or anything like that we're going to just limit it to local code and then you can check this box at the bottom and then we can go to next and then simply create the access key here now this page here gives you your access key and the secret access key so what we're going to do is copy these and these need to be added to our settings.pi file but what I'm actually going to do here is create a dot in file in our application because these are the types of things that we definitely don't want to be stored in GitHub or in any Version Control System we need to keep these secrets we're going to use a DOT end field to do that so let's go to this empty.in file that I've got and I'm going to paste in two settings here firstly the AWS access key ID and secondly the secret access key as well we can then copy and paste these values that we've got here into our DOT in file and then we should be able to load them into settings.pi so let's copy both of these and we can then paste them into the dot info now as I said before it's very important that we don't commit these to GitHub if we do commit them anyone has access to the Keys and they can actually use them to store their own files or do anything they want on AWS or at least anything they want based on the permissions assigned to the user who has this key and in our case that permission was S3 full access anyone could use our s3d service in any way they want so we're going to go to our get ignore file here and I'm going to add this dot in fail to or get ignore found just to make sure that it's not committed so that's the two keys that we need for this user that we're going to be using to actually interact with S3 through our Django application what we now need to do is add another couple of settings we need to reference the S3 bucket that we're going to use to store the files but we don't currently have an S3 bucket in the application so let's go back to the Amazon console and we're done with IEM so we're going to now go to S3 and we can see what that service looks like now the main page here lists out the different buckets that we have in our AWS account what we're going to do is create a new bucket using this button here now you can see that we need to create a bucket name here and the bucket name must be globally unique it must not contain spaces or uppercase letters so what we're going to do I'm going to paste this one in here and I'm going to call it bug bites Django S3 demo 2 and I'm going to set the AWS region to one that's nearest me and that's EU West one and the closer the region the faster your requests to this bucket are going to be so you want to set this to something as close as possible to your current location or at least the location of most of your users once we've done that we have a section Below on object ownership keep with the recommended option to disable ACLS we don't need those if we go down here we can see another setting here and this one blocks any public access to the resources that we're going to store in this bucket we want to keep this on and that's because we want all the items that we're going to store in this Bucket from our web application to remain private we don't want to expose them to anyone who's not using our application only those who access this with the keys that we've created here through the application are going to be allowed to actually access these resources and that's basically all we need to do here we can now scroll to the bottom and then we can create the bucket in our Amazon account that will take us back to the bucket list page and you can see the new bucket has shown up here and currently it is empty we don't have any objects in the bucket let's go back to the bucket list page what we're going to do now is add another couple of settings to our settings.pi file or rather the dot end file if we go back to the Django storages documentation you can see that one of the settings here is this one it's the AWS storage bucket name and that tells Django storage is which bucket we actually want to use to store the files that are uploaded from our application so let's copy that and go back to our DOT end file I'm going to set that equal to the name of the bucket that we get here from the console and it's this one here so let's paste that into the dot end file and we can clean up the formatting here a little bit and there's one more setting we want to add at the moment if we go back to the Django storage's documentation and we scroll down here you can see that there's our region name AWS S3 region name if we copy that we can paste that in and remember I set this to EU West one which is the region closest to where I am so you might want to set that to whatever is closest to you that you used when you created the bucket so now we have these values in or Dot N file we want to actually read them into our settings.pi file so what we're going to do is install a package that's going to let us do that and that package is called Django Environ and this is a python package that allows you to use 12 Factor methodology to configure your Django app using environment variables and in our case the environment variables are going to be defined here in this dot end file and we're going to read them into the settings.pi file so I'm going to scroll to the top and I'm going to clear out this comment at the top here so we can get a bit more space to work with let's stop the server and we're going to use the PIP install Django Environ command and that will install that package into a virtual environment now let's go back to the documentation for Django Environ if we go to the quick start on the left hand side you can see that this gives us a guide to using this package with a dot in fill in the project root of your Django application what we can do is import Environ at the top of the settings.pi file so let's do that here once we've done that go back to the documentation and we can basically copy these lines of code here firstly this one here will define an inv object so we can do that at the top of settings.pi let's paste that in all we're doing here is taking the Environ package and instantiating an end object once we've done that we can go back to the documentation and we can copy this line of code here we can paste that below the base directory of our application and I'm going to remove the OS calls we're using pathlib and Django by default now so we can reference the base directory and the dot end file that lives within that directory now this statement here will read our environment file the dot end file and we can then reference the variables defined in the dot end file when we use this object so let's scroll to the bottom of the file and we can then use the values from the dot end file so for example we need to define a setting called AWS access key ID so we're going to use the NV object that we've created from that package and then we can reference the name of any key that we have here in our DOT end file so we're just going to reference this one here for the access key and paste that in now I'm going to do this for every single setting that we have in the dot in file so I'm going to paste these below here we have the secret access key the region name and the storage bucket name and these are just referencing those dot end values and you can see the security benefit of use a package like Environ here we are loading these values from an end file that's in our git ignore file it's not on GitHub and then we can push the settings file to GitHub that references those values but it does not expose them to anyone who can read a repository to verify that we can run the shell plus command and we can reference the settings module and then I'm going to copy the name of this one and make sure that it's being loaded in correctly and you see that it is so let's exit the shell and I'm going to run the Django development server again and we're going to go back to our page I'm going to try uploading this file again if we refresh the page we can resubmit the form and hopefully it's now going to upload to S3 so let's see if that works and you can see we're taken back to this list page if we go back to AWS and we go to that new bucket that we just created you can see we now have objects in the bucket one object and if we click this cats folder we can get taken to the dog.jpg image so this upload is actually what it's been sent to a cats folder and that's because if we go to models.pi you can see that this is uploading to this cats directory which we used in the last video so I'm going to change that back to dogs and remember upload 2 will specify where from the media route we want to place the uploaded Assets in our case the media route is basically the S3 bucket and that's why it's sent it to this kind of subdirectory in the S3 bucket now SD doesn't actually have directories all objects are just defined by the path to that object but you can think of that as similar to a directory and if we uploaded a new object we would store it in a docs directory because we've changed this upload to a parameter so that's just an explanation of how this upload to parameter works with Amazon S3 let's go back to the bucket and we're going to go to that new object that we uploaded dog.jpg and on the detail page you can see it's got an object URL but if we click that URL you can see we get an access denied error and that's because all of the objects in our bucket are private remember when we created the bucket we blocked all public access to the items in the bucket so we cannot access any objects through the URLs that are defined so our objects have been uploaded and we cannot access them publicly they are secure in our bucket let's move on with the video If we go back to our web app and we click the view all dogs link we get taken to a list page for the application and you can see this dog is appearing a lot larger than I thought it would actually so I'm going to try and amend that now okay so I've added a new Tailwind class to limit the size of this dog and the styling here is not the best but we're not going to focus on that in this video now what I want to do here is right click the dog and we're going to open the image in a new tab and we can have a look at the actual URL for this dog you can see that the image URL here is referencing the dog.jpg image that's in the cats directory of our bucket so it's referencing that but it also has some query parameters in the URL you can see that the access key ID is embedded into the URL and that comes from what we have in our dot info we've set this access key ID so that's then added to the URL and we also have a signature and an expires key in this URL now what this means basically is the these URLs are generated by our Django application in order to view these resources on S3 they are temporary URLs they are signed and at some point they are going to expire so if you were to share this URL on the web at some point this URL is going to expire based on this expires parameter and after that anyone who clicks the URL will not be able to access this resource now this can be beneficial if you have private assets that you want to expose in a web application but you don't want them to be available forever to anyone who has the link you can use this mechanism now the expires parameter is set based on a setting in Django storages and I'm going to search for the settings called AWS query string expire it's this one here and by default is 3600 seconds so by default these URLs that we get when we access the resource here they expire after one hour so we're going to copy the name of this variable here it's called AWS query string expire and we're going to paste that into the settings.pi file and we're going to set it to something a lot less than one hour we're going to set it to five seconds so we can now go back to our page here and when we refresh the page we are going to see that the image appears but when we right click this image and open it in a new tab we are now getting an error saying that the request has expired and that's because our expiry time in the URL is now a lot less than one hour in the future and it's now just five seconds in the future so these particular images here the links expire very quickly in this case and you can use that and set that value to whatever you want in order to protect your resources online from being shared and distributed so let's change it to 600 which is 10 minutes and we can go back to the page and go here we're going to try deleting the image now so if we click this button and you can see the image has been removed from the list View and if we now go back to the AWS console and we go to our S3 bucket here when we click the name of the bucket you can see we no longer have any objects in the bucket so the delete button is still working and that's because in our models.pi file what we're doing is we're referencing the image field when we delete the rule from the database this function is called and we call the image.delete method which will then remove it from the underlying storage which now has the S3 bucket so that also deletes the image from the bucket so at the moment we've covered how to upload assets to S3 using the Django storages package we've seen how to create a user in the Amazon identity and access management service and also how to create these access and secret access keys that we can then use in our Django app to upload resources to the cloud and we've also seen this setting here called AWS query string expire and the number we set this to will determine how long our URLs are valid for and therefore how long objects and our buckets are accessible for Via that particular link that's generated so we've integrated Django successfully with the S3 service let's move on to something a bit different now we're going to have a look at the Amazon cloudfront service so let's go to the cloudfront documentation here or rather the homepage it tells you that we can securely deliver content with low latency and very high transfer speeds and cloudfront does this by delivering our data through 450 globally dispersed points of presence and because of so many of these and the distributed throughout the world are assets from our S3 bucket can be stored on these servers and cached and therefore used to very rapidly serve our content to users that are requesting it so that's the benefit of Amazon cloudfront the question is how do we actually use this in a Django project how do we connect the cloudfront distribution as it's called to our S3 bucket let's go to the AWS console and we're going to search for the cloudfront service here as you can see it's a global content delivery Network so let's click this link and we're going to be taken to that page now on this page you can read about some of the benefits of cloudfront and an interesting security benefit of having resources distributed worldwide onto cloudfront's Network and its servers is that it can help prevent DDOS attacks and that's because rather than having a single point point where all requests are coming into your application or coming in to get resources these requests are actually distributed geographically based on user locations and therefore it can help to balance out any attacks that are going on on your site or applications or whatever else you may be using so it can help prevent DDOS attacks and Amazon offers some additional services that can help with this we're not going to go into that now but if you're interested let me know on the right hand side we're going to create a cloudfront distribution and what we need to do with cloudfront distributions is connect them to An Origin domain and that can either be an AWS origin for example an s3d bucket or you can have a custom origin that's a bit more complicated and we're not going to do that in this video what we're going to do is we're going to select our S3 bucket that stores the application resources and we're going to follow the recommended settings for origin access here and select the origin access control settings and that means that the bucket will restrict access only to cloudfront so to get a little bit more information you can click this info tab and you can see on the right hand side with origin access you limit the access of your origin which is the S3 bucket to only authenticated requests from cloudfront and cloudfront recommend using origin Access Control in favor of an older version of that called origin access identity so we're going to select origin access control and when you select origin Access Control we actually need to create a control setting so we can click that one here and it gives you a name and we can give it an optional description and for the signing Behavior we want to keep this one here to sign the requests and the origin type here is S3 we can also use something called media store but we're going to keep it S3 and create that origin control setting now notice below here for bucket policy it says that we must update the S3 bucket policy in order to use this cloudfront distribution with S3 and that's because we need to give cloudfront access to our S3 bucket so we're going to need to update that policy and it says here that cloudfront is going to provide us with the policy statement for the bucket after we create the distribution so we can scroll down and I think we can basically keep everything else the same here there's a lot of settings here that go deeper into cloudfront but we don't need to use them for this particular use case and one of the things you can do is specify a cache policy and this is the one that's recommended for S3 it's called caching optimize so we can just keep that as it is and finally we're going to scroll down to the very bottom and just finally create this distribution now that's going to take a little bit of time so I'm going to resume the video when that is complete okay that distribution has now been created if we go to the distributions page you can see a list of all of your distributions in cloudfront this is the one we've just created and you can see that as well as some other things it has a domain name and that's a bunch of random numbers and letters and then dot cloudfront.net so that's a domain that's associated with our cloudfront distribution that's quite important here you can also see the origin for this domain and that's the S3 bucket that we have been using in this video so all cloudfront distributions they take resources from a given origin which is usually really an S3 bucket but can also be a custom origin such as a server that you're running so it takes the resources from that origin and it distributes them to cloudfront locations throughout the world and that allows these assets to be rapidly served to users so now we're going to integrate this into our Django application but before we do that remember that cloudfront needs to be able to access the S3 bucket that serves as the origin so let's go to our distribution and we're going to go to the origins tab at the top here and we can select the S3 bucket that is our origin and then we can go to the edit tab here and if we go down to the origin Access Control section and remember this is just giving us access to whatever origin we've set up in our case it's an S3 bucket and we can copy this policy so let's copy that and we can then go to the S3 bucket permissions by following this link here and that's going to take us to our origin S3 bucket if we scroll down here we can see there's a bucket policy section if we hit the edit tab here we can then paste This Bar policy that we've copied from cloudfront into this bucket policy here so let's quickly read over this policy we have an ID here of allow cloudfront service principle and we're allowing access by setting the effect to allow and what we're allowing access to is this resource here and it's the AWS S3 bucket that contains our media files we're allowing cloudfront to get objects from that bucket by specifying the get object action but we're not allowing just any cloudfront distribution and that's what this section here does it specifies a condition and we are only letting the distribution that we've just created access the S3 bucket so basically this bucket policy will allow our new cloudfront distribution access to get objects from the S3 bucket we don't really need to know the details here because cloudfront gives us this code but it's always useful to understand what is a bucket policy and what is it actually doing so let's now save the changes and let's go back to cloudfront now and we're going to go back to the distribution page here I'm going to call copy this domain name if we paste that directly into the browser we get an access denied now what we're going to do here is we're going to upload a new image to our STD bucket and we're going to do that through this form here let's call again the dog Bruno and we're going to attach an image and this time we'll go for dog 2 and we'll submit that form when that's been submitted we can then go to view all dogs here and we can see the dog in our list view let's now go back to the S3 bucket that's on AWS here I'm going to open that in a new tab at the moment and we can right click and do that if we go into this bucket you can see there's a dogs directory and it contains that image that we've just uploaded now what we're going to be able to do from our cloudfront domain name is we're going to be able to access that image of a dog through cloudfront by adding to this domain the path to the object in our bucket so the path that we have here is we have our dogs subdirectory and then it's dog underscore2.jpg so what I'm going to do is go back to the other browser and from our domain name we can specify the dog's subtitles actually and then we're going to paste in the name of that dog and you can see we now get access to the image and that access is coming not through S3 directly but through cloudfront cloudfront is accessing the origin and it's loading that image through one of its Edge locations or points of presence and cloudfront actually caches that object on its servers throughout the world so when we refresh this page you can see it's very quickly loaded back up so we have this caching going on and we also have this geographic distribution of resources so we can access this directly through cloudfront's domain name but how do we do it from Django if we go back to the list view on our Django application if we right click this dog and open you can see the URL is still coming directly from the S3 bucket so we need to change that so what we're going to do is close the link here and go back to vs code and we're going to Define some new settings from Django storages in this file here so let's go back to the Django storages documentation again and there's a section of this documentation on using library with cloudfront so we can click that and we get taken to this section here and as it says here if we're using S3 as a CDN via cloudfront we want this storage to serve the files using that domain so we can set an S3 custom domain to whatever our cloudfront distributions domain is so let's copy the name of that variable and we can paste the name of our distribution in so let's go back to cloudfront and we're going to copy this distribution name and we're going to paste that in as a string so now we've specified a custom domain name for our images we're not getting these directly from S3 we want to get them from the cloudfront distribution so let's save that file and we can go back to our web application and refresh this page and hopefully we're going to see an image of the dog and this time when we right click the image we're taken to that same URL that we constructed manually a minute ago and that shows the dog and it's coming from our cloudfront distribution now to demonstrate exactly what cloudfront does what we're going to do is add a new dog here and I'm going to call this dog Bruno number two because I can't think of anything better at the moment and we're going to attach the other door here and submit that if we now go to the list view we should hopefully see both of those dogs and again the CSS really could do with some work here this is the second dog here and what happens is when we first upload the image when we fetch it cloudfront is then going to get it directly from our S3 origin at first and cloudfront does this by firstly checking whether or not the object already exists in the cache for that distribution if it does not exist it then goes to the origin and it fetches that object and then stores it in its cache throughout the different locations in the world so the first time we look for the object we don't get that cache hit on cloudfront but the second time when we access the image we're going to get that directly from cloudfront and it's going to be served up much faster and that's the benefit of using a CDN like cloudfront to serve your media files and other assets so this is all working quite well with cloudfront now but there is a problem this link that we have here is not going to expire the way the S3 links did so so this link here can then be shared on the web and it might be vulnerable to spam and if you want to keep your assets private and only accessible through your web application then you might want to protect that by signing the URL as we did with S3 so we need an expiry time and what we're going to do is go back to the Django storages documentation and you can see that there's a section here for cloudfront signed URLs so what we need to do is modify settings.pi we need to include two new settings and that's the cloudfront key and the cloudfront key ID so before we do that we need to go back to cloudfront and we need to actually create those keys so let's expand the left hand bar here and we're going to go to the key management section and what we're going to do is we're going to create a public key here so let's click the button here I'm going to create a key and I'm just going to call this key Django cloudfront public key and then we need to add a public key to this setting here and I'm going to link to our website that allows you to generate these public and private key pills what we need is a 2048-bit key pair so select that and we can then generate the new keys once that completes you can copy the public key and we can paste that into this box here and then we can create the public key and our cloudfront distribution now notice the ID for this public key we're going to need that in our Django storage setting for this particular setting AWS cloudfront key ID but there is one more step on cloudfront in order to get this working we need to go to the key groups section under key management and here we're going to create a new key group and let's give it a name of Django cloudfront keygrip and we're going to select that public key that we just created and we're going to add that to the key group so that has created a key group in cloudfront now the purpose of this key group is to restrict access to the resources that we're serving through cloudfront to only those who have valid keys for this particular key grip so what we need to do is associate the key group with the cloudfront distribution that we've created so let's go to distributions and we're going to go to the page for our distribution now at the top we're going to click the behaviors Tab and we're going to to select this default Behavior here I'm going to edit that now these are the settings for the closed front distribution that we set up earlier there's a section here called restrict viewer access and as it says here if you select yes to this option viewers then must use cloudfront sand URLs or signed cookies to access your content and there are two trusted authorization types and one of them is the recommended option which is to use trusted key groups and we've just created a key group now so we're going to select that option and we're going to select the key group that we just created here so once we've selected that we can scroll down and save the changes to this distribution and then we can go back to the distributions page and as you can see it's now deploying the new changes to that distribution so we're going to wait on that completing and then we'll resume the video now let's go to our DOT in file and we're going to add a couple of new keys to this file firstly it's the cloudfront key ID and secondly the key itself so to get the cloudfront key ID we're going to go back to close front and we're going to load up the public Keys section here and we're going to copy this key and paste that into the dot info now the value of the cloudfront key we need to set that equal to the private key that is part of this key pair so what we're going to do is copy that and we're going to go back to our DOT end file now it turns out that if we paste this it's actually quite tricky to reference a multi-line value in a dot in file so I'm going to remove that and I'm going to open the python interpreter and we're going to expand this terminal here and I'm going to format this value so that it can be used in a dot info now what we're going to do here is set a variable called private key and we're going to set it equal to that string that we've copied from the website and I'm going to paste that into the terminal and we can close off that string so when we access the string private key you can see we get back the key and all we need to do now is just copy this value from The Interpreter and we can paste that into our DOT in file here now there might be an easier way of doing this I don't know but this is what worked for me what we now need to do is reference these values in the settings.pi file so at the bottom we're going to do that let's create the variable called close front key ID so we're going to use the end object from Django Environ and we're going to reference the key of the same name in the environment file and just in case we're going to use the string.strip method to get rid of any white space at the start or end of that string and we're going to do the same for the cloudfront key let's copy that and paste it into the terminal and we're going to reference that key here now importantly we're going to set a keyword argument that's available on the environment object if we look at the dot end file you can see that we have these new line characters in The String that's been generated in the python interpreter we're going to specify a keyword argument called multi-lane and set that equal to True here and basically that's going to tell Django and vionon that we expect this to be a multi-line string and that's important because these RSA keys that you see here the new lines are actually part of the value of these keys so we need to interpret them with the new lines let's go back to the application after we read in the cloudfront key we're going to encode that as ASCII and then finally we can call the dot strip method to remove any leading or trailing white space and that's all we're going to do at the moment let's exit the shell and what we're going to do is reload the Shell with these two new variables but you can see that we're getting an error here and that's that we have an unexpected keyword argument multi-lane and the reason for that is because we need to use the end dot string function rather than just referencing the environment object itself so we're going to do that for both of these two variables here so it's nth dot string rather than just the end object let's try and reload the shell again and now we get it successfully and we can try and reference the private key here by copying the name of this variable and we're going to reference it here so we get that in the Shell let's now exit and see if this works in our application so what we're going to do is run the Django server and go back to the application here and we're going to refresh this page now you can see we get an error here and that's an RSA backend is required to sign cloudfront URLs so we need to install one of these packages we're going to install the cryptography package and our terminal so let's run pip install and we'll specify that package once that's done we can run the server again and we're going to refresh this page and hopefully this is now going to work so now you can see we get the two images if we right click one of these images you can see that we're loading it from a cloudfront distribution but we also have an expiry date on that particular URL so now we don't just directly get the images from cloudfront we also have an expiry date we have a signed URL that is not going to be around forever and that can help prevent sharing these resources on the web and can help secure your assets so let's close this image and we're going to finish this video by demonstrating that this query string expires still works with the cloudfront URLs if we set this to five seconds if we go back to the application and refresh this page the images are now generated but if you right click one of the images and try and access that we now get an access denied and that's because the expiry time is now in the past the URL has expired and therefore we cannot access this resource and as a final demonstration of what the key groups and the keys have done here let's now try and access a resource directly through our cloudfront domain name so we're going to go back to cloudfront distributions here I'm going to copy the domain name and we're going to follow a link to the dog that we uploaded earlier and you can see we now get the missing key error code we can no longer directly access this we need a secure signed URL to do so and that's how you set that up with Django and with cloudfront and S3 as your origin so that's all for this video we've learned a lot about how to integrate Django with AWS in this video we've learned how to store media files on S3 and how to use signed URLs and we've also learned how we can change how long these URLs are valid for using settings and the Django storages Library we've also learned how we can use cloudfront to server assets from An Origin which is an S3 bucket in our case we've learned how to set up cloudfront with that origin and amend or S3 bucket policy to give cloudfront access to the audio and we've also learned how to create signed cloudfront URLs which use these concepts of a key grip and a public private key pair in order to sign the URLs and therefore limit the amount of time in which they can be accessed without causing an error so this has been a dive into S3 and cloudfront if you are interested in any particular topics that are related to this let me know in the comments if you want me to dive deeper into things like cloudfront let me know too and if you're interested in seeing how Django can integrate with other Cloud providers such as Microsoft Azure and Google Cloud platform let me know in the comments as I'm preparing some content around us and it'll be interesting to know what you would like to see so thanks for watching if you've enjoyed the video please like And subscribe to the channel and we'll see you in the next video

Channel: BugBytes

Views: 8,872

Keywords:

Id: RsiXzwesNLQ

Length: 38min 44sec (2324 seconds)

Published: Fri Mar 24 2023