Deploying Django to Google App Engine using Docker

in this tutorial i'm going to show you how you can deploy a django app to google app engine using docker what we're going to cover in this tutorial is first we're going to create a new django project and we're going to be setting it up with docker which we can use as our local development server then we're going to be creating a very basic placeholder app and this is going to include a static page with an image on it and a style sheet so we can demonstrate using static files with google app engine then we're going to set up our google cloud account so that we can deploy to google app engine and then we're going to be configuring our project for deployment to google app engine again using docker and finally we're actually going to deploy our app to app engine let's get started the first thing we're going to do is create a new directory on our system that we can use for our project files so head over to the terminal or if you're on windows the command prompt and then navigate to the location where you want to create the project so i'm going to do it in cd slash document slash workspace and i'm going to call the project deploy hyphen django hyphen g a e short for google app engine hyphen docker so i'm going to just make a directory here using the mkdir command mk dirt deploy django gae docker then i'm going to switch into that directory cd deploy and i'm going to initialize it as a get project now this is optional if you don't want to use git then you don't need to do this i do recommend doing it though because it helps keep track of the changes that we make to the project so do git init and then i'm going to open the project up in my code editor by typing code dot and this is just going to open up in visual studio code of course you can use whichever code editor you prefer to use when you're working on your coding projects now we've got our new project we're going to add a git ignore file to exclude certain things from git again if you're not using it then you don't need to follow this step but i'm just going to create a new file here and call it dot git ignore and then i'm going to open up my browser where i have a link to the default python get ignore file i will also put this in the description of the list of resources for the video and it's just a handy template for python projects of a get ignore file so i'm going to copy the contents of this and then i'm going to go back to our project and i'm going to paste it in and then save the file i'm just going to add one more item to the end here which is forward static and save that and i'm going to explain why we need that later on in the tutorial now we have our project set up we can go ahead and add a requirements file for installing django which can be installed inside a docker container so i'm going to create a new file in the root here and i'm just going to call it requirements.txt and inside the requirements.txt we're going to add two requirements one is django and the other is g unicorn so we'll type django and we're going to install more than equals 3.2 comma less than equals 3.3 and what this will do is it will install django with the latest patch version of 3.2 so i like to do it this way so every time we rebuild our docker container we get the latest patch version which might contain some security fixes and so on we're going to do the same but for g unicorn so i'm going to type g unicorn more than equal 20.1.0 comma less than equals 20.2 and g unicorn is going to be the whiskey server that we use on app engine so it's the default one that we use on app engine for deploying django or any python application now save the requirements.txt file and next we're going to move on to creating our dockerfile so let's go ahead and create a new file in the root of the project and that file is going to be called docker file this is going to define the development server that we're going to be using so we don't actually use the docker file for the deployment this is just used for our local development server and there are many many reasons why this is good practice to do one of them being that you have a consistent development environment for all developers that work on your project so this is my preferred way and the way i recommend creating all django projects we're going to start by defining the base image by typing from python colon 3.9 hyphen alpine so alpine is just a very lightweight version of linux that is recommended when you're running docker containers so we use that as our base image then i'm just going to say label maintainer equals and then londonappdeveloper.com you can of course put your name or your email address or whoever the maintainer is of this docker file that you are creating next we need to copy the requirements.txt file to the docker image when it is getting built so we'll type copy dot forward slash requirements dot txt forward slash requirements dot txt then below that we'll do copy dot forward slash app forward slash app and then workda forward slash app so what this does is when we build our image the requirements file will be copied to the image and stored at forward slash requirements and then we will copy the contents of a folder called app and we haven't created that yet but that's what we're going to be using for our django app and all of our django code is going to be stored in there and then finally we set the workda so that's the working directory for our docker container so this is the directory that we are working from when we run commands on our container and we switch it to forward slash app just so that we don't need to use the cd or change directory command to change to it every time let's go ahead and create an empty folder here because otherwise our build is going to fail so the folder is just going to be called app and you want to make sure it is in the root of the project and we're going to be populating this folder with some contents in a bit these hairs everywhere next we're going to add to our dockerfile a run command that will set up a virtual environment for our dependencies install our requirements and also create a user for our django application so we'll do that by typing run and then python hyphen m v env forward slash py this is a new virtual environment and it places it in forward slash py in the container then two and signs to add a new command and then forward slash or sorry backslash you want to have a backslash here and then below that we'll add a second command which is forward slash py slash bin slash pip install hyphen r forward slash requirements dot txt and then the double and signs again backslash and then add user hyphen hyphen disabled hyphen password hyphen hyphen no iphone create iphone home django hyphen user let me just explain what this does so the first line here is it creates a virtual environment called forward slash py so at the location on the image as for such py then we run the pip install command but we run it from this virtual environment by specifying the full path forward slash py forward slash bin for sas pip and then install and then the requirements file that we copied here on line four then below that we use the add user command and we pass in disabled password and no create home because this isn't a linux user that we're going to be logging in as we're just going to be using it to run our application and this step is optional but it's recommended for security because you don't want your application having full root access over the image and it's not really a big deal for our local development server but just in case you do deploy this application to a production server at any point you might want to have this in there just to be safe now we're going to add to the path so the environment variable path in the docker container we're going to add a new entry for our virtual environment so whenever we run python it runs from that virtual environment with all the requirements that we've installed there so we can do that by typing env path equals and then these double quotes forward slash py forward slash bin colon and then dollar sign path so this just adds a new entry to our path so that whenever we use python it uses it from this directory by default finally we're going to type user django hyphen user and this ends the creation of our docker file and what this does it just switches to the user that we created here so anything we run after this when we run our application it's going to be running from django user and not the root user on our linux container okay make sure you save the file and now let's move on the next thing we need to do is create a docker compose file that is going to run docker for our project so in the root of the project i'm going to create a new file it's going to be called docker hyphen compose dot yml yaml file and then we're going to type version colon 3.9 services colon we're going to define a service called app before i do that i'm just going to change my spaces to two spaces because i prefer using two spaces for yaml files this is totally optional it just makes it easier for me to read them so underneath services i'm going to type app colon and then underneath that i'm going to type build colon and underneath that i'm going to type context colon dot and then level with the build line here i'm going to type ports colon hyphen eight zero zero zero codon eight zero zero zero then underneath that volumes colon and then hyphen dot forward slash app codon forward attach app then finally we're gonna add command colon python manage dot pi run server 0.0.0.0 colon 8 000. so what this does is first we specify version 3.9 that's the version of the docker compose syntax that we're using and it's the latest version at the time of making this video then we define our services block and these are the services that we have defined in docker compose the first service is called app which is defined on line four and then we have underneath app build and then context colon dot and that all that does it says build based on the context of the current directory we're in and that will be this directory so it will by default use the dockerfile in that directory if we had a custom dockerfile that was named something different then we could specify that here in this build section here but we just called it dockerfile which is the default so just by setting the context it will automatically use the dockerfile that we created then below that we map port 8000 in the container to port 8000 in our local machine and this allows us to access the server via http when we run our development server next we have the volumes and this maps a volume of our current directory app here and it maps it into the running container and what this does is it allows us to access the code in real time from the container so when we make changes to our django application that will automatically be mapped and refresh the development server that is running inside docker finally we have the command which is the django run server command for starting a development server and then we specify the port 8000 and we bind it on 0.0.0.0 which just makes it accessible on the network so then we can map the port 8000 to the container and we can access it on our browser on the host machine now we've done that we're going to save the file and then we're going to open up our terminal or if you're on windows you'll use either powershell or command prompt whatever you're familiar with and now we're going to run a command that is going to create a new django project in our project so we'll do that by typing docker hyphen compose run hyphen hyphen rm app sh hyphen c and then the quotation or double quotation marks open them and then django hyphen admin and then start project and then app and then dot at the end and before we close the command we are going to add double quotation again so what this does is it runs the docker compose command we're using the run command which is for running a single command on a container then we're specifying rm which removes the container when it's finished running it just prevents our machine from filling up with containers that we're not using anymore then we specify the name of the service which is app and that's the one we defined in dockercompose.yml then we have sh hyphen c which is just for running a single shell command on the container and then in the quotes here we pass in the command that we're running so we're passing in django admin start project app and then dot so django admin is the cli for django when it's installed and this allows you to start a project start project as you imagine is to start a project app is the name of the project and the dot says create the project in the current directory so that's the working directory of the container then we close out the command and this is all we need for creating our project so let's hit enter on that and hopefully it should build the container for us now and then it should go ahead and create the project if this is the first time you're running the command then it's going to take a minute to download the required alpine image from the docker hub and then it will build the container and then it will create the django project so this stuff happens the first time you run the command when you run it subsequent times so times after this it should automatically be cached in the system so it should be a lot faster almost instantaneous we'll just wait for this to finish and then we'll continue now that this is finished you can go back to the project and if you click on app you can see that it's created some template files here so this is a basic django project that is created for us with the django command line tool after this is done we're ready to go ahead and test our development server so go ahead and head over to the terminal or the command prompt or powershell window i'm going to type docker hyphen compose up this should start an instance of our development server on our local machine so you see something like this and this means it has been successfully started we can go over to the browser so i'm just going to open up the browser here create a new tab and i'm going to go 127.0.0.1 colon 8000 which is localhost on the machine if we hit enter you should see this page so this is the placeholder page for a new django project and this is just because we haven't added any urls or anything to it so far so we're going to be doing that in a moment so now that we have our django project running in our development server we can move on to the next step the next thing we're going to do is we're going to be creating an app inside our project so django projects are structured into multiple apps and we're going to be creating a new app inside the project that we're just going to use to put our demo code so we'll do that by heading back to the terminal and you can push control c to stop the server here or you can open a new tab or window and leave it running if you prefer and then we're going to type docker hyphen compose run dash dash rm app sh hyphen c and then python manage dot py start app demo and what this does is it uses the same syntax as before to run the container in docker compose and we're passing in a slightly different command here so this is python manage dot py star app demo just move this over here so you can see it and what this does is it starts a new app in our django project and it's using the manage.py which is created in the project when we run the start project command i'm just going to open up the code editor here so if we observe the code editor as we hit enter i'm just going to shrink this down so you can kind of see both you hit enter here what should happen is it will go ahead and it will create a new directory in our project with the name of the app which is demo so here we have a new file here or a new directory and inside it we have a bunch of other different files that make up the django app so we're going to start by removing the ones that we don't need so the various of these that we aren't going to be using for this tutorial one of them is migrations so we'll delete that another one is admin.py so we'll delete that and then we have models.py we'll delete that and finally tests so we might cover using that in a future tutorial if you're interested then please leave a comment in the video below and let us know that you're interested in learning about databases with google app engine or whatever you're interested in let us know and we'll try and create some content around that okay let's continue what we're going to do now is just open up the app and then settings.py and inside that we need to add our new app to installed apps so this way it'll be picked up by django as an app that is installed in our project the way we do that is just add a new line below here and then just add a string using single quotes and type demo so this is a new app that we're going to be adding to our project now we save the file and we can move on next we're going to configure the static files for our project so static files are things such as images javascript and css that is needed for our html templates that are rendered by django and the convention in django is to store them within each app inside a directory called static so inside demo we're going to create a new directory called static you can just go ahead and do that now actually make sure it is in the demo app and what you would do is you would add potentially multiple different apps to your project or you may install external apps in your project that also have their own static files so in order to serve them through google app engine we need to use a command to collect all of the static files into one single location that we can then upload into google app engine in order to serve the static files so this is how django handles static files when you're using it in production in order to collect all of the static files in one location we use something called the collect static command this is provided by default by django applications and we can use this to gather up all of the static files from every single app that's enabled in our project and copy it to a single location but first we need to tell django where we want that location to be so where do we want to store all of the static files that are being collected we're going to store it in the root of the project in a directory called static that we're going to keep outside of git or keep outside of our source control and we're going to use the collect static command before we do any deployment we need to add a new line to the bottom of settings.py and that is called static underscore root and we're going to set the value equals in quotes forward slash static what this will do is when we run the click static command it tells django we want all of the static files to be collected into the directory called forward slash static because we're using docker that is going to be put inside a docker container so we need to create a volume inside docker compose in order to get those files accessible in the project so when we run our deploy command from google app engine they can be pushed up to google app engine let's save the settings.py file and then open up docker compose and we just need to change one line here so below the volume here where we have the app we're going to add a new line and it's going to look similar but it'll be dot forward slash static colon forward slash static so this says map any files or map the directory in our container forward slash static to the project and create a new directory called static there and map that content so when we call collect static django will store all of them where we specify it here in the static route forward slash static and in docker compose we'll map a volume from that directory here to the directory in our project save the docker compose file and now we can move on next we're going to be creating a template page that we can use to deploy so inside our demo forward slash static directory and it's very important that you make sure the static directory is inside demo i know we have a lot of different files and directories here it's important to make sure that they're inside the correct nesting so you want static to be inside demo which should be inside app here we're going to add a new file inside static and that file is going to be called style.css and then we're just going to add a simple css file to style our page and we'll start by defining body and open up the braces and then font hyphen family we'll use the aerial option here so ariel helvetica sans serif and then color we'll set the font color to ffff which is just white text hyphen align to center we'll center everything in the page and then background color colon 3d 3d 3d which is just a dark gray then below that we'll do img which will handle the image styling and we'll do border hyphen radius colon 3px and then border hyphen or colon 1px solid white so this is just some basic css i am far from a professional designer so if you want to make it look prettier than this and you want to use your css skills then go nuts here because you can basically do whatever you want at this point now we're going to choose an image and store that in our project so i've provided a link to an image in the resources of the video feel free to use that image or whatever image you want really and we're basically going to save this image and we're going to save it in the project inside our directory so i'm going to do workspace and then the deploy django ga docker and then i'm just going to put this inside demo and then inside static so we have river sunrise call this.jpg in case it doesn't add that automatically okay so now we have riversunrise.jpg which is just an image that we're going to put in the page that we create for our template now we can go ahead and create a html template file so we'll do that by creating a new directory inside demo or we'll create a new file inside demo and if you're using an editor like mine you can create the directories and subdirectories as you go so inside demo we want it to be called templates then forward slash demo and forward slash demo.html so templates is just the standard name for all the html templates that you create in django then you need to specify the name of the app again and this is just the limitation of django the way that it collects up or identifies the different templates you need to specify the name of the app for the template and you do this in a subdirectory of the templates directory then you can open up the demo.html file which is just the name of the template we're going to add some contents here so i'm just going to add a basic html file but we'll start by using the load static tag which allows us to load static files inside our django template we'll do that by using these squiggly brackets here and then we want to do two percent signs and inside that we're going to type load static then below that we're going to add the document html tag and then below that we're going to add html lang equals en and then inside that we will do i'm just going to change my spaces to 2 again because i prefer the smaller spaces easier to read and then we have head title django gae placeholder and then link rel equals stylesheet href equals we will add here the syntax for retrieving a static file which is these open and close squiggly brackets and then the percent signs and then static and then style.css so what this will do is it will get the url so the path for the star.css static file and it will automatically map that for us in django and you're going to see how this works in a bit then we're just going to close the link tag here close the head tag and then create a body tag and inside the body tag we'll just add some contents here so h1 django gae placeholder page and then below that p and we'll just say here's a sunrise and below the p tag we'll add img src and again we're going to use the same syntax here for getting the static file so static and then river hyphen sunrise oops sunrise.jpg and then just close the tag out here and if you save that file the next thing we're going to do is just create a view and we'll do that by opening up the project and again making sure we're opening up views.py inside the demo app and we're just going to add a simple view here so we'll type df demo request and then return render request demo forward slash demo.html download.html and this just simply has a single line view which is a function that this is the entry point that the url maps to in django so when we make a request to our url it's going to call this function which is then going to return the render function it's going to call the render function here pass in the request which is just the request object it has all the details about the request that's being made and we're going to render this template so it's the template demo for demo.html you can save that and the next step is to actually connect this up to a url so we can go ahead and test it so we'll open up the app directory here so app forward slash app this is the kind of project configuration for our project and then we'll open up urls.py and inside urls.py we are going to add a new line here below the imports and we're going to do from demo import views so from our demo app we're going to import the views that we have created our view in then in the url patterns here at the bottom we're going to add a new line that's path and we'll just add an empty string so we'll handle the kind of root path for our project and then comma views.demo so this said whenever we get the url for the root of the project call the views.demo function which will then go ahead and call the function insideviews.py and return the render function here which should theoretically render our template you go back to urls.py just make sure you've saved the file here now open up the terminal or the powershell whichever command line you're using and do docker hyphen compose up and this will create the development server again and then we're going to go back to the browser and i'm going to open a new tab i'm going to head over to the 127.0.0.1.8000 hit enter you should see the placeholder page that we have created so this is a very basic django app that just renders a single page and this is what you should see on the screen the next thing we're going to do is we are going to set up our project to toggle debug mode so if you head over to the project in visual studio code you open up settings.py and you scroll to the top what you should see here is there is a debug equals true now it says above it a warning it says don't run with debug turned on in production and the reason for this is that the debug mode is intended for just developers working on the project and all it does is when the project crashes it allows you to see lots of information about the crash and all of this information should help you debug what the problem is however this is not good for production because it helps attackers or potential attackers get more information about your app than they need so this could help them find vulnerabilities in your app so on so the best practice is when you deploy your application whether it's to google app engine or any service that you're deploying to you want to make sure this is set to false but when you're using it locally you want it to be true so i find the best way to do this is to use an environment variable to toggle the debug mode we can do that by adding a new import to the top of the file this is import os and then we're going to replace this true here i'm going to change it to ball and then open and close the brackets we're going to call the ball function which is the boolean function then int open and close brackets os dot environ dot get and then inside the get parameters or the arguments that we pass in is going to be debug as a string comma 0. so what this does is we get the environment variable called debug and if it isn't set it doesn't exist we're going to default to zero so by default we're not going to enable debug mode then we convert that to an integer using the int function and the reason we do this is because all environment variables are always strings so even if you set it to a one or a two it's going to come in as a string and this is going to confuse us when we convert it to a boolean and because we want it to be a zero to be false and one to be true once we've converted it to an integer using the int function we then convert it to a boolean which converts it to a true or false value so zero converts to false and one converts to true now that we've updated the debug configuration there's one more thing we need to do and that is to modify this allowed hosts so a loud host is a feature that is provided by django which is designed to protect against header attacks so there's a certain type of attack that they can do on the django application and this could be avoided by specifying the specific host names that are allowed to access your application when it's not in debug mode and the way you normally do it is you would define the different host names here so you would provide a list of all the host names it could be app.example.com or whatever the hostname is for the django project and then only that hostname would be allowed to access the app when it's running in production however google app engine apparently according to their documentation has a built-in protection against this and their recommended approach is to allow a wild card so allow all the hosts in the django configuration because that protection is done at the google app engine layer so this is okay when you're deploying to google app engine but it's not okay if you're deploying to any other type of deployment so if you're deploying in any other way you want to make sure you specify your allowed hosts whereas because we're using google app engine we can just put an asterisk here in a string and this will allow hosts from any host and google app engine will protect us against the header attacks so save the settings.py file and then we are going to head over to dockercompose.yml and you can navigate to it here if you want open up that file and we're going to add the environments block to the bottom of the file so we'll do environment colon hyphen debug equals one so this means when we run in docker compose it's gonna set debug to one so that we have debug mode enabled when we're running our development server locally but if it's running and debug equals one is not set it is going to disable debug mode which will be when we're running on app engine so save the project and just to make sure it's working okay you're going to need to close out of the server so you can do that by doing control c if your server's still running mine's not and then you can run docker compose up and it should restart the django development server and just to make sure it is all working okay once it starts open up your browser and just refresh the page okay so everything's working as expected so far the next thing you want to do is sign up to google cloud platform if you haven't already signed up when you sign up you should see this my first project as a default project that's created now don't worry if your screen looks a little bit different from mine it's possible that you're just a different stage in the kind of registration process if you've got a brand new account you should have my first project if you already have an existing account and you have a project on there that you've used for something else i recommend creating a fresh project just for this demo so that we can make sure there's no conflicts or issues with the code so if you need to do that you can click on the drop down here and you can create new project and you can go ahead and fill out the form here and this will go and create you a new project once that's done make sure you have the project selected so in my example i'm going to be using my first project and the next step is to enable google app engine so if you click on this hamburger stack here navigation menu you should be able to locate app engine underneath the serverless section of the left hand navigation when you click that for the first time on a new project you should be taken to this page here so this page allows you to enable a new app engine project if you've already enabled app engine then you won't need to complete this you only need to do it once for each project that you want to use app engine on in your google cloud platform we're going to click on create application and then we need to specify a region so i'm going to specify a region near me it's recommended that you use whichever region is local to you or more importantly local to the users that will be using your application it's important to remember that as of the time of creating this video you cannot change the region once it's been set so if you create it in the wrong region and you want to change the region the only option you have is to delete the project and create a brand new project in the region that you want once you've specified the region you can click on create app this will go ahead and create the google app engine app it's going to ask you the language and the environment so we're going to use python because that's what we're deploying here and we're going to use the standard environment so there are two different options here standard and flexible standard is the best one in most cases and what standard is is it basically means that google is managing the servers entirely for you you cannot connect to the server that is running the code that you push to app engine and this is good in a lot of cases because you don't need to worry about the management overhead of that server you don't have to apply security fixes you never have to log into it and install dependencies or worry about it if it breaks this is all taken care of by google and this is one of the amazing things of using app engine is it totally removes that overhead of having to maintain servers which can be a headache because you need to maintain them make sure they get the latest patches and fixes and so on however if you do want to maintain the server because you do need to for whatever reason it could be because you need some kind of bespoke dependency running on the server there are various different reasons you would choose flexible but that's kind of like an advanced tutorial so we're not going to be covering that here we're just going to be using the standard environment and it's what i recommend in most cases once you've selected that you can click on next and then this will take you to this page here which gives you some tips for getting started so kind of next steps we're not going to need them here because we're going to be doing that in this tutorial so you can just click on i'll do this later it will take you to this page that says url not found and this is because we haven't deployed anything to app engine just yet the next step is to configure our project to actually be able to deploy to app engine so we can do this by opening up our project and we're going to create a new docker compose file here in the project so i'm going to close all of these and i'm just going to shrink these down so you can see the full view and i'm going to create a new file here in the root of the project it's very important it's in the root of the project and you're going to type docker hyphen compose hyphen deploy.yml this is going to include the docker compose configuration for our deployment now you might be wondering why do we need so many docker compose files isn't this just a whole lot of overhead without much benefit i would disagree i would say that there are lots of benefits to this small amount of overhead in comparison one of the main benefits is that you can isolate the dependency of the google cloud sdk from your local machine so you don't have to have it installed locally you can use it through docker and therefore you can create a reproducible environment for all the developers on your project i've worked on many projects that developers had to spend days if not weeks setting up their machine because they have to have different versions of the google cloud sdk for different projects they're working on multiple projects and there's all these different configurations that takes a long time to figure out and can get very confusing especially when you're working in a professional environment so i always recommend doing everything through docker so you can have that consistency for your project and you can control the versions that you're using of the different tools that are needed for your deployments this way you don't get one developer saying oh i can't do deployments because i don't have the right dependency on my machine or i have a conflict or something everyone can do it no one has an excuse it's all set up as long as they have docker installed so inside this docker compose deploy dot yaml we're going to create a new configuration specifically for the deployment process so what we need to do is start with version hyphen 3.9 and services colon and i'm just change this to two spaces and the first service or the only service we're going to need is gcloud i'm going to talk you through what each line does in a moment let's just type it out first we use image colon google forward slash cloud hyphen sdk colon 338.0.0 then volumes colon hyphen gcp hyphen creds forward slash creds or creds plural with an s and then dash dot and then codon forward slash app working and then level with volume so we need to make sure we're level with volumes here actually i've got the indentation a little bit wrong here so let's just indent this volumes this volumes should be level with image and it should all be underneath g cloud the indentation is very important with the yaml files below volumes we're going to have working underscore dir colon forward slash app and then environment colon dash cloud sdk underscore config equals forward slash credits and it's very important this is spelled exactly as i typed it here so cloud sdk underscore config equals forward slash creds now finally we're going to add a new block at the bottom here so delete the tabs all the way to the beginning of the file i'm going to type volumes colon and then gcp hyphen creds and then a colon at the end so it might seem strange but there's a reason for this we'll just save the file now i'll talk you through it so first we have the syntax version of docker compose 3.9 then our services block and then the service g cloud which is based on the image google forward slash cloud hyphen sdk and then version 338.0.0 which we're pulling in via a tag now you could change this version here to just latest so if you deleted three three out and change it to latest it would use whatever the latest version is at the time and put in the container however sometimes this can be a problem because there are any breaking changes that would break your deployment that deployed to the sdk then you might want to be in control of being able to manually increase the version and make sure you can test it and that nothing breaks before you go ahead and do that and that just helps you to keep the stability of your project up but if you do like living life on the edge and you want to use the latest version all the time feel free to change this to latest and you'll get the latest version of the cloud sdk next we have volumes and we're mapping a volume here called gcp creds so because we don't have any dot or forward slash before gcp credits this maps to a named volume which we've named below here and what this does is it creates a volume with docker compose for our credentials that we'll use to authenticate with google app engine and the reason we do this is just so that every time we run our docker container we don't need to authenticate again so it allows us to have a persistent set of credentials that we can use for the project until we delete the volume manually and this is quite useful to do in a project because it helps you to save a lot of time from authenticating multiple times we're mapping it to a directory inside the container called forward slash creds and then we have another volume here which maps the root of our project so just having a dot here maps the root of our project so everything inside our project directory into the forward slash app directory inside the container and this is needed because the cloud sdk needs access to this in order to deploy our app then we specify working dir that's the working directory and we do it to forward slash app and then the environment so we set an environment variable cloud sdk config this tells the cloud sdk to store all of our credentials inside our forward slash creds directory which is mapped to the volume here then here we map the volume so this tells docker compose to create a new volume specifically for this project that we can use to store the credentials now we have the docker compose that will give us access to the cloud sdk we can go ahead and create an app.yaml file in the root of the project so we'll do app.yaml and it's important to use the yaml so there are two different types of name that you can give a yaml file one is just yml and one is yaml the google cloud sdk likes the longer version here so they're both the same the same file extension just one's a longer version and the google cloud sdk prefers yaml i don't think it would work if you just put yml and feel free to try that but i don't think it would work so make sure you do yaml now this is going to define our app engine deployment this is where we put all of the details that tells app engine how to actually run and serve our application so at the top here we're going to type runtime colon python 3 9 which uses the python 3.9 runtime which is the latest at the time of making this video then we specify entry point colon and we type g unicorn hyphen b colon and then dollar sign port dash dash chdir app forward slash app.wsgi colon application the entry point is the command the app engine will run to start our project so we're going to use g unicorn here we're going to pass in dash b which i believe stands for bind so we're going to bind to the port in the environment variable port and this is set automatically by google app engine then we have chdo which is change directory and this tells g unicorn to run from the app directory that is going to be pushed up to the google app engine file system that is going to contain the django project that we're running then we specify the wizki file so when you create a django project inside app forward slash app you see you have this wizki.py you need to tell it where that is located within app so if you're already in the app directory it's located in app dot wsgi and then we run this application here so this is all automatically generated by django you don't need to manually change anything inside the wizki file so that tells app engine how to run the project next we need to define the handlers so this is how does app engine handle the urls when people make requests to google app engine so i'm going to add a space here it's totally optional you don't have to add a space i just like to keep it neat and tidy new handlers colon and then dash url colon forward slash static and then below that static underscore dir colon static forward slash and then below that we're going to add a second url which is the same level with this url underneath handlers dot url colon forward dot and then the asterisk symbol here the wildcard asterix then below that we're going to do script colon auto okay save the file i'm just going to talk you through these handlers here so as i said handlers handle the url requests made to the app that's running in app engine so we have url and this is the url for capturing all the static urls so the static files in our project then we're going to start with forward slash static so we can tell app engine anything that starts with forward static serve it directly from the static directory static forward slash so this is the one that's going to be created when we run our collect static command and it's going to be uploaded to google app engine and this means all the static files can be rendered directly without having to run through the wizkid service because the whiz key service is ideal for serving python code but not serving static files so if you want to serve them efficiently you need to serve the static files separate outside of the wizkid command here that we're running in gu unicorn finally we have this wild card so this says any other url that hasn't already been matched so anything that is not static we're going to send it to the entry point script so we're going to use the auto script and this will run the g unicorn command here so this says anytime there's a url that isn't static it must be a python code that needs to be executed so pass it through the g uniform corn request and then that will return the result of the page so we can save the file and now we can move on the next thing we're going to do is we're going to add a g cloud ignore file so inside the files of the project make sure it's in the root of the project i'm going to add dot g cloud ignore and here we're going to add all of the files we want to exclude from app engine so these are the things we don't want to be uploaded to app engine by default the google cloud sdk will upload everything we don't need everything we only need to give it the bare minimum it needs in order to run the code so that includes the static directory that we have here which is going to have our static files and then the app directory so we basically need to list every other file we'll start by dot get then dot get ignore then underscore underscore pi cache underscore underscore forward slash so any pi cache files that are these hidden files that are created here we want to exclude them because they'll be generated by app engine when the code is run and then uh forward slash setup.cfg this is in case we add a setup script to the file we're not going to do that but this is just a kind of standard file that we want to create then docker compose.yml docker hyphen compose hyphen deploy dot white dot yml and then docker file make file and readme.md just in case you add these files to the project we don't currently have them but it's just good to have them here in case we do add them so save that and that's everything we need to do to set up the project now we're going to move on to actually deploying the application before we deploy we need to authenticate with google cloud in the sdk so this we only need to do one time before we do a deployment and then we don't need to do it subsequent times because as i explained earlier the credentials will be cached in a volume that are going to be reusable for the project so to authenticate we're going to open up the terminal and open up a new tab here you can feel free to close the dot the docker compose server and then just use that one i'm going to create a new one i'm going to do docker hyphen compose it's going to expand this here so you can kind of see the full line docker compose hyphen f docker hyphen compose iphone deploy dot yml run dash dash rm and then gcloud which is the name of the service sh hyphen c and then g cloud auth login so this is the command for running a docker compose command it's quite long if you're worried about the length then you can always create a make file which has some shortcuts if you click on the blog post that accompanies this video i should i have an example of a make file that you can use in there that simply just reduces the different commands down to just make login or make deploy so you don't have to type this every time but for this video we're just going to type it out just so we can kind of learn what the commands are here we have docker compose we're telling it to use the docker compose deploy.yaml file then we're using run removing the container after we run the command we're calling the g cloud service and then we're passing in sh hyphen c gcloud auth login so this is the cloud sdk command for logging in to the cloud account so hit enter and the first time you run it it's probably going to pull the container down and this can take a few minutes depending on your internet speed once this is done it should be cached on your local machine so only happens the first time you run it every time you run it after that it should be a lot faster because the image is already there ready to just be run we're going to wait for this to finish and then we're going to continue [Music] once the command finally finishes running and the image is downloading you should see a screen like this so this contains a url that we can use in order to authenticate with google cloud to access our google google cloud platform via our account we're going to copy the contents of this and then open up the browser and then in a new tab we're going to paste it in and it will ask you what account you want to authenticate with so we're going to choose the account and then it will ask you to confirm that you want to give access to the account so this is allowing you basically full access to the google cloud platform that you are authenticating for so we're going to click on allow because that's what we need in order to deploy to app engine then you're given this code so this is like a random code that we can use to complete the authentication process and we can do it by clicking on copy here heading over to the terminal pasting the value of the code and hitting enter this will go ahead and complete the authentication so here you can see that we are logged in and this means we're now authenticated and the credentials for this will be stored inside the gcp hyphen creds volume that we set up in docker compose so i'm just going to clear this screen and now that we're authenticated we can move on to the next step which is actually deploying to google app engine there are two commands we need to run in order to complete the deployment so the first is collect static so this is the command i was talking about which is a django command for collecting all of the static files needed for our project in one single location if we open up the project and then we can see the static directory is already created here this was created when we run our container last because it maps a volume and it creates it automatically so we're going to just run the command by typing docker hyphen compose run hyphen hyphen rm app sh hyphen c and then python manage dot py collect static make sure you spell it correctly click static hit enter and this will go ahead and run the click static command so what we should see here is after a minute it should gather everything together and you can see it populating the files now so here are all of the static files for the different apps we have so this is the river sunrise and the style.css that was taken from our demo and you might wonder well why don't you just put them directly in there well it's because there are various different apps in your project and you want to make sure as you add more apps they can easily be curated in one location when you run this command which is typically part of the deployment pipeline you can see here we have admin and this is for the django admin app that is automatically installed for all new projects we're not actually going to be using it but it's actually included it because we still have it in installed apps in our project configuration but the point is that the collect static command collects up all of the static files and puts them in one place now we can run the actual deploy command for google app engine do that by typing docker hyphen compose hyphen f docker iphone compose iphone deploy run dash dash rmg cloud sh hyphen c and then open the quotes here gcloud app deploy dash dash project and now we need to head over to the web browser again so the console that has the google cloud and we should be logged in here you can get the project id by clicking on the project drop down here and choosing the id for the project you want to deploy to so we're using my first project so i'm going to copy this id here copy that to clipboard head back to terminal and paste it in here after the project so what we do here is we run the docker compose command without deploy.gamol configuration we're passing in rm so that we remove the container when it's finished running using the gcloud service and then sh hyphen c which is to run a single shell command when we run the container and we're calling gcloud app deploy which is the command in the cloud sdk for deploying to google app engine and then we specify the name of the project by doing dash dash project and the name of the project we're deploying too once this is all typed out you can hit enter and this will go ahead and start the deployment so we should see that happening now and it can take a moment it will ask you if you want to confirm that you want to deploy i'm going to hit y for yes hit enter now it should start the deployment so you can see it's uploading all these files this includes all of the static files all of the python files inside our app and it's uploading them to the google app engine file storage and then it's going to be able to actually run our application or at least hopefully it should be able to run our application we'll find out in a minute let's just wait for that to finish and then we'll continue [Music] it's going to change these batteries out hopefully it keeps running actually we should be good we've got two bars keep an eye on that i lose volume now i'm gonna be angry [Music] okay you can see now that the deployment has completed successfully the first time i ran this before i recorded this video i got an error saying the cloud build api was not enabled i think this was just a glitch in app engine because the first time i ran it it gave that error and then when i run it again the error just went away and it didn't happen again after that so i think it was just a issue with the app engine sdk for when you're deploying so if you do get that then just run the command again and hopefully you don't get it again once the kind of api has had time to enable itself in the cloud project now that the deployment's done you should see a url here so you see deployed service default you can copy the contents of the url and open up the browser paste it in in a new tab and we should see the app work if everything worked correctly now there's a chance that there was an error somewhere and it doesn't work but you can see that here it works correctly we have our app now deployed to google app engine so every time you need to run a deployment you need to run the two commands for collect static and then the app deploy command and you can consolidate these into a shell script or you can add these to your continuous deployment tool like github actions or whatever if you don't want to type them out every time some people just like to create a make file and as i said earlier there's an example of the make file in the blog post that accompanies this tutorial so that's how you deploy a simple django app to google app engine using docker i hope you found this useful if you have any feedback or comments then please leave them in the comments below they always generate an interesting discussion and hopefully we can learn from each other if there's anything else you would like to learn about this process like for example how you can handle django migrations or how you can integrate with the cloud data store then please comment below and let me know so i can figure out what content to create next because there's so many different options that you can do from this point and it's good for me to get a guide from you as to what you're interested in learning thanks so much for watching and i'll see you in the next video [Music]

Channel: London App Developer

Views: 3,842

Keywords: python, docker, google app engine, gae, google cloud, tutorial, deploy, deploy-compose, django

Id: TFsFLzNL5Fk

Length: 59min 10sec (3550 seconds)

Published: Mon May 03 2021