How to use GeoDjango (import GIS shapefile with Python)

in this video I'm going to show you how to use geojango which is a geographic web framework that allows you to do a lot of cool GIS stuff but we're going to keep it simple initially and just show you how to take a shape file import it into a postgres database and then display that map data in the Django admin console so it's a big one let's go ahead and get on into the tutorial all right I'm going to be using a remote server in the cloud running Ubuntu in this tutorial I'm going to show you how to set up that server I actually have account credits down below that you can use to set up a free server for yourself in the cloud as well otherwise you can use your own system but some of the commands might be a little bit different since I'll be using Ubuntu so go ahead and Skip forward if you have your own system otherwise let's set up our remote server and I'm going to do that with digitalocean so once you're here you can create a droplet and our droplet is going to be located in Singapore for me because I'm located in Kuala Lumpur and that's the closest data center to me physically so pick the one closest to you like I said I'm going to be using Ubuntu latest version 22.10 is good we don't need too much resources so we'll just pick a regular CPU with uh one gigabyte of ram 25 gigabytes of solid state drive storage and then I'm gonna pick a password okay and the only other thing I need to do on this page is to rename my hostname you can choose whatever you want I'm just gonna call mine Ubuntu okay so click on create droplet that's going to go ahead and create your virtual private server in the cloud somewhere and when that's finished we'll catch back up with you our Ubuntu server is up and running at this IP address so I'm going to copy that and what I'm going to do on my Mac is open up a terminal window to log into this remote server via SSH if you're on Windows I have other videos about getting set up with SSH on Windows pretty much all you have to do is type in SSH the user we're going to be operating under initially is root and then we're just going to do at that IP address that we copied hit enter the first time we connect is going to ask are you sure you want to connect we can type yes hit enter and now that password that we just created which I have saved over here I'm going to copy and paste that into the terminal window hit enter and now we are on our remote Ubuntu system so first thing I want to do is to install the postgres database okay so there's also an extension called post GIS which includes postgres let's go ahead and do that and I'll show you what I mean so we'll do a pseudo apt install post GIS hit enter and this is not only going to install post GIS but all the dependencies all this good stuff and most importantly postgresql version 14 with post GIS version 3. so that's 84 new packages let's go ahead and hit enter and let that do the install okay now that we have postgres installed let's set up our database and first thing we're going to do is switch over to the postgres user right now we're using the root user whoops not sure what happened there let me put this back to a normal reasonable size okay so yeah we're going to switch over to the postgres user you can see that happen instantly instead of root before now we're postgres and what we're going to do is to create a database and the database that I'm going to choose to name it is going to be gisdb so my database name is going to be jsdb so we'll execute that and that database has been created and now let's configure that database and to do that we'll access the postgresh Shell with psql and we want to access that with the gisdb so we'll hit enter and now all these commands will be postgres commands so there's a few things we want to do I'm going to use my cheat sheet over here to copy and paste them in so we want to create the extension postgis which if you remember that's the extension for postgres associate that with this database we're going to create a user my user my database user is going to be called George he's going to have the password geography is fun and a few other things here we are going to set the default client encoding to utf-8 we're going to set the transaction isolation to read commit it don't worry about too much what these mean we're going to set the time zone to UTC and then we're going to Grant privileges all privileges on the database to George so that's all been set up that's a once and done process we'll go ahead and exit out of the postgraph shell and exit out of operating under the postgres user so now we are back to running under root which on that note root is not a good user to operate under to make a Django database under so let's make a new user with the ad user come in and I'm going to make a user with the username Tony and I'm going to give him a password confirm my password and give him a name and all this other stuff is pretty much optional so you can just click enter or hit enter is the information correct yes hit enter okay so we now have that user created let's give that user pseudo privileges so we can install additional packages as necessary so we can do user or mod Dash a capital G sudo Tony to give that user pseudo Privileges and now let's switch on over to that user so s u Dash Tony and now you can see instead of root we are operating under Tony all right so let's install some more packages here we need Django we need Python and we need a way for python to talk to postgres so we can do that with sudo apt install Python 3 Dash Django and then we also want Python 3 Dash p s y c o p G2 do you want to continue go ahead and type enter and we'll let that do its thing it's now time to set up our Django project and we can do that with the Django admin command and we'll pass it start project and then you can call your project whatever you want I'm going to call mine GIS and what that does is creates this GIS folder so let's go into that folder and in here we have another GIS subdirectory which we'll be looking at in a bit and this manage.pi file which is going to allow us to interact with our Django project so let's actually use that right now we'll type in Python three manage dot Pi start app and I'm going to call the app within my project world so you can have multiple apps within a Django project but in our case we're just going to have one app and I'm going to call mine world so let's do that and I spelled start app wrong it's s-t-a-r-t-a-p-p so try that again and what that command does is creates another subdirectory under here called world and in here we have a few different python files a couple of them we'll be looking at throughout this tutorial is admin dot pi and models.pi so we'll do that later on but for now there are a couple settings within the gis subdirectory that we have to change so inside of the gis subdirectory we have this settings.pi file so let's open that and you can choose whatever text editor you want if you're not familiar with Vim I recommend to use Nano so I'm going to open up that file and in this file there is a section called databases and here's that section so we basically want to tell Django about the postgres database that we set up so we'll get rid of these two lines which default to a SQL Lite database and instead of that I'm going to paste in these few configuration lines which basically says the engine that we're going to use is a post GIS backend which is actually built into Django the name of our database which we created earlier was jsdb our username for the database is George his password is geography is fun and this is going to be hosted on the Local Host which is the server that we're on right now so that's one setting that we want to change the other one is down here in installed apps oh nope it's up above okay here is the installed apps settings so we added an app called GIS to our python project so we want to make Django aware of that so let's do GIS here and we also want to leverage the Django JS package app I should say so we're also going to include in this list django.contrib.gis all right let's save that file and at this point it's time to start looking at the data that we're going to be working with so I'm going to open up my web browser and go to naturalearthdata.com this is an open source set of data that we can download here let's go to the downloads Tab and they have large scale medium scale small scale and they have a couple different options cultural physical and raster we're actually going to be working with the cultural data set which defines country borders and actually below that too sometimes they have um as depending on how specific you want to get they do have States and provinces and for some data sets like the United States I believe they have counties as well but we're going to start with a high level countries View and specifically we're going to look at Sovereign countries okay so this data set with 209 sovereign states so what we're going to do let's download this locally on our computer first and look at it in a GIS software and then just to explore it get familiar with it and then we'll work with it in Python so let's go ahead and download that and that downloads as a zip file so I'm going to extract that and open it up and what do we got in here we got some a bunch of different files but the one that we're interested in is this shape file so let's open that with an open source GIS software called qgis so we'll do that and this is a basic overview of the data so you have countries all these Sovereign countries with their outlines that are clickable each one of these corresponds to a row in the data table and has all these different uh metadata associated with it so if we right click on the shape file and go to the attribute table we can see each one of these countries in here so let's take a look at Kenya right so we can click on the Kenya row right click on it and pan to that feature Zoom to that feature and this country right here is Kenya and looks like it got that one right so you get the point let's minimize this and pull that data into our python Django project so I'm going to get out of here get out of here and we want to pull this data onto our server so we can right click this and copy the link address which as you can see down here is a ZIP file so let's copy that minimize this and in our Django project let's make a new directory inside the world directory let's make a directory in here called Maps let's go into that directory and let's use the wget command to download that zip file so you can do wget and then paste the path to that zip file hit enter and that's going to go ahead and download that onto our remote Ubuntu server so here is that zip file let's unzip that file to extract it but we need to First install the unzip command so we can do that with sudo apt install unzip okay now that we have the unzipped command let's unzip it with unzip and then pass it the name of the file that we want to unzip so unzip hit enter and that has extracted that into a folder I'm sorry that not a folder into the individual files including our shape file which is what we're interested in sorry guys at this point I realized I made a mistake early in the tutorial uh let me show you what I mean inside of our JS subdirectory there is the installed app section in our settings.pi file where I said that we installed an app called GIS which was wrong our app name was World our Django project name was GIS so we want to change that from GIS to world and save that and now we can continue the tutorial by pulling in that shape file to our Django project so to do that make sure you are in the root of your Django project and we're going to use the manage.pi command to do that so let's go and type python3 manage.pi and we're going to use a command called ogr inspect to pull that shapefile in so this takes quite a few arguments so let's make sure we get it right so the first argument is the path to the shape file so that's located at world maps and then Natural Earth admin zero sovereignty dot shape now the second one is going to be the name of the Django model that we want to use so I'm going to call my Django model country because each row in that table represents a country okay the next argument is dash dash mapping and then dash dash multi dash dash name Dash field is going to be name and dash dash null true okay so there's a lot to explain here a couple important ones are the name field we're pulling that out of the attribute table that we saw there's a column in there called name so that's going to be the name of the individual row whatever row whatever country we're looking at that's going to be the name we want to explicitly allow null values because some some of the columns don't have values for some of the rows so null equals true and that looks good so let's see if that works let's execute that command and that prints out basically two different things okay so the second part right here it's all every single column in our shape file is being mapped to what it's going to be considered in our database or in our postgres database so that's the country mapping so that's the second part and the first part is defining a Django model for that shape file so we basically want to take this first part all the way up to here copy that go into our world sorry not the apps into world and then our models.pi file and we're going to get rid of everything in here and paste that in so what do we just do we created a Django model which is a representation of the database in here for the country so basically think of this as a good way to interact with the database via python so let's save that file and now we have this second half of the file that we have to do something with which is the country mapping so I'm going to copy that and basically at this part of the tutorial we're going to be following pretty closely to this tutorial on the official Django documentation website where we create this python script where we Define our mapping between the model and the database and then we give it a path to the shape file and then we run it we run that command to pull in row by row each one of those countries into our postgres database so we're going to take that again one step at a time so let's as it says here let's create a file called load.pi with this data in it so I'm going to do that inside of world load dot pi open that up and then I'm going to add a few spaces up top here and paste that in now right below that in the example that they have here you want to specify the path to your shape file on your computer and I have already done that in my cheat sheet over here so our country shape file is located up a level in the maps directory and it is called this dot shape okay so a couple other things we have to do in here we want to grab this and make some modifications to it this is our function that's going to actually do the heavy lifting for us and import that shape file and we have to change a few things so instead of world shape we called it country shape instead of World border we simply called it country that's the name of our model and instead of world mapping our dictionary above is called country mapping so I think that looks good let's go to the top of our file and pull in what we need to pull in like they do in the example here so we'll copy this pasting it up top and again instead of world border our model is called country so from basically saying from the current directory that we're in we have a file called models.pi and in that file there is a definition for what a country model looks like so um I think that looks good let's save that and best way to find out is to run it to see if it works so a couple other things we have to do before we do that though like I said this is a long tutorial so basically we have to we didn't really interact with the database yet we have to tell the postgres database about our definitions of what a country is and all that good stuff and Django takes care of all of that stuff so we don't have to know any SQL language or database language to do that we can just tell Django that we want that to happen and Django is going to take care of that behind the scenes so in order to do that we can type python3 manage dot Pi make migrations and that's going to go ahead and as it says here it created the model called country which it just looked in our models.pi file to see what that definition looks like and in order to actually apply those changes to the database we can do python3 manage.pi migrate hit enter and that's going to go ahead and actually commit those changes to our postgres database okay so we're now at the time where we can pull in the actual shapefile data into our Django database so let's do that we can do that with python3 manage.pi shell and this is going to open a python shell where you can execute python commands now and in here we want to do from world import load so this is basically like I said from there's a subdirectory called world and inside there there is a file called load.pi which we just created so we're going to pull that in here and load has a function in there called run we created that as well so when we run that it's going to execute that command and pull in all the individual records from the shape file so let's see if that works hit enter and it is one by one saving each country to our database okay that has finished up it pulled in all those countries and let's confirm that let's interact with our database with python and this is where the true power of Django shines because like I said earlier you don't have to know any database languages or anything like that you could just use Python to see what data you have in your database so from World dot models import country okay so that's kind of what we did up top we did took a shortcut we did from dot models import country but we can be explicit and say from world.models import country if I spell it right okay so now we have access to that so we can see all of our countries or at least get a count of them so let's do country dot objects dot all or I'm sorry country.objects.count and that's going to say that we have 209 countries so that looks good so far let's do country dot object X dot all that's just going to give us this truncated list of all the countries and that looks good too let's get a single country let's do country dot objects.get name equals Italy and that's going to return the country object for Italy one of the columns in there was population estimate so we can access that with the property pop EST and there's your population estimate for Italy which looks like 6 million or so we can also filter the data so let's do country dot objects dot filter continent equals Europe and that's going to reach oops I missed the S on objects there so we'll do country.objects.filter continent equals Europe and that again displays that truncated list of all European countries but let's get a count countries out of this Europe accounts and that returns 45 countries which sounds about right let's shift gears now and look at some actual map data okay I want to see a map on my screen so let's get out of our python shell here with exit and Django comes built in with a web server and we can start that web server up with python3 manage.pai run server and because I'm working on a remote server I'm going to pass in 0.0.0.0 which allows the Django web server to listen on all public IP addresses with the port 8000 so when we execute that I can come back into well not that one I can come back into um digital ocean where I set up my server copy this IP address open a new tab go to that IP address colon 8000 and that's going to load our Django website not quite yet we have to change one thing in our settings.pi file we have to add our IP address to allowed host so let's do that quick close out of our server with Ctrl C open up the gis settings.pi file and in here I think it's at the top allowed hosts it's empty right now so let's add our IP address to that save it run our server again and now we can load the page and we see the default Django admin landing page or not admin landing page we're going to look at that now so yes this is the default public landing page if you go to that same URL slash admin you will see a way to log into the admin site and by default there isn't a user for that we have to create that user so let's go ahead and do that we'll Ctrl C out of here we can do python manage.pi create super user oops python3 manage.pi create super user and I'm going to use the username Tony give them an email which honestly really doesn't matter and then password confirm that okay and now we can use those credentials to log in so let's start back up our admin site or our Django development server and then we can try to log into the admin site so we'll give it the username Tony which I just created and the password log in and now this is our admin site so by default our model our country model is not displayed in here so we have to actually enable that so let's go do that inside of our world directory we have the admin.pi file and in here we have to register our model so let's do that it's really simple we'll get rid of all this and say from Models import country and then admin Dot site dot register country so this is basically registering our country model with the admin site so we'll save that start up our development server reload the page for our admin site and now we can see access to our country so let's click on countries and we do see a whole bunch of list of countries there's 209 countries which checks out let's look at one of these let's look at which one do we know Singapore all right there's all your metadata associated with Singapore such as the sovereignty a short three-letter code for that the official name Republic of Singapore and then all the way down here at the bottom we see that shape of the country the border of Singapore and you can zoom out and see that with context with the entire world let's try one more let's do um New Zealand so same type of thing all that metadata and down here at the very bottom we see that GIS map data with New Zealand highlighted in the next video right here I'm going to show you how to make a public facing website where each country has its own page with the name of the country and a map thank you

Channel: Tony Teaches Tech

Views: 5,154

Keywords: geodjango, geodjango admin, geodjango examples, geodjango install, geodjango tutorial, gis data python, gis in python, gis python, natural earth data, natural earth data set, natural earth gis data, naturalearth, naturalearthdata, python for gis, python gis, python shapefile, read shapefile in python, shapefile python

Id: Wju7FPGcNO0

Length: 26min 8sec (1568 seconds)

Published: Thu Sep 14 2023