Download building footprint dataset from @Google | Open Building Dataset | GeoDev

people recently release the building footprint data set from almost all around the globe in this video I am going to show you how to download that building footprint data set in your required format either in zip file or geosition with your custom area of Interest so let's get started in order to download the data set you need to go to site dot research.google slash open buildings and you will enter into this website so don't worry about this URL you can find that URL in the video description below and here they have some description about data set so they mentioned data they they have 1.8 billion building detection from 58 million kilometer Square within Africa South Africa southeast Asia and Latin America and Caribbean you can see the data set in this map so they have mostly covered the southern part of the globe and then they have the building density uh shown in the map and then this data set can be used in population mapping humanitarian response environment science addressing system and vaccination planning and also in the statical indicator so in order to download this data set so you can see all the license and then data set create some things so they have two type of license so creative common attribute CC by before license and also common open data set license so in order to use the data set you can you don't need to pay or you don't need to do anything you can just use it it's since it's the open source data they have discussed about the data accuracy in terms of three type low density medium density and high density so you can see the accuracy scores precision versus recall value in this graph for Africa South South East Asia and then Central America Caribbean Islands and yeah so by default they are giving the data set in the CSV format with the threshold score but of course you can download this data set as the csb and I mean zip file and use Json format later with the help of python don't worry about it I'm going to show you how to do that so in order to like download the data set you need to go to the like almost bottom of the website and you will see the link is download from the map and then there they have the like tiles and based on if you click the any of these Styles you can actually download the data set for that tile for example if I want to download the data set from like Indonesia near Jakarta then I can click on this link and then simply click on this link it will uh actually start downloading and then your data will be downloaded so in order to demonstrate these things so I have like downloaded the three data set one is from like this Indonesia Jakarta which is 22 MB a size and sorry uh and the data set I downloaded from the uh somewhere uh near India so which is around 3 GB and and the data set is around 8.4 GB so I'm going to show you how to like deal with this big amount of data and then how to convert that to the uh reasonable size in the save file and geosition file all right as I said before I have downloaded the three files one is around 20.6 MB and the one is around 1GB and another one is really huge file it's around 8 GB and here I already extract this files so first of all I want to show you this one so which is the smaller size if I try to extract this it's a 56 MB and then uh this 56 MB data set I think we can open it in the Excel sheet and then try to visualize it slowly but for the other data set like the huge amount of data set of several GB data set you cannot open it in the CSV file and you cannot even see the file so here if I show you this latitude longitude means the building centroid latitude and longitude coordinate and then area in meters mean the building footprint area and then this is The Confident score that means if the accuracy is higher near to one that's actually the perfect matching of the building if the accuracy is low that means it might not be building and there is no guarantee and there is like geometry attribute here you this is the actual polygon size and I am not sure about this full Plus Code attribute but anyway so we have the data which have uh five attributes latitude longitude area confident geometry I think one two three four five six attributes so I'm going what I'm going to do is I'm going to convert the CSV files into the jio Json or zip file and then uh try to like try to load it in qgis or whatever um whatever visualization platform we have in order to like create the save file I'm going to use the python so I'll write first of all I'll create the uh create the file is building footprint Dot ipy and B if you are not familiar about this file format make sure you watch my some of the previous data science related content which are might be in the description below so make sure uh you are familiar with this python notebook and here I'm going to use the jio pandas uh in order to com convert this uh building footprint CSV file to the zip file so if you don't know make sure you watch my crash course on jio Panda jio pandas for absolutely beginner which is also in the description below and since I already installed my uh jio pandas package so first of all I want to import pandas aspd and then maybe jio pandas is GPD and then hit run so I have to select the python kernel uh sorry I selected the wrong Cardinal so it should be the dial one python environment and then jiral so this is my virtual environment if you are not familiar about this policy virtual environment and jio pandas make sure you watch my crash course on jio pandas so I hit prawn with the digital empowerment and now it will run and then I can use these packets all right now my uh jio pandas is loaded successfully now I can create the data frame so with the uh pandas first and then I can provide the data link over here so in order to like provide the data link so I can switch back to my file and then copy the file location file path and then write it here so and then if I say DF dot head then it will print out the five first five row of this CSV file all right now we need to convert this data frame to the jio data frame in order to convert that so I'm going to write like GDP and then gpd.go data frame in data frame will be DF which is the above one and then geometry will be the uh sorry this is not my geometry actually geometry will be the geometry column and my CRS will be the uh sorry CRS will be the 4326 which means uh wgs 1984 coordinate system and then if I write gdf dot head then everything will be same but this is actually the jio data frame with the coordinate system is wzs 1984. and then here if I write gdf dot to file and then I can like write it as a CSV I mean save file and now my location is inside download 687 buildings and 687 buildings dot SHP file oh hit run so okay so now it give me the wording but don't worry about it warning so what it will do is if the column name is longer than 10 character then it will truncate to 10 characters since that's the property of the zip file and it will take a minute and uh you will see the file over here in this period and in a minute I'm going to load that zip file in the qgis or whatever system we have all right now my output is generated so if I show you in a qgis so it will automatically uh align with my base map and since this is my first area from Indonesia so you can see my building footprint like this so this might not be like exactly fit with the osm building footprint I don't know which file have the error but it might be due to like little bit shift in the training imagery data set but that's how you convert it to the set file and I have also like tried to generate the zip file from another file which is 39f I think so it's around like not this one uh 30f so it's around like 2.5 GB so about like size of this size of data so you can easily generate this file or geosystem file using this jio pandas method uh but in order to like deal with this High I mean the 20 GB of CSV file you need to use the Dax so the jio I'm going in this video I'm going to show you how to use the jio pandas Dax function to like load the 20 GB of CSV file and then clip it into your area of Interest so yeah so in order to do that so you need to like install the jio pandas Dax package so here I have already installed with this command line so Honda install minus C conduct for task jio Panda so if you have the Anaconda environment uh since I have the gdal sdn condyle environment so I already installed this package so now I can import it and then use it as uh like uh is my uh model so okay let me zoom in a little bit so that you can see it properly so here and now I'm going to import the dash dot data frame sdd and uh also I'm going to import the jio pandas docs sorry it's uh Dax jio pandas okay and then not data it should be d a s k and then hit enter so now my that's model is already imported now I need to like create the darks data frame which is represented by DDF normally so I can read the CSV file uh from sorry um from this location but not from this location so I can copy the location path and then paste it over here okay so now if I say DDF dot hit so you will see the exactly similar output it's simply the Dax data frame what it will do is it will not consume the memory too much memory to render it and then uh it's helpful to like solve the memory issue and now I need to like convert it to the uh since it's the normal data frame so in order to convert that to the geodata frame I can write G DDF equal to das Dax geodata frame dot uh from Dax data frame and then uh my data frame name is DDF and of course we need geometric column and for the geometric column this time it will be DDF and then geometry and also I need to like map it dot uh map map partitions and then it should be mapped from GPD dot jio series all right this is my jio data frame Dax geodata frame and uh I also want to like reset it index g-e-t-e-f dot reset index and I don't need drop equal to true and if you write the G DDF dot hit it should be the same like similar things all right so now it's time to like create the area of Interest so in order to like create area of Interest so you can go to jio system.io website and here you can like create your own boundaries make sure that boundary file under the building data set link so here if I zoom in to this particular location so maybe you can choose any one of area but I'm think I think for this particular tax I'm going to like download the data set from some place of Kathmandu so simply make the rectangle or whatever shape you have or simply make make available that file in the uh zip file format so that we can easily import it in the geodata frame and then load it so here I can save it as the save file and then uh I can get start with that one but fun for me uh since I already have the data set over here so this polygon dot SSP so if I show you this polygon so it's the I think same is this polygon below so Zoom to here yeah so this is the right this this save from the Kathmandu City so now I'm going to clip my data set into this particular polygon shape so in order to do that so I can write the a y equal to GPD dot Freight file and then uh focus on to that particular zip file all right so here copy path and again I need to go here and then I can take a y dot head and then it will be simply the polygon right so if I type the aoi.crs so it will be the wgs 1984 which is ebst426 right so I can set my this jio data frame with the same coordinate system so here I can write gddf dot CRS so if I simply run it there is nothing but I can assign it with the aoi.crs so it will automatically take the aoi coordinate system is the gddf coordinate system so right now I have the 4326 coordinate system from my geodata frame from this 20 GB of CSP file so here now what I can do is I can write the clip and then I can like clip the docs jio pandas and then dot clip and then I can write which feature I want to click and then which area of Interest and if I say clip dot head then I'll see the exactly single output but this time it the buildings are only from this particular area of Interest all right so now it's time to like um export this clipped file is the uh file so in order to do that so I can write clip dot to file and then I can provide the location where I want to save this file and for now I can like provide this link is the link to save this data set and then I can write aoi Bui l d i n g s dot SSP right sorry it should be SSP okay so if I do so it will automatically like clip my save file but yeah before doing that so I need to like compute this clipped file because uh it's the Dax geodata frame so it cannot be like exported as the simply by the to file method so if I run this I will get the error here data frame object has no attribute to file right so in order to like make it workable so you have to like equal to clip dot compute and then uh you need to provide a clip.2 file and then the file location so if I run it it will run but since this is the advantage of data and it has lots of foundation actually I already run this function it took me around 15 minutes to run this function but it's really depend on your data size and your area of Interest so since I already run so I'm going to cancel this and then show you the result because I already run it in the different environment and then the code is exactly the same okay here so if I show you the result so it's in the download folder and then it's the aoi footprint and then if I load it here so this is my area of interest and these are the like buildings so yeah that's that's how we like download the exact required footprint uh there is like no limitation we can load whatever file of data we have let's say for me it's a the actual size of data Watts around 20 GB so yeah that's the really good way to like load the data so this is the 2mb one and then actually uh this one is the 20.2 GBC file a CSV file right so yeah in this way you can simply like download the building footprint from almost all around the world only on the southern part from this Google data set I hope you enjoy this video If you enjoyed it please hit the like button and subscribe to my channel thank you for watching

Channel: GeoDev

Views: 8,598

Keywords: tekson, gis, web gis, web mapping, leaflet, geodjango, note, bookmark, latitude, longitude, django, osm, models, template, maps, map, arcgis pro, arcgis, deep learning, neural network, unet, image segmentation, object detection

Id: R0ElIJS4C70

Length: 22min 4sec (1324 seconds)

Published: Sat Jul 15 2023