Python Geocoding - Address to Latitude & Longitude

welcome everybody uh we have a problem uh we have a beautiful list of addresses of mcdonald's and burger king restaurants in berlin london and paris and we want to put them as markers on a map like this but to be able to put them on a map like this we need latitude and longitude values which we don't have we only have the address values and that's exactly what today's video is about and we'll figure out together today how we can transform and address to let it to longitude values um another term for it is scale coding and we'll use python requests and pandas this helper library and mapquest api which offers a geocoding api to actually transform these addresses to lady to longitude values again you can watch this video standalone if you're interested in how to transform addresses into latitude longitude values or as part of my series where we scraped these addresses from the web in the video before and we'll have a look on how we get the ready to longitude values of these address today and then in the next video put them as markers on the map have fun before we start coding let's have a look on mapquest api which is the api we'll be using map so simply type and map quest api and it's it should be the first hit on google you can directly get your free api key here and register and you see they have a free tier here with 15 000 transactions per month for exactly zero dollars which is really nice and they have a good documentation over here with a lot of apis one of them is the geocoding api which we'll use and you see here whenever you provide an address to the api it returns latitude and longitude values like this so let's start coding here is our starting point and the list of addresses of mcdonald's and burger king restaurants in a csv file and i provide this file in the video description that you can directly start here if you haven't watched the video up front first let's create a new file getgeo.p second let's import the libraries we need we need the pandas library for data analytics data display data wrangling and so on we need requests to actually do the requests to the api and we need json to be able to handle the result from the api accordingly great once we have done the imports we will structure the code in three steps first step would be reading in the restaurant csv second step would be do the actual api calls and attach latitude longitude values to the csv and last step is again saving this file to a csv back all right let's start at the beginning first we need to read in our restaurant csv into a pandas data frame so let's do this so once we've read in the file let's quickly print it to console to get a feeling about the data so here you can see all our restaurants from zero mcdonald's in berlin germany until 150 earth 45 uh burger king restaurant in london uh to not always show all the data we can make use of pandas head function that's executed again then we only get the for our first five records here all right reading the data is quite easy but now we can directly come to the complex part doing the api calls getting the lady to hit longitude values and then in the end saves the file back let's have a look on how the api works so fortunately the documentation is quite good so when you look here for the geocoding api you can click on the get you get the url you need to call and let's directly copy it and we already see the parameters we need to hand over in the request which is a key and you'll only get this when when you register here with mapquest api and the location which is actually the address itself so let's try to call it first thing is to do a get request to the url which we just copied second is we need to hand over certain parameters which we of course need to define first so we'll do the get request here to the ul and hand over these parameters which we define here the parameters itself contain your api key which you get once you register and the uh the location contains address itself so let's demonstrate the api and i'm entering my api key here and the security phonetics around you don't worry i will revoke the the api key right afterwards and let's hand over an example address here from our csv file perhaps right this one here which we use is address uh let's try to execute it but first let's print the response to the console all right let's start it invalid syntax line 13 seems we did an error here oh yes we forgot this character in here let's try again a voila we are getting a response 200 which means the call itself was successful um but let's perhaps understand what's in the response for this we slightly need to change the code so first of all let's comment out this here and let's get the response text and execute it one more time et voila we'll get the actual response content with a lot a lot a lot of data and somewhere within here you see we get the latitude and longitude values which we need so let's try to figure out how to get them out of all this content here in the next step at this point it's time to make use of our third library we imported json so like this is really a mess to read all this and and dig through it so let's actually um transform the response text and we're getting here into a json object which makes it more easy to read so like this we define a new variable data which contains yeah like it says our data and we'll transform the response text into a json object with adjacent loads let's simply print it out and execute it again and you see our data again but now in adjacent format we already recognize that the latitude longitude values here seems to be within the results array here and within here the location and then we get ready to longitude values here below on the led long so perhaps let's directly access the results as we don't need the status code copyright and and so on let's directly try to access the results here and execute it one more time so you see the stuff in the beginning um status code and so on we got rid of it we directly start here at the location and let it do longitude values come closer to what we need all right now to actually get the latitude and longitude values it becomes a little tricky so what we need to access is this long with which holds the latitude and longitude values for this actually we need to access the zeros object of this array the provided location within we need to access the locations object again with it the zeros entry this array here and then within this element the led long and from within the let long we need the led and long values um sounds complex but it looks quite easy right away let me quickly type it into the the code so what i did here is what i just told you and basically it's accessing this led and lng value and printing it to the console right afterwards let's run it you see we're getting our latitude and longitude values printed to the console now we are basically prepared to execute it for all our entries of the csv file from within our data frame so this would be the next step right now so we start with a loop to loop through each and every row of our data frame and let's delete the print command first of all here so this command will basically go through each and every line of the data frame and let's execute the same steps we just did for a random address for each of the rows basically but we don't want to have this hard-coded address anymore we basically want to construct the address based on each row individually so let's quickly do this and create a new variable containing the address so basically we are accessing the data frame here at each row and we vary the required columns with with the term street zip city and country here so let's start with the street and then do the same for the zip code the the country and the city all right let's see if this works as expected and let's quickly comment out the api calls to avoid unnecessary unnecessary calls here so let's first of all see if the loop is working and you see it is so for each of the row of the data frame we are getting the address great so then we can actually delete the comment again delete the printout here and actually hand over the api address into our location of the api call itself so this should give us the latitude and longitude values for all the addresses let's again try it and print it out and you see and this might now take time we get for each of the 150 45 entry the latitude and longitude values let's wait for it finally that's great so it works as expected but actually after closing the script those values would be gone so we need to save them to our data frame and then back to our csv let's do it so instead of printing them to the console we are storing them into the csv into the columns of lead and long here and we're using data frame comments for this so basically uh we just specify for each row in the column ready to longitude we are storing the latitude and longitude values and once we did this after the loop we can save back the data frame to csv and let's give it a new name and execute it let's see again it takes now a lot of time because it needs to compute all the requests for all the rows it might take a time but we only need to do it once because afterwards we save it in our file and good to go we are done now i'm curious let's have a look on the file it's created here restaurants geo.csv let's go in and you see we got the latitude and longitude values for all of our addresses great that was a target of today the next video will be putting these latitude longitude values on the map as markers so to get markers for each and every mcdonald's and burger king restaurant in berlin paris and london thank you for watching

Channel: Code & Dogs

Views: 6,437

Rating: 4.9720278 out of 5

Keywords: python, python3, leaflet, leaflet.js, folium, gps, longitude, latitude, programming, webdev, coding, development, 100daysofcode, webscraping, web-scraping, beautifulsoup, selenium, api, pandas, web, scraping, web scraping, web crawling, geocoding

Id: tKy-IHAxt4s

Length: 14min 31sec (871 seconds)

Published: Sat Oct 31 2020