AI Learns to Play GeoGuessr

geoguesser is a game that drops you in a random street view location and your job is to figure out where you are the closer your guest is the more points you get and there are five rounds per game at first it may seem pretty hard to pinpoint your location but by using signs flags and other contextual information it is possible to get a perfect score however some of the best geoguesser players like geowizard only need a few seconds in a single image to determine where they are with incredible accuracy and this made me wonder could an ai play geoguessr [Music] before we create an ai we have to think about how we'll design it in order to simplify the problem i chose to create an ai for the united states map on geoguesser rather than the entire world however this doesn't automatically mean we'll get a higher score since the scoring formula is adjusted and therefore will need to be more accurate additionally the average score is still under 10 000. one possible way to design this ai would be to give it a street view image and have it guess the latitude and longitude of the image another possibility is that we could give it an image and have it guess the state it belongs to after some experimenting with different designs the best solution that i found was this let me explain instead of having an ai guess the exact location of an image we'll have to predict the square it belongs to on this map each square is approximately 340 miles tall and 360 miles wide so being able to predict the correct square would result in a fairly accurate guess however the way ai's like this work is that they won't predict exactly one square it belongs to but give a confidence level to each square for example consider an ai that has been trained to recognize animals if given this image the ai won't simply say that it's a dog instead it will give its confidence for how sure it is of the type of animal for example it may be eighty percent confident it's a dog ten percent confident it's a wolf seven percent confident it's a fox and three percent confident it's a cat these confidence levels should add up to a hundred percent therefore when we give our ai a street view image it should give us something like this the darker the squire is the more confident the ai is that the image belongs to that square we can determine a final guess marked by the pink dot by taking a weighted average of the centers of each square using their corresponding confidence levels the higher the confidence level of a square the closer the final guess will be to the center of that square the basis of any good machine learning project is the training data the ai can only be as good as the data it uses to learn each dot on this map represents the location of a street view image that the ai will use to learn in this case there are about 4 000 street view images that the ai will train on and each square contains a similar amount of images i will now show a series of predictions made by the ai throughout its training in order to show its learning progress the image on the left is the image that the ai will be predicting the location of and the image on the right is the output of the ai the pink dot represents the ai's predicted location using the weighted average method i described earlier and the green diamond represents the actual location of the image at first the ai's guesses will look very random giving a low confidence to every box but it will continually learn by making adjustments to itself with each training example after repeatedly doing this many of times with different images the ai will start to recognize patterns and features in the images that will help it to determine the location of an image eventually it will get to a point where it stops making progress in its learning and that's when we will stop training the ai [Music] for anyone interested the way we are doing this is by using a convolutional neural network and a method called feature extraction i won't go into detail about how these work but i'll leave useful resources in the description where you can learn more about these if you wish now there's only one thing left to do which is to use the ai to play geoguesser the goal being to beat the average human which is a score of just under 10 000. i decided to play five games to get an idea of how the ai performed on average the first game i'll show was the best of those five but before i do if you want to play along take a moment to look at these five images and try to determine the location that these images were taken at then you can compare how you do to the ai this was the first location that geoguess had put us in as you can see the area is pretty flat and dry i decided that moving around a little bit was fair game to try and get the best picture possible once i found an image that looked good i took a screenshot of it i gave the image to the ai to predict its location and this is what it gave us as you can see there are a lot of darker boxes in the center of the united states showing that the ai is confident that it's somewhere in this area using the weighted average of these boxes and their corresponding confidence levels gives us a guess of a latitude of 39 degrees and a longitude of negative 100 degrees i then put these coordinates into google maps and try and get as close as i can to that location on the geoguesser map by slowly zooming in on both maps the final guest ended up being pretty close at just 153 miles away earning 3700 points for this round and as we can see we did get it in the correct state of kansas since that's what we're trying to beat is ten thousand we only need two thousand points per round to beat it so getting three thousand seven hundred and the first guess is a really good start the second location that geo gesture gives us is also a pretty dry area with some mountains off in the distance this is the image i used for this location and this is what the ai gave us as you can see there's a group of darker boxes around new mexico arizona utah area and this will give us a final guess around 43.7 degrees latitude and negative 111.8 degrees longitude our final guess for this round was a little bit further at 287 miles only getting us 2866 points but it is still more than the 2000 points per round that we need and as we can see where we were only one state over in new mexico when the actual location was in arizona this is the third location that geoguesser gave us as you can see it again appears to be in a pretty dry area with a few mountains in the background this was the image i used for this location and this is what the ai gave us as we can see the ai is very confident that it's in this northwest region of the united states there are two dark boxes here that the ai is confident that this location is in so the weighted average is pretty much in between those two boxes this guess was originally surprising to me because i would have thought it might have been more around the new mexico arizona utah area like the other guests because it sort of looks similar to that however our final guest shows that it actually was in this northwestern region of the united states we were over 400 miles away so we'll only get 2 000 points for that round but it will keep us on track to break 10 000. this round highlights one of the challenges that the ai has which is predicting locations close to the edges of the united states since we are using the weighted average method to determine our final guess the final guess often gets pushed more towards the center than stay out at the edges however we were still only one state away in idaho as it was actually in washington this is the fourth location that geoguessr gave us and it does look somewhat similar to previous rounds while also having some distinct features as well this is the image that i use for this area and this is what our a i predicted as we can see again there are two dark boxes that the ai is pretty confident in so our final guess will be in between those two boxes around utah [Music] the final guest for this round ended up being extremely accurate at just 84 miles away earning over 4 000 points for this round as we can see the correct location actually was in utah where we predicted with this guess we broke over 12 000 points in just four rounds this is the final location that geoguesser gave us and it looks very different than the other four locations that it previously gave us with this one being much more green with many more trees this is the image that i used from this area and this is what the ai gave us unlike some of the previous guesses the ai isn't super confident in any box but it's just sort of spread out around the southern area of the united states this final guess was pretty decent at only 181 miles away giving us a score of 3500 points this means that we finish the game at 16 000 points easily beating the average human of only 10 000 points we're also able to see that each guess was either in the correct state or just one state over showing that the ai was able to find the general location of each guess this makes sense for the method we used of just predicting the box it's in since we weren't trying to be exact but just trying to get the general location of the area overall this game showed the ai's ability to differentiate between different types of images and find the general location of images how did it do in the other four games the second best game was this 15 thousand point game the ai started off with an extremely good guess in new mexico just 33 miles away from the correct location [Music] its second guess was also pretty accurate at just 126 miles away putting it in arizona instead of utah the area around arizona new mexico utah and colorado is pretty distinct from the rest of the united states so the ai generally has pretty good accuracy in this area however its third guess was pretty far away when i put it in nebraska instead of texas however by looking at this landscape in texas it's not hard to understand why i might have thought it was in nebraska its forecast again was another very good guess when it actually was in nebraska just 121 miles away its fifth guess was a bit further off when it put it in iowa instead of arkansas however it was hard to find a good picture for this location the third best game was this 14 500 point game its first guess was pretty close when i put it in south dakota and said in nebraska its second guess in wisconsin was extremely close at just 48 miles away the ai put the third and fifth guesses around the center of the united states when they were actually both in arkansas this shows a pattern that the ai is often misclassifying arkansas as being in the middle of the united states it seems like the ai may be misinterpreting the flat fields found in arkansas to be similar to the fields found in nebraska and kansas for its fourth guest the ai put it all the way in idaho when it was actually more central in colorado the fourth best score came from this 14 000 point game in general for this game the ai's guesses were pretty average as they got the general location correct for most of them it did have one good guess in new mexico of only 146 miles away again showing that the distinct area of new mexico is easier for the ai to identify however it did have one slightly bad guess where it put in wyoming instead of utah this is another pattern that i noticed where the ai was often confusing the dry mountainous areas of wyoming and montana to utah the final game was this 13 700 point game as you can see there were three good guesses that were worth over three thousand points however there were two poor guesses one was where it guessed in the center of the united states when it was actually in texas and the other one was where guests in colorado when it was actually in iowa after five games the ai had an average score of just under 15 000 points per game this is noticeably better than the average human score of just under 10 000 points per game we were able to create an ai that plays geoguests are better than the average human by finding a design that worked and collecting good training data for this video i just focused on making an ai that identified the locations of images in the united states however this could be done for other countries continents and even the entire world so if you would like to see an ai try and outperform the average human on geoguesser for the entire world then leave a like and i might consider doing that and if you enjoyed this video then consider subscribing but if not and thanks for watching

Channel: adumb

Views: 614,888

Keywords: computer science, artificial intelligence, ai, machine learning, convolutional neural network, feature extraction, geoguessr

Id: mM_dC1HVAQ4

Length: 13min 55sec (835 seconds)

Published: Fri Jul 24 2020