AI Learns to Play Super Mario Bros!

hey everyone chris here welcome back to another episode of ai learns today i've got something that i'm pretty proud of an ai that learns to play different levels of super mario bros say i didn't just learn to complete a level either i learned to optimize its path to complete the level as quickly as possible and i learned some pretty cool techniques along the way these techniques include a pixel perfect wall jump and an enemy flight pull glitch i'll explain this in further detail when we get there i'm going to pause here for a moment before i showcase all of what mario was able to learn i want to talk about how we got here what you see on screen and more importantly what mario sees this is the real super mario bros game running on an nes emulator rather than using raw pixels i decided to split them into 16 by 16 pixel sections which form a block each block can be seen here for a total of 13 rows and 16 column although i show 15 rows here the top two rows won't contain anything since they're used to display the on-screen information like score points collected world and time left i chose not to use all the blocks as inputs for mario since i found myself really only focusing on a certain region when i played that region is outlined here in pink and is what i've chosen to feed as input to the ai the ai is also given what row of the block it is in and altogether there are 80 inputs for this ai 70 from the blocks that are in pink and 10 from a one hot encoded variable depicting the row these inputs are then given a specific value based off the block type and fed through a neural network at the end any number of buttons can be pressed at first these button presses are going to be completely random since the ai has no idea what mario should be doing or even which way it should be going i'm choosing not to display the connection from each input because otherwise there'd be way too many lines on screen since each input is connected to each of the nodes here instead i just show pink lines to represent many connections the blue lines below represent a positive weighted connection while the red lines indicate a negative weight each individual is given a score based off this fitness function don't worry if you don't understand it i created it mainly to reward early exploration maximize distance through the level and minimize the amount of time taken at the end each individual is given a probability to be selected for reproduction based off their fitness from here a population of individuals played different levels of super mario bros over a period of three weeks i started with a larger population to promote exploration and slowly decrease the population size over time to promote exploitation at first the ai completes levels in a pretty inefficient way since it's only concerned with finishing the level after a while it will need to improve its time in order to improve its fitness i also let a small portion of the population start other levels to help with initial learning this allows for techniques learned in one level to be carried over [Music] here is a more advanced generation on world 2-1 again notice there is less hesitation with jumping over pipes and a lot more confidence with going through the level in general also pay close attention to how the ai finishes the level as this plays an important role on other ais in different levels in case you didn't quite catch that let's rewind the ai was able to figure out how to do a wall jump a wall jump is a technique i've seen some speedrunners do but i've never been able to pull off myself in order to do a wall jump you need a few things one you need some speed toward the wall you also need to hit a block at a pixel boundary which happens once every 16 pixels finally you need to go into the wall for at least one pixel i'll play it in slow motion to really help show the moment the ai lands on the pixel boundary and performs the wall jump because the ai now knows how to wall jump that technique can be carried over to other levels there are some exceptions due to limitations with generalizing across blocks i won't get into that here but i'll link my blog post in the description below in case you want a more in-depth technical read [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] [Music] so [Music] [Music] [Music] [Music] [Music] [Music] you

Channel: Chrispresso

Views: 178,661

Rating: 4.9285841 out of 5

Keywords: machine learning, super mario bros, neural network, genetic algorithm, python, AI, artificial intelligence, generalized intelligence, AI learns to play mario, AI learns to play super mario bros, NES, AI plays super mario bros, Mario AI, Super Mario Bros AI

Id: CI3FRsSAa_U

Length: 9min 4sec (544 seconds)

Published: Wed Aug 19 2020