Can I teach my computer to drive? (coding vlog)

if you're anything like me you may have watched some videos on YouTube about coding where someone has created a program that teaches their computer to play snake or play Tetris or generate YouTube video scripts I've watched a lot of these and they've always focused on the end result it makes the end result seem inevitable like here's the thing which I did and here are all the steps I did perfectly in order to make it I don't think that's how most people program so in this video I want to try something different I want to try and do a coding vlog I wanna make this video we're going to take you in this camera around with me as I try and program as a particular task I've been challenged by Robo race this is a sponsored video sponsored video everyone this was sponsored by Robo race I've been challenged by them to teach my computer how to drive now I'm not actually gonna put my computer behind the wheel of a car it can barely run Skyrim as it is so instead I'm gonna be teaching it how to drive on a 2d racing circuit a bit like a kind of flash mini game and this blog is gonna focus on the journey of how I did that because well if you've watched my videos before you may be aware of I'm not the world's greatest programmer why are you I'm filming this at the start this is probably gonna take two maybe three weeks so you'll see me change clothes a couple of times maybe maybe you'll see me get a haircut oh and we'll find out if one well I can actually do the programming because we felt convinced that I can and two if this format works if the coding vlog works no one's ever tried this as far as I could tell so I've identified some key things which we need to learn how to do in order to make this project work the first of them is how to make a continually refreshing graphical representation of what's going on inside the code in other words only to learn how to make a game window something that refreshes a certain number of times a second is populated by objects and allows me to see what's going on secondly I need to learn how to make the object in that representation interact with the environment and specifically if objects are cars I need them to know the distance between them and the wall of the racing track in front of them to the side of them you know in a couple of directions that information then goes into how those objects behave in other words how they drive and you know we're gonna have the typical controls for a flash game for Driving's of W a s and D something like that then after comes the actual machine learning where I'm gonna have to learn how to do some genetic programming where we generate a huge number of cars at the start of a race given the ones which perform best we breed them and produce a new generation and we do that several times until we hopefully end up with some cars that actually know how to drive then we'll need to test if the cars have learned to drive or if they've learned to drive on the specific test track that I've been working on by testing them out on other tracks I have no doubt that this plan will not survive contact with the data because that's what my plans tend to do but that I think is what I need to do that's the broad outline oh and I'm gonna be working in Python because life's too short and I already know my way around the language apologies if this isn't the best language to use but it's the one I'm gonna stick with okay so after that initial foray and about a day or so of on and off programming here's where I'm at so this is a game window which I've coded up in PI game I started out in arcade but that was just to simple module I had to switch over to PI game because I had so much more functionality those who don't know what programming modules are basically it's a series of instructions it's like a toolkit the toolkit I started out with was great because it was easy to understand but it wasn't complicated enough to even replicate something like Super Mario Brothers so I switched over to PI game because you could do so much more with it where at the moment is that I have a particle which represents the car in the game and I could actually import any number of them that completely scales to any number of cars I've won and that car has a position it has a velocity it has an angle for this velocity of that in the space and it has other characteristics you can give it like acceleration for example what you've seen on the screen here is its distance to the black walls the white here represents the track and the black represents the track walls it was just something that I drew freehand and in Photoshop and it's a bitmap the numbers on the top right corresponds to the distance from the center of the car to the wall looking at the left hand field of view the center of its field of view and the right of its field of view and the field of view is a custom number that can be dialed in custom for each car what you may notice is that it doesn't bounce off of the walls of the track it just bounces off the widt edge of the window I need to implement that I was I was very pleased to get it bouncing off of the walls that's a job that I'm gonna come to you tomorrow and when that's done that's all everything that I need to do in terms of setting up the game done and we can move on to the actual machine learning that's something that I'm going to need to do tomorrow though because because today is my anniversary it's our fourth anniversary I actually did on the channel for a while oh yeah as pixel girl and we are currently in the Gherkin in London because there is a restaurant on the very top [Music] unfortunately the way that's not amazing for the best view of all is right here okay so much has happened since yesterday and I do have ever been so excited for a coding project in my entire life let me take you through what I've done this is the latest version of the game running so what you see is that the cars are now bouncing off of the walls I implemented that it was actually surprisingly easy to do I basically just copied the interactions between pixels when they hit balls but just with checking if the pixel that they're running into is black or white I've also updated the race so you can see actually how far it is between that particle and the wall in its field of view well you might also notice this of the field of view how narrow the cone is is specific to each car right so it varies from quake to Stevie Wonder that's a random number that gets assigned within the class of the particle when you call it now the cool thing about this though the really cool thing is that these cars are driving based on the information that they're getting from the environment so depending on how far they are in each of those directions and also how fast they're going at the time they're choosing whether or not to press WASD in other words whether they want to accelerate brake turn left or turn right the way that I've accomplished this is a simple bit of linear algebra you have a vector representing your distance in each of the direction plus your speed that's like a column vector and then you have a matrix that you're multiplying that by when you multiply that vector by the matrix you're going to get another vector which gives you a series of numbers if those numbers are beyond a certain threshold which I've just said to be zero completely arbitrarily that corresponds to whether they press a button or not on that particular tick that like time step of moving forward in the game whether at that moment they are pressing a key down or not those key presses then go into this line of code where it shows you whether or not the effect of acceleration or brake or turn left or turn right is being implemented so referring back to the list that I do at the start of the video all the things I need to do we now have the game engine working and we have the particles interacting with their environment like this is one way smoother than I thought it was now Herber comes the technically difficult bit now we need to do the genetic programming we need to take the cars which do best on this track and then make more them the question is though how do you assess how well a car is doing on this track what we need is a school function so they paint one or on my lunch breaks by the way it helps me relax a score function is a generic way of keeping track of how well a particular version of the algorithm is working so humans have a pretty good idea of what a school function for race driving looks like it means you go around the track fast that's a problem though for the computer when it doesn't know what the track is or for that matter which is the right way to go around the track to it it's just a bunch of pixels and in many ways a lot of successfully using machine learning is choosing the right school function to try and maximize or minimize so I tried loads of different versions including the angle that the car's gone around the track so if it starts off at zero degrees it completes the track and has gone through 360 degrees but that didn't quite work god this is multitasking this apps the finest I'm painting terrain by the way because I got so sick to death of the black and slightly off black of my highfalutin Nostromo tyranids so what I decided to use was the way that we assess it in video games and in real life races I suppose checkpoints it's you can reach certain checkpoints around the track fastest and one of those checkpoints is the finish line the key advantage of this technique is that it it has a direction built into it the cars can only go around the track one way they have to clear the checkpoints in order so yes in theory you could go all the way around the track and hit the first checkpoint but you're gonna do so much slower than a car that goes the right way around the track so if you build in number of checkpoints hit and the amount of time taken to hit those checkpoints then you're gonna have some winning combination the exact details you can work out after the fact with some experimentation so here are the checkpoints on the screen you'll see these red dots which corresponds to the individual checkpoints and you also the cars and our different covers I thought would add a color to each car because it allow us to see which cars are successful in going forwards to the next generation the way them the way how the way that the code works to check if it reaches a checkpoint is that there's a function now which is update score that's associated with each particle and what it will do is check if the next checkpoint and it starts out as one if the next checkpoint is within 30 pixels of the position of the car if that's the case then it counts as having gotten that checkpoint and the next checkpoint is updated and it's now looking for checkpoint two and it means that it's got a score of one if it's reached the first checkpoint that gets divided by the time taken now comes the fun part because now I need to write a function which is going to first of all work out which of these particles does the best which are the which of the cars is best performing over say 30 seconds blanks all of their scores and it'll take the top 10% and that will be our parents for the next generation much much sooner than expected I think we've got it working so I wrote a function which allows the particles to breathe basically it will for each of their characteristics for the elements of the array there's a random chance of inheriting each of those from his parents plus a little extra mutation chance on top so there's like a small variability on top of it but it's like much smaller than what it inherits from its parents and the color is a mix of the two and the field of view is also a random chance between the two that's working I think I'm getting good at programming guys I think it's actually happening if you moments later that me that has that made it work I know hang on hang on being stupid yeah at least five second generations let's just see if this works oh my god programming is amazing oh how sick is that so now what I want to do is based on that initial experiment the cars weren't really learning to drive throw basically did bundling around the track so I'm just gonna make them lose a lot more momentum when they crash so that's basically changing the elasticity of the I'm using lose more kinetic energy when they crash I never thought I'd be able to do this this is so cool what's interesting is after watching this for a while all of the cars have become Brown that's probably just because if you take an average of lots of colors they're eventually just gonna become Brown rather than you know it was a brown car that did particularly well this is fascinating to watch I know that's interesting in that generation there was a car that did quite well that was blue that was a wild car so it was a new one and in fact the new cars on this generation are doing well are slightly like brownie blue the ones that are being successful other ones that go forward and turn right because the first checkpoint is a right turn that's super interesting though that some of them like a significant fraction are still going on when we're on the track I need to tweak this okay right bit later made some changes so what I realized was there was a left over in my code for when I initially sort of set up the individual drivers initially I was just testing them in random angles and random locations so the starting angle was still random which is why so many drivers were going the wrong way around they weren't all pointing in the same direction now I'm on generation three of the new version they're almost all going the right way around the track and what's interesting actually is that the field view that they need has narrowed massively there's still some which have gotten a wide field of view never ever thought that I would sort of get so into watching tiny pixels get on screen for yeah they've all mutated into having like a really narrow field of view they only see what's in front of them they didn't see what's on the side of them ah there's a purple guy that's just made it in though the big field of view is doing quite well oh I say that it's just compact quick I also made a change to how the breeding works basically um the way that it worked before was if you were in the cutoff of the top 10% or whatever it was you stood an equal chance of having offspring you produce the same number of offspring as everybody else but I wanted to make it such that the higher your school was within that cutoff the more children you were gonna have so rewards performance within that bracket so now the number of children that you have is inversely proportional to your rank I think it's happening I tweaked how it learned I tweet the variant the weights of the parameters like I realized that the scale of for example the speed versus the distance was gonna be completely different I've noticed that they weren't turning which kind of told me that the distance to the walls wasn't important to the drivers it was just the speed but with fog this has just started the fifth generation is like a bunch of lilacs there's a little bit green showing through but look they're turning they do move in herds now what's interesting is they've now reached the one can turn they have to turn left just pretty much everything else I just had to turn right and so what they're doing is they're all going into that turning and going to move straight into the wall they have to learn to turn left as well like there has to be the possibility in their programming I suppose they do turn left actually there's like a kink before oh I also added a lot more checkpoints I realized that there just weren't enough checkpoints and then I how important reaching each one was by the time taken to reach it yeah oh my God look at them actually learn something I built in code works there's a bunch of front runners okay I know good good into the wall like a bundle Elevens all it takes is like one driver to work out that they can turn left oh oh one of them learns how to look what I made it and what was it it was particle what 11:04 whole new sport so that's interesting the previous generation there was a car that went almost the entire way around but this time around didn't work that was just a chance mutation you know like having a really gifted parent doesn't mean you're gonna have a really gifted child like my mum amazingly musically talented I'm not so much although I got look he's gonna give a that's to that we're going all the way around please tell me that it's not gonna crash because it goes all the way around oh my god watch because there's two of them that means that there will be two over there will be an offspring of those two so how many of each generation do we keep we're keeping ten the first ten of each generation as definitely tenders made it all the way around now so now it is this is when the timing aspect comes into it it's like all of you have made it around no it's which one of you is fastest this is so cool okay I think we've taught a computer how to drive around a track around one track I am dead chuffed with that that is so cool okay next though that's that's I think three of the things which I said I need you to do we've got the game working we've got the objects interacting with the game world and we've got them going around a track now we need to test if the cars can go around another track or if they've just learned this one right so it's much later in the month I've spent the last week or two just tweaking the models here and there and training staff and experimenting actually we're not to see pizza girl's family and snuck away for a couple of hours in a coffee shop rated most of this work I've trained a final generation of drivers based on all this experimentation and I've drawn a new map in Photoshop which means that we have our final test we're gonna find out oh so now we're gonna find out if the cars have overfit to the training days or if they actually know how to drive so I've pointed the scripts to the new map and let's see what happens ah a bunch of cars are stuck in the walls I think the the star of the race may have been a slightly different location let me just let me just fix that you may have noticed by the way that I've changed the look of the track so there is now an indication of what each car is doing so whether it's pressing WASD or not so it's interesting to watch that actually because it doesn't behave quite as a human would okay I should have moved the starting position the cars let's see how this works right so we've got all the cars so the cars have learned to go around the track not necessarily in the right direction some of them have got confused and gone the wrong direction but they are driving they oh I don't think any of them have actually got all the way around yeah okay no at least one has gone all the way around so that's interesting this was a wildly different track ahead long straights I had way tighter turns than they had to deal with in the training track but they are going around the right way and for the most part they're actually picking a direction and sticking to it what you may notice by the way is that the cars always are always practicing w which is what humans do when they're playing on these games but a lot of the time instead of pressing a or D they actually just tend to press both at the same time and then they'll lift off whichever key they is in the direction they don't want to go in which to a human sounds insane but to a computer kind of makes sense so it makes no difference to them but I think we can count that as a success I mean yes the cars weren't necessarily going the right way around the track well I'm quite sure why that didn't happen but they were avoiding the walls they were taking good racing lines and taking corners nice and tight I mean I see this as an absolute wind now what done here is actually pretty simple it's taking in a couple of input parameters applying some linear algebra and then letting a car decide if it wants to press W a s or D real driving is a lot more complicated than that and there are cars which can autonomously go around racetracks and I went to go and see them in action rubber race for testing out a variety of their cars at the shells Li Walsh Hill Climb these cars are just like my virtual ones and that they take in the distance from the car to the track wall and make a decision about where they're going to drive but they're much more complex they also use lidar these GPS they take into account the friction coefficient between their tires and the road they also look absolutely stunning if you'd like to learn more about the science behind these cars and how they drive themselves check out a link in the description to videos about the technology and also about season alpha the first round of competitions between these cars taking place around the world you can also check out their Instagram and there'll be a link to my github where you can take a look at the code I use in this video thank you robo race for sponsoring this series of videos and of course thank you for watching I'll see you in the next one

Channel: Simon Clark

Views: 17,551

Rating: 4.9720101 out of 5

Keywords: programming, vlog, vlogging, coding, how to, computer science, programmer, roborace, devbot, race car self-driving, ai, artificial intelligence, python, how to learn programming, code, simon clark, dr simon clark, drsimonclark, simonoxfphys, machine learning, autonomous driving, coding for beginners, python bot, coding projects, roborce, tech, how to code, coding vlog, how to drive, self-driving cars, genetic programming, python for beginners, pygame, pixel girl

Id: Qv7dhms5dDQ

Length: 20min 16sec (1216 seconds)

Published: Thu Jun 27 2019