Understanding AI from Scratch – Neural Networks Course

learn the fundamentals of neuron networks from one of the most popular machine learning instructors Dr Ru in this unique handson course you'll learn Core Concepts and manually tweak Network parameters to create a self-driving car while this is phase three of his machine learning series it's fine to start here so let's get started I've been wanting to make this course for six years I got the idea when Grant from three blue and brown posted his video on neural networks the moment he said this one thought experiment that is at once fun and kind of horrifying is to imagine sitting down and setting all of these weights and biases by hand so welcome to my fun and horrifying new course where we'll play with neural networks inside this special playground I created the goal is to teach the car how to drive and we'll do that by manually changing the network parameters using the mouse wheel we'll start with a simple Network that just stops the car from going off Road and gradually increases complexity to teach the car different traffic rules as well there I say you don't need any prerequisites to start this course I think the playground and the lessons do a good job explaining the math I use human language and only introduce fancy terms so you know they exist in case you find them elsewhere but if I'm wrong and you still get confused just ask in the comments or on Discord Now by changing these parameters manually will understand exactly what the neural network does this is really different because courses normally teach neural networks in a machine learning context where they're automatically generated from data that makes them work really well but it becomes impossible to tell what they do exactly it's why we call them blackboxes but I want you to understand what the neural network can do before I teach those complex algorithms for generating them and the best way to understand I think is by playing with them like this now this course is good for those starting with AI but also those with some experience who want to understand things better I've been working with machine learning for over 10 years now and some things still surprise me I think it's because training complex models is so easy nowadays just write few lines of code and you're done makes us overconfident thinking we know more than we do I've seen many solutions that fall short are overly complex and use unnecessary resources if you took my machine learning course you know what I'm talking about there we use the huge neural network to get the best results but one with the fraction of the size could have good accuracy as well and we could explain what that does so I think there's real value in revisiting Basics from time to time throughout the course I'll give you homework assignments to improve your logic and deepen your understanding I'll also give you a final challenge toach teach the car how to race instead of following the rules nicely I'll host a live stream event where I race against your AI cars and there will be prizes I just haven't figured out what those are yet but stay tuned and while you're designing your race car I'll continue to teach how to code some things as well for that part you do need to know some math and JavaScript this playlist can help with that and we will continue the self-driving car project so it's good to be somewhat familiar with it but will mostly just add new functionality into it so if you feel confident in your skills take the last version from GitHub and try to follow along what I'll teach then is how to implement drra shortest path algorithm so the car knows how to reach its destination I'll also teach how to make the game mechanics we'll control the main car but the others will be AI I'll show you how to monitor their progress and make the scoreboard now this top view is not great when racing I think I'm really used to seeing things from the car's perspective so I'll teach you how to code the camera sensor where we render what the car sees this might be useful someday for object recognition but for now I just like it and what I like even more is this view from behind the car I'm going to teach you how to do that as well and I'm not done controlling the car with the keyboard is not ideal I'll teach you how to implement analog steering Twice first by turning this into a mobile app and using the device orientation sensor to turn I think it's better than using the keyboard but I have an even better one using the camera some basic image processing and these blue wristbands we're basically going to become Iron [Music] Man oh yeah and the sound it's procedurally generated from scratch all of this is just plain JavaScript no libraries like everything else on the channel so you can learn all inner workings of a complex system like this and all of this is AI the new camera sensor the path finding the fancy controls using the smartphone or this thing image processing augmented reality they all make the system more intelligent that's what I want you to get from this course AI is often a combination of things not just neural networks and artificial intelligence is not the same as machine learning many people confuse the too there's no machine learning in this course which reminds me someday I'll teach you how to add machine learning into the system as well but for that we need data so if you want to help go to this link and raise try to beat my time from there if you make an account and do a good job your name will appear there and others will see you racing next to them that's actually me racing I'm recording every move we do so I can replay it like that but don't worry about making mistakes for machine learning we need to teach the car what not to do as well so mistakes are more than welcome excited great now get ready to put your neurons into overdrive this is the playground and look at the car it's doing something it's applying the so-called right hand rule for solving mazes so if you're inside a maze put your right hand on the wall and you'll eventually get out of it most of them anyway there are some mazes where it doesn't work but we'll get to that later now just look at this the car is exploring it's going everywhere and it's actually great it's not easy to do this you can press this manual override button on the right and now the car is controlled by the keys on the keyboard and just try to use the arrow keys and make the car do the same things without crashing it's really difficult like a very tedious thing that you have to do and um if it happens so that that you crash then you can always press this other button here and the simulation will restart so for me this is quite challenging to get it to go even half as good as what this neural network can do so turning off this manual override you can see it's going much more confidently than me and knows how to turn just right now let me put back on this manual override and teach you about the sensors next so you can see these two lines coming out of the car these are what I call sensors and you can see what happens to this front sensor here when the car gets close to the Border it lights up it's a proximity sensor so it knows when something is is nearby and how close that thing is now the sensor also has a range so this tip here is as far as it's going to read anything and the values in that case are going to be zero so now this front sensor doesn't read anything and you can see these sensors as input values here to the network and if you hover the value is written there so if I'm going to go now up here you can see how that value is increasing from zero to one essentially or 100% but not exactly 100% because if I'm going to zoom here a little bit you'll see that the sensor starts in the middle of the car so it's not going to go all the way to 100% because the car is going to crash before that happens so let's see where it crashed it was at 93% or 0.93 let's restart this simulation and the other sensor is here on the right and it's the second input to The Matrix and this one might actually go to 93 um percent maybe without crashing because the car is not as wide as it is long but I think soon after this 0 93 now um oh 0.94 okay so it can go a little bit closer than the other one just because of how the car looks like let's restart this simulation and even without hovering you can see the intensity of these inputs like this one is is colored yellow this one is grayed out it indicates that this one is reading a larger value and see how this one is now lighting up as well the last input here is a different kind of sensor it's the speed of the car in phase one of the self-driving car course we only had these kind of sensors implemented via Ray casting but there are other things that the car can know like it knows its speed so this value here is going to have negative values when going backwards and positive values when going forward so it's a little bit different than the other two sensors it can also have negative values the other ones couldn't have those and look at the color coding when I'm going backwards that meter there is blue and when I going forward it's yellow so this color coding yellow for positive values and blue for negative values is going to appear everywhere so you can see here negative weights that are blue and this is a positive weight and here this spinning bias value here is a positive value this one here you know it's a negative value because it's blue and these up here are the outputs so the car can go forward left right and in reverse and it only goes like that if these neurons light up so these two are lighting up now it means that the car would like to go forward and to the left but because of this manual override is is on it doesn't do that I don't let it do that but it wants to do that so let's see what happens if I move the car forward and that right sensor here doesn't read anything anymore what happens to the outputs there you can see it changed a little bit it tries to go to the right now because it wants to look for that border for that you know that right hand rule there it looks for the border so it's going to turn right let's see it do that if I turn off this manual override the car continues like that and you can actually play with these values here on the right by using the mouse wheel so for example you could change the value of this weight maybe make it negative weight instead and you can see that this is a very very delicate system here so the car doesn't work now as as before anymore but you could do that the playground lets you play with these values any way you want like maybe let's make this bias also uh negative it actually did something it started working again but then it crashed so again really sensitive and I'm not going to explain to you what this network is doing right now you will eventually understand what everything here is doing and be able to implement this Logic for the right hand rule and even the more complicated logic like with the stop signs and traffic lights and things like that you'll see but we have to take things slow and step by step so go to the top of the URL here where it says s is equal to default and type S is equal to [Music] FWD and this here is a much simpler scenario from before first the car has only one sensor this time this front-facing sensor here but it works like before so if I turn on this manual override and move the car forward you can see that it's lighting up and it also shows here as the input to the neural network so this part is exactly the same as before but only one input and the network is really simple it's just one neuron here making the decisions and the only decision it can make is to go forward we can control the car to do more than that like we have access to all its capabilities here if I'm going to use the arrow keys but this neural network can't decide all that it only has the potential to go forward or not go forward and at the moment it's doing nothing like if I'm going to turn off manual override the car just stands still this neuron is not lighting up and the reason for that is how this thing works so this input value coming from the sensor here we can call this x it's a variable this value is multiplied by the weight in this case it's zero and then if this value x times this weight is greater than the bias here which is also zero then this neuron lights up but it can't be greater than the bias because this is zero here so any value for x multiplied by 0 it's going to be zero and 0 is not greater than zero so with this neural network here it can never light up no matter what the input says we could make it light up by lowering the bias here for example let's make it minus 0.10 it crashed there but basically any value here that is negative is going to work and zero or any positive value for the bias is not going to turn this network on ever because the result of x * W here is always zero because W is zero so let's just leave this bias to something like minus 0.2 and restart the simulation and see the car going forward like that it never stops now okay because again no matter what the value for x here it's never going to be anything else than zero in one multiplied by zero so this also needs to be something for it to work and let's turn on manual override and restart and go somewhere up here with the car for example this location and let's try to modify this weight so that this neuron turns off because we want the car to stop going forward when it sees something so that it doesn't crash you can try playing with this weight and having large values like this and you'll see that nothing happens this output neuron still says go forward but if you go lower values negative values minus 0.2 0.3 4 5 6 minus 0.6 in this case then this neuron turned off and if I'm going to move the car a little bit here you can see that there is this kind of Sweet Spot there where this neuron turns on and off let's restart this simulation and turn off the manual override and the car still hits that place so we need a different value for w let's put back manual override restart and let's try to get maybe somewhere here here so that when we are further away it turns off so I'm going to modify now this weight and go lower looks like minus 0.8 is a value but we can go actually even lower than this so maybe - 0.9 minus one and let's see what the result is here you can see now this sweet spot is much lower here let's try to see what happens now so restart this simulation and turn off the manual override and it stopped it didn't crash anymore it stopped exactly at this dotted line here nice so let's turn back on the manual override restart and see where is the spot where it changes so here this is the moment where it changes from on to off and if you look at the input value this 0 19 and play a little bit with the car at that point you will see that it changes at 20 basically this 20 is the same value as here but minus 20 in this case and it's clear why that is if you look at this weight value because it's just -1 so whatever X is multiplied by -1 means - x has to be greater than- 0.2 or X less than 0.2 because the sign switches if you multiply by a negative value so that's why these values are matching right now because this weight weight is one minus one but basically it doesn't have any scaling effect other than flipping the sign so having weights of one or minus one are making the math easier in a way but basically what this means is that if x is going to be less than 20% Then This neuron stays on otherwise the neuron stays off now there is a way to visualize things so that we don't have to do this kind of mental math all the time I'll show you you just go here in the title and say s is equal to FWD uncore D and it's the same thing as before if you will turn off the manual override you will see that the car is going to stop at this dotted line but now this thing appears here at the bottom this axis here is the same thing that we are visualizing here and here so 0.79 or this 79% it means 79% of this Arrow that's where this yellow dot is present so this arrow is for the input value changing back to manual override you can see that I'm moving this point up and down because that's how the relation ship with the sensor is changing this point will always be in the positive side of this axis because if we go far away like this the smallest value that the sensor can read is zero so it aligns like that and there is also this other line here and this lighter region this is controlled by the weight and the bias the bias if you're going to lower it make it even smaller it's going to to move that region up and increasing it is going to move that region down now we had it at minus 0.2 let's keep this value and let's see what happens when we modify the weight increasing it does this and decreasing it does that it changes the slope of that line so this describes a line and the lighter region that I have here I made it so that when this yellow dot is inside of this lighter region it means that this neuron turns on so now if I'm going to move the car up a bit you can see that that place where the lighter region intersects the axis is the same spot where the neuron turns on and off so you can tell where that point is just visually this is 20% of the whole distance here without doing any mental calculations and this network that we have here that solves our problem is not the only one that solves our problem let me show you you can put here a value of minus 01 and then this value let's change it to minus 0.5 and you can see that this point lines up again the difference now is that this slope right here decreases at a slower rate at half the rate that the value on X is and this network works exactly as before so if I restart this and remove the manual override you can see that the car is stopping exactly at the same spot so there are two Sol solutions for the problem now homework task for you think about it how many solutions are there really let me know in the comments let's do the math as well to confirm that this point here is at 20% so x * - 0.5 has to be equal to - 0.1 so dividing by - 0.5 means that X is equal to 1 / 5 basically and that is 20% and you could write this in code you could make a simple if statement and say if -0.5 x if the variable X is the input is greater than minus 0.1 then accelerate otherwise do nothing so that piece of code is the same thing as what this neural network is doing it's important to understand that the neural netor works are capable of doing these kind of if statements now there's one more thing I want to show you here if you click on this diagram is going to show you a simplified view where it only uses one dimension so these two Dimensions here are important if you want to display this slope as well so the simplified mode loses some information we don't know what that slope is but for understanding what happens here and seeing that the neuron is on when the sensor is Reading in this section here this is enough and reducing from two Dimensions to one dimension like this will mean that in the future we'll be able to show higher Dimensions easier so losing some information but it will let us do some nice visualiz ations later on play around with these values again and see how the visualization changes there maybe we use the minus 0.2 value and the minus one weight the one with the simple math for previously and you see it still looks the same as before but the slope now decreases much faster at the same rate as X and what I mean by that is if I'm going to go here and put this bias to zero whatever the value is here it's going to be the same here so this is essentially a square now if this weight is going to decrease by 05 for example then this rectangle here is not a square anymore it actually fits two squares because the rate this decreases is half of this x now believe it or not I'm not done talking about just one output there next time we'll still have only one output there and there's a lot to talk about it but uh I want to leave you with something more fun to do so go here at the top and say s is equal to BC K and now you can practice having two outputs going forward and backward and see what you can do with that let let me just show you something real quick so here this is the same as before you can control this lighter grayish region but the other is going to control this green region here so if I'm going to increase this bias so that it goes like that and decrease this one now the car is going to go back backwards in the beginning your homework is going to be to make it go forward and then bounce back like that people teach neural networks in different ways and that can make things confusing I'm going to try to clarify some things now you don't really have to listen to this part it won't matter in the rest of the lessons but it might help you someday now some people say that for the neuron to light up x times the weight must be greater or equal to the bias not really a big change but that neuron would light up now with everything set to zero and in ours it doesn't but you can always make a small change to the bias and it's going to work pretty much the same there's no real reason to choose one over the other so you will find both implementations out there just don't be confused another thing people do is move the bias here on the left same thing so far but they also changed the minus to a plus this doesn't seem right I mean it's not the same thing anymore but again it doesn't really matter any network you find that works in the first case can be transformed into one that works in the second case just flip the value of the bias the reason you find this implementation is because it looks like the line equation something people are familiar with I like to use the neuron fires when stimulated above a threshold idea so it's just personal preference another thing you may find is people removing the bias entirely but that would break things we saw earlier that we really need a bias so for it to still work they add here an extra weight connected to a fake node that is always on so this weight here acts exactly like the bias and everything is all right I think people like this because it groups together all parameters in one thing so to speak now this here is called an activation function and this one in particular is a step function there are others that are more powerful like a sigmoid here would produce values between zero and one so the car could be more like in the real world and accelerate more or less depending on the situation I don't want to teach smooth activation functions because our controls here are binary and it would make things more complicated our car will still be able to control the speed by pressing the acceleration at different rates so we don't really need those but they are important when doing machine learning a gradient like that helps optimize the networks that's why you'll find them everywhere like in this other playground I found while building mine it's really great a bit more abstract but a really useful learning tool so check that one out as well we learned what one neuron can do with one input next we'll see when one input is not enough we'll try using two inputs and even a hidden layer prepare for some neurons in the shadows open the playground from the link in the description it should have here fwdg is the session name now let's have the same Network as before so- 0.5 for the weight and- 0.1 for the bias and the car is going to stop right here at this dotted line let's change the objective so that it goes up as much as possible so can we figure out a network that lets the car go as much as it can before hitting the Border there and we could try playing with this basically Ally It's kind of logical that this needs to be a little bit to the right so maybe moving the bias like this this might be too much let's let's try it too much we probably need to change also this maybe something like that I don't know still too much this is tedious we can also do something else to figure it out we can use the genetic algorithm from phase one it's right here let me show you when it optimizes here it's going to try to create many different cars simultaneously and each of them have a different network how different from the one we started with is controlled by this slider here so if I'm going to go down all the way and press this button again you will see apparently just one car going up but it's actually 100 or whatever going up and this network is the same that we were playing with before here in this simulation when the car touches the Border it disappears entirely because there are many of them and I don't want it to be too crowded so let's try to give a little bit of a mutation here and that means that the network is going to be similar to the one before but not exactly the same so now you can see that we found a car that is pretty much touching the Border but it didn't disappear it's still here so it's a good car and when we find a good car like this we press this save button and now we have this new car to work with like if we close this Optimizer from here you can see it going up and then stopping exactly there pretty much let's see close is it really very close or could it go even higher it might go even higher a little bit let's try optimizing again let's save and try again every time we run this optimization the previous best car is not lost so if it looks good saving in the worst case it's just going to give us the same thing we have previously we don't lose anything valuable like this so let's save again and the last time I just want to be sure that I'm getting a as good value as I can get there safe I think it looks kind of like as before so maybe it was good the first time already and these values minus 0.56 and- 0.14 let's figure out what this value is here turning on the calculator and 0.14 divided by 0.56 [Music] 0.25 so that seems to be the point here 0.25 but this weight and bias are quite complicated we already start to have now the second decimal there and these are some things that we can expect from neural networks they don't give us the easiest way to write 0.25 there there are several easier ones that I can think of right off the bat for example here I could put minus 0.1 now if you press the zero key on your keyboard it's going to zero that and lower like this minus 0.1 and this could be minus 0.4 so zero that minus 0.4 so this should be the same thing as before let's try to restart and it is but it's a simpler one because 1 iD 4 is 25% so easier for us to understand the values but the end result is the same there are other values here that are easy to understand like Min -1 and- 25 for these small values when scrolling hold down the shift key this is going to make you scroll more precisely like that so refresh and this one works as well there are many solutions here the one found by the genetic algorithm is just one of them and and these for me at least are easier to understand when I look at the numbers what is really going on here now it's important to stop and think what is the car really learning here what is this function for and one thing that can help with that is changing here the title FWD Corb and we get the same thing but now the car is a bus and you can see it doesn't work the reason why it doesn't work is that the bus is longer so the distance here from the center of the sensor to the tip of the bus is longer than we had before so it is going to touch we could run the same Optimizer again and find the solution that works for the bus so that's no problem but but the same brain that we had previously doesn't work also for the bus and for the previous car that we had let's save this one and now close and let's see the bus going smoothly and it stopped perfectly there but if we go back now we can see that the car is stopping not anymore in the last moment where it could stop there because it's using the brain that we made for the bus so this is important how big the car is and in some cases real life cases people are making the same kind of cars equipped with the same kind of brains and then they have different brains for other cars that are different just because the car has a different size uh to it if you want to have one brain that does everything then the size of the car has to be an input here here an additional input we won't be doing anything like that here it's not very complicated but I don't want to teach you about the buses here I just want to focus on the cars but basically we need to be able to understand that a longer car needs to stop accelerating earlier than a shorter one now let's go back to these values here minus1 and minus 0.25 five like this restart just to see that it works and another thing that doesn't work is if you write here FWD uncore 2 so we have the same car from previously and it goes just as expected but there's another car on the left and both of them have the same brain so interestingly the car on the left does nothing stop and try to figure out why maybe pause the video and think about it I'm going to start talking again in five 4 3 2 one so there are two dots here on the right right one is for the car on the right the one that is moving and the other one is outside of this region to begin with so this car is so close that it's already inside of that black region where the neuron doesn't fire so it's never going to start if the car is too close there it's going to start and you can really tell that something else is needed here I mean what happens if we do make this car move just like this for example we can lower this bias more like that it's moving right but if we restart the simulation that means that this other car on the right is going to crash because it's going to stop accelerating much later so we can't have them both not with one input here let's see what happens if we introduce the speed as well so here at the top s is equal to SPD underscore 2 and now we have some extra things here we're going to get to them in a little bit but first let's just put some values for these weights and biases let's put minus 4 here minus 0.4 and minus 0.4 here as well and here minus 0.3 and let's refresh and something happens and arguably it's better than before it's not perfect this car is not going all the way till the end this one is also not going till the end it's it's worse than the one on the right but both of them move this time so it looks like something now before we get into what is happening here I want to show you a nice tool this uh Desmos calculator has a 3D version now so desmos.com sl3d here and let's write the same function we had previously so m - 0.4 x - 0.4 y has to be greater than -0.3 and you can see a flat surface here this red region in the XY plane now just a little bit of exploring here this is the same thing as if you write Min -04 x -04 y + 0.3 greater than 0 right so let's write that plus 0.3 greater than zero it's exactly the same thing we just moved that one term now if we remove the greater than zero we move out of the XY plane and we're in 3D now we are defining a plane this now what we have here is not defining a line anymore it's defining a plane and only when we do that kind of comparison like equal to zero means the line where that intersect the XY plane there or greater than zero means all the parts of this plane that are above water so to speak so positive values now this 0.3 here is called the Z intercept it's actually this point right here where the plane is touching this z-axis because if X and Y are zero here then you get just that so that's why it's called the Z intercept and this here controls the rate of decrease in this case on the X and this one on the rate of decrease on the Y so this plane is tilted in the same way on X as it is on y but if you would have here minus 0.8 y for example is going to be tilted on X less than it is tilted on y it's much more tilted this way than this way but let's go back to our original one so- 0.4 Y and let's go here and have this XY orientation and zoom in a little bit because we are only interested in values between minus one and one now this is not centered really well you can go to these settings here and say Center origin we can also make these surfaces translucent like that so we can see through it let's zoom out a bit so that this one appears here and minus one is here so this what you see here is the same that what I am plotting here I'm just using a little bit of shading here so kind of like the plane going under the water there you can see it like that and there is a slider here at the bottom that actually controls how pixelated this image is it takes effort processing power to calculate this image right here so I don't really recommend getting this smooth version from the left side sticking to something like this and above if your computer starts to burn up but um basically This Plane where this plane is above the water that's when this is going to trigger now so if these dots appear to be on land then that is going to trigger for them and this mode also has a simplified view because this is showing a 3D space here right but we don't really care about what is going underwater so if you click on this once it's a simplified view showing just the part of the plane that is above water or you can consider it like that so back to the 2D space coming after we have an inequality sign there so let's rerun now this simulation and see what is happening here it's very interesting right you should take a moment and try running this many times and paying close attention to what is really going on there so this car on the left that is stopping early it's this point it's moving on the edge of this plane here and then somehow going down and stopping early the other one the car on the right goes higher here and somehow slingshots away from this plane like it it jumps from there kind of and it falls down later interesting isn't it now if you look at the top here I can only show one of the two brains they are the same brain so the same value for uh this weight the same value for that weight and for the bias but the inputs for them differ because now this one is closer so it has a larger value for for this 0.86 this is 0.86 here this is going to be smaller right so the brain that you see on the right here is going to be from this car here and if you turn on manual override you can see that brain changing while I'm moving that car and look at what the point is doing under the right so I'm moving the car forward and backwards and forward and backwards and because I'm changing the speed like that it kind of looks like it's drawing a circle because it moves the point up and down also below the zero point vertically because the speed can be negative in this case but uh combined with the sensor there is also moving it from the left to right so somehow it's drawing a circular um pattern there of course if you would do this somewhere below here where this doesn't read anything then that point just moves up and down like that because there is nothing moving it left to right the sensor isn't picking up anything and of course if I'm stopped then the point is just going to lie on the horizontal axis like that so really there is a lot going on here and I recommend you play with this it's um it's really fun but let's try to optimize this so that the cars go as much as possible and let's use the optimizer from here so hopefully we're going to get cars that are going to be as close as possible to the Border there and now we are getting something better actually so let's save this and let's try again now one thing when doing this simulation the cars there are two sets of them right on the left and on the right they have the same brains what I mean here is that the cars here are all a little bit different mutated slightly but exactly the same brains from this set are copied in this set and the car showing in red here is the best one of both sets so you might see here a car that is a little bit better still in front of this one and it didn't yet touch but its counterpart from here it's maybe below somehow so the added distance that they traveled is less let's save and try one more time let's see if we can get it really really close there it's still doing something so wait for it to stop before saving anything otherwise it might collap eventually let's save this and I think I'm happy with this it looks really good now let's close this and see just two individual cars and it's nice they are stopping there and they are doing some kind of crazy things here but if you're like me something bugs you at this point the car on the right actually seems to arrive faster than the car on the left which doesn't make sense I mean look at this it's overtaking it about now and stopping it let me show you that again so it's overtaking it now and stopping there and if you look here in the diagram this one is slowly making its way there and you can kind of see the overtaking moment from this slingshot guy that just lands there where it should be ending so the fitness function described inside of this optimization is go as far as possible but it doesn't say anything about the speed like it doesn't matter if they are going there as fast as possible or not and integrating both of those in a fitness function is not easy because it's a kind of a two parameter optimization function so we are just going to not use the optimizer anymore but I do want to show you still this like if we decrease this value even more the rate of decrease on the speed here it's going to move that plane kind of like that even more so here this apparent slope is going to go down like that and refreshing now is going to make the cars start to do that even earlier and now they have this really really slow movement at the end here you can go even further than that like this and now you see very soon they start doing that and they start to really really crawl there in a in a funny way of course increasing this too much like this is a big problem because this is going to make them slingshot like that really much uh it makes them slingshot because they accumulate speed and then the friction is not enough to slow them down on time so there are very many things here to consider when doing this optimization like how the slope should be here exactly but the really important thing is this point right here where the car must stop let's refresh now and we see that we got something good so this point here is really important it needs to be outside of this lighter region so it shouldn't be on land because if it is then it's going to accelerate more and that means that it's going to get so close that it collides but now let's focus on the car on the right let's refresh and it's this one it's sling shotting we kind of want it to slingshot more right so that it lands perfectly in that one spot could we do that so I need this to be a little bit higher there I think that's too much so I'm going to hold shift and maybe say minus 35% ah too much so let's try minus 38% on this one it's now doing that thing it's fixing it getting close so it should be a little bit higher minus 0.37 maybe let's refresh still doing that 36 and now it crashes there so it's not easy to figure out the exact values we would need even more decimals there or we could play with this value as well increasing this is going to lower this a little more okay now it stops nicely without it triggering anymore it could still go a little bit higher so maybe this one we try to increase a little bit okay this is perfect for me but let's now focus on this other car what is happening with that so you can see it doesn't go as far and it even moves a little bit after that one has stopped so it's not as good let's see what happens here here it's sliding down like that right and you can see it's stopping before the other car um closer to this zero here so that means that the sensor is reading a smaller value here so it's further away from the border we could get closer to the Border if we move this a little bit like that so we could go here and make it higher but hold down shift so let's make 29 and you could see it moved a little bit forward let's try 28 and the one on the right is also moving now and it's too much it actually collides to the wall so let's just stick to minus 0.29 and rerun this simulation and this one got closer but it's still not exactly there and it's doing that thing where it's sliding down the hill instead of jumping somehow slingshotting like that so that it lands perfectly there let me run this with a higher quality here just to show you better what is going on because that pixelation is interfering a little bit basically we would like there to still be some land here so that this car can accelerate a little bit more and then slingshot exactly there but there is a problem with that you say this other car needs this point here to be the border so if that car needs that point to be the Border and the one on the left needs a point there and this point here needs to be outside of that region it's not possible to do this with one single line is it and that's where hidden layers come in a hidden layer is a row like this where we can add more neurons they're called hidden neurons or hidden nodes if you prefer the graph terminology because their computations are not usually seen from outside the blackbox now here we do see everything but I still use the word hidden when I mean not an input or output node more terminology this is also called a perceptron when we add hidden layers it becomes a multi-layer perceptron it's important to know this because some libraries use it like when we use neural networks in the machine learning course we used the MLP classifier inside the psychic learn library in Python so good to know that means a neural network go up here and put underscore H instead of two now go here and make sure that this pixelation is on because we're going to change some things here and it's going to be slow otherwise and let's do something click first on this point and now you can actually create a new Point here and this one here connect to the same one so we created a new neuron here with these inputs and let's add the same values here so minus 0.29 I'm just going to go to minus 30 and then hold shift and go up one and here 0.35 and 1 2 3 4 five like this and the bias here minus 0.26 like so this neuron is the same as this one here you're going to see if we rerun this simulation that both of them are going to Blink exactly in the same way now we can actually remove these links by right clicking on them and this one and let's move this one under right click on it and connect it to the output now reset the bias here to zero you can press zero on your keyboard and you can put any positive weight here anything will work because basically the value of this neuron here when it's on of value of one multiplied by any positive weight is going to be greater than zero so these now are in sync I'll show you we go here and refresh and this is the same thing as before you can see the same shape appearing here in this diagram now this solution here with these neurons Works nice for the car on the right but not for the one on the left that's the one that we try to fix so let's create a copy of this and try to modify it so that it works well for the car on the left I'm just going to click on this and create a new neuron here and same values so minus 0.29 minus 0.35 and minus 0.26 so now if I'm going to connect this one up here and let's set this weight to zero and this one to 0.1 or anything positive we get the same thing here so just to test this should work in the same way the car on the right is stopping perfectly the one on the left still has a little bit of work to do let's get this slope to go a little bit like that so we're going to increase this value here slightly so holding shift and basically I want to get this other car here to slingshot I don't care if this one breaks anymore I just want this one to work and it doesn't really work it does slingshot a little bit a little bit maybe until here but it needs to be a little higher maybe 26 it's better but still a little bit more 24 even more okay so now it looks perfect it stops perfect but the other one breaks so let's compare now the two functions and I'm going to put back this high quality diagram here and let me turn off this one and on the other one let's rerun the simulation and see where the car on the right jumps it's this point right here and now with the other one let's see where the other one jumps it's this point right here now turn on this pixelation more because it's really quite slow and let's create another neon where we try to define a line that touches both of those points let's set this one to zero this one to some positive value and here let's try with this left one first maybe minus 40 the one on the right minus 90 and the bias here let's say 60 and now I'm going to teach you how to do the conditional and so we want this one and this one to be on because then this region will essentially intersect with this region giving us a curve here not a straight line so to do that we put both of these values like this and you can already see a curve appearing here but we don't want it like that we want it at the intersection somehow so now this is actually an or because it's either this one or this one so we also learned how to do the conditional or but to do the end we are going to increase the bias here like that so the value here if it's on then it's going to be multiplied by 0.1 so we have 0.1 from here and and if this is on 0.1 from here so adding these is going to be 0.2 which is greater than 0.1 but if only one of these is on like now then this doesn't turn on because it needs both of them to activate so let's just try to see how this one works probably we need to do some fine tuning yeah looks like this car is crashing because this is too high we need to lower it down a bit so minus 92 maybe if you look closely what happens here it touches that tip a little bit you can see here after it sto oops it blinks a little bit so I think that we need to also adjust that one slightly so here where it says minus 0.40 let's make it maybe minus 0.43 okay I think this is perfect and we don't need this one anymore because this is anyway connected to zero here it was just used as a as a reference so let's remove it by right clicking on this node then it's going to remove all the other links as well now your homework go up here and say SPD uncore 3 try to get it to work for or three cars this one seems to be crashing at the moment one quick note here I've found many resources claiming you need smooth activation functions to have curves here one of them in a lecture by top university now what I'm sure they meant was a smooth curve because we got a curve here already using a step function a curve just means something deviating from a straight path doesn't NE necessarily have to be smooth it may sound like a small thing but I think that false information like that is confusing especially because it seems like it should be true so people pass it around a lot and it makes understanding things harder I think the playground doesn't support smooth activation functions and as I said earlier we won't be studying them in this course but if you're curious that shouldn't stop you from learning nowadays there are a bunch of resources that can help it's lesson three and we're still using one output but that's about to change we'll start using two and then four so the network can tell the car to go forward backward left or right now this is called a multi-label neural network and you don't need to know this name to follow along but I do want to tell some terminology and don't confuse it with a multiclass neural network a multiclass neural network is what we had in the machine learning course where the drawing could be classified as one thing but here the network can say to go forward and left at the same time for example so it's a different thing okay finally the car will start exploring open the playground from the link in the description and you're going to see this scenario with three cars but the difference is that now we also have the possibility to go in reverse and I'm going to show you how easy it is to solve this problem when we can do both of these things I'm going to set this weight here to minus one this one also to minus one and up here the bias Min - 0.9 like this now this by itself crashes two of the cars but if we also go under right here and put the weight of one one and 0.9 essentially filling this remaining space and restarting the simulation see what happened let's check this again it looks really good the cars are stopping where they should and they are slowing down only when they need to at at the very end this one arrives first this second this third if you just look at what happens here it looks all right and we don't have any curves or any fancy things but what we do have is this other ability of the car to slow down quicker so even if you could see a little bit of that kind of bouncing effect here only for the fastest moving car I don't know if you can even notice it uh it's not a big deal anymore because this is such a powerful maneuver that it's basically making all of these points just follow this line separating the two surfaces here and at the very end you can see these things blinking like crazy because we're at the edge this is a bit annoying here if you look at the neural network but if you look here on the left even if you zoom in the cars don't really move because the acceleration is so small there and going in both directions so I'm not really bothered by it but if you are you could set for example this to minus 0.89 and now there will be a small Gap there maybe if I'm going to do this you can see this very narrow Gap here forming where the points stabilize nice now let me show you the next homework go to S equals to revcore H it's going to be the same thing but just one car in the middle and the goal for you is to make it do this where the car goes up and down like that forever try to find a network configuration that does that I think it's pretty pretty cool but now we're going to move on and I'm going to teach you about steering so let's change the link at the top to S equal to sdore 0 and you're going to see this new scenario here and the car has one more sensor the one on the right you can see this extra input appearing here and it's now sensing something because the road border is quite close but the other two are zero because the car is not moving and because this is not sensing anything now here at the top there are two new outputs for turning left and for turning right and all these weights connecting them now these are really confusing to work with like that so let's remove them from this button right here and move things around to make things a little bit clearer so let's separate this into two parts one on the left here will be what we just did previously going forward and being able to stop so we don't Collide and then on the right let's keep the steering part it's going to work with this new sensor that we have here and let's do the same thing that we did before so let's connect this one and say here minus1 and this one - one and here minus 0 9 and for the reverse one one and 0 point nine now this is the same thing that we had before and the car should stop nicely at the end and it did but if you look here something strange happens focus on this diagram and when we refresh crazy things are are going on this doesn't look anything like what we had previously right and that's because here we we don't have the speed now we have this sensor if you click on it it's going to change to the speed and show what we had before exactly so now you can recognize this pattern here but what were we looking at before so this is what you're going to have to understand next it looks like these regions are somehow changing but they're not you can only change those things if you play with the weights and biases from up there what happens is that this is really a 3D space now and one axis that goes like that is the speed in this case it's the one that is missing from these two labels here and what we are seeing here is a slice of the cube at that speed somewhere inside of this Cube at the value of the current speed so if I'm going to change to this it's like we are watching the cube now from the from the top and what we see when we change this we see the cube from the side so this amount of green that is still on the right of this point is actually the same in this case but as the speed of the car changes this amount of green that we see in this other View is going to start growing and then shrink back because here as the point is moving up if you think about some kind of horizontal line at that point location the amount of green on that line is growing and then shrinking again so growing and then shrinking again so this is what we see in this view growing and then shrinking again it's not easy maybe it's going to help if we use Desmos again and let's type in the functions so we have -1x - x -1 y minus y and there is no weight connected to Z at the moment so I'm just going to put plus 0 Z and nothing appears here and this is because what we are describing now is a hyper plane it's something in four dimensions and here Desmos is saying something try adding an equal sign to turn this into an equation because it can't show four dimensions there so you could put equals like equals to our bias which was -0.9 and this now turns into a plane in three Dimensions but we're really interested in the values that are above this bias so greater than minus 0.9 so what we have now here is essentially the gray area in our diagram but let's look at it from the top like so let's make it actually gray it's possible to do that here and let's zoom in here so that we focus on the minus1 + one area and let's add the green one next so that was x + y + 0 Z we are not using that at the moment and this is greater than 0.9 and this is our green area let's make it green so it's similar to ours and it looks pretty much the same as what we have here looking from the top but if we look from the side like this and slice the cube somehow I'm not sure if we can do that let's try going here in the settings let's try putting here zero okay it looks like we can so zero means that we are basically having a zero speed now and the car has a zero speed at the moment let's turn on this manual override and when I go in reverse you see this green part disappears from here it disappears from there because when you have a negative value for the speed like minus one for example then we are watching this part this slice of the cube that doesn't have any green in it but if we go forward and increase the speed we get more green appearing there and that's because we are slicing more in front there like let's say one would be very much maximum speed of the car but let's put 0.5 for example so this is half as much as the maximum speed and let's see if we can get to that if I start all the way at the bottom here so let's see how much green is coming here yeah I could get to that but I don't want to crush the car I really hope this is clear for you because because these things start to not be very easy to understand and the tools can only do so much now let's think about turning so I want to turn based on this sensor here if it doesn't sense anything or if nothing is close if the border is not close here I want the car to turn to the right so I'm going to connect here this sensor and to keep things easy I'm just going to put minus1 Here and Now vertically I can move that red region as a percentage so this 20% minus 20% here just means 20% upwards here so let's move this all the way to the top maybe minus 0.9 like this and it's still off because this dot is above it here but if I'm going to move the car somewhere where this doesn't appear then it's now moved in this red area so this will make the car turn and let's see how it goes I'm going to remove this manual override and the car is turning a bit too much and then crashes in the wall so maybe a value of- 0.8 is better let's try try to restart okay yeah looks like we got it to turn so it's really nice and see what happens here in the diagram so what did we add this red region let's try adding it in Desmos as well so this one doesn't depend on X at all it's 0 X Plus 0 Y and minus Z because we have a minus one weight there greater than minus 0.8 and we can change the color here to red and this is what we get now the option from here to have translucency to see through these things doesn't really work when we have these solid objects I think it shouldn't really work it's kind of hard to to think about these solid elements and seeing through them somehow it's going to be very confusing anyway but this didn't affect the other things they are still there if I'm going to toggle it from here it's just a new thing that was added so here you have a combination of green and red really and that combination of green and red is what I'm showing here as this orange when they are overlapping there the gray and red makes the red appear brighter like this but from here you can actually remove some of these elements if you find them confusing like if you just want to see the red area here you can turn off the gray and you can see it now well and the green and you can focus on what happens with the point and in relation to this control here but let's turn them back on and change this to the previous View and see what is happening so you see this crazy blinking that is happening there it's because the point goes inside the red region and out outside of the red region sometimes really really quickly here when it's actually doing the turn it's it's doing that so what happens is the point is basically somewhere at the border here and then moving like that really fast that's what makes everything blink red when seen from this side here again really hope you get it play with these things it really helps I think now if you look at this something is weird because the car is not really driving on its side of the road it's driving in the middle like like crazy so let's try to do that probably what we need to do is decrease this value here let's try going slowly because minus 0.9 crashed it so I'm just going to try 81 8 two and focus what happens on the car as I'm changing this 83 you see it got closer got closer 84 it's getting closer but it's also sometimes moving on this kind of trajectory that is probably going to hit the wall eventually let's see 86 ah almost almost hit there let's try 87 maybe this is as much as we can go okay so at 87 this thing happened basically it turned in a way that it was on a collision course and we need to correct for that if it gets too close to the side of the road we have to teach it to turn left so let's connect this here and do the opposite so minus1 turns to + one and then here we have + 87 okay and probably it helps to switch back into this other mode where we can see now this so red and blue are the two different sides if it's not going to turn right it's going to turn left and if we restart we see the car correcting itself it's actually doing it like crazy if you see here very quick Corrections happening and that's because there's no spacing here between the blue and the red it also reminds me that in phase one of the course some people were asking why does the car shake like that so now you have an answer it was in between regions like this now it's better it's on its side of the road but it's touching that lane a little bit I think we can do even better than that so maybe let's try changing this to minus 0.9 like so and this one to 0.9 so forcing it even closer I think this looks good if this shaking bothers you then you can always put here a higher value like minus 0.89 and now the car is not going to correct itself as often let's see how it does in a different scenario so change here to Str strore one and now you see the car going in circles like that and this is what I meant in the beginning that the right hand rule in mazes doesn't necessarily guarantee that you're going to get out of the maze you can also get stuck in situations like this so I just wanted to show you a situation where it doesn't work let's move on to a more complex scenario S strore 2 and see what happens here something kind of crazy happened there right let's look at it a little bit more so these things are quite nice it's really keeping its lane and then here oh no oh even crashing horrible so let's look again and see what happens here this is fine but here it's doing something crazy and that's because the front sensor doesn't see anything when moving here but it saw something when moving there so it slowed the car down this is the problem here as well the front sensor doesn't see anything so the car only sees that it has to turn right but it has large speed and the same thing happens here picks up speed even more and then sees that it has to turn right but then when the front sensor sees something eventually it's too late for it to turn now the solution for this is really to add more sensors so that it can see things in front at different angles and figure out that something is coming but I don't want to increase the complexity of this structure too much because we will still add things to this and it's going to become more and more confusing if we have more things to work with what we will do is Cap the speed so that the car cannot go as fast as it can but at a speed where this problem doesn't happen anymore think about it the car sensing things on the right really it finds out in the last minute that it has to turn so if it's going some crazy speed there it is going to fail it's too late so let's see how we kep the speed and let's change here so that we see this chart and let's remove blue and red because they are confusing and let's remove these two links from here we will do something else with the reverse section later for now it's best to focus just on this green area and cap it so I want this section here to be black so that the car doesn't accelerate to the speed that it could manage mechanically and for that I'm going to have to add here a new node like this and let's just copy these values here so minus 1 - one and this one here is - 0.9 and now these nodes are doing the same so let's remove these links put a zero here you can press zero on your keyboard to do that faster and let's connect these with some kind of positive WID 0.1 for example and you see this shape appearing here again so now whatever this value is this one is going to be the same now let's put this to zero and create a new shape here that is just a horizontal flat line here and the bottom part gray so I'm going to take the speed here only it doesn't depend at all on this one right it's going to be a flat shape and for Simplicity I like to put here minus one so here I can control a percentage and let's try to put minus 0.4 and connect this one to the gray output and add some value there so now this percentage here if it's minus 0.5 it's exactly in the middle this is higher this is lower let's start at minus 0.5 and see if the car Behavior changes but we also have to do the combin so the intersection between this region and this region it's going to keep just this kind of trapezoid here to do and you put both of these to 0.1 and at the moment this is actually doing or so it's either that or that and here we can put 0.1 so that means that this needs to be on this needs to be on for this one to activate great and another homework for you would be to do the same thing for the green region here you would have to use or so it's a good chance to practice that logic and you need to duplicate these make similar logic many many complicated things maybe clean this up a little bit organize things differently and connect in the same way to that one or do what I'm going to do now and use the not operation so what will happen is that when this one is on this will be off and when this one is off this will be on which kind of makes sense I mean when you drive a car you typically accelerate or decelerate you don't really let the car slow down by itself in almost any situation unless you think about conserving gas or or something like that but if you want it to be fast then you probably want to apply the brake if you are not accelerating so having a not here does make some sense and let's see how to do that I'm going to create another node here it's easiest to teach you like this and let's copy these values now so 0 1 0 1 0 1 and remove these other links right clicking and zero this one positive here so these nodes are now in sync we have the same thing as before we introduced another hidden layer here and for the not we are going to connect this one here with a negative weight and here we have to use a negative bias as well if you're going to use a value of minus 0.1 it will work because when this is on so a value of 1 multiplied by - 0.1 so we do get here - 0.1 and it's not greater than minus 0.1 so when this is on this is not going to be on but when this is off then that's going to be 0 * something which is 0 is greater than minus 0.1 so basically we have a way here to get the opposite of this node and that's why the green area is appearing here nice like that and there's no overlap or anything going on like if you remove the gray area here it's clear that the green is the opposite of the gray okay so does it work let's restart still doesn't work it's still going too fast there on that section so we're going to have to cap the speed even more maybe minus 03 let's say and let's see what happens here these were the most problematic things and the car is going to go now quite slow compared to before yeah it still did a little bit of a of a loop there but it's not crashing anymore so maybe even a smaller speed minus 0.2 let's see if it crosses no okay it doesn't cross the line anymore so with this Logic the car moves relatively slow to what it can do but if you think about it and if you would think that these are real cars and you have you're watching them from the helicopter or whatever um they would look like that they wouldn't speed up crazy like what we had previously so this is really a decent speed for a car inside of a city so we did it the car can now navigate these complex situations it's even going on its own part of the road and maybe it's a good time I show you these other buttons up here so you can save the current car Network structure that we have here we put a lot of work into this so maybe we don't want to lose it and saving from this button and you can also load from this other button so you can try different configurations save them under different names and then load them to see how they work in different scenarios which by the way you can load from this button right here so if you took the virtual world course you have the editor there and you can create any world you want for the cars to drive in and you can use this playground to load those worlds into it and see this car or any car you buil here manage it now one thing that I still want to show you is that the network here doesn't look like the one I showed you in the very beginning of this course like if you type here default the network here is quite different it has just one hidden layer right and then this right sensor is connected to something here and to something there not directly to these outputs and there are significantly many weights here on the right this is the common structure neural networks have really and the reason why everything on this line is connected to everything on this line and here the same thing has to do with how neural networks are represented internally as matrices and speeding up some of the operations and also a few other things I'm not going to get into but basically we could convert our Network into this network by just adding zero weights where they are needed and making this kind of relationship here so that this node is in sync with this node and and so on the problem is this one more hidden layer is missing here so how can this car do something here and let's go back I can show you that we can actually remove this node from here as well so right click on it connect like this and just put here the. 1.1.1 so now we have that gray area here but it's covered by the green so we can't see it and the green has to be a knot again but somehow created without that extra node from there so we can can connect this as such put a negative 0.1 here a negative 0.1 here and it's not quite right here this needs to be minus 0.2 I'll let you do the math for this one but you can see the end result is exactly the same and we didn't really need another node there another hidden layer now you may want to save this version of the network as well because it's different than the previous one and what I'm going to show you next will kind of destroy it so focus on this network here and change the url to S strore three and notice that it looks different but it's really the same thing it's doing the exact same things as previously like you can recognize this shape here and adding on these other two you can see them here whatever Network you create in Str strore 2 is going to be converted in this kind of full Network that you see here and these weights are essentially zero all of them that are added new and you can recognize the other connections there plus these new nodes two new nodes that are needed to connect to the turning right and turning left and they are just in this kind of synced relationship so whatever the note here is this one at the top is doing the same and this network looks much more like the one on the first page there but that one had also some values for these weights and the reason why that is is that I optimize this network a little bit more by using the genetic algorithm so I did this I lowered the mutation here and I tried to generate cars that are very similar to that one but a little bit different so that maybe they act better and this is really slow uh maybe it's acting a little bit better if we put this to the max but basically this is really slow because it's trying to process this large Network and do these visualizations and yeah but you can see this car that is leading here the one that covered the most distance is not the one that we previously had that we were previously working with because it has some values for these other weights it might crash at some point I don't know but it's at least a car that is going for now better faster than the previous one the one we designed and I think I saved one of these cars and I've used it as the intro there you can see some of these connections here are still quite obvious that they are that the network is based on what we designed but there may be some relationships between these notes that we didn't even think of and this network is potentially finding some of those it's also making this much harder to explain what is going on so this is the truth whenever you optimize using automatic methods like this you will start to get a network that you can't explain every single thing from there usually you can't explain anything from there but in this one because of all the things that we have done we do still have some IDE a of what is going on now this car seems to be doing quite a good job I'm going to save it and let's close this and see it going and it might be faster but it's not as smooth as what we had previously it's doing this kind of fidgeting here um sometimes it's pressing acceleration and break at the same time but you see how these small changes in the weights are affecting the overall result even though it's kind of doing the same thing some of these structures are totally different so here there is not anymore that kind of Separation the gray area looks different it's doing a an an or here now it's not doing an and anymore more so probably yeah this bias here has been made lower so this is interesting an interesting change and the green region is very much the same as before so now this is not anymore when not breaking when not accelerating you break um it's also this these two sections appearing here where both of them are pressed at the same time and there's much more going on here like when the car is turning at some point I noticed that the green region is you saw that it was changing a little bit the other sensor is somehow interfering with the logic from the component that we had independent previously and there might be some good reason for that like if you do see that it's empty on the right maybe you want the car to break break because you don't want it to go crazy even if the front sensor is not doing anything so it is possible to come to some better logic more complex logic automatically like this and here this this thing here is really strange before we just saw the clear separation between blue and red and what is this thing if we remove the red you can see that this is actually blue appearing here so the blue somehow makes this shape now and it depends on the speed as well if it's going to act like that so depending on the speed this steering left which we only use to correct its Direction has a different effect hopefully with all this you can see that we can still understand some of the things that the car is doing even though the complexity has grown especially because it's based on something we created but also because of the tools that we are using here and the understanding that we gained during these lessons it's important to remember that this is a small Network I mean compared to that deep neural network we had in the machine learning course this is nothing and the one there is nothing compared to powerful ones out there so just imagine the complexity those can handle if ours can already do so much and by the way in case you're curious I did teach how to implement that chart and the decision boundary plot in the machine learning course the diagram that we're using here is that but regions can also overlap because of what I said earlier multic class multi label remember the car is now moving nicely staying on the right side of the road and navigating this maze quite well but what will happen if we add some markings well nothing the car has no idea there even there we need to add more sensors and update the logic for them as well the new network will have five inputs this time so get ready to navigate in higher Dimensions start the playground from the link in the description and you're going to see this scenario which is pretty similar to the last one but it has also these markings on the road like the stop marking and the traffic lights now the way we're going to solve this problem is the car is equipped with two new sensors they are not really visible here but if you're going to turn on the manual override and go forward like this you're going to see that this red sensor appears here it's overlapping the other one the way I implemented this on top of the code from phase one is I just added a stop sensor another sensor to the car and it has only one Ray here that is casted in front but it's looking for borders from these markings instead of the road one other thing that I did is the car is only seeing this border if it's looking at it from this angle like if I'm going to go up here slowly and look at the stop like that it doesn't see it it doesn't turn red you could model this logic in the neural network as well but that will make it even more complicated and you'll see today it's going to be quite complicated by the end now let's go back here and see the input node associated with this sensor so it's this one here it works pretty much the same as the front one but it's the distance to the stop marking there not to the side of the road so you can see that value changing like this and it's zero if it doesn't see anything now for the traffic lights the same thing but a different input node a different sensor in the car object as well and you can see those values changing but if the traffic light is green then this sensor is not reading anything so it only looks for yellow or red traffic lights and the same thing applies as before if you're looking at some traffic light from the wrong angle like this then it's not going to to trigger anything you need to look at them from the correct angle like this assuming that you're driving on the right side of the road so two new nodes here and stop to think about it when we had one node we were using that to define a line two they were defining a plane three we had this hyper plane what are these they're also called hyper planes anything after three dimension is just hyper but visualizing or understanding what is going on now when we build this new network is going to be much more challenging than anything else because we are not anymore in the 3D world that we are used to so let's start solving this problem and we're going to start from scratch I'm not going to load any version of the car from previously because I think recapping what the logic is helps to understand how we complement the logic so we will build the same card that follows the right hand rule of solving mazes as previously but a little bit different he'll see let's move this sensor for detecting the the right side of the road there and the turning left and right let's get them out of the way for now and here let's focus on moving forward and stopping we're going to focus on moving forward actually and for that we need the forward sensor so that we can detect when the car is going to stop and the speed so that we can control stopping at different moving rates now let's create a new node here like this and uh two more one like this and one like this and one like this you can probably tell something crazy is going to happen but we begin with the very basic logic here so let's define this weight to be minus one minus one and this one was Min - 90% like this and all of these weights here I'm just going to put plus 0.1 so that this node just whatever it is it's going to sync with this node and this will sync with that node and this will sync with that node so the logic here is really simple what is happening here whatever this value is just transfers to all the other ones and you can see here on this axis that that just means the car is going to ex accelerate until it's very close to the border so like if we are going really really close to the Border then it doesn't accelerate anymore and this line moves here when the car is moving oops I crashed it let's restart this the reason why that line moves like that is because of the speed so because this is still a 2d scenar Ario we can put here this one like that and see the decision border here so now the car should be able to go forward and stop whenever oh it's going to crash because I didn't connect the reverse so here we are going to do a not whenever this one is on this will be off and vice versa I'm just going to put here - 0.1 and - 0.1 and if this is off 0 multipli by something is zero it is greater than- 0.1 but if this is on then minus 0.1 is not greater than minus 0.1 so this node whatever it's going to say this will be the opposite and now restarting is going to stop like that and I'm not going to bother with this kind of blinking here because it's not possible to solve without doing a lot of redundancy and changing some biases a little bit but to stop the blinking because it it is bothering me watching it like that I'm going to press this button for the manual override and now it's going to stabilize to some position there and let's do the steering logic next so basically in this situation it shouldn't steer because it sees something on the right so let's change this here to show this sensor pointing to the right and connect it to steering right I'm going to put here min-1 and here a good value was minus 0.9 so at the moment it's not steering to the right but if I'm going to move the car a little bit in front then this point here moved in that reddish region so that means that it does want to steer to the right and it's going to want to steer to the right a lot like that and eventually it's going to crash if we don't teach it also to steer to the left as well and we solved that by filling the remaining part here with blue so it was like this now let's restart this and turn off the manual override and the car is shaking a little bit like like crazy there but it is kind of steering except for this crazy thing that happens because let's see this again because now it's just speeding up and all of a sudden it found out it has to turn so the speed it had was so much that it made it jump to the other side and then it it got confused so our solution to that was to cap the speed to make it go not as fast as it can but at a reasonable speed that the sensor is enough to solve this problem nicely and we did that by doing this adding a new node here for the speed let's change this to see what's going on and I'm going to remove the blue and red so that they are not confusing we have to cut this gray plane here like that and the good value for this was uh 20% so I'm going to put this to minus one and here - 20% like this and this was now connecting here let's just remove this um set this weight to zero turn this one to on to see how it looks like so 20% means that from here and we want to combine the two so not like that this is now doing the or so it's either below this slanted line or below this section we want to combine them so that we get the intersection of the two and we can do that by increasing the bias here by 0.1 like this let's stop also that fidgeting here so we can put minus 0.89 for the turning right and that's going to give a small Gap here for when turning so that it doesn't correct its location all the time like that let me show you here if we put the sensor to the right and turn on the other two colors and we really need to use high resolution here let's take off also these other things um it's still not very visible but um a gap like this appears there even if it's minus 0.89 it's a small Gap that prevents it from fidgeting like that okay let's put this back here because my computer is starting to breathe heavily and turn off this manual override and now the car should be doing very nice nice things here let's have a quick check to see if it's turning properly and if it's visiting all of these regions as well everything looks nice on the right yeah turning properly and not crossing the border so this is exactly what we we had last time but the extra nodes that we have here are to help with the New Logic that we are going to create for the stop marking and for the traffic lights yeah so what should happen now is that the car should stop at the stop marking not just cross there like like it's nothing and the way we do that is the same way that we stop at the road border like we are going to call this Logic for the Stop node here as well so let's try to do it this is minus1 this is minus1 and this is minus 90% like that and now let's investigate if this works we're going to put here the speed like so let's not worry about turning anymore and just keep these gray and green regions there I think it's actually clearer if we keep just the gray region because the green one is always going to be the opposite now so here with the gray region emphasized we look what happens when the car is reaching the stop sign and let's turn on the manual over right here so when it's reaching the stop sign here we still have this front sensor telling us that it's okay to go because it doesn't see the stop sensor but if we change this to be the stop sensor you're going to see that this point is going to be much closer to this side because the STP marking here is closer than the road border so let's see that right so this did move like that but now it doesn't have that region that tells it to to slow down so we have to give that to the end result here and I'm going to create a new node I think it makes it clearer like this and let's just copy whatever happens here on the right for now 0 1 0.1 0.1 and let's disconnect this one so I'm going to put here zero or let's just remove it entirely and connect this one instead so now we do get that thing here but we don't get it anymore for the front sensor because that part is disconnected all right let's still focus on the stop for now and just check if it works so I'm going to remove this manual override and it goes and it stops at the stop marking and it never goes anywhere else from here so this is something we all have to fix let's put on manual override so that this doesn't blink like like crazy and now because we are stopped at the stop we want to move again so looking at this we would like to have some gray region here on the right it's okay to move now after we have stopped at the at the stop so let's add another link from here it's only going to be a vertical region it doesn't depend on the speed or or anything like that when we are really close to the stop marking it's okay to go so we can set here a value of + one like this and a value here of maybe let's try 90% so that this part from here 90% from here gets gray so 90% now this node needs to be connected as well and we are going to do an or here between this one and this one because if this is okay or this is okay it's going to accelerate and you can see this region appearing here it's not capped but actually we don't have to worry about capping for now I think it makes it more complicated so let's remove this line and here instead of having a bias of 0.1 we just have it zero so that this value transfers like that and let's refresh to see what is going on remove manual override so the car is slowing down like this and at some point it continues like that so it's great it crashes there but the logic from here is going to prevent that it's going to do the capping and and everything now it's up to you if you like what happened with the stop I think that it actually took a little bit too long so maybe a value of 85 let's try 87 I think this is now growing too much I don't want it to be a full stop and to wait seemingly so long there so I think this is this is decent how it looks like in the real world a stop should mean a full stop checking if other cars are there and if not you can go but I like things to be a bit faster here and uh speaking of other cars in the demo I have on the website cars give way to other cars on the ride that works with another sensor like this it looks fancy but they just a bunch of rays in front and to the right that check for cars if any of them sees something a sixth input is used to make the car stop we won't be adding more cars here because it's quite messy but we will do the traffic lights logic and that's exactly the same think about it waiting for a car to pass or for a light to change is the same thing really just the sensor is different we'll move on to the traffic lights in a moment but first let's reconnect this side so the car doesn't crash like that I'm going to replace this link from here to this node in the same way nothing changes really so that I can move this one here on this side and connect nicer to this one here we have to do an and so both of these need to happen for it to work properly let's restart and we can see now how this looks like combined with the cap there it's slowing down at the stop and now moving to the right and now as it's moving down let's change this back to the up Arrow which doesn't have that special part there for accelerating again which would just make it crash there but you can see that it's slowing down and it's doing the turn properly so both Logics are combined now in a in a proper way it's good if you stop and play with these values try to inspect things and get your own understanding of this before moving forward but the last part here really is going to be quite easy it's um a trick that we're going to do to make the traffic lights also work with as little effort as possible for that we are going to move the traffic light here in the middle and the stop to the right and the traffic light is just going to connect to this node with minus one here and just like magic it's going to work let's have the car go back to the starting point and see what the traffic light is showing here you saw it was slowing down and wanted to stop but then the traffic light just became green so basically what we want the traffic light to do is like the stop but we don't want it to start all of a sudden while the traffic light is still um red it's going to stop there and wait there until the traffic light turns to Green by itself so that's the logic it's actually a simpler Logic for the traffic lights than it is for the Stop even though you think that these are more complex it's the traffic lights themselves that are complex not the logic that the car is doing car just stops and this is a little bit cheating because I'm assuming that there are no traffic lights and stops next to each other so that's why this logic works but it works in most cases so I'm happy with this now the logic here this or and this and and this and can actually be simplified so we don't need these hidden layers I'll show you let's remove this node and car is wow it's doing some crazy things now and connect these ones here as well so everything connected to this one note there now this is a value of 10 and this is 10 let's make this one 30 30 and this one here let's make it 60 because we're going to put these weights here to 0.1 and 0.1 and if only one of these is on it's enough to turn this one on so we are essentially doing an end here combined with an ore somehow with one single neuron there it's uh it's interesting but it does help to simplify this and if you restart now it doesn't work because I forgot this bias here so now I just want to transfer whatever is here to here and this bias has to be zero okay so now it should work exactly as before but with a simpler structure here we can also get rid of this node so if your goal is to have less hidden layers we can do that like so and connect this one here and this one here the minus and a plus and it's the same logic as before let's just check it and you may wonder why is this moving here well it's because there are other things there are other sensors appearing on some other axis so now this slicing here here it's not anymore a slicing of a cube it's a slicing of a hyper Cube we have not just one more axis the forward sensor we have the forward sensor we have the right sensor we have the stop sensor all of them are there somewhere and here we are just visualizing a slice of that complex hyper Cube and you may see some of these movements happening there from time to time but yeah you can actually get rid also of this Noe and simplify this even more so you could connect all of these directly to that and all of these directly to that and figure out how to do here the not by a combination of of links like that I'm not going to show you that that's going to be your homework but it is possible to Simply y this even more so that you need only one hidden layer here but in my opinion the logic that I'm reading now is actually clearer for for me for a human to understand so I can definitely understand here this kind of compound operation that is happening like this will turn on when it's greater than 0.6 these two will make it 0.6 exactly it needs something else and these are not enough to get anywhere close than that so this logic here is clear it needs to have both of these and one of these so it's actually even simpler than having the more more nodes there and here we just transfer the value here and not to that if I'm going to have all of the connections copied to the other one that's just going to make more math in your in your head but here the not is very easy for me to understand I don't know about you please keep in mind that we are not really solving self-driving here this simulation is very simple and these markings in real we don't recognize them so easily I mean we could like Road markings could send signals and smart cars could pick those up but that would be very costly to maintain so car companies have a different approach they make cars see the world as we do with a bunch of cameras they have good reasons for doing that the roads were made for us so seeing the same way we do should be enough to solve self-driving but that gives another layer of complexity here large neural networks are used to look at the scene like that and figure out that there is a stop sign there not too different from the drawing recognizer we built in the machine learning course where to get the best accuracy we used a monster of a network and there we didn't get to 100 100% no big deal for the drawings but here we kind of have to right luckily stop signs look the same everywhere so never mind anyway the problem is far more complex and we're simplifying a lot is all I'm trying to say we have a nice car now going around and obeying all kind of rules but the car just goes around forever usually they go somewhere so we'll experiment with adding a kind of a compass sensor that tells where the target is Now spoiler alert this Compass approach is not great so I won't insist on it too much I just want to get something that kind of works so you see it's not enough in practice and why we'll need to study path finding algorithms soon speaking of which I'll have to show you how to update the self-driving car code base to load the cars we make in this playground and also how to get it ready for pathf finding about time we get back in the code settle start the playground from the link in the description and you're going to get this scenario that also has a Target the car has three sensors this time because we're going to teach it to turn left or right depending on where the target is so it's going to need to see on the left as well and there is another sensor that is not really displayed anywhere but you can see it here as this node this value here tells the angle that you need to turn to to face the target kind of it's not really an angle it's again a value between zero and one but let's check it out so I'm going to turn on this manual override and hovering this you see there a value of minus 20 before this just means that you have to turn left minus means left and positive values mean you have to turn right so now you have to turn right 25% and then you would be facing this so a zero value means that you're pointing exactly at the Target there and if you're going to go all the way like opposite direction then it's going to say minus 1 but as soon as it's more convenient to turn right you're going to start getting positive values there so one essentially and it's just telling you that now the car should turn right so that you turn to face the target that's what this sensor says and really it's kind of like a compass if you look at a compass then you get the same information from that as well many devices have these kind of magnetic sensors nowadays including smartphones okay so with this setup let's try to make the car head to that Target and I'm going to start by cleaning everything and we're again going to work with some components here like last time so let's move now this up sensor and the speed sensor here and and put the reverse in this top left corner and now turning left turning right and it's going to use these sensors here the left and right sensors and it's also going to use this one but let's set it aside for now the logic will Implement here is going to be really easy I want just the car to move somehow and control the steering based on this other sensor it won't be anything nearly as polished as what we did in the last lessons I just want to give you the idea how you might do such a thing and let's do the simple Logic for moving forward so let's focus here on the speed and the forward sensor and make that slope appear there so minus1 and -1 and here we're going to set minus 90 and this is going to be just the opposite so I'm going to put here + one + one and positive 90% like that now if we restart and turn off the manual override we should see the car going there and stopping like that good I'm going to turn back manual override and move the car down a bit like this and let's Implement steering with a similar logic as as here for that I'm going to put here this sensor and here this sensor so we are focusing on that and let's connect this one here with the Min - one weight and a + one here this time and now here we are going to use + one and minus one so we get this very simple looking structure here and what this does really is now the point is here so the car wants to turn left and it's essentially trying to balance out the ratio between between these so the end result is going to be that the car wants to line up in the center of the road most of the time let's try to see how this works so the car is in the middle of the road and now these are equal so the card doesn't decide to turn at all it's really perfect here how how equal those values are normally you don't get something like that let's turn off this gray and green so that there is no blinking happening and see this again all right and now let's see what happens if we add the bias here so we're going to add a bias maybe 20% here for this one and minus 20% for this one and this is going to mean that the car wants to turn right here and it's going to prefer the right lane actually let's try to restart and now you see the car keeping the right lane there and then at the intersection it's steering left because of its initial position when entering the intersection it's acceleration and actually at the intersection if you look here this point is somehow going a little bit crazy but anyway this kind of configuration means that that the car tends to turn left at the intersection and not right and if you do the opposite like put here minus 20 and here + 20 then the car wants to drive on the left lane and here at the intersection what's going to happen is that the car is going to turn right in this case and it actually does a pretty good job even with this very simple logic here but um fails at the end because it this proportion here tells it to turn left anyway uh this is not important I just wanted to get here some logic that actually works and decides how to turn at the intersection currently based on these biases but we are not going to use those we're going to use the value of the sensor instead so I'm going to put here Zer and zero as we had before and let's move this one here because it's somehow a natural place for it to sit and here I'm going to connect this one with a positive weight let's say 20 and a negative weight here like so and look what happens here I'm going to turn on manual override refresh and now I'm going to to move the car somehow here as Center as I can there it's not fantastic but anyway and you see this falls into the red region now because of how this target is influencing the car has to turn to the left so it needs to now turn right so that it's on the right lane for turn to work so if I'm going to turn off the manual override then the car is going a little bit to the right and then is able to manage that left turn but if you change this link to TGT uncore 1 then there's a Target on the right and uh the car is already doing crazy things there let's see this again so the car turns to be on the left lane and prepares for the turn to the right and then it continues to do some things here and it is affected somehow by this um Target still like you can see that it starts on the left lane but after passing this it moves mostly on the right lane because now the target is on its left so it's still preparing to turn left to head back to the Target but it never gets another intersection to try anyway this does work to make the car turn in different directions according to where the target is but it's not always a good solution like if you go here and put TGT uncore 2 you see the target is now on the left and the car is going left at this intersection because it's where it thinks the target is but it has no idea of the road structure here so this is a big problem we either have to teach this road structure to the car make it part of the neural network somehow a lot of inputs weights biases very very complex stuff or rely on some other solution that gives us the path to follow go up here here and type S is equal to SP for shortest path and you're going to see very different things here this logic is not the one we were playing with now it's actually the one that teaches the car to do the right hand rule that we played with previously and that Target the compass sensor is completely gone so the car is just exploring here this Maze and nothing else but there is this added button here that tells the car to follow a specific path and let's try restarting this and pressing this button and it's going to obey now this Corridor that appeared here so these lines here that you see are going to be new borders that the car is going to try to listen to this Corridor is the shortest path from where we press the button to where the target is and you can actually remove it and press it anytime you want like now and see what the car is doing we just told it go to that Target and it's doing it it's going to that Target no matter um where it is in this maze so to speak I'm going to teach you how to implement this algorithm for for finding the shortest path between two points in a graph but for that we need to take this brain back into our code base from the end of phase two so we're going to need to set things up a little bit there and in the next lesson we're going to implement D algorithm this part is mostly for those who followed along with coding the self driving car simulation and want to get the full connection here if you just want to learn about dice algorithm them you can skip this part I'll post a new code on GitHub and you can take it from there before the next lesson this is the code from the end of phase two and index HTML looks like this there are many cars there in this simulation and they're trying to navigate this more complicated world the best one is always highlighted now the first thing we want to do is bring the logic for the right hand rule here inside of this system and we're going to go to the code and create a new folder here we are going to name it saves and in it I'm going to paste the car we saved in that one lesson mine looks like this and if we open it we can see a lot of different things here it's B basically Json information and it's stored inside of this car info Global variable the only important things here really are the max speed the acceleration and the friction because they have different values in the playground than in our code base but also the brain um let's see where that is not these but that one so the levels the inputs outputs biases and the weights we have to load these from here now and the sensor is also important like the car here has five of these Ray casting sensors we have to use just two of them at the right angle and um with an offset this system doesn't yet support that but it's an easy fix and we have to include the speed of the car as well right so a little bit wor to get this to work exactly like in the playground but it's not very complicated the sensor values are here as well this Ray count Ray length Ray spread and the new offset so let's go back to index HTML here and um load the car just here beneath the world so it's going to be in included like a basic JavaScript file right hand rule. car the extension is just to mark that it's a car file and let's save this and go inside main JS close this and go to the place where we generate the cars so there is this generate cars function down here and we want the car to load the information from inside of that car info I'm going to say here car is equal to and let me copy this from here I'm going to cut it here and this is going to be what we push inside of the cars array but before that we will say car load the car info like this and this load function inside of the car object is what we need to implement so we go here and below the Constructor we just have to type load given some info let's set the brain to info. brain and those proper is Max Speed info Max Speed friction info do friction acceleration info do acceleration and the ray count from the sensor from info sensor Ray count and we have to do this for the ray spread length and offset so I just copied this three more times and this is going to be here rray spread let's copy it here array length let's copy it here and array offset here and this offset is something that the sensor doesn't support at the moment so let's fix that real quick they are all made to be symmetric at the moment so here I'm going to say this array offset is equal to zero in the Constructor by default but it can be assigned something else if we load it from that file and when we cast the Rays this is the important part we need to add that offset to the to each individual array angle so here I will say plus this array offset and we're almost done the last thing we need to do is set the speed as the last input to the neural network so in car JS where we update here we get these offsets as inputs and that's just fine but here we also have to add the speed at the end and we can use Dot conat and it has to be an array that we give here it's going to be an array with just one element so the speed it's going to be a normalized value of the speed so we take this speed and we divide by this Max Speed so that it's going to be between zero and one and if we save this and test we can see here the network working uh it's the same thing that we have in the playground but visualized a little bit differently you can probably recognize here the weights and the biases how they look like and this is the speed value that we have in the right and these two are the front and right sensor respectively and it's doing that right hand rule now but one thing that you might be curious is where are the other cars so we see that all cars are loading this car info they have the same brain and they are actually overlapping in the simulation now here we do have this code that is doing a mutation but it needs to have something in local storage for it to work so if we go back and save this car in local storage and refresh we see that it does now some mutation like that and it actually gets other cars as well not just the one that we originally created many of these can fail depending on the mutation and how much change has happened there but it is what it is and uh you can control the mutation from here if you want but I think a value of 0.1 is quite quite good most of the time now to teach the car to follow a path to do the path finding we will have to create a world that has a Target so we're going to go inside of the world folder and open this index HTML next this one should show empty like this and we're going to start clicking to create a graph that is useful to teach about pathf finding now I'm going to start clicking here and it would be good if you try to follow the same kind of structure if not you can also get this world from GitHub but uh basically I'm going to start here like so and let's zoom out a little bit so here if you haven't followed the previous course we just click to add points and zoom uh out and use the middle Mouse button the one with the scroll wheel to pan around and after doing this we can create now few more points so maybe something like this let's also add a new Point somewhere here on the right and connect this one back to the start now right click to deselect this point and clicking to select this one let's add a link also here and few more of these segments up here like so so this should be complex enough for teaching about pathf finding and the starting point is going to be here so we select the car and we click here at the bottom somewhere where we want the car to be in the very beginning and the target this one is going to be all the way here at the top somewhere here like this now when saving the world it's important to choose some good position to start at and maybe the zoom level this will make some things easier later on when testing so maybe something like this is what we're going to see at first and let's save this I called this path finding and you can find it here pathf finding. world in the SA from the world folder so we have two different saves one for cars and one for worlds here now to load this new one we go to the index HTML from the root not inside of the world so this one and here instead of including this big world we are going to make a copy of it so that I remember that big world exists and include the new one so pathf finding world now if you save this and go to our car simulation and refresh you're actually going to get an error turn on the developer tools and you'll see this kind of errors here and the reason for these errors are here in the way the world was saved so pathf finding world is just adjacent object but big world has world is equal to world. load and inside of that function we have the Json object and it ends with this parenthesis and semicolon you could add these manually to that but I'm going to teach you how to make the world Editor to save in this format because it's more useful I think so inside of the world project we open index HTML and we look for the save function so this save function is here and it just writes here the stringified version of the world we are going to change this a little bit so we are going to say here like like this and now we stringify here like that and then we close this and now save this file go back to the world editor refresh and press save again overwriting the file is just fine and if you look now at pathfinding World it does start like that so we made this so that we don't have to do any manual editing anymore like what we did in phase two and if we go back here and refresh this page we can see now the cars going in this new world but before we check things out there let's also fix the load function inside of the index HTML from world so what we need to do is go to the load and here prepare to cut out the extra strings at the beginning and at the end there so I'm going to go here above this Json data and I'm going to take the Json string first from this file content and I'm going to take the subring like so and starting at the index of the first open parenthesis plus one because I don't want to include that parenthesis and then ending at the last index of the close parenthesis so basically taking the same thing that we had previously otherwise loading worlds will not work with this new format we just need to take this Json string and put it here instead of the original file content now now if you save this go to the world editor and refresh now load for example the big world that we already had there something crazy happens so it's still not perfect if you zoom out and um go down a little bit you can actually see that that other world is there but if you open the console you'll see that there is an error happening and this error is because after this world is loaded it's set into local storage and that part fails this world is too big to fit into local storage so we're going to implement the same solution we used to load the world in the simulator let's remove entirely this load function from here and at the top I'm going to also remove this button for loading the world instead we are going to go at the bottom and add a new script from saves big.or and this is going to load the world in the same way that we had previously in the simulator but we need to take out from here these attempts to read from local storage so all of these things can be removed like so now if you save it works but there is one small problem with the code and that is that the roads here at the roundabout are supposed to be one way and there is a mistake I omitted last time so to fix it we have to go in World JS World JS here and see where the loading is happening so here we load the graph with this function and this is where the problem is so we actually have to open math graph and load and here the segments are just loading what is P1 and P2 but not also the information if this is um oneway Road or not and we can fix that by typing this here and saving refresh and you're going to see that the lines here look different because they are supposed to be oneway segments it's a small thing but really important because I'm going to teach you about about path finding and we will have to take into consideration oneway roads as well now maybe we want to have a Target in this world as well to have a more realistic scenario this is my hometown after all so let's add the target here and figure out a nice zoom and center point maybe here where the car is and save this new world big with Target let's go to the index HTML from the root so not inside the world anymore and add a way to include that one as well I'll comment out the path finding for now and this will be big with Target save and go to the simulation refresh and now it's loading the Big World hopefully with the target let's see yeah you can see it there in the top right okay so this is what we are going to be using to study about pathf finding but we will work with this pathf finding world to begin with it's easier than the big one we just created and uh helpful when explaining how the algorithm works so let's make sure we have it here and also loaded by default in the world editor let's add these other ones for reference save and refresh and seems to work quite well D algorithm for getting the shortest path is really important when I was at Microsoft I used to hear about it every day now it's not strictly speaking an AI algorithm but when added to an intelligent agent like this it makes it more intelligent because it can find its way to Any Given location so I like to count it this one we will implement the algorithm for calculating the shortest path in the graph object which is in World JS and then math and the graph is here now this file is pretty big but it's just a graph implementation it has points and segments or nodes and links or vertices and edges depending on the terminology you want to use I call them points because this is a spatial graph and the nodes do have physical locations there so I prefer this terminology in this case and all these functions are not really important for us all because they are for constructing the graph adding removing points and segments and things like that the only method that we will use is this get segments with point so given a point in the graph we will get all the segments that are connecting to it and below this one we are going to start to implement our algorithm so get shortest path between two points so let's call these start and end and this method is going to create a path in this path array and to debug to see how the algorithm is progressing as we are coding I'm just going to add to it the start and the end and see if we can make these light up in the interface let's return the path save the file and we are going to be modifying the graph editor here to also display the shortest path so let's open this one and uh all the way at the bottom in the display method at the end we are going to calculate the shortest path on the fly so path is equal to get shortest path between two points and I'm just going to take out the graph point of zero the first point and the last point so graph points of this graph points do length minus one this is just for debugging a first version here now let's display these we are going to Loop through every point of the path like this and draw it on this context the canvas associated with this uh graph editor now let's save and um open the index HTML from world not the one here in the root folder the one from World here above this one and it's going to look like this and doesn't seem anything special but if you look closely this Point here is darker than the other ones let me zoom in a little bit you can see it's clearly darker because it's drawn twice we are also drawing it now the start and end locations and this is the last point that was added when constructing this graph so this is also darker but let's mark them a little bit better we can add here for example a size Maybe be 50 and let's give it the blue color as well and when you refresh you're going to see this much clearer here and the end point is now there so the algorithm works by expanding starting to expand here at the start point and growing and growing and growing until eventually reaching the end point let's start this expansion from this start point and see if we can get to this other point here going back to graph JS and here I'm going to get all the segments that are connected to the start point in this case it's only one there but there can be more so seg is equal to this get segments with point the start point and that's this method from here and looping through all of these segments like this I'm just interested in the other point the point that is not the start point so let's take it out like this other point we check if the first point of the segment is equal to the start and in that case we return here P2 otherwise P1 so whatever start is we get the other point and let's add this one to the path and see if it lights up so path push other point like this let's save this and refresh and you're going to see here both of these lighting up now when we have traveled from here to here it's important to keep a note of the distance that was traveled because eventually we want to get to the end point there and we want the shortest possible travel distance so the distance here is just the length of this segment in this case we can go here and say other point distance is equal to the segment length simple as that and now this point knows how far it is from the start point and after this for Loop here we are done with the start point we can mark it as visited we don't need to worry about it anymore all of the points connected to it have been processed now these attributes distance and visited we should set them in the beginning so initialize them to something here let's Loop through all of the points in the graph and the distance we set it to something think really large and I'm going to use the maximum possible value that we can use in JavaScript so Infinity pretty much and they are not visited and now we expanded already from the start point and we need to choose a new point something that hasn't been visited yet and expand from that one so let's prepare for that and Define here current point set it to the start point and let's give it a distance of zero because we are starting from here we haven't traveled at all so makes sense to put this here like that and you can probably tell that this is going to Loop and this current point will update from start point to something else so let's change now everything here where start appears to current point and get a new point to Loop this section and it's going to be an unvisited point so to keep things clear let's define here unvisited and we get that by using the filter method on the points and saying that we want the points that are not visited we always want to expand the one with the shortest distance that we currently have so it's also useful to take out these distances and I'm going to use the map method here remapping each point to just its distance so this is an array of distances from all the points and now to find the unvisited point with the smallest distance we are going to find inside of this unvisited array the point with the distance equal to the minimum distance from this other array here so the minimum function expects many parameters here not an array like that so we have to destructure it and then it finds the point with the smallest distance and this is the thing that Loops so we can try writing here a loop for example and let's just Loop twice so starting at one going to two I ++ and everything here inside of this for Loop like so and if you refresh you're going to see that these two new points are marked there and that's because in the first Loop we visited this one so expanding to process its neighbors and in the second Loop this one expanded as well now the distance that we are calculating here needs to be a cumulative distance we need to add add what was previously here to the new length of the segment that we are processing in this step so here I'm going to type current point do distance plus the segment length because in the second Loop this point here has already a distance and that is important to be to be kept so that this point knows the travel distance from the starting point along this path and this point now knows the travel distance from the starting point along this path if you look closely you're going to see small issue here that this point is visited twice it looks darker because of the transparency and that is happening with the current logic but it makes no sense because we move from here to here and then from here to here so instead of this point having a distance of zero which it originally had it was the starting point now it has double the length of this segment we don't update these distances if we find larger ones so here let's also not add a point to the path in that case so if this distance that we are calculating now this current Point distance plus segment length is less than other point distance then we do these things like so if you save and refresh you're going to see that this point is not emphasized anymore because when expanding like that it doesn't go back here it's counterproductive let's try to expand more let's put here three and see what happens you can see this expanded here this one didn't expand here just yet because we only expand from the point with the smallest travel distance that took to get there so this one expanded and most likely this is going to expand again and again and again it's going to take some time until these distances here cumulate to the same value of this distance here so let's try and see see if that's true put the four okay let's see five growing here again let's see six yeah let's see what happens next I think that this distance starts to be almost the same as this one here seven okay so this did this expand or did this expand let's see aha it looks like actually this one expanded now and these are the two new points it helps to understand if we visualize the previous point it was connected to highlighting this segment let me show you where we're updating this distance here I'm going to type other point previous is equal to the current point and now to visualize these in the graph editor I can check if there is a previous there might not be like for the start point then we create a new segment from the point to the previous and we draw it we can draw it on this context and uh let's emphasize it with uh larger width so that it's obvious save and re refresh and now you can see that this point is connected to this one so at the moment it seems like the shortest path to get to this point is from here but we haven't processed this point yet this is not yet visited that happens in The Next Step so let's put here eight and now you can see this update here and it's showing twice because this point was added into the path array once when visited from here and then it's updated it's added again to the path with uh an update to its previous value here so a little bit confusing visualization here but this is correct now it shows that this is the best way to reach this point and the algorithm goes on and it's going to end up eventually to the end point there so we have to think about the stop condition and it's not very complicated we can just replace this with a while the end is not visited and save refresh and now you can see the full path here but we are not interested in this part from here we just want to know how to get from the start to the end right and the trick is to backtrack from here so from the end point we are going to go back back back back back back back back back all the way to the starting point and that will be the shortest path from the start to the end let's go back to our code and we don't need to create the path early on like this we only do it after this Loop ends so let's remove also adding this other point here it was just for debugging and here we can create the path again like so as an array and let's loop again with this current Point let's set it to be the end point and now while there is a current point because the start point doesn't have a previous So eventually this Loop will terminate we unshift the current Point into the path and this is the same as push but it's putting it in the beginning of the array so at the end we're going to have the path in proper order and update this current point to be the previous save and refresh and you're going to see the path is correctly forming there and this shortest path calculation is being executed on every frame so if you're going to change the structure of the graph like this you will see the path is updating as well and if we're going to add here a new segment then this new shortcut is going to be preferred over this one so it seems to work but one thing that I still want to do is we have added here these attributes to the points of the graph and they are things that relate to this algorithm and we should remove them from there so that other people working on this code seeing that these attributes exist there don't get confused and actually doing this is not really proper like adding attributes like this is not the best practice in the world but um alternative would be to create different structures with the with the metadata and um that would be just too much work so this is a quick quick fix for that and if you refresh you will see that it still works but those segments are not visible there anymore because this um previous was removed now we were able to test what happens on different graphs but we should also be able to test what happens with different start and end points testing properly is really important no matter what you're developing so let's try to add this this feature next into the graph editor we're going to go up where we are adding the event listeners here and I'm going to add the way to Mark a start point and an end point and we'll do this by pressing the S and E keys on the keyboard while hovering uh point so let's add the event listener for key down to the window and the call back function first it checks if we are hovering a point so like that and now if the key that we have pressed is an S key then we are going to set the start to hovered we're also going to copy this below and same thing for the E key and this will be end as the attribute that we are going to store the value in and now the display at the bottom is going to do here the shortest path not between these hardcoded values there but um this start and this end and it makes no sense to do this code when we don't have a start and an end we will get an error so we have to do here this check first now let's save and refresh and you're not going to see anything yet but if I'm going to press for example s while I'm hovering this this point here s and now if I press e after hovering this other one I'm going to get the shortest path from s to e let's say if we put here this as the end point and this as the start point you can see this Loop appearing here so it seems to work let's try also the real world example so going here in index HTML the index from the world again and uh replacing the path finding world with this one so I'm just swapping these here so that I have a reference of what is available save and refresh and you're going to get this world but uh a little bit of a warning here you can't move points here or add new points click on this world or anything you can but if you do that it's going to block the interface because this is a really big world and adding things to it manually um it's a really slow process so safest thing to do is actually go down here where the world is generating inside of the animate function and comment that part out when you refresh now nothing is different but if you try to add new points to this world or make modifications like drag this point around it's not going to regenerate everything around it like the roads and buildings and and whatever and if you accidentally click on something you don't have to wait like I don't know 2 3 minutes for it to process so let's see does this work here let's say that this is going to be the start point I'm going to press s hovering this one and let's put an a destination an end point somewhere here close to the Target so maybe here e and you can see it works um almost almost perfectly the problem is this was the start point it's going up like that and then here at the roundabout this is bad it's going in the opposite way here in Finland you drive on the right side of the road so it should go on this way and it's choosing the other one because it's apparently faster like um less distance to travel this other place looks good but we need to incorporate this these um oneway roads here so these are essentially one-way roads here you can only go this direction on this one and this direction on this one and currently it's going opposite there so we need to do a fix for that and it's not complicated we just have to go back to graph JS in our algorithm here and instead of getting segments with point we have to get segments that are leaving from the point so get segments leaving from point like this and we need a function for that it's not available here in the object so I'm going to copy get segment withd point Bel low and rename this one get segments leaving from point like that and the logic is going to be quite similar so here in the loop if the segment is one way we're going to have to do something special for that but otherwise if the segment allows to go both ways then this is same as before if the segment is one way we have to only push into this segments array if the first point of the segment is equal to point so if segment. P1 equals point then push into the segments this segment that we are looping and if you save refresh and now press here s and let's add also the end location also close to this target as before now the path looks like this it looks proper so this is how you solve the issue about oneway roads how was it did you understand D algorithm by coding it like this or do you need a better explainer let me know in the comments I could also teach more about it if you want like how to make it more efficient or how to use it to compute some other path like the fastest path or how to include traffic information whatever my PhD was on location based applications so you can ask pretty much anything here just having the shortest path is not enough the car needs to obey it somehow now there are several ways you can do it in practice first thing that comes to mind is to cover the path in targets and have the car navigate to the nearest one each time using the compass sensor if these targets are close enough the problem I showed before goes away if you like this idea that's your new homework because I'm going to teach you another approach will'll generate a corridor around the shortest path like that so the car from earlier doing the right hand rule has no choice but to follow it to the destination now get ready to go down the corridor of knowledge we are going to add the functionality to generate a corridor in the world because we need to have information about how wide the roads are and things like that so in World JS and then World JS below this generate method from here and implement the method generate Corridor it's going to be a corridor that links again a start and an end point so start end and We Begin by Computing the shortest path between the start and end using the method we implemented before now from this path we are going to get the segments the consecutive segments that build the path so let's generate those in this array we are going to Loop starting at one and all the way until the length of the path and add to these segments a new segment with the point from IUS one so the previous point and the current Point path of I and to get the corridor we're going to wrap these segments in the envelopes so going here let's say temporary envelopes because we're going to unite them later we remap each segment to an envelope created from this segment using this segment as a skeleton so each segment is remapped to an envelope using the road width and the road roundness like this and let's let's just set the corridor to be these envelopes for now we still have to do something to them but let's debug so that we know these things are working at the moment we're going to draw this Corridor Below in the draw method here and uh before drawing the cars so that we don't draw the corridor above the cars now we have to check if it exists maybe we don't have one and then looping through all the elements these are eventually going to be Road borders so segments I'm going to use this variable name here to Loop through the corridor and um just draw it on the context but both the segments and the envelopes have a draw method so this is going to stay the same now save this file and we have to generate this Corridor somewhere and we'll do that inside of the graph editor so in our event listener we defined before we have to check do we have here a start and and location and if so we tell the world to to generate the corridor between the start and end locations like this now because the world has a way to display its Corridor we don't need this code at the bottom here anymore and uh generating the corridor is done only when we press SN e over some points there so not on every frame it will be displayed on every frame but not regenerated on every frame it's important because generating the corridor is more processor intensive than just figuring out the shortest path let's save this and refresh and now let's press s here and let's go somewhere maybe just here and press e and these are now the envelopes let's try a few more times maybe e here s here e here seems to work but when generating these corridors we really want to use the car's location and the Target location and the car can be not where the point of the graph is it can be anywhere like it's here and the target can be at a house so we need to implement way that the start and end points can be anything along these segments here and we'll make that work by essentially pressing SN e no matter where the mouse is here but then projecting the mouse to that segment and that will be the start location I'll show you we go back to our graph editor here and we don't need to be hovering over anything anymore so we can do this and instead of this hovered here we can say this mouse both for this one and for this one now of course we can't use this point directly so we need to project it on the graph and actually add it to the graph it's going to be a little bit work to do but not horrible you'll see inside of world JS where we are constructing the corridor so in the generate Corridor method up here we have to do something to these points before passing them to the shortest path let's take first the nearest segment to the mouse click so we can use here the get nearest segment from the start and looking inside of all the segments from the graph like that this is going to give us the start segment and we can do the same thing for the end segment but using end here and now we need to project this start Point onto the start segment to get a point that is on the graph somehow we can do a projection using the segments method to project the point like this now this here returns two things it Returns the point and the offset how far from the start of the segment we are and I want just to use the point and I don't want to complicate the syntax too much afterwards so we will destructure this directly to get just a point and we will refer to this point as appr start and we can do the same thing for the end for the Target essentially and now these two new points that are over some segments in the graph we need to add them in the graph as actual points and they need to be connected to other points in the graph so let's do that here taking the graph points and pushing the projected start and doing the same thing with the projected end and now the segments we need to Define as well they will be temporary segments after we do this path finding here we don't need to have them there anymore we should remove them from there we don't want to alter this graph structure forever it's just a temporary thing that we need to do right now so the first one is going to be a segment from the first point of the start segment to the projected start and then the second one will be from the projected start to the second point of the start segment then we're going to have the same things going on for the end segment and the projected end Point like so and the graph segments also need to know about these temporary segments now I'm going to append them at the end using concat like so and very important here when finding the shortest path we don't use start and end anymore instead we're going to use the the projected start and projected end the points that we just added to the graph so that we can have any start and end location on the road now after the shortest path is found we can clean up so let's remove the points that were added and we can do that with the remove Point method from the graph let me copy this for pro n and this is everything we need to do really because when removing a point with this method it removes also all the segments that are connected to it so these temporary segments will be removed as a result of this refresh and now if I press s here and E here you can see it doesn't need to be anymore on a point in the graph it can be anywhere like s here e here and my mouse can be anywhere like I can press s here and it's going to map to this point because that's the nearest point from the projection there is one problem that happens like um if e is here and S is here you can see this thing we need to also connect the projected start and the projected end if both of them are on the same segment if the start seg is equal to the end seg essentially so I'm just going to go here and um if they are the same segment I will add to these temporary segments a new segment connecting the projected points like so now save refresh and if you press s here and E here you can see there's no more that problem and everything from before still works but we still need to unite these envelopes to do a polygon Union a union on their polygons so let's go back to World JS after we have the envelopes here we are going to generate the segments the road borders let's say segments is equal to polygon Union and we are going to take out from the envelopes just their Poes because the union method here Works only with polygons and let's set the corridor to these segments now save and refresh let's press s here and E here and this is now the corridor and it doesn't have those kind of artifacts where the intersections are so it's good but I want to emphasize it a little bit more so down where we are drawing the corridor here we are going to also pass a color let's make it red and thicker like so save re refresh and now when pressing s here and E here it looks better I think the next thing we'll do is close this world project Here and Now focus on the simulation and make the cars in the simulation respect this Corridor let's open main JS and close the other two files so we don't get confused and and where we are creating the road borders here we're going to have to do something different so first we need to find where the target is because the corridor will be between the cars like the best car for example and the target wherever that is so I'm just going to take out the Target by looking through the markings like this and taking the mark marking that is an instance of Target like so and only if the target exists we are going to take the road borders as the uh Corridor otherwise we just do as we do now and everything is a border so if there is a target let's tell the world to generate the corridor between the best car which has an X and y so it's going to work and the Target Center like this and now the road borders are going to be equal to taking the segments the corridor which is the segments but we have to map them like you see below there because the code from phase one was working with this kind of structure not segments that have a P1 and a p too but an array with two points in it and now else we essentially have this thing from here but we remove the const let's close this parenthesis and um above here we need to Define Road borders and I just set them to nothing because they will be populated here now let's save this and open this other index HTML and it works somehow maybe this is the obedient car those were just mutated ones I think this is the one that is that we created previously in the playground I recognize these structures here and it seems to work just fine going all the way from the start location to the Target exactly the way we want I wonder what happens at the end I guess it just starts going back let's see yeah it's just going to go back it doesn't care that the target is there it just follows the corridor until the end then there is no end it's an infinite Loop but this is actually a way to train cars to get them to be faster or different because this code also has mutation and now this one is better than the previous one this is a different network it's slightly different because it's doing this fidgeting here I don't really like the fidgeting but uh maybe it has some other components that are useful so it seems like it is faster than the previous one and if you think about the racing scenario then it is better um so this would win yeah and uh you might be able to increase the number of cars here like this is 100 I think that this code supports even more like maybe 500 yeah seems to be just fine so in the playground the number of cars is more limited because there are a lot of crazy things going on in the background and uh it's using up a lot of uh CPU but wow what is some crazy things are happening here it kind of looks like a race already wow that one is speeding up so somehow it's cornering care Yul but then it was really really speeding up at some point and now it's not doing it anymore I wonder what happened okay now it's speeding up again interesting behavior from some of these cars uh this one crashed but it already passed the finish line so I guess it's it's okay I'm already thinking about the game even though it's not yet a game that's what we're going to do next time now for the Big Challenge open the playground using the link in the description and try to design the logic for a racing car instead of this obedient one we have now you can use as many sensors as you want and try it on as many scenarios as you can I've added several there but if you have the world editor we built in Phase 2 make more of them it's important to test a lot because you don't know what the track will be on race day there's a danger your car becomes really good in some specific cases learning the track basically but it won't do good in general so again remember to test well and when you're happy with the car send it over on Discord deadline is in May I'll be done posting tutorials for the racing game around that time and the live stream will be soon after that detailed rules and info about the prizes are on Discord so check those out and ask if anything is unclear thanks for watching and see you guys

Channel: freeCodeCamp.org

Views: 219,697

Keywords:

Id: VgzHT9quo5c

Length: 224min 17sec (13457 seconds)

Published: Tue Apr 02 2024