CS50 2020 - Artificial Intelligence

[Music] [Music] this is cs50 harvard university's introduction to the intellectual enterprises of computer science over here computer science uh so today we are joined with by cs50's own brian yu this is an unusual week a look at artificial intelligence or ai you might recall that some weeks ago when we first introduced python we talked about writing programs that answered you by saying hello if you said hello or by saying goodbye if you said goodbye but those programs were all implemented with if conditions and elsif's and elsif's and so that really wasn't artificial intelligence if you wanted to have a whole conversation with a computer program like that that would be a huge number of ifs and else ifs just to anticipate all of the things the human might say so we can do better and indeed humans have been doing better in this field of artificial intelligence and i'm so pleased to say that brian's here to lead us along that way now cs50 zone brian you thanks very much welcome everyone to cs50 and today as david alluded to the topic of discussion is artificial intelligence which is all about taking our computers and trying to make them intelligent somehow trying to get them to be able to act in a way that's somewhat rational somewhat human and that could take a number of different forms one example of artificial intelligence for example might be game playing you might be familiar with the game of tic-tac-toe where you've got this 3x3 grid and x and o take turns trying to get three in a row and if for example x started the game and played in the middle square of this grid and then it was o's turn and o played in the top it turns out that at this point in the game x has a very strategic move and a human that's very good at the gain or a computer they can maybe try and figure out how to play this game well might make an intelligent move like playing in the top right corner for example and if x plays in the top right corner well then o is going to need to play in the bottom left corner in order to block x from getting three in a row and here maybe you can see one of a couple possible good moves that x has now but x could play a move like moving in the right square over there and now o is in a tricky position x has one way that could potentially win the game horizontally uh and another way they could potentially win the game vertically and so o is going to have to block one of those ways and maybe they go to block horizontally but then x is going to win no matter what just by playing in that bottom right corner and so a human playing this game could reason through the game thinking about how the opponent might respond and how x would respond in turn and a computer might try to do the same thing for a game as simple as tic-tac-toe or a game even more complex but ai goes beyond just game plan you might see examples like handwriting recognition where nowadays computers are pretty good at taking human handwritten text which can be messy which is different from person to person and somehow figuring out with pretty high accuracy exactly what characters the human was actually writing ai comes up in areas like uh spam detection maybe in your email inbox you have your inbox and your spam email usually gets sorted into a separate folder where you might get a whole bunch of emails coming into your email inbox and somehow your computer is able to figure out with reasonable accuracy which emails are good and which emails are spam now of course the computer is not perfect at this there are false positives where the computer thinks that an email might be spam when it isn't actually and there are false negatives too where a spam email might accidentally end up in your inbox because the computer doesn't catch it but those sorts of false positives and false negatives are the types of things that ai researchers are working on trying to reduce to make these systems more and more accurate you see these kinds of systems show up as well if you've ever been on a video watching website like youtube or netflix for example where you have watched a whole bunch of videos or tv shows or movies and then software behind netflix or behind youtube is able to give you recommendations suggest other videos that you might be interested in watching based on the things that you've watched already and in more recent years ai's gotten pretty good at doing even more sophisticated things things like generating data take a look at these two images for example of people and see if you notice anything strange see if either of these people look strange to you in fact can you figure out which of these two images is not a real person that is to say a computer-generated person that looks like it's human but it's not actually a photo of a real person so you can look at these two images carefully maybe look at the eyes and the hair and the mouth and the nose and see if you can figure out which one it is it turns out neither of these two images are images of real people they're both computer generated not photos of real people but a computer has been trained to generate images of people that look like real people that could fool someone into thinking that it's a real person but it's all just ai generated information so today we'll take a look at all of those ideas how it is that artificial intelligence works and ultimately one of the big takeaways is that ai is not just one algorithm or one principle it's really a collection a category of approaches to problem solving that can all be used to try and solve some of these problems of trying to make computers intelligent so let's begin with one of the first areas where we might want to make our ai work and that's in the area of decision making that very frequently we want to train a computer to be able to make a decision well that decision might be deciding if an email is spam or not spam or deciding whether or not to recommend a video to you or it could be deciding what action to take in a game for example so let's take a simple game maybe you've played a game like this before where you control this paddle along the bottom and you're trying to bounce this ball in order to hit all of the bricks along the top imagine if you were trying to program a computer to be able to play this game strategically to play this game well and the computer observed the ball move that way so the ball's moving that way what should you program the computer to do well it makes logical sense that if the ball is moving to the left then you should program the computer to move the paddle to the left as well to try and catch that ball before it falls through the ground and so you could take that kind of logic and encode it into a computer program using what we might call a decision tree decision trees are just a way of representing a decision that a computer might make by asking questions and depending on the answers to those questions we might ask another question or make some sort of decision so for this game of the paddle we could create a decision tree by asking a question like this we could ask is the ball to the left of the paddle if the answer to that question is yes well then the action we should take is we should move the paddle to the left because the ball is moving left the paddle should move left as well if the answer to the question is no well then we need to maybe ask another question we might ask a question like is the ball to the right of the paddle and if the answer to that question is yes then we'll go ahead and move the paddle to the right and if the answer to the question is no well that means the ball's not to the left of the paddle and it's not to the right of the pedal so we may as well just not move the paddle at all in that case so this again is that decision tree that can allow us to make these choices about what it is that our ai should do in this situation and we could take that decision tree and turn it into a kind of pseudo code something that might look like something you would write in c or in python i might say while the game is ongoing if the ball is to the left of the paddle go ahead and move the paddle to the left else if the ball is to the right of the paddle move the paddle to the right and else if neither of those are true then don't move the paddle at all so one of the advantages of this decision tree is that it translates quite nicely to the conditions that you're familiar with from the world of programming so let's give this a try with something a little more complex let's take the game of tic-tac-toe for example and imagine you were trying to program an ai to strategically play the game of tic-tac-toe what questions might you ask in your decision tree what yes or no questions might you want to ask to create an ai that can play this game well that can play this game strategically and maybe i'll take a volunteer if anyone wants to suggest one possible question i might want to ask of my ai as i'm trying to teach this ai how to play this game any thoughts for questions that i might ask in this situation and let's go to let's see uh let's try basiley if you have an idea um maybe let's ask what is the uh probability of winning given a certain move great and what would winning mean how could you detect what it would mean to win like what are you going to look for on the board um three consecutive um yeah x's or circles great you're looking for some opportunity for like three consecutive x's or three consecutive o's so one of the questions you might ask is checking to see whether or not you could get three in a row maybe on the next turn if you already have two in a row and there's an empty space that could be an opportunity for something we might want to try any other questions that we might want to ask as part of our strategic decision making if we're trying to play this game of tic-tac-toe any thoughts about other things we could try other types of things we should be looking for or asking as we're training our artificial intelligence as we're trying to program it to play this game strategically let's go to uh santiago you have an idea hello ryan if there is any possibility to win the other player for the other player yeah so you might want to check the other player to see if they have some possibility to win and if they have some possibility to win you could try to block them so those are great questions you could ask and we could start to formulate a decision tree for tic-tac-toe based on those questions i might start by asking can i get three in a row on this turn and if the answer is yes well then my action is pretty straightforward i should play in the square that will get me to three in a row if the answer to the question is no well then i might ask santiago's question i might ask can my opponent get three in a row on their next turn and if the answer to that is yes well then we better play in the square to block them from getting three in a row otherwise they're gonna win the game if we don't block them now if the answer to this question is no though if i can't get three in a row on this turn and my opponent can't get three in a row on the next turn well now it's a little bit less clear you could maybe try and come up with additional questions you might ask but these two questions are very clear and the rest are maybe a little bit less obvious but ultimately we don't want to be doing any of this as we're starting to make our ai more and more intelligent as david was alluding to earlier ultimately rather than us have to program every condition into the computer telling it what to do in every situation we'd like for the computer to be smart enough to just figure out on its own analyze on its own all of the possibilities and figure out what move it should make and ultimately we want our computer to make the optimal decision the best possible decision when playing this game and so to do so we can introduce our first algorithm in artificial intelligence that we'll look at today and that algorithm is called minimax minimax is a game playing algorithm that's useful anytime you have two competing players that are trying to play some sort of competitive game like tic-tac-toe but it'll work for other games as well and ultimately as with we've seen all throughout cs50 we need some way of representing this game inside of the computer and ultimately you'll recall way back from the beginning of cs50 we've been trying to take everything and represent it just with numbers and we can do the same thing with games as far as our ai is concerned there are only three possible outcomes of the game that our ai is going to care about either o wins or x wins and x gets three in a row or it's possible that neither side wins that it ends up being a tie and so what we're going to do is take each of these three possible outcomes and assign a number to each of those outcomes we'll say o winning we're just going to say let's call that negative one x winning we'll call that one and a tie well that's somewhere in between the two so we'll go ahead and call that zero nobody wins that game and now each player has a goal some objective that we can quantify using these numbers the x player who we might also call the max player aims to maximize the score what's best for x is a score of one meaning x is going to win but if x can't win well then tying the game is zero that's better than losing the game which is negative one meanwhile the o player we're going to call the min player the minimizing player their goal is to minimize the score negative one is the best outcome for the o player but if negative one can't happen if o can't win well then tying the game is still better than x winning because x winning would mean our minimizing player player o is ultimately going to lose the game and so we can take every board in tic-tac-toe and assign a score to it it's either one or zero or negative one so this board for example this game is over x has already won the game and because x has won the game we're going to give that a value of one one corresponds to x winning negative one to o winning zero for a tie uh so let's now consider this game board this game board isn't over yet but we can still assign a score to it let's assume that it's x's turn what score would we give this board well when we're giving each board a score we're considering what would happen if both players were playing optimally in other words if both players are playing the best possible moves and in this case if x is playing the best possible move well then x is going to play in this square to get three in a row and so this board also has a value of one because if x plays their best move ultimately x is going to win the game that has a value of one so let's take a look at another board maybe one that's a little bit trickier now in this board let's assume it's o's turn so o gets to make a move and then because there's one empty square x will get to make a move after that what value would you give to this board maybe we'll ask for a volunteer remember the possibilities are one if x is going to win negative one if o is going to win and a zero if it's going to be a tie if both players play the best moves and it's o's turn right now what value would you give to this board and why maybe let's go to santiago go ahead um yeah i would say that um the value would be zero because if both players play their best move no one in the end will win because o would block x and then there would be nothing left yeah absolutely correct it's certainly possible that x could win because x has two in a row they could get three in a row but if both players play their best moves well then o is going to block here and then x is going to play in the upper left and it's going to be a tie which means this board is going to have a value of 0. and the way a computer might figure that out is by reasoning it through exactly the same way we did by considering both of the possible options if i want to know the value for this board where it's o's turn well then i'm going to consider both of the possible options o has two choices o could play in the top left or o could block x by playing in the bottom there and the computer would consider both of those possible choices and try and figure out what value each of those boards has so what happens if o plays in the top left well then x is going to play in the bottom and x is going to win the game x winning the game that has a value of one remember x winning that's a value of one which means this board also is going to have a value of one so x wins that's not great so let's consider the other possible choice o could also have blocked x by playing in the bottom here and in that case that's going to lead to x playing in the upper left it's the only other possible option that board has a value of zero it's a tie since nobody won which means this board also has a value of zero and now the minimizing player is going to look at these two options if i play in the top left that's going to be a value of 1 that's not good for me if i play in the bottom that's going to be a value of 0 so that's better and so ultimately we can conclude that this board up top as santiago correctly stated is also going to have a value of zero if both players play their best moves it's ultimately going to be a tie and this is what we might call a game tree where you're exploring all of the possible branches all of the ways this game could go and from here we're two moves away from the end of the game but you could back up and consider what would it look like three moves away from the end of the game and you'd end up with a bigger tree because you now need to explore even more possibilities as you're trying to figure out what the value of any particular game board is and so that's the way the minimax algorithm works consider all the possible moves you could make and then recursively in a sense consider if i make my best move what's my opponent going to do in response and then what could i do in response to that and we could formulate that as a little bit of pseudo code by saying if the player is x well then for all of the possible moves let's go ahead and calculate a score for that board the same way we were just doing by recursively following all the possible moves let's get a score for that board and let's choose the move with the highest score and otherwise if the player is o well then we'll do the same thing for all the possible moves we'll calculate a score for that board but then we'll choose the move with the lowest score so the x player is picking the move that maximizes the score the o player is picking the move that minimizes the score and using this approach you can create an ai that can play a game like tic-tac-toe perfectly it'll never lose the game if you've programmed it correctly and this works not only for tic-tac-toe but for other games as well but do you see any problems with this approach this approach of considering all of my possible moves than all of my opponent's possible responses to those moves and then all of my possible responses to those moves and recursively until we get down to the end of the game any possible problems that might arise let's go to sofia maybe it can take a really long time to like explore all the branches and actually come to a conclusion yeah the amount of time it's going to take to explore all of those possible moves it could be quite large depending on how complex the game is for a game like tic-tac-toe maybe think about how many possible games of tic-tac-toe there are it turns out there are about 255 000 possible games of tic-tac-toe which seems like a lot but computers are pretty fast and computers can make it through all 255 000 of these possible tic-tac-toe games usually in a matter of seconds on most modern computers but if you imagine a game like chess for example known for being a fairly complex game how many possible games of chess are there if there are 255 000 possible games of tic-tac-toe how many possible games of chess are there well it turns out that only after the first four moves if you only look at the first four moves of chess there are 288 billion possible chess games which is a lot for any computer to try to deal with and that's only the first four moves if you consider the entire game of chess nobody knows the answer exactly but people estimate that 10 to the 29 000 is probably a lower bound on the number of possible chess games the actual number is probably higher than that that is far too many for any computer to be able to reasonably get through in any amount of time computers as they exist now are not going to be able to consider all of the possible moves going all the way to the end of the game in order to figure out what the score for any board is which is why a game like chess is much harder for an ai to play than a game like tic-tac-toe but it's not impossible and in fact the best computer chess players are far better than the best human chess players at this point so what could we do what maybe improvement could we make or what change could we make to the algorithm so that our ai can still play a game like chess even though there are so many additional possible moves there are a lot of possible answers but any ideas anyone want to offer a thought or a suggestion as to how we might address this problem there's so many moves to consider too many for a computer to ever reasonably try to attempt any thoughts on something we could try let's go to kurt yeah you have an idea uh yeah just if there was some way along the way um to maybe assign some point values where you could below some threshold maybe you could like discard those paths yeah we want some way to be able to more intelligently not explore all of the paths but somehow like discard some of the paths or not consider some of them or stop ourselves somewhere so that rather than consider all of the 10 to the 29 000 possible games of chess we just consider fewer of them exploring fewer possible games and exploring fewer possible levels and so there are many variants on the minimax algorithm one of the most common being what's called depth limited minimax where in depth limited minimax we consider what's called an evaluation function rather than follow the game until the very end we follow the game some number of moves and chess maybe you're going 15 16 moves but not all the way to the end of the game and then you're just asking a question like all right at this point in the game who seems likely to win even if we're not going to calculate it all the way you can maybe make some judgment by counting up how many pieces of what value each side happens to have and you might come up with other strategies for different games but this now is where we need to start being even more intelligent just thinking about how can we come up with a good evaluation function that's able to take a complex game and estimate who's potentially likely to win so that then is minimax an example of an algorithm that can be used to play games and i'll pause here for any questions about minimax or game playing artificial intelligence before we move on to an entirely different type of artificial intelligence uh yeah sophia go ahead yeah i just have a question like this algorithm is kind of following like how we would think about the game but are there any other algorithms that like don't necessarily look like at step-by-step evaluations yeah there are definitely other algorithms and other approaches in fact right now what we've given an example of is really just an example of an exhaustive search searching for all of the possible different moves and then making a judgment based on that we'll see some examples of more intelligent algorithms or algorithms that can learn later today in fact and we'll take a look at some of those other possibilities and how we might apply them and so this then is really an example of what we might call a search algorithm a search algorithm is just a type of algorithm where we're looking for some solution or some answer it could be looking for the best move to make in a game or you might imagine a search problem more abstractly is something like finding your way through a maze you're trying to get from one point in a maze to another point in a maze and you have to make decisions about which way to turn and how to go and the real world application of this might be something like driving directions if you go into google maps for example and you type where you are and where you're trying to get to google maps pretty quickly can give you a fairly optimal route some path to take which way to turn and when when to make which decision that will get you from one point to another this is an example of a search problem google maps needs to somehow search through all of the possible routes you can take to figure out how to get you to the place that you're going and so we could come up with an algorithm for trying to do that imagine we have a maze like this we're trying to get from point a to point b but of course we have some walls here these grayed out squares or walls that we can't cross through but we'd still like to try and find some path that will get us from point a to point b there are a number of algorithms that you might try and you could think about if you were solving this maze as a person how would you do this how would you decide what decision to make whether to go left or whether to go right but as far as the computer is concerned we need to give it a very precise algorithm and one of the simplest algorithms we might come up with is one called depth first search and the way the depth first search would navigate its way through amazing is to say the ai is going to just follow a path and if ever the ai needs to make a choice it reaches a fork in the road so to speak where it could go left or it could go right it's just going to choose one randomly it doesn't know which way is better so it's going to pick one path and it's going to try it and if ever it hits a dead end it reaches some wall where it can't make any more progress then our ai is just going to back up and it's going to try another path instead so i can show you what that looks like we're starting at point a we're trying to get to point b and the way our ai might work is that we might start by just following one square after another initially we don't have much choice in the matter we haven't reached any branching points or any forks in the road but at this point right here now we have a decision we could go left or we could go right and as far as depth first search otherwise known as dfs is concerned dfs doesn't know which way to go it doesn't know whether to go left it doesn't know whether to go right so we pick one randomly we might choose to go left for example so we're going to go left and we're going to keep following the path until we get to another fork in the road we could go left or right dfs doesn't know which to pick so we're going to choose randomly maybe we'll go left and now we hit a dead end at that point our algorithm knows this was not a good path to take so it's going to back up to the latest decision point and make a different choice so it's going to back up to right here and say all right i tried going left it didn't work let's try going right instead so it's going to go right it'll hit a dead end realize that okay this was not a good path to take it just led to dead ends it didn't go anywhere so instead of going left let's try going right so we'll try this path maybe we'll try going up but hit a dead end so we'll try going to the right here again we hit a decision we're going to make some choice and again we're just going to repeatedly hit dead ends and try new paths until eventually we make our way to the destination and so that is depth first search and as long as they're like a finite number of squares this algorithm is eventually going to find a path to the destination if such a path exists like if there is a way to get from point a to point b then this algorithm will eventually find it because it tries something and if we reach a dead end then we try it again and we keep doing that until eventually we find our way to the destination but this algorithm isn't great there are certainly some problems and maybe room for improvement so maybe i'll ask all of you what problems do you see with this approach maybe reasons why this algorithm might not be ideal areas where we might be able to improve upon this algorithm as it stands right now this again is depth first search let's go to uh joy uh i think it is very time consuming yeah it's very time consuming and it's time consuming in a number of ways in one sense we're exploring a lot of paths that really don't lead anywhere in the sense that you know we did we explored all of this area but ultimately that didn't help us towards trying to find the goal and it's also time consuming in the route that it ultimately finds like if you imagine using google maps for example to navigate our way from one place to another uh it's likely going to be the case that you want to find an efficient route you want like the fastest way to get from where you are to where you want to go and if google maps were to give you like a long and winding route with lots of detours that got you to the destination but took much longer than it needed to that's probably not the best user experience for you and depth first search unfortunately could run into just that situation imagine a maze like this where from point a we could go up or we could go to the right we don't know which to take so depth first search we'll just randomly choose we might choose to go up and maybe we'll go right and we'll find our way to the destination sure depth for search was able to find us a path from a to b but remember that often what we want to do is we want to make the optimal decision this is a correct path for getting from a to b but it's not the best path if we're looking for the shortest path from a to b well that's going to be this one it's going to be this path here that goes from a to b there so we'd like some way to fix that we'd like an algorithm that is able to find us the shortest path not just finding us any path and so for that we can use an algorithm called breadth first search otherwise known as b fs and the way this algorithm works is that instead of whenever we reach a fork in the road pick one until we hit a dead end and then pick the other what breadth first search is going to do is whenever we hit a fork in the road let's take both paths let's take one step on the left and then one step on the right and then another step on the left and another step on the right and effectively just search outwards from the starting point we're going to start from the beginning and look for all of the places we can get to by taking one step away from a and then everywhere we can get to taking two steps away from a and then three steps away from a so on and so forth so we're always looking at the things that are nearby before we look at the things that are further away so the way this might work is that from a we're going to take one step up and then we're going to search one square to the right and then we'll look a second square on this direction and then a second square along the path to the right and then a third square and a third square and we're going to repeat that process effectively alternating between all of the different options that we have until eventually we find this optimal path from a to b and because we're looking for the shorter paths before we look for those longer paths eventually we're going to find that shortest possible path now there are still problems with this too as joy pointed out these algorithms have a tendency to explore a lot of paths unnecessarily let's go back to that original maze for example and consider what would breadth first search do if i presented it with this maze we'll consider what might happen we might go until we reach the first decision point right here we could go left or right and remember depth first search picked one and just followed it until a dead end what breadth first search is going to do is it's going to pick both it's going to go to the left and to the right and then to the left and to the right and basically alternate between all of those until we reach another decision point and then it's going to consider all of those possibilities let's go left or left here and right and consider these possibilities too and it's going to keep exploring any time we reach any decision point it's going to explore this way and that way this way and that way over and over again and ultimately if we repeat this process and consider what breadth first search is looking for sure it's going to find me the shortest and in this case the only possible path to get from a to b but it took such a long time to be able to do so it looked at so many different squares in fact it looked at all of the squares in order to do something as simple as find the shortest path to get from one point to another and so here we can try to start to be a little bit more intelligent what we'd ideally like to do is that when we reach this first decision point go left or go right most humans if you gave them this maze and told you to try to solve it at this decision point you wouldn't just pick randomly you would choose to like go to the right because you know the goal is somewhere to the right and so it's probably the case that we should follow that direction if we're trying to find our way to the end of the maze and so these algorithms we've looked at so far depth first search and breadth first search are examples of what we might call uninformed search they're not using any specialized knowledge about the problem they don't really know anything about the maze they're just basically blindly guessing and hoping that eventually we make our way to the solution but in ai we can improve upon this by using an informed search an inform search uses something that we know about the problem to try and improve the way we search to allow us to search a little bit more effectively and to do so we're often going to make use of what's known as a heuristic some way of estimating how good a particular state is going to be so in this maze for example if i'm trying to get from a to b a heuristic would allow me to answer a question like if i see point c over here and point d over there which one of those points would i rather be at which one is better for trying to find our way to the destination and between c and d breadth first search and depth for search they have no knowledge of which one of those is going to be better as far as it's concerned every squared is just a square but if you're looking at this as a maze most people would intuitively tell you well d is better even if i don't know exactly how to get to the destination if i could be at either c or d i'd probably pick d because it just looks like it's closer to that destination and so the heuristic we could use is one that's often known as the manhattan distance the manhattan distance between any two squares effectively says ignore the walls ignore all the boundaries just consider how many squares like in this case up and to the right would i have to go to get from where i am to the destination and so for c we would go up this many squares and then all the way to the right whereas from d it doesn't matter whether you go up or to the right first but it would go right four squares and then up two d as far as manhattan distance is concerned is much closer to the goal than c and so i would rather be at d than at c and as soon as we have that notion of like between any two choices which one of those would i rather be at that gives us a way to improve upon the random guessing that the other algorithms do remember the depth first search when it reached a fork in the road and it could go left or right it didn't know which to pick so we just randomly picked one now that we have a heuristic like this we can make an informed choice we can say i don't know whether left or right is the correct solution but right is probably going to be better than left because of this heuristic and so i can make those sorts of judgments and so for that we'll take a look at another algorithm known as greedy best first search so in greedy best first search what we're going to do is consider for each of the squares what its heuristic value is according to the manhattan distance in this case so this square for example is one away from the goal so we've labeled it with the number one this square meanwhile is two away from our goal so we're labeling it with the number two this is three away this is four away you'll notice though that we're ignoring any of the walls any of the boundaries because those are going to be difficult to compute we want something efficient this square here even though in order to get to the goal we have to follow this winding path to go around all the boundaries we're still giving it a score of two we want something efficient right now it just looks like it's two away so it's not a perfect heuristic the heuristic is just an estimate but we're using it as an approximation that's going to help us as we go about this search process and so what we'll do is we'll start the same way starting from point a looking at all of the squares we can get to until we reach our first decision point so here we could choose to go left or we could choose to go right and in this case going left according to the heuristic this squared is 13 away from the goal and this one over here is 11 away from the goal so between those two being 11 away from the goal that sounds a whole lot better so greedy best for a search is going to make the choice to go to the right so we'll go 11 we'll keep following until we reach the next decision point here from the eight we have two choices up or to the right as far as the heuristic is concerned both of these are equivalent right going up we're seven squares away going to the right we're six plus one that's a total of seven squares away so here even greedy best first search sometimes needs to pick randomly if both squares have the same heuristic value we just have to make a choice and maybe we make a bad choice and hit a dead end but then we can just try the other one so seven six we're going to keep following here we have another decision point we can go down or we can go to the right but the one to the right has a smaller heuristic value it's a six instead of an eight so we're always going to try and pick the one that looks like it's going to be closer so we'll pick the six we'll go to the five and here we reach one more decision point we can go up which has a heuristic of four and we can go down which has a heuristic of six so even here because the four is the smaller value we're going to go up even though you the human can see we're ultimately going to run into a dead end the computer doesn't know that yet the computer just has to judge it based on the heuristic what it thinks is the best thing to do but eventually it's going to run into that wall realize that it can't make any progress and then go back down here and we'll follow that path until ultimately we arrive at the solution and so here we arrived at the same solution that breadth first search did but we didn't need to consider all of the squares we could ignore all of these off to the left we could ignore all of these down below just by being a little bit smarter about what choice we make instead of just choosing randomly we make an informed choice based on that heuristic a pause here for any questions then about the algorithms we've looked at so far depth first search breadth first search and now an informed algorithm greedy best first search any questions about these algorithms yeah let's go to let's see sophia in the like the i guess decision to go randomly is it possible to like go further and see if they're short like lesser values um like go one more step you certainly could like maybe try and look ahead a couple of steps and look at what might be following it and that might offer numbers that would improve upon the situation but even this algorithm is not perfect in fact when you're just looking at these heuristic values and following the heuristic values sometimes the heuristic values will lead you in a good direction like sometimes in this case it we ultimately we made a couple of wrong turns but ultimately we kind of found our way but that's not always going to be the case there are some cases where because the heuristic is really just an estimate the heuristic can sometimes be wrong take a maze like this for example if you were to follow greedy best first search you would go 16 15 14 13 12 at this point you could decide either the 11 or the 13 and greedy best for a search would look ahead and say you know what this path looks pretty good and it's gonna and none of these values are any bigger than this 12 and so it would find this path that takes you from a to b but even that path isn't actually the optimal path to take the optimal path to take the shortest path between a and b is actually this one here which looks a little counterintuitive and the reason we didn't catch it is because it involves like going to the left first and then winding around and normally according to this heuristic we usually don't want to be going to the left because we know that our goal point b where we're trying to get to it's off to the right so even if you're looking at these heuristic values and looking ahead this algorithm might not be perfect and so we might try to improve upon this algorithm even more and so for one final search algorithm that i'll show you the more most complex one that we've seen yet is an algorithm known as a star search a star search tries to build upon the ideas of greedy best first search which is to say we're going to use a heuristic to try and intelligently make choices but we're going to try and solve the problem we just saw which is the greedy best first search isn't always optimal when it just takes into account the heuristic it's not always going to make the best choice because we might end up choosing a path that's ultimately longer what a star search is going to try to realize is that if we've made it like all the way down here and now we're at a spot where we could be like 11 squares away from the goal that's okay but honestly being 13 squares away from the goal much sooner is probably better so we can't just take into account the heuristic value we should also take into account how far have we gone already if i've already traveled many squares and i find myself pretty close to the goal i would rather have traveled fewer squares and find myself close to the goal and so a star search is going to try to combine these two pieces of information combine the heuristic information that we have seen here and also combine how many steps have i taken so far because that factors into the ultimate optimal solution too and so here's how a star search is going to work it's going to be the exact same idea as before just looking at the heuristic value but instead of only considering the heuristic value we're going to consider taking the heuristic value and adding to it how many steps we've already taken so we take our first step and now we've taken one step and we're 16 squares away from the goal according to the heuristic for a total of 17 and then we take our second step and we're 15 squares away from the goal and we take our third step and we're 14 squares away from the goal and we keep going until we reach a decision point after five squares we're now 12 away from the goal and now we have a choice we could go six squares and be 11 away from the goal or we could go six squares and be 13 away from the goal which is worse so we're gonna make us the decision to go up so for now it doesn't seem like we've done any better we haven't found the optimal solution just yet but notice what will happen eventually if we follow this path for long enough we'll end up at a point where we've made 14 steps and we're estimated to be five away from the goal one two three four five ignoring the walls and now we have a choice we could be six squares away from the goal after having walked 15 steps so 15 plus 6 that's a total of 21 or this option is still available to us we could be six squares away from the start but be 13 away from the goal 6 plus 13 that's a total of 19 and that 19 is a smaller number than this 21 so this 19 is ultimately a better choice as far as a star is concerned so by combining information about how far we've traveled and how far is left to go we can make a more intelligent choice we can say you know what let's go ahead and try this path and a star then will ultimately be able to find its way to get from the starting point ultimately to the goal and this algorithm then relies upon having a good heuristic function and it just so happens that you can prove that if you pick a good heuristic function this algorithm will be optimal it will always find the shortest path from point a to point b and it's not going to get stuck like greedy best for search might on a path that isn't actually optimal and so there are many of these sorts of search algorithms that are designed to try and find intelligent ways to find a solution to some problem to navigate its way through some landscape and so i'll pause here for any questions then about search algorithms in general we've taken a look at what we call classical search navigating our way through a maze for example or finding driving directions as well as what we might call adversarial search where there's an opponent and you're trying to search for the best move in a game of tic-tac-toe or a game of chess brian one question from the chat does the assignment of these heuristic values also take a lot of time for the computer or is that automatic ideally you want to find a heuristic that's going to be efficient to calculate so if the if the time it takes to calculate the heuristic takes a long time then yeah it could be the case that this would be even worse but ideally you want to look for a heuristic that's going to be very quick to calculate and this is why sometimes when we're looking at heuristics we're going to ignore some of the details that make this more complex like when we're calculating these heuristics we're ignoring the walls because having to deal with the walls is going to make it even it's going to make it take longer and longer amounts of time to be able to figure out these values and so by ignoring the walls we can pretty quickly calculate just like x y coordinate wise how many coordinate squares are we away from that destination also i've been labeling all of the squares in this grid with their heuristic value just so you can see what those heuristic values are in practice if our search algorithm never touches a square like it never touches any of these squares it's not going to bother computing heuristic values for them because it's not relevant to the search process so it'll never actually calculate the heuristic values for all of these squares which will save time too i have just shown them to you visually so that you can see all the squares and what numbers would be assigned to them but yeah really great question all right and uh kurt was there another question uh yeah i was just wondering if i mean uh the example that you showed it's uh we're using like a map to actually search through a map so it kind of like maps directly onto the um sort of the concept but um like could this also be used for like textual search or other kinds of searches through like different kinds of problem spaces or no yeah this can be used for a lot of different problem spaces for text they're usually different approaches in the world of like natural language processing but you can use these kinds of search algorithms anytime you have some computer which we usually call like an agent something that's making decisions that has to make certain decisions at certain points even if those decisions aren't like geographic decisions about which way to walk for example as long as it's making some decision that moves it into like a different position you can often apply these types of algorithms in order to in order to solve problems and so these algorithms are not just good for maze solving that can be used in other domains too so ultimately what i want to move on to now is less about just coming up with these algorithms that let the ai figure out exactly what to do right away but looking at a type of artificial intelligence that you've probably heard of called machine learning and machine learning is all about trying to take our ai and trying to get it to learn learn somehow from data or learn from experience in much the same way that humans might learn that we learn from looking at our environment we learn from our surroundings we learn from our experiences and we get something out of those experiences and so one type of machine learning is known as reinforcement learning where you learn from positive or negative rewards you do the computer does something well and it gets a reward the computer does something poorly it gets some sort of punishment and the computer tries to learn from that let's imagine for example a very simple game that we might want our computer to play the computer is going to try to navigate its way through these tiles and it's going to try and get to the goal much similar to what we've seen before but this time the computer doesn't know where the obstacles are let's imagine that there are some obstacles in its way highlighted in red here and if our computer agent this yellow dot here ever hits one of those obstacles the computer loses this game so it hits an obstacle the computer loses but the computer doesn't know where the obstacles are i'm showing them to you visually but the computer has no idea how would the computer try to solve this problem how would you try to solve the problem well all it can do at first is randomly make a choice randomly guess like let's choose to go to the right okay we hit an obstacle that was no good but the computer now can learn from experience as long as it knows once it hits the obstacle that was a bad outcome well now in the future the computer can learn i better not do that anymore rather than go to the right let me try something else i'll try moving up all right that seemed to have worked better let me try making another action and let's try another one maybe i'll go down this time okay that led to an obstacle so the computer learns from that too it learns that was a bad thing to try so let's try something else maybe we try going up this time that two leads to an obstacle that was no good so maybe we'll try going to the right that seemed okay maybe we'll go to the right again all right that led to another obstacle and so over and over it's just making mistakes and we let the computer make those mistakes but every time it makes a mistake the computer is learning something from that it's learning what to do or what not to do the next time it goes through the same process and so in the future it can navigate its way around and eventually with enough trial and error it can find its way to the goal and once it's found its way to the goal it'll remember that too it'll know i now know exactly what sequence of actions will take me from the starting point to the goal and so in the future i can just keep doing that again and again and again so that then is an example of reinforcement learning but even with this example here do you see any potential problems with this approach limitations or downsides to what we've just done here reasons why this ai might not be perfect any thoughts let's go to lex lean it might not be taking the most efficient path yeah absolutely because it's randomly trying eventually it'll find its way to the goal but it might not necessarily find the best path because here there was a faster path there was a faster way to get to the goal and that was to say once you get here go up instead and that will lead to a more efficient path but because our ai is learning like whenever it reaches the goal it learns to do that when it doesn't reach the goal it learns not to do that our ai doesn't have that ability to find that better path in the future and so we could improve upon this and this is what we call a trade-off between exploring and exploiting in artificial intelligence we want our ai to do both of these things we want ai to exploit the knowledge it already has it knows what to do it knows what not to do we want it to use that information but we also want it to explore a little bit we want it to sometimes try some new action that it hasn't tried before because maybe that'll be even better than the things i've already tried in the past so so far our ai in the example just now was only ever exploiting the knowledge it already has but it was never exploring something new and so often when we're training ai we wanted to find some sort of nice balance between the two we wanted to make good choices but occasionally take a risk try something else see if maybe we can find a better solution and so one strategy for doing this is known as the epsilon greedy approach to trying to solve these problems where we assign a value epsilon which is equal to some proportion some probability that our computer is just going to make a random choice and so we might say if we generate a random number and it's less than epsilon which in this case happens like ten percent of the time then let's go ahead and make a random move rather than make an intelligent smart move just pick a move randomly and the rest of the time ninety percent of the time make the move that we know to be the best the one with the highest value we know of so far and this will often result in a nice balance ninety percent of the time our ai will make good choices that it knows to be good choices but ten percent of the time it'll make a new choice something random and maybe it'll stumble across something bad and know to avoid that in the future but maybe it'll come across something better and know to do that better in the future as well and so i'll show you a real life demo of this actually years back the italian institute of technology was working through an example of trying to do something just like this but before we move on i think there's a question from the chat brian someone asks is this related to genetic algorithms this approach genetic algorithms are another type of this type of learning we're actually going to talk about genetic algorithms in just a moment so we'll get there very shortly but yes definitely something related so the italian institute of technology a couple of years back tried to teach a robot how to flip pancakes something that you might have done before or seen someone else do before but in in practice it's difficult to encode in a robot exactly how to do that exactly what moves to make exactly how to move its robotic muscle to be able to flip a pancake effectively so rather than try to program every last movement into the computer they just let the robot learn via reinforcement learning every time it flipped a pancake incorrectly it learned what not to do in the future and every time it flipped a pancake correctly it learned what to do in the future and so i'll go ahead and pull up an example of this now and so what we're going to take a look at here is a artificial pancake and initially the human researcher shows the robot what success looks like the robot needs to know what is success and what is failure so the human demonstrates here is what it looks like to actually do the pancake flipping by demonstrating exactly what motion to make and what it feels like to successfully flip a pancake and then we let the robot try it all right that was his first try not too successful we'll try it again because after three tries okay not great but it's learning every time it does something wrong it learns what not to do in the future he has five tries he has ten tries not great right and now after 15 tries or so 11 tries i guess we'll see what happens okay so not great right it takes a while to learn these kinds of techniques to learn here's 15 tries all right but finally after enough tries once you do enough practice with this here's after 50 tries we now actually have a robot that is learning to successfully perform this task not because human programmers told it exactly how to do so but because it learned from that failure and learned from experience and once it knows how it knows exactly what to do in the future it can continually replicate that behavior over and over again once it's trained to be able to perform that task so that then is how you might take advantage of this idea of reinforcement learning but someone in the chat brought up another approach this type of thing known as genetic algorithms or genetic learning and this is where a lot of machine learning has taken inspiration from the human the human brain how humans learn and how nature works and because nature has managed to evolve intelligent beings uh why couldn't we try to do the same thing in a computer as well and so in nature you have generations of populations that evolve over time where the most fit organisms survive and are able to evolve and mutate and change over time to become better and better at adapting to their environment and so one strategy one approach to trying to build intelligent machines is rather than program one algorithm that is really good at performing a task let's just create a lot of algorithms that are really bad at performing the task because that's much easier to do but we'll let them evolve we'll let them try some tasks and after they do it they won't do very well but we'll take the ones that did the best and replicate them and mutate them and let them try again and then repeat this generation after generation eliminating all of the computer programs that don't perform well but duplicating and mutating the ones that do the pseudo code for which might look a little something like this we're going to start by making an initial generation of candidates randomly where each candidate is some program designed to try and solve some task but rather than program it intelligently the way you've been doing all semester we're just going to program them randomly to just make random choices and we're going to repeat this process until eventually they're successful we're going to for each one of these candidates calculate its fitness calculate how good is it at performing the task that it's designed to do and then we're going to remove the least fit candidates maybe take the half of them that didn't do very well eliminate those but take the ones that did do well and make a new generation from the remaining candidates duplicate them make a few mutations to them randomly just to change things up and to see how things work in order to try to create a better generation and over time as we repeat this process in much the same way that evolution is able to continually produce better and better organisms that are more and more fit we can do the same thing with our algorithms too producing better and better algorithms over time and i have another demo to show you of this here is a simulation of some self-driving cars that have not been programmed how to drive but are starting out driving entirely randomly each of these rectangles you see is one example of a virtual self-driving car each of the little crosshairs you see the little x's are sensors that the car has access to so the car has access to data about how far away in any given direction particular obstacles are and what these cars are trying to learn is how to drive through some sort of environment while not crashing into anything and initially they didn't do very well you noticed they were crashing almost immediately but now we're about six generations in seven generations in they're starting to do a little bit better they're starting to get a sense for like turning when you run into a wall um even when they get to new environments they haven't necessarily seen before they're starting to do a little bit better but of course they're still not great they're still crashing quite frequently sometimes a generation does even worse than the generation before it because it's not always going to be the case that when you make random mutations those mutations are not necessarily going to be better sometimes the mutations actually end up being a little bit worse and so they might move backwards a little bit but over time generation after generation you're hopefully seeing that now 10 generations in these cars in general are doing a whole lot better than they were before and every generation we're taking whichever cars made it the furthest duplicating those eliminating the rest and moving forward is there a question in the chat yeah with regard to the pancakes how did the robot know what was wrong with the previous pancake flips in the case of the cars how does the car know that it aired yeah so whenever you're doing anything with reinforcement learning whether it's the pancakes or these robots here the programmer still needs to tell the ai what does success look like and what does failure look like so the in in the pancake example we trained the the pancake flipper to be able to know uh that when you're flipping this pancake um when it is first taught what does it look like when it's successful so it gets a feel for that and it was probably also told that if the pancake falls that's not a good thing for example and that's not something that the ai should try to do and in this case i assume these cars have been programmed to be told that if you crash into something that is not good the car presumably can detect after it's crashed into something and so it probably also has some sense of like how far it was able to drive such that we could duplicate the ones that did drive the furthest and not the ones that didn't and let's see how this car does oh it was so close to the end didn't quite make it uh maybe give it one more generation and we'll see how this generation does and all right so it's navigating these turns it looks like a whole bunch didn't make it but as long as we get one successful one we can learn from that successful one in the future here's the ending looks like they crashed but and all right it looks like one car was finally able to make it to the end and was able to successfully learn that task of how to navigate uh through this environment of mazes so i'll pause here for questions then about reinforcement learning and how these ideas might have worked um and samuel did you have a question uh yes um so with the genetic algorithm the car one specifically so all the car the cars are learning from each other as each one crashes it's not so much that the cars are learning from each other that then that we're generating new cars based on the cars that were previously successful the idea being that if we run ten cars and see how far they go we take the five that went the furthest we eliminate the other five that didn't make it very far but then we duplicate and repeat the ones that did do well such that in the future hopefully this new generation of cars will make it a little bit further and generation after generation the hope is that we're able to make it a little further along the road until eventually as you saw like 15 generations in we were able to get some cars that could perform the task successfully uh let's go now to josiah the cars specifically learning just from the track i mean when you change the track do we need another i mean do we need to start from zero again or yeah so if there were the track we're different would it have to learn again hopefully not hopefully what the cars were learning was in this case was learning based on sensor data like how far away a wall is in any direction which way to turn and the goal is for this type of approach to generalize and i mean i mean hopefully actual self-driving cars in the real world are not trained this way but you would hope that as they're trained on sample environments that when you put them in a different environment they would be able to generalize their knowledge to be able to apply similar techniques that if in a real world setting you take a car and put it on a road that it's never seen before hopefully it's seen enough examples of things that are like it sensor data that it recognizes that it's able to make an intelligent decision based on that so yeah the hope is and the goal in many cases with ai is to be able to generalize this knowledge beyond just a specific example any other questions uh yeah let's go to yerganj [Music] yeah so can these algorithms ever reach like a bottleneck just like in a real-life evolution there are branches and some branches just reach a bottleneck so is that possible here as well yeah it's definitely possible that there might be that the the algorithms might end up converging to something that seems pretty good and it doesn't seem like any mutations are doing any better but it turns out a totally different algorithm might actually do better than that that's what we will often call a local maximum in the context of artificial intelligence where there is some better approach there is some better algorithm but we can't necessarily find it there are other strategies for trying to deal with that problem but it is definitely a challenge we're thinking about uh and one more question let's go to sophia any question about how the fitness is calculated like in both of these cases um is it like the certain motors that are running or here like the distance yeah in the case of the pancake it was probably like a binary of just like did the pancake end up landing in the pan or not was our way of calculating the fitness of that particular example in the case of the cars we would probably calculate fitness based on distance traveled whichever cars ended up traveling the furthest that would be our definition of fitness and that's really the part the programmer needs to be involved in the programmer needs to decide what does it mean to be most fit what does a bad example look like and using that once the computer knows what success looks like and what failure looks like then it's able to learn from that to be able to do better in the future all right so that was genetic algorithms which is one example of computers that are able to learn from the way that humans learn learning from the way that nature works for example but computer scientists didn't really stop there there are other examples that we can do to add to this as well and so one other example of using this idea of reinforcement learning and using genetic algorithms might be in video recommendations for example where you could have some watch history and the way that an algorithm like youtube or netflix might suggest videos for you to watch is via this reinforcement process that it will just try suggesting videos to you and learn from experience if it suggests a video to you that you like that you click on that you watch all the way through well then the algorithm learns in the future let's recommend more of those sorts of things that's like the car traveling very far so we learn to do more of that in the future if they recommend a video to you and you don't click on it you don't ever watch it well then it's probably not going to recommend that to you in the future and it's probably going to learn from that experience as well and so this is one example of computer science learning from nature learning from the way that humans are another place that's happened too is by looking at the human brain that the human brain consists of neurons and those neurons are connected to one another and they pass information from one to another electrical signals flow through one neuron to another neuron and that's how brains are able to make these very sophisticated kinds of computations and this is what we might call a neural network a collection of these neurons and one place that ai's been looking into especially recently gaining in popularity has been trying to develop artificial neural networks instead of using a biological neuron we just use what we might call an artificial neuron or a unit and you can think of this unit as just storing some value like some electrical signal for example like you might find in the human brain now just in a digital format and these artificial neurons these units can be connected to each other in sort of an input that translates into an output sort of format where this arrow represents some sort of calculation some calculation that is taking this value on the left and transforming it into that value on the right and neural networks are usually not just one input unit and one output unit but they can be more complex you might have a neural network with two different inputs each of which connects to an output or even more sophisticated neural networks like this one where you have multiple layers of these units that are all connected to each other each of these arrows performing some sort of calculation and if you've ever heard the term deep learning this is often what that's referring to this idea of deep neural networks with many layers each of which is performing calculations and ultimately using some linear algebra these neural networks are able to figure out exactly how to tune these calculations to translate input into some output if you give a neural network enough data it can learn from that data it can learn exactly how to set these various different arrows to be able to calculate some task to be able to translate some input into some output so for example that might take the form of handwriting recognition how exactly do we train computers to be able to learn how to recognize handwriting given all the different types of handwriting well one way to try to do that is by using a neural network setting up a network of all of these neurons and all of these connections and then you give to that neural network a whole bunch of data i give to the neural network a whole bunch of already existing handwritten digits each of which is labeled so the computer knows which image corresponds to which digit this data set you're looking at here is a very famous data set of handwritten digits called the mnist data set and given all of this data the computer can start to train the neural network and start to figure out exactly what calculations to run at each layer of the neural network to be able to translate the input into the output to be able to translate like a screenshot of what looks like a handwritten number eight that we all could tell is the number eight but it might be difficult for us to describe exactly how a computer could figure that out but via the neural network by training the neural network on all of the sample data it's able to learn some sort of function that can translate this input this handwritten digit into the output what the actual digit happens to be and these neural networks have proved to be incredibly versatile and we won't get into all the math here because it does get a little bit more complex but it's used all over the place it can be used for email spam detection where if you give the computer a whole bunch of data a whole bunch of emails some of which are labeled spam and some of which are not it can learn a function it can learn exactly how to tune that neural network to be able to predict for any given email whether it's spam or not and really the key factor here to making these networks work is having access to large amounts of data and this is why a lot of companies now are doing a lot of work and trying to get a lot of data and use that data in training their machine learning models it's because the more data that you have the better you can make these algorithms because the better you can tune them to all the different types of inputs there might be to help to make them even more accurate in the future to be able to test these networks to see how well they work and every time you're interacting with websites online you're often helping to provide some of that data every time you go to your email app for example and you mark an email as spam that email app might be learning from that learning okay this type of email that's an example of a spam email and so it learns in the future to do a better job of trying to do that classification and likewise every time an email is marked as spam and you have to tell the computer you know what that wasn't spam the computer is learning from that too it's more data that the computer can use to help to inform its algorithm and the way that it's working and so for one final example we can take a look at how images like this were actually generated how could a computer possibly get an image like this and generate it and make something that looks very much like a real person even though it's not actually a real person well it's done using the exact same technique using a neural network that learns how to translate inputs into outputs by having access to a large amount of data in this case having access to many many photos of real people so the computer can start to get a sense for what a real person looks like it can start to train the network in that way and then rather than build the entire person all at once build it up step by step a computer generating a photo like this this is a pretty complicated task it's pretty difficult to do but you know what's easier is generating that that is 16 pixels it doesn't look like a person at all but this a computer could do pretty readily just generate what seems to be a whole bunch of kind of random looking pixels but then you would train the computer to add a little bit of detail to this image to be able to learn if this were a real image how would i add a little more detail to it to make it look a little bit more like what a person would look like and it does so again by having access to large amounts of data many many photos of people so they can begin to learn from that experience so the computer learns how to take this image and turn it into this for example still doesn't really look like a person but it looks a little more like a person it's got a higher resolution you can maybe make out that there's some hair here and a face here and then the algorithm learns how do you take an image like this an 8 by 8 grid and turn it into a 16 by 16 grid now this i mean it still doesn't look like a person but it looks a little more accurate and over time we can follow these steps one after another adding more and more detail until eventually the computer is able to generate an entire image that really looks like a person that's very hard to distinguish just by this input to output process learning some mapping from inputs to outputs by having access to a whole lot of data so before we wrap up here i'll pause for any final questions about artificial intelligence any of the algorithms we looked at for searching for learning or anything like that any final questions yeah sophia go ahead i had a question about like with the fitness how is all the i guess like data you mentioned they get regenerated like the successful ones but i was wondering like how is that all that data stored of like what the success was exactly because if fitness is just like a score it's hard to know like exactly what decisions it made such that it was successful yeah and that's actually one of the trickier things about machine learning that we can train these machine learning models to be very very good but it's not always immediately apparent to us like what it was doing in order to be successful and this is an active area of research within machine learning including some faculty at harvard that work on this too which is the interpretability of machine learning results like the algorithm becomes very very good at recommending a video to you or generating an image of a person but it's hard for a person to look at that machine learning model and understand how it arrived at that conclusion ultimately in many cases people just throw their hands up and say we don't really care how the algorithm is doing it as long as the algorithm is doing it successfully and eventually produces good results we'll just take the ones that are doing the best and use those even if we don't necessarily understand them exactly how those algorithms are working and that's definitely an area of concern for some people in an area of research for others that are looking into these types of tools and technologies is there a question in the chat maybe indeed a final question from the chat brian every time i choose the parts of a picture that contain traffic lights or crosswalks to prove i am not a robot robot am i training google's driverless car uh maybe not google's necessarily but certainly a lot of times when you're doing that type of thing i mean in one part it's to verify that you are in fact human that is one of the purposes of those things to make sure that robots aren't signing up for websites for example but certainly it could be just giving more examples of labeled data that often times what machine learning models rely on is labeled data where you have like a a digit a handwritten digit and you have a label of this is the number two or this is the number eight and so the computer can then use those digits along with those numbers in order to figure out uh how to learn how to take handwritten digits and convert it uh to individual numbers and so having access to that kind of labeled data ultimately ends up being really really valuable all right so that's it for artificial intelligence for today thank you all so much and we'll see you next time [Music] [Music] [Music] you

Channel: CS50

Views: 90,750

Rating: 4.9785905 out of 5

Keywords: cs50, harvard, computer, science, david, j., malan

Id: eey91kzfOZs

Length: 72min 34sec (4354 seconds)

Published: Thu Dec 31 2020