Finite Math: Markov Chain Example - The Gambler's Ruin

hello and welcome to the next video in my series on finite mathematics now a few points I always got out of the way before we start number one if you're watching this video because you are struggling in a class right now I want you to stay positive and keep your head up if you're watching this it means you've come pretty far in your schooling up at this point and you may have just hit a temporary rough patch I know what the right amount of practice perseverance and hard work you can get through it I have faith in you and so should you number two please feel free to follow me here on YouTube and or on Twitter that way when I upload new videos you will know about them and on the topic of the videos if you like this video please give it the thumbs up that encourages me to keep making them and on the flip side if you think there's something I can do better please leave a constructive comment below and I will actually try to incorporate that into future videos and finally just keep in mind that the examples I work in this videos are geared towards individuals who are relatively new or just reviewing the basic concepts and finite mathematics so I will be going over each example quite slowly and I will be explaining everything we do so you know what's going on and why it's going on so all that being said let's go ahead and dive right in so this is the next video in a series of videos on Markov chains and I'm gonna do a couple of videos in the future just on specific types of problems or specific examples that have some application to the real world so this video is going to Center on a problem that is common to almost every finite math book you're ever going to see in the Markov chain chapter and that is a common problem called the gamblers ruin and we're going to set up a sort of a gambling or betting process and model it in a simple Markov chain okay so we can actually see how this game will progress into the future long term so I'm going to assume a few things I'm going to assume you know the basics of Markov chains I'm not going to re-explain them in this video because I've done several videos before this one explaining what Markov chains are I'm going to assume you know how to do some basic work with matrices and we're not going to do any matrix operations in here but you at least know how need to know how to read them and things of that nature so let's go ahead and look at the problem I do want to give credit for this problem I did adapt it from one of the number of finite math books I have and you can see the authors in the year down there in the lower right it's a really good book if you're looking for a supplementary book I highly recommend picking it up or maybe even looking to adopt the current versions I really like it ok so let's go ahead and talk about our problem so a reluctant gambler is dragged to the riverboat casino by his more freewheeling friends now he's very conservative when it comes to gambling and things like that so he only takes $50 to gamble with and of course it means he leaves his wallet and debit card and credit cards and everything else locked in the trunk of the car so he can't be tempted to go get them so he's only taking $50 into the casino to gamble with now since he doesn't know a whole lot about gambling and I don't either so don't ask he decides to play roulette remember that's the wheel that's spun what the little marble in it it bounces around and then it lands on whatever number and of course are all kinds of bets you can place on the table but he's going to do something pretty simple I'm at each spin he's in a place a $25 bet on red that whatever number comes up is just simply red now if that red occurs he wins $25 but if black comes up he loses his $25 therefore the odds of winning or losing for that matter are 50% or 50/50 kind of like a coin flip so just keep in mind that the odds in casinos are not even and kinda they have to be uneven because if the odds were even the cocina would never make any money so the casino sets up games that are slanted in the casinos favor and the payouts are slanted in the casinos favor that's how they make their money if there is one field of study that you need to know to understand how a casino works believe it or not it's finite math so so anyway so this is the game he's going to play now he sets up some simple rules he will quit playing when he either has zero money left so he goes broke which is fairly obvious he can't play anymore or is up twenty five dollars so if he has 75 dollars he's going to quit that's sort of the self-discipline of going into the casino so under those two conditions he will stop playing so let's model model this process as a Markov chain and examine its long-run behavior or its long-term probabilities so the first thing we'll do we'll set up a transition diagram now in this problem there are four states so remember he's going to come into the casino with $50 so he could lose all the money and go down to zero or he could win go up to $75 and then he stops and he calls it a night and just kind of hangs out and does whatever so there are four states that could happen because he's always betting $25 and the chances of winning or losing are 50% so at the end we have broke game over and on the right we have up 25 dollars I'm going to quit while I'm ahead so those are our two end points now if he's broke he's can't gamble anymore so he's stuck in the state of broke so the probability of being broke once you're broke is 100% so 1 now on the right hand side if he stays true to his self promise once he wins his $75 the chances of him having setting $5.00 are 1 because he stopped playing that that money is in his pocket and it's always going to be there so those are sort of the end states in our transition now at each step if he actually has the money to make a bet the chance of winning or losing is 0.5 so if he has $25.00 and he makes a bet the chance of winning and going to $50 is 0.5 likewise if he has $50 and he bets 25 the chances of going up to $75 are 0.5 because he has a 50% chance of winning on the downside if he has $50 that's 25 and loses that 25 now he's at the $25.00 state and of course if he has 25 dollars that's it then loses it now he's down to zero and then the game is over so those are all the possible paths through our transition diagram with The Associated probabilities again it's very simple it's not that complicated there are only four states and the probabilities of going between the states except for the ends there is 5050 so let's go ahead and set up a matrix using this transition diagram now on the ends we have what are called absorbing States and the way to think about an absorbing state in this diagram a transition diagram or a transition matrix is that once you enter that state there is no leaving it it's kind of like a black hole so once he is broke he's always broke once he wins a $75.00 he quits and therefore he always had the $75 so he never leaves once he's in either state he never leaves it again so we set up a transition matrix and it looks just like this is very similar to what we've been doing in the previous videos on the left hand side we have the state we're coming from and on the top we have the state we're going to so you can see the one there in the top left that just means if he's broke now he'll be broke next time and on the lower right that just means if he had $75 he will always have $75 because he stopped playing and then the point fives just aligned with our transition I agree so their probability of going from the $25 state to the broke state is 0.5 so it's the second row first column now the chance of going from the $25 State to the $50 state is also 0.5 so that's the second row third column element then you go down to the 50 if he starts in the $50 state he has a 50% chance or 0.5 of going down to 25 and he has a 50% chance of point 5 probability of going up to 75 so that is the exact same information in the transition diagram just put in matrix form and by now you should know how to do that it's actually pretty straightforward so let's go ahead and look at some long-run probabilities as we run this transition or this process many steps in the future so this is what we expect in the next state now we take the transition matrix two steps into the future so how do we interpret this let's look at something that says new to this matrix let's look at the second row second column that 0.25 so how do we interpret that well that means if our gambler starts with $25 when he walks into the casino and plays this game the probability of having the $25 it's still in his pocket two spins of the game from now is 0.25 or 25% now let's go down to the next row that is if he walks in the casino with $50 at state 3 so let's go ahead and look at the third row first column so he walks in the casino with $50 the probability that he will be broke 2 rolls from now is 0.25 so think about how that would play out actually in the game he walks in the casino with $50 any places of bet and he loses well that's the first step so now he is $25 then he places that $25 then he loses again so now he's broke so the probability of that occurring is 0.25 or 25% and that's how you actually interpret that from the matrix let's go ahead and run this further so this is what we would expect 10 spins into the game so what would this how we interpret this right here well after 10 spins of the wheel if he walks in with $25 the probability that he will be broke 10 spins from now is point 6 6 6 6 6 7 ok now if we go down to the next row if he comes in with $50 the chance of him being broke after 10 spins is 0.33 3 3 3 3 3 now you can see that those are actually fractions that we'll talk about here in a minute so that's just the beginning where he starts and 10 spins of the wheel we would expect him to either be broke or to have won $75 it just depends on what he starts with and that's sort of the point of this problem so let's go ahead and do it 25 spins into the future now look at our probabilities and this should be fairly obvious what's happening here so if he walks in with $25 that's state 2 so our second row the probability that he will be broke after 25 spins of the roulette wheel is 0.667 and as you can guess that's 2/3 now the probability that he has $75 after 25 spins if he comes in with 25 is 0.33 3 as you can see there the probability of them having $25 or $50 F for 25 spins is 0 okay let's go ahead and look at the next if he comes in with $50 the chance of being broke 25 spins of the wheel end of the feud in the future is 0.33 3 or 1/3 now also if he walks in with $50 the chance of having 75 25 spins in the future is 0.667 or 2/3 so you can see how this matrix develops over time over the long run and that's what we're trying to figure out in this problem so it would appear that our matrix looks like this as though as we run the matrix into the future towards basically infinity we have fractions in certain states so state 2 to state 0 is 2/3 state 2 to state 4 is 1/3 state 3 to state 0 is 1/3 state 3 to state 4 is 2/3 of course we still have our absorb or absorbing rows on the top and bottom so what can we say about this this game now the best thing we can do here is we can't really say that there is one specific answer to this gambling problem all we can say is that whatever is in the first column is what it is and then whatever is in the fourth column is just 1 minus that and you'll know by now that each row has to add up to 1 so the best general form we can do for this problem is this long run or steady state matrix over here on the right whatever is in column 1 that is X so that means whatever is in column 4 has to be 1 minus X now let's look at two examples and or two starting points and then we're done so let's say he walks in with $50.00 like he planned on doing so remember our gamblers are 250 dollars which is state 3 he starts in state 3 therefore we can set up an initial state vector that looks like this okay so we put a 1 and state 3 denoting that he is starting in state 3 which is in this case $50 so now we can take that initial state and then multiply it by our transition matrix and then raise the transition matrix to some crazy exponent so I did I did 50 for this example now we know that effort 25 it it steadies out but I just use 50 because I don't know like 50 so if we do this in our calculator this is what we get we get one-third and two-thirds so what does that what does that tell us therefore if this gambler starts with $50 he has a one-third chance of going broke and a two-thirds chance of coming out ahead $25.00 meaning $75 and we did that just with a simple transition diagram with transition matrix we set up an initial state vector that says he starts in state three that's what the 1 means and then we did some simple multiplication so you're welcome to Cocina with $50 and do this exact game you have a one-third chance of going broke and a 1 a two-thirds chance of coming out ahead so that's actually those are actually pretty good odds to tell you the truth of coming out ahead now let's say he comes in with $25 but does the same game so our gambler starts with $25 he would begin state to remember $25 to state 2 therefore we can set up an initial state vector it looks like this 0 1 0 0 and again the 1 means he is starting in state 2 so do the exact same process we'll take our initial state vector multiply that by our transition matrix raised to some crazy power in this case 50 and this is what we get now we have to birds in the first spot and one third in the fourth spot so when the gambler walks in with $25 now he has a two-thirds chance of going bust and only a one-third chance of coming out ahead $25 okay and this is she kind of make intuitive sense if he walks in with $50 okay the chance of sort of bouncing around and bouncing up to $75 is greater because of where he started now with the $25.00 it's kind of the opposite because he only starts with 25 the chance that he goes bust very quickly is you know it's pretty high so the only thing that changed here is what he started the game with okay but the transitions are the same the odds are fixed in the game the only thing we did was change what the gambler started with okay so that wraps up our example of the gamblers ruin and again it's just a simple way to set up a Markov process or a Markov chain using simple probabilities using example that we can actually picture in our minds actually happening so again it's just a one of several examples I plan on doing to actually demonstrate these Markov chains sort of in a real-world context so you can see actually how they work you know in your mind so just a few reminders before we sign off number one just remember if you're having problems in a class right now I want you to stay positive you are very smart and very talented and if you're having problems it's just temporary with a little bit of hard work patience and practice you can get through it and again I do have faith in you and so should you number two please follow me here on YouTube and on Twitter that way when I upload new videos you know about them also if you like the video give it a thumbs up because it does encourage me to keep going or if you think I can do something better please leave a constructive comment so all that said the most important thing is for me to wish you the best of luck in your studies and to look forward to seeing you again next time you

Channel: Brandon Foltz

Views: 176,471

Keywords: markov chain, markov system, gamblers ruin, finite math, finite mathematics, brandon foltz, brandon c foltz, brandon c. foltz

Id: afIhgiHVnj0

Length: 20min 25sec (1225 seconds)

Published: Fri Nov 16 2012