Today we're going to talk about something emotional, which is what happens when you go to the doctor, have a basic screening for a test and you get that letter in the mail saying you have tested positive for this disease. If you get that letter, if you're anything like me your stomach falls into your feet and you start to worry; a question that mathematics can help us answer is how worried do you really need to be? Maybe very worried, maybe not at all. We start with the question about what disease we're talking about? Is it common like the flu or is it a rare disease, usually the kinds were really scared of, some kind of cancer or some unusual genetic disease? Maybe something auto-immune, affects a very small fraction of the population. So if it's a rare disease and you get a bad result well the second thing you worry about after how rare is the disease is how accurate is the test? There are two possibilities for you: either you're sick or you're healthy. And there's two possibilities for the test, either says you're sick or it says you're healthy. So you may be sick or you may be healthy; and the test may say that you're sick or it may say that you're healthy. So if you're sick and the test says you're sick - well for one thing it's getting it right. I think right should
maybe be- right in green. This is a true positive. And this is a bad state, because you're sick but maybe at least you've got the disease caught so now you can begin to treat it. Another possibility is that the test says that you're healthy and in fact you are healthy - so this is a true negative.
- (Brady: Again that's good news) That's the best news, right? But there are two ways in which the test can make a mistake; it can make a mistake by telling you if you- that you're healthy when you're sick - it's called a false negative. And there's another case that can happen which is you're healthy and you're told you're sick, and that thing is called a false positive. So what we've got here is we've got the correct diagnoses in green and we've got the false diagnoses, the mistakes that the test can make, we've got them in red. So I want to talk about when a letter comes in the mail after you've had this screening for, maybe a rare disease, something that's pretty scary but very remote, and you get a positive result in the mail. So you've tested positive for x, I won't name it too upsetting, and you wonder what to do; you think well I should call the undertaker, I've got to let my relatives know, this is terrifying I'm now going to have a dreadful treatment followed by an untimely death. This is, I think, a very normal reaction to such a letter. And the question is, do you need to have your day ruined? Could be that the test is correct and you're sick and that's very very sad, and it could be that the test made a mistake. And so the question that you might ask yourself is, if you were so inclined mathematically inclined, you might say what is the probability that I'm sick given that I had a positive test? Maybe that's very high, maybe 99%, and then you really do have to worry. Maybe 0.01% - then you don't need to be worried. How big is that probability, right? You've got the bad news, so that's a given for you now, and the question is what's the probability that you're sick given this information? Yesterday when you didn't have this information you weren't worrying about the disease at all, right? There's a remarkable answer to that question which dates back to the 1700s. So- and more surprisingly than it's so old is that it was brought to us by a Presbyterian minister named Thomas Bayes; maybe you've heard of Bayes' rule. It has a way of interchanging what you're trying to find the probability of and the given information. So I'm going to write Bayes' rule, for this case, and say that the probability that you're sick given that you tested positive is equal to the probability that you have a positive test given that you're sick - well that's not quite right I just flipped the things over - but if you take this inverted probability and scale it properly you do get an equality. So what do we do here? We say that's times the probability that you're sick, divided by the probability that you got a positive test. And this is the
famous Bayes' rule that- this simple formula was brought to us, you may have learned it in a basic statistics course or a basic math course, computer science, data science - it's a very important part of statistics. Rather simple to derive in formulas and I think hard to absorb in terms of its broad implications. We want to figure out the probability that we're sick given that we got this lousy letter, and now we've said that that's equal to something that's got three things we've got to fill in. So to start Brady with what you asked about, the probability that you're sick, let's suppose that it's a rare disease, one in a thousand, so I'll say the probability that you're sick is 0.001. And let's suppose that the test is really really accurate; that when you're sick it catches it. So just to make this totally simple I'm going to say, suppose the positive test given that you're sick, if you were sick, it would catch it 100% of the time. Now most tests aren't quite that good but a lot of them are really good, like 0.99 or 0.98, so my simplification here is not so far off. And then what we've got to do is stick something in the bottom here, what's the probability that you're going to get a positive test? Probability of a positive test has got two parts to it. One of them is the probability that you've got a positive test given that you're sick, and one of them is that you've got a positive test given that you're healthy, that's those false positives. Now this isn't quite right the way I've written it, because it- here I've just added up the positive tests given that you're sick and the positive tests given that you're healthy. But I need to scale those numbers by who they apply to. Now we've said that the probability of a positive test given that you're sick is 1, but we know that the probability that you're sick is very small so that's going to make only the tiniest tiniest contribution. So we have a 1 here for the positive test that you're sick, 0.001 for the probability that you're sick. Then there's the positive test given that you're healthy times the probability that you're healthy, that's a very
big number, 0.99. Now what about this positive test given that you're healthy? We go over here, that's the red, that's our false positives. And let's suppose this is a pretty good test; that it's going to tell nine healthy people that they're healthy and one healthy person that he's sick out of every ten. So it might catch all of the disease but it also might tell a few healthy people that they're sick as well, so this goes back over to this box - there's two ways to make a mistake. It can still make mistakes by telling healthy people that they're sick. And I'm worried that in this particular test that that happens in one out of ten. So the probability of a positive test; well this number because of that rarity of the disease is effectively zero, and this number over here is effectively 0.1, so this is very close to 0.1. So I'm going to put that 0.1 here and say that now I've computed that the probability that you're sick given a positive test is effectively 1 times 1 in a thousand divided by 1 over 10, and this is
1 in 100. So how worried should you be when you get that bad letter? Well, if the test is virtually all the time right when you're sick, if it makes a one-in-ten rate of false positives and it's relatively rare, affects only one in a thousand in the overall population that's called the base rate, then your chances of being sick or one in a hundred. Now seems a little premature to call the the funeral home I would think. [Extras] ...applications. So we can do the exact same analysis in the case of somebody who's trying to identify criminals in lineups, right? Criminals are a small part of our population, at least we hope so, I think they seem to be.
- You have forced Monty's hand. He knows where the one remaining goat is, he knows it! And he has to show it to you.
