The following content is
provided under a Creative Commons license. Your support will help
MIT OpenCourseWare continue to offer high quality
educational resources for free. To make a donation, or to
view additional materials from hundreds of MIT courses,
visit MIT OpenCourseWare at ocw.mit.edu. JOHN GUTTAG: Hello, everybody. Some announcements. The last reading assignment of
the semester, at least from us. Course evaluations are still
available through this Friday. But only till noon. Again, I urge you all to do it. And then finally,
for the final exam, we're going to be
giving you some code to study in advance of the exam. And then we will ask
questions about that code on the exam itself. This was described in the
announcement for the exam. And we will be making this
code available later today. Now, I would suggest
that you try and get your heads around it. If you are confused,
that's a good thing to talk about in office hours,
to get some help with it, as opposed to waiting till
20 minutes before the exam and realizing you're confused. All right. I want to pick up where
we left off on Monday. So you may recall that we
were comparing results of KNN and logistic regression
on our Titanic data. And we have this up using 10
80/20 splits for KNN equals 3 and logistic regression
with p equals 0.5. And what I observed is that
logistic regression happened to perform slightly
better, but certainly nothing that you would
choose to write home about. It's a little bit better. That isn't to say it
will always be better. It happens to be here. But the point I
closed with is one of the things we care about when
we use machine learning is not only our ability to make
predictions with the model. But what can we learn by
studying the model itself? Remember, the idea is
that the model is somehow capturing the system
or the process that generated the data. And by studying the model we
can learn something useful. So to do that for
logistic regression, we begin by looking
at the weights of the different variables. And we had this up
in the last slide. The model classes are
"Died" and "Survived." For the label Survived,
we said that if you were in a first-class
cabin, that had a positive impact on
your survival, a pretty strong positive impact. You can't interpret these
weights in and of themselves. If I said it's 1.6, that
really doesn't mean anything. So what you have to look at
is the relative weights, not the absolute weights. And we see that it's a pretty
strong relative weight. A second-class cabin also has a
positive weight, in this case, of 0.46. So it was indicating
you better had a better-than-average
chance of surviving, but much less strong
than a first class. And if you are one of those poor
people in a third-class cabin, well, that had a negative
weight on survival. You were less likely to survive. Age had a very small effect
here, slightly negative. What that meant is the older
you were, the less likely you were to have survived. But it's a very
small negative value. The male gender had a relatively
large negative gender, suggesting that
if you were a male you were more likely to die
than if you were a female. This might be true in
the general population, but it was especially
true on the Titanic. Finally, I warned you that
while what I just went through is something you will read in
lots of papers that use machine learning, you will
hear in lots of talks about people who have
used machine learning. But you should be very wary
when people speak that way. It's not nonsense, but
some cautionary notes. In particular,
there's a big issue because the features are often
correlated with one another. And so you can't interpret the
weights one feature at a time. To get a little bit
technical, there are two major ways people
use logistic regression. They're called L1 and L2. We used an L2. I'll come back to
that in a minute. Because that's the default
in Python, or in [INAUDIBLE]. You can set that parameter at L2
and do that to L1 if you want. I experimented with it. It didn't change the
results that much. But what an L1 regression
is designed to do is to find some weights
and drive them to 0. This is particularly
useful when you have a very high-dimensional
problem relative to the number of examples. And this gets back
to that question we've talked about many
times, of overfitting. If you've got 1,000
variables and 1,000 examples, you're very likely to overfit. L1 is designed to
avoid overfitting by taking many of
those 1,000 variables and just giving them 0 weight. And it does typically
generalize better. But if you have two variables
that are correlated, L1 will drive 1
to 0, and it will look like it's unimportant. But in fact, it
might be important. It's just correlated
with another, which has gotten all the credit. L2, which is what we
did, does the opposite. Is spreads the weight
across the variables. So have a bunch of
correlated variables, it might look like none of
them are very important. Because each of them gets a
small amount of the weight. Again, not so important when
you have four or five variables, is what I'm showing you. But it matters when you
have 100 or 1,000 variables. Let's look at an example. So the cabin classes, the way we
set it up, c1 plus c2 plus c3-- whoops-- is not equal to 0. What is it equal to? I'll fix this right now. What should that have said? What's the invariant here? Well, a person is in
exactly one class. I guess if you're
really rich, maybe you rented two cabins, one
in first and one in second. But probably not. Or if you did, you put your
servants in second or third. But what does this
got to add up to? Yeah? AUDIENCE: 1. JOHN GUTTAG: Has to add up to 1. Thank you. So it adds up to 1. Whoa. Got his attention, at least. So what this tells us is the
values are not independent. Because if c1 is 1, then
c2 and c3 must be 0. Right? And so now we could go
back to the previous slide and ask the question well, is
it that being in first class is protective? Or is it that being in second
or third class is risky? And there's no simple
answer to that. So let's do an experiment. We have these
correlated variables. Suppose we eliminate
c1 altogether. So I did that by changing the
init method of class passenger. Takes the same arguments,
but we'll look at the code. Because it's a little
bit clearer there. So there was the original one. And I'm going to
replace that by this. combine that with
the original one. So what you see is that instead
of having five features, I now have four. I've eliminated the
c1 binary feature. And then the code
is straightforward, that I've just
come through here, and I've just enumerated
the possibilities. So if you're in first
class, then second and third are both 0. Otherwise, one of them is a 1. So my invariant is
gone now, right? It's not the case that
we know that these two things have to add up
to 1, because maybe I'm in the third case. OK, let's go run that
code and see what happens. Well, if you remember, we
see that our accuracy has not really declined much. Pretty much the same
results we got before. But our weights are
really quite different. Now, suddenly, c2 and c3
have large negative weights. We can look at them
side by side here. So you see, not much difference. It actually performs maybe-- well, really no real
difference in performance. But you'll notice that
the weights are really quite different. That now, what had been a strong
positive weight and relatively weak negative weights
is now replaced by two strong negative weights. And age and gender
change just a little bit. So the whole point
here is that we have to be very careful, when
you have correlated features, about over-interpreting
the weights. It is generally pretty
safe to rely on the sign, whether it's
negative or positive. All right, changing
the topic but sticking with logistic regression,
there is this parameter you may recall, p, which
is the probability. And that was the cut-off. And we set it to 0.5,
saying if it estimates the probability of survival
to be 0.5 or higher, then we're going to guess
survived, predict survived. Otherwise, deceased. You can change that. And so I'm going to try two
extreme values, setting p to 0.1 and p to 0.9. Now, what do we think
that's likely to change? Remember, we looked at a
bunch of different attributes. In particular, what
attributes do we think are most likely to change? Anyone who has not answered
a question want to volunteer? I have nothing against
you, it's just I'm trying to spread the wealth. And I don't want to give you
diabetes, with all the candy. All right, you get to go again. AUDIENCE: Sensitivity. JOHN GUTTAG: Pardon? AUDIENCE: The sensitivity
and specificity. JOHN GUTTAG: Sensitivity
and specificity, positive predictive value. Because we're shifting. And we're saying, well, by
changing the probability, we're making a
decision that it's more important to
not miss survivors than it is to, say,
ask gets too high. So let's look at what
happens when we run that. I won't run it for you. But these are the
results we got. So as it happens, 0.9
gave me higher accuracy. But the key thing is, notice
the big difference here. So what is that telling me? Well, it's telling me that
if I predict you're going to survive you probably did. But look what it did
to the sensitivity. It means that most
of the survivors, I'm predicting they died. Why is the accuracy still OK? Well, because most people
died on the boat, on the ship, right? So we would have done
pretty well, you recall, if we just guessed
died for everybody. So it's important to
understand these things. I once did some
work using machine learning for an
insurance company who was trying to set rates. And I asked them what
they wanted to do. And they said they didn't
want to lose money. They didn't want to
insure people who were going to get in accidents. So I was able to
change this p parameter so that it did a great job. The problem was they got to
write almost no policies. Because I could pretty much
guarantee the people I said wouldn't get in an
accident wouldn't. But there were a
whole bunch of people who didn't, who they
wouldn't write policies for. So they ended up not
making any money. It was a bad decision. So we can change the cutoff. That leads to a really
important concept of something called the Receiver
Operating Characteristic. And it's a funny name, having
to do with it originally going back to radio receivers. But we can ignore that. The goal here is
to say, suppose I don't want to make a decision
about where the cutoff is, but I want to look at, in some
sense, all possible cutoffs and look at the shape of it. And that's what this
code is designed to do. So the way it works is I'll
take a training set and a test set, usual thing. I'll build one model. And that's an important thing,
that there's only one model getting built. And then
I'm going to vary p. And I'm going to
call apply model with the same model
and the same test set, but different p's and keep
track of all of those results. I'm then going to plot
a two-dimensional plot. The y-axis will
have sensitivity. And the x-axis will have
one minus specificity. So I am accumulating
a bunch of results. And then I'm going to
produce this curve calling sklearn.metrics.auc,
that's not the curve. AUC stands for Area
Under the Curve. And we'll see why we want to
get that area under the curve. When I run that,
it produces this. So here's the curve,
the blue line. And there's some things
to note about it. Way down at this end
I can have 0, right? I can set it so that I
don't make any predictions. And this is interesting. So at this end it
is saying what? Remember that my x-axis
is not specificity, but 1 minus specificity. So what we see is this corner
is highly sensitive and very unspecific. So I'll get a lot
of false positives. This corner is very specific,
because 1 minus specificity is 0, and very insensitive. So way down at the bottom,
I'm declaring nobody to be positive. And way up here, everybody. Clearly, I don't
want to be at either of these places on
the curve, right? Typically I want to be
somewhere in the middle. And here, we can see, there's
a nice knee in the curve here. We can choose a place. What does this green line
represent, do you think? The green line represents
a random classifier. I flip a coin and I
just classify something positive or negative, depending
on the heads or tails, in this case. So now we can look at an
interesting region, which is this region, the
area between the curve and a random classifier. And that sort of tells me how
much better I am than random. I can look at the whole area,
the area under the curve. And that's this, the area under
the Receiver Operating Curve. In the best of all worlds,
the curve would be 1. That would be a
perfect classifier. In the worst of all
worlds, it would be 0. But it's never 0 because
we don't do worse than 0.5. We hope not to do
worse than random. If so, we just reverse
our predictions. And then we're
better than random. So random is as bad
as you can do, really. And so this is a very
important concept. And it lets us evaluate how good
a classifier is independently of what we choose
to be the cutoff. So when you read the
literature and people say, I have this wonderful method
of making predictions, you'll almost always
see them cite the AUROC. Any questions about
this or about machine learning in general? If so, this would be a
good time to ask them, since I'm about to
totally change the topic. Yes? AUDIENCE: At what
level does AUROC start to be statistically
significant? And how many data
points do you need to also prove that [INAUDIBLE]? JOHN GUTTAG: Right. So the question
is, at what point does the AUROC become
statistically significant? And that is, essentially,
an unanswerable question. Whoops, relay it back. Needed to put more
air under the throw. I look like the
quarterback for the Rams, if you saw them play lately. So if you ask this question
about significance, it will depend upon
a number of things. So you're always asking, is it
significantly better than x? And so the question is,
is it significantly better than random? And you can't just say, for
example, that 0.6 isn't and 0.7 is. Because it depends how
many points you have. If you have a lot
of points, it could be only a tiny bit
better than 0.5 and still be
statistically significant. It may be
uninterestingly better. It may not be significant
in the English sense, but you still get
statistical significance. So that's a problem when
studies have lots of points. In general, it depends
upon the application. For a lot of applications,
you'll see things in the 0.7's being considered pretty useful. And the real question shouldn't
be whether it's significant, but whether it's useful. Can you make useful
decisions based upon it? And the other thing
is, typically, when you're talking about that,
you're selecting some point and really talking about a
region relative to that point. We usually don't really
care what it does out here. Because we hardly ever
operate out there anyway. We're usually somewhere
in the middle. But good question. Yeah? AUDIENCE: Why are we
doing 1 minus specificity? JOHN GUTTAG: Why are we
doing 1 minus specificity instead of specificity? Is that the question? And the answer is,
essentially, so we can do this trick of
computing the area. It gives us this nice curve. This nice, if you
will, concave curve which lets us compute
this area under here nicely if you were to take
specificity and just draw it, it would look different. Obviously, mathematically,
they're, in some sense, the same right. If you have 1 minus x and x, you
can get either from the other. So it really just has
to do with the way people want to
draw this picture. AUDIENCE: [INAUDIBLE]? JOHN GUTTAG: Pardon? AUDIENCE: Does that
not change [INAUDIBLE]? JOHN GUTTAG: Does it not-- AUDIENCE: Doesn't it
change the meaning of what you're [INAUDIBLE]? JOHN GUTTAG: Well, you'd have
to use a different statistic. You couldn't cite the AUROC if
you did specificity directly. Which is why they do 1 minus. The goal is you want to have
this point at 0 and this 0.00 and 1.1. And playing 1 minus
gives you this trick, of anchoring those two points. And so then you get a
curve connecting them, which you can then easily
compare to the random curve. It's just one of
these little tricks that statisticians
like to play to make things easy to visualize and
easy to compute statistics about. It's not a fundamentally
important issue. Anything else? All right, so I told you I
was going to change topics-- finally got one completed-- and I am. And this is a topic I
approach with some reluctance. So you have probably all
heard this expression, that there are three
kinds of lies, lies, damn lies, and statistics. And we've been talking
a lot about statistics. And now I want to spend
the rest of today's lecture and the start of
Wednesday's lecture talking about how to
lie with statistics. So at this point, I usually put
on my "Numbers Never Lie" hat. But do say that numbers never
lie, but liars use numbers. And I hope none of you will
ever go work for a politician and put this
knowledge to bad use. This quote is well known. It's variously
attributed, often, to Mark Twain, the
fellow on the left. He claimed not to
have invented it, but said it was invented
by Benjamin Disraeli. And I prefer to
believe that, since it does seem like something a
Prime Minister would invent. So let's think about this. The issue here is the
way the human mind works and statistics. Darrell Huff, a
well-known statistician who did write a book called
How to Lie with Statistics, says, "if you can't
prove what you want to prove, demonstrate
something else and pretend they are the same thing. In the daze that follows
the collision of statistics with the human
mind, hardly anyone will notice the difference." And indeed, empirically,
he seems to be right. So let's look at some examples. Here's one I like. This is from another famous
statistician called Anscombe. And he invented this thing
called Anscombe's Quartet. I take my hat off now. It's too hot in here. A bunch of numbers,
11 x, y pairs. I know you don't want
to look at the numbers, so here are some
statistics about them. Each of those pairs
has the same mean value for x, the same mean
for y, the same variance for x, the same variance for y. And then I went and I fit a
linear regression model to it. And lo and behold, I got the
same equation for everyone, y equals 0.5x plus 3. So that raises the
question, if we go back, is there really much difference
between these pairs of x and y? Are they really similar? And the answer is,
that's what they look like if you plot them. So even though
statistically they appear to be kind
of the same, they could hardly be more
different, right? Those are not the
same distributions. So there's an
important moral here, which is that
statistics about data is not the same thing
as the data itself. And this seems obvious,
but it's amazing how easy it is to forget it. The number of papers
I've read where I see a bunch of
statistics about the data but don't see the
data is enormous. And it's easy to lose
track of the fact that the statistics don't
tell the whole story. So the answer is the
old Chinese proverb, a picture is worth
a thousand words, I urge you, the first
thing you should do when you get a data set, is plot it. If it's got too many points
to plot all the points, subsample it and
plot of subsample. Use some visualization tool
to look at the data itself. Now, that said,
pictures are wonderful. But you can lie with pictures. So here's an interesting chart. These are grades in
6.0001 by gender. So the males are blue
and the females are pink. Sorry for being such
a traditionalist. And as you can see, the women
did way better than the men. Now, I know for some of you
this is confirmation bias. You say, of course. Others say, impossible, But in
fact, if you look carefully, you'll see that's not what
this chart says at all. Because if you look
at the axis here, you'll see that actually
there's not much difference. Here's what I get if
I plot it from 0 to 5. Yeah, the women did
a little bit better. But that's not a
statistically-significant difference. And by the way, when I plotted
it last year for 6.0002, the blue was about that
much higher than the pink. Don't read much
into either of them. But the trick was
here, I took the y-axis and ran it from 3.9 to 4.05. I cleverly chose my
baseline in such a way to make the difference look
much bigger than it is. Here I did the honest thing
of put the baseline at 0 and run it to 5. Because that's the
range of grades at MIT. And so when you look
at a chart, it's important to keep
in mind that you need to look at the axis
labels and the scales. Let's look at another
chart, just in case you think I'm the only one who
likes to play with graphics. This is a chart from Fox News. And they're arguing here. It's the shocking
statistics that there are 108.6 million
people on welfare, and 101.7 with a full-time job. And you can imagine the rhetoric
that accompanies this chart. This is actually correct. It is true from the
Census Bureau data. Sort of. But notice that
I said you should read the labels on the axes. There is no label here. But you can bet that the
y-intercept is not 0 on this. Because you can see how
small 101.7 looks like. So it makes the difference
look bigger than it is. Now, that's not the only
funny thing about it. I said you should look at
the labels on the x-axis. Well, they've labeled them. But what do these things mean? Well, I looked it
up, and I'll tell you what they actually mean. People on welfare
counts the number of people in a household in
which at least one person is on welfare. So if there is say, two
parents, one is working and one is collecting
welfare and there are four kids, that counts
as six people on welfare. People with a full-time
job, is actually does not count households. So in the same
family, you would have six on the bar on the left, and
one on the bar on the right. Clearly giving a very
different impression. And so again,
pictures can be good. But if you don't dive deep into
them, they really can fool you. Now, before I should
leave this slide, I should say that it's not the
case that you can't believe anything you read on Fox News. Because in fact, the Red Sox
did beat the St. Louis Cardinals 4 to 2 that day. So the moral here is to ask
whether the things being compared are
actually comparable. Or you're really comparing
apples and oranges, as they say. OK, this is probably the
most common statistical sin. It's called GIGO. And perhaps this
picture can make you guess what the
G's stand for GIGO is Garbage In, Garbage Out. So here's a great,
again, quote about it. So Charles Babbage designed
the first digital computer, the first actual
computation engine. He was unable to build it. But hundreds of years
after he died one was built according to his
design, and it actually worked. No electronics, really. So he was a famous person. And he was asked by Parliament
about his machine, which he was asking them to fund. Well, if you put wrong
numbers into the machine, will the machine have right
numbers come out the other end? And of course, he
was a very smart guy. And he was totally baffled. This question
seems so stupid, he couldn't believe anyone
would even ask it. That it was just computation. And the answers you get are
based on the data you put in. If you put in garbage,
you get out garbage. So here is an example
from the 1840s. They did a census in the 1840s. And for those of you who are not
familiar with American history, it was a very contentious
time in the US. The country was divided
between states that had slavery and states that didn't. And that was the dominant
political issue of the day. John Calhoun, who was
Secretary of State and a leader in the Senate,
was from South Carolina and probably the strongest
proponent of slavery. And he used the census data to
say that slavery was actually good for the slaves. Kind of an amazing thought. Basically saying that
this data claimed that freed slaves
were more likely to be insane than enslaved slaves. He was rebutted in the
House by John Quincy Adams, who had formerly been
President of the United States. After he stopped being
President, he ran for Congress. From Braintree, Massachusetts. Actually now called
Quincy, the part he's from, after his family. And he claimed that
atrocious misrepresentations had been made on a subject
of deep importance. He was an abolitionist. So you don't even have to
look at that statistics to know who to believe. Just look at these pictures. Are you going to believe
this nice gentleman from Braintree or this scary
guy from South Carolina? But setting looks aside,
Calhoun eventually admitted that the census
was indeed full of errors. But he said that was fine. Because there were
so many of them that they would balance
each other out and lead to the same conclusion, as
if they were all correct. So he didn't believe in
garbage in, garbage out. He said yeah, it is garbage. But it'll all come
out in the end OK. Well, now we know enough
to ask the question. This isn't totally
brain dead, in that we've already looked
at experiments and said we get
experimental error. And under some circumstances,
you can manage the error. The data isn't garbage. It just has errors. But it's true if the
measurement errors are unbiased and
independent of each other. And almost identically
distributed on either side of the mean, right? That's why we spend
so much time looking at the normal distribution,
and why it's called Gaussian. Because Gauss
said, yes, I know I have errors in my
astronomical measurements. But I believe my errors are
distributed in what we now call a Gaussian curve. And therefore, I can
still work with them and get an accurate
estimate of the values. Now, of course, that
wasn't true here. The errors were not random. They were, in fact,
quite systematic, designed to produce
a certain thing. And the last word was
from another abolitionist who claimed it was the
census that was insane. All right, that's
Garbage In, Garbage Out. The moral here is that
analysis of bad data is worse than no
analysis at all, really. Time and again we see people
doing, actually often, correct statistical
analysis of incorrect data and reaching conclusions. And that's really risky. So before one goes
off and starts using statistical techniques of
the sort we've been discussing, the first question
you have to ask is, is the data itself
worth analyzing? And it often isn't. Now, you could argue that
this is a thing of the past, and no modern politician would
make these kinds of mistakes. I'm not going to
insert a photo here. But I leave it to you to
think which politician's photo you might paste in this frame. All right, onto another
statistical sin. This is a picture of a
World War II fighter plane. I don't know enough about planes
to know what kind of plane it is. Anyone here? There must be an
Aero student who will be able to tell
me what plane this is. Don't they teach you guys
anything in Aero these days? Shame on them. All right. Anyway, it's a plane. That much I know. And it has a propeller. And that's all I can tell
you about the airplane. So this was a photo taken
at a airfield in Britain. And the Allies would
send planes over Germany for bombing runs and fighters
to protect the bombers. And when they came back, the
planes were often damaged. And they would inspect
the damage and say look, there's a lot of flak there. The Germans shot
flak at the planes. And that would be
a part of the plane that maybe we should
reinforce in the future. So when it gets hit by
flak it survives it. It does less damage. So you can analyze where
the Germans were hitting the planes, and you would
add a little extra armor to that part of the plane. What's the flaw in that? Yeah? AUDIENCE: They didn't
look at the planes that actually got shot down. JOHN GUTTAG: Yeah. This is what's called, in
the jargon, survivor bias. S-U-R-V-I-V-O-R. The planes they really
should have been analyzing were the ones that
got shot down. But those were hard to analyze. So they analyzed
the ones they had and drew conclusions,
and perhaps totally the wrong conclusion. Maybe the conclusion they
should have drawn is well, it's OK if you get hit here. Let's reinforce
the other places. I don't know enough to know
what the right answer was. I do know that this was
statistically the wrong thing to be thinking about doing. And this is an issue we have
whenever we do sampling. All statistical techniques
are based upon the assumption that by sampling a
subset of the population we can infer things about
the population as a whole. Everything we've done this
term has been based on that. When we were fitting
curves we were doing that. When we were talking about the
empirical rule and Monte Carlo Simulation, we were
doing that, when we were building models,
with machine learning, we were doing that. And if random
sampling is used, you can make meaningful
mathematical statements about the relation of the
sample to the entire population. And that's why so much
of what we did works. And when we're
doing simulations, that's really easy. When we were choosing
random values of the needles for trying to find
pi, or random value if the roulette wheel spins. We could be pretty sure our
samples were, indeed, random. In the field, it's not so easy. Right? Because some samples
are much more convenient to
acquire than others. It's much easier to acquire a
plane on the field in Britain than a plane on the
ground in France. Convenient sampling,
as it's often called, is not usually random. So you have survivor bias. So I asked you to do
course evaluations. Well, there's
survivor bias there. The people who really hated this
course have already dropped it. And so we won't sample them. That's good for me, at least. But we see that. We see that with grades. The people who are
really struggling, who were most likely
to fail, have probably dropped the course too. That's one of the reasons I
don't think it's fair to say, we're going to have a curve. And we're going to always
fail this fraction, and give A's to this fraction. Because by the end of the term,
we have a lot of survivor bias. The students who are left
are, on average, better than the students who
started the semester. So you need to take
that into account. Another kind of
non-representative sampling or convenience sampling
is opinion polls, in that you have something
there called non-response bias. So I don't know
about you, but I get phone calls asking my
opinion about things. Surveys about
products, whatever. I never answer. I just hang up the phone. I get a zillion emails. Every time I stay in a
hotel, I get an email asking me to rate the hotel. When I fly I get e-mails
from the airline. I don't answer any
of those surveys. But some people do, presumably,
or they wouldn't send them out. But why should they
think that the people who answer the survey
are representative of all the people who stay in
the hotel or all the people who fly on the plane? They're not. They're the kind
of people who maybe have time to answer surveys. And so you get a
non-response bias. And that tends to
distort your results. When samples are not
random and independent, we can still run
statistics on them. We can compute means
and standard deviations. And that's fine. But we can't draw
conclusions using things like the Empirical
Rule or the Central Limit Theorem, Standard Error. Because the basic assumption
underlying all of that is that the samples are
random and independent. This is one of the reasons
why political polls are so unreliable. They compute statistics
using Standard Error, assuming that the samples
are random and independent. But they, for example, get them
mostly by calling landlines. And so they get a bias
towards people who actually answer the phone on a landline. How many of you have a
land line where you live? Not many, right? Mostly you rely on
your cell phones. And so any survey that
depends on landlines is going to leave a lot
of the population out. They'll get a lot of
people of my vintage, not of your vintage. And that gets you in trouble. So the moral here
is always understand how the data was collected, what
the assumptions in the analysis were, and whether
they're satisfied. If these things
are not true, you need to be very
wary of the results. All right, I think
I'll stop here. We'll finish up our
panoply of statistical sins on Wednesday, in the first half. Then we'll do a course wrap-up. Then I'll wish you all
godspeed and a good final. See you Wednesday.
