The following content is
provided under a Creative Commons license. Your support will help
MIT OpenCourseWare continue to offer high-quality
educational resources for free. To make a donation, or to
view additional materials from hundreds of MIT courses,
visit MIT OpenCourseWare at ocw.mit.edu. JOHN GUTTAG: I'm
a little reluctant to say good afternoon,
given the weather, but I'll say it anyway. I guess now we all do know
that we live in Boston. And I should say,
I hope none of you were affected too much by the
fire yesterday in Cambridge, but that seems to have been
a pretty disastrous event for some. Anyway, here's the reading. This is a chapter in
the book on clustering, a topic that Professor
Grimson introduced last week. And I'm going to try and finish
up with respect to this course today, though not with
respect to everything there is to know
about clustering. Quickly just reviewing
where we were. We're in the unit of a
course on machine learning, and we always follow
the same paradigm. We observe some set
of examples, which we call the training data. We try and infer something
about the process that created those examples. And then we use inference
techniques, different kinds of techniques, to
make predictions about previously unseen data. We call that the test data. As Professor Grimson said, you
can think of two broad classes. Supervised, where we have a
set of examples and some label associated with the example-- Democrat, Republican,
smart, dumb, whatever you want to
associate with them-- and then we try and
infer the labels. Or unsupervised, where we're
given a set of feature vectors without labels, and
then we attempt to group them into natural clusters. That's going to be
today's topic, clustering. So clustering is an
optimization problem. As we'll see later,
supervised machine learning is also an optimization problem. Clustering's a
rather simple one. We're going to start first
with the notion of variability. So this little c is
a single cluster, and we're going to talk about
the variability in that cluster of the sum of the distance
between the mean of the cluster and each example in the cluster. And then we square it. OK? Pretty straightforward. For the moment,
we can just assume that we're using Euclidean
distance as our distance metric. Minkowski with p equals two. So variability should look
pretty similar to something we've seen before, right? It's not quite variance,
right, but it's very close. In a minute, we'll look
at why it's different. And then we can look
at the dissimilarity of a set of clusters, a group
of clusters, which I'm writing as capital C, and
that's just the sum of all the variabilities. Now, if I had
divided variability by the size of the
cluster, what would I have? Something we've seen before. What would that be? Somebody? Isn't that just the variance? So the question is, why
am I not doing that? If up til now, we always
wanted to talk about variance, why suddenly am I not doing it? Why do I define this
notion of variability instead of good old variance? Any thoughts? What am I accomplishing
by not dividing by the size of the cluster? Or what would happen
if I did divide by the size of the cluster? Yes. AUDIENCE: You normalize it? JOHN GUTTAG: Absolutely. I'd normalize it. That's exactly what
it would be doing. And what might be good or
bad about normalizing it? What does it essentially
mean to normalize? It means that the
penalty for a big cluster with a lot of variance
in it is no higher than the penalty of
a tiny little cluster with a lot of variance in it. By not normalizing,
what I'm saying is I want to penalize big,
highly-diverse clusters more than small,
highly-diverse clusters. OK? And if you think about it,
that probably makes sense. Big and bad is worse
than small and bad. All right, so now we define
the objective function. And can we say that the
optimization problem we want to solve by clustering
is simply finding a capital C that minimizes dissimilarity? Is that a reasonable definition? Well, hint-- no. What foolish thing could
we do that would optimize that objective function? Yeah. AUDIENCE: You could
have the same number of clusters as points? JOHN GUTTAG: Yeah. I can have the same
number of clusters as points, assign each point
to its own cluster, whoops. Ooh, almost a relay. The dissimilarity of
each cluster would be 0. The variability would be 0, so
the dissimilarity would be 0, and I just solved the problem. Well, that's clearly not
a very useful thing to do. So, well, what do you think
we do to get around that? Yeah. AUDIENCE: We apply a constraint? JOHN GUTTAG: We
apply a constraint. Exactly. And so we have to
pick some constraint. What would be a suitable
constraint, for example? Well, maybe we'd
say, OK, the clusters have to have some minimum
distance between them. Or-- and this is the constraint
we'll be using today-- we could constrain the
number of clusters. Say, all right, I only want
to have at most five clusters. Do the best you can to
minimize dissimilarity, but you're not allowed to
use more than five clusters. That's the most
common constraint that gets placed in the problem. All right, we're going to
look at two algorithms. Maybe I should say two
methods, because there are multiple implementations
of these methods. The first is called
hierarchical clustering, and the second is
called k-means. There should be an S
on the word mean there. Sorry about that. All right, let's look at
hierarchical clustering first. It's a strange algorithm. We start by assigning
each item, each example, to its own cluster. So this is the trivial solution
we talked about before. So if you have N items,
you now have N clusters, each containing just one item. In the next step, we find
the two most similar clusters we have and merge them
into a single cluster, so that now instead
of N clusters, we have N minus 1 clusters. And we continue this
process until all items are clustered into a
single cluster of size N. Now of course,
that's kind of silly, because if all I
wanted to put them all it in is in
a single cluster, I don't need to iterate. I just go wham, right? But what's interesting about
hierarchical clustering is you stop it, typically,
somewhere along the way. You produce something
called a [? dendogram. ?] Let me write that down. At each step here, it shows you
what you've merged thus far. We'll see an example
of that shortly. And then you can have
some stopping criteria. We'll talk about that. This is called
agglomerative hierarchical clustering because we start
with a bunch of things and we agglomerate them. That is to say, we
put them together. All right? Make sense? Well, there's a catch. What do we mean by distance? And there are multiple plausible
definitions of distance, and you would get a
different answer depending upon which measure you used. These are called
linkage metrics. The most common one used
is probably single-linkage, and that says the distance
between a pair of clusters is equal to the shortest
distance from any member of one cluster to any member
of the other cluster. So if I have two
clusters, here and here, and they have bunches
of points in them, single-linkage distance
would say, well, let's use these two points
which are the closest, and the distance
between these two is the distance
between the clusters. You can also use
complete-linkage, and that says the distance
between any two clusters is equal to the greatest
distance from any member to any other member. OK? So if we had the same
picture we had before-- probably not the same
picture, but it's a picture. Whoops. Then we would say, well, I guess
complete-linkage is probably the distance, maybe,
between those two. And finally, not
surprisingly, you can take the average distance. These are all plausible metrics. They're all used and practiced
for different kinds of results depending upon the
application of the clustering. All right, let's
look at an example. So what I have here
is the air distance between six different cities,
Boston, New York, Chicago, Denver, San Francisco,
and Seattle. And now let's say we're-- want
to cluster these airports just based upon their distance. So we start. The first piece of our
[? dendogram ?] says, well, all right,
I have six cities, I have six clusters,
each containing one city. All right, what happens next? What's the next level
going to look like? Yeah? AUDIENCE: You're going
from Boston [INAUDIBLE] JOHN GUTTAG: I'm going to
join Boston and New York, as improbable as that sounds. All right, so that's
the next level. And if for some reason I only
wanted to have five clusters, well, I could stop here. Next, what happens? Well, I look at it,
I say well, I'll join up Chicago with
Boston and New York. All right. What do I get at the next level? Somebody? Yeah. AUDIENCE: Seattle [INAUDIBLE] JOHN GUTTAG: Doesn't
look like it to me. If you look at San Francisco
and Seattle, they are 808 miles, and Denver and San
Francisco is 1,235. So I'd end up, in fact, joining
San Francisco and Seattle. AUDIENCE: That's what I said. JOHN GUTTAG: Well, that explains
why I need my hearing fixed. [LAUGHTER] All right. So I combine San
Francisco and Seattle, and now it gets interesting. I have two choices with Denver. Obviously, there are
only two choices, and which I choose depends upon
which linkage criterion I use. If I'm using single-linkage,
well, then Denver gets joined with Boston,
New York, and Chicago, because it's closer to Chicago
than it is to either San Francisco or Seattle. But if I use
complete-linkage, it gets joined up with San
Francisco and Seattle, because it is further from
Boston than it is from, I guess it's San
Francisco or Seattle. Whichever it is, right? So this is a place
where you see what answer I get depends upon
the linkage criteria. And then if I want, I can
consider to the next step and just join them all. All right? That's hierarchical clustering. So it's good because you get
this whole history of the [? dendograms, ?] and
you get to look at it, say, well, all right,
that looks pretty good. I'll stick with this clustering. It's deterministic. Given a linkage criterion, you
always get the same answer. There's nothing random here. Notice, by the way,
the answer might not be optimal with regards
to that linkage criteria. Why not? What kind of algorithm is this? AUDIENCE: Greedy. JOHN GUTTAG: It's a
greedy algorithm, exactly. And so I'm making
locally optimal decisions at each point which may or
may not be globally optimal. It's flexible. Choosing different
linkage criteria, I get different results. But it's also potentially
really, really slow. This is not something you want
to do on a million examples. The naive algorithm, the one
I just sort of showed you, is N cubed. N cubed is typically
impractical. For some linkage criteria, for
example, single-linkage, there exists very clever N
squared algorithms. For others, you
can't beat N cubed. But even N squared is
really not very good. Which gets me to a much
faster greedy algorithm called k-means. Now, the k in k-means is the
number of clusters you want. So the catch with
k-means is if you don't have any idea how
many clusters you want, it's problematical,
whereas hierarchical, you get to inspect it and
see what you're getting. If you know how many you
want, it's a good choice because it's much faster. All right, the algorithm,
again, is very simple. This is the one that Professor
Grimson briefly discussed. You randomly choose k examples
as your initial centroids. Doesn't matter which of
the examples you choose. Then you create k clusters
by assigning each example to the closest centroid,
compute k new centroids by averaging the
examples in each cluster. So in the first iteration,
the centroids are all examples that you started with. But after that, they're
probably not examples, because you're now taking the
average of two examples, which may not correspond to
any example you have. Actually the average
of N examples. And then you just
keep doing this until the centroids don't move. Right? Once you go through
one iteration where they don't
move, there's no point in recomputing them again
and again and again, so it is converged. So let's look at the complexity. Well, at the moment,
we can't tell you how many iterations
you're going to have, but what's the complexity
of one iteration? Well, let's think about
what you're doing here. You've got k centroids. Now I have to take each
example and compare it to each-- in a naively, at
least-- to each centroid to see which it's closest to. Right? So that's k comparisons
per example. So that's k times
n times d, where how much time each of
these comparison takes, which is likely to depend
upon the dimensionality of the features, right? Just the Euclidean
distance, for example. But this is a way small number
than N squared, typically. So each iteration
is pretty quick, and in practice, as
we'll see, this typically converges quite
quickly, so you usually need a very small
number of iterations. So it is quite
efficient, and then there are various ways
you can optimize it to make it even more efficient. This is the most commonly-used
clustering algorithm because it works really fast. Let's look at an example. So I've got a bunch
of blue points here, and I actually wrote
the code to do this. I'm not going to
show you the code. And I chose four centroids
at random, colored stars. A green one, a fuchsia-colored
one, a red one, and a blue one. So maybe they're not the
ones you would have chosen, but there they are. And I then, having chosen
them, assign each point to one of those centroids,
whichever one it's closest to. All right? Step one. And then I recompute
the centroid. So let's go back. So we're here, and these
are the initial centroids. Now, when I find
the new centroids, if we look at where
the red one is, the red one is this point,
this point, and this point. Clearly, the new centroid
is going to move, right? It's going to move somewhere
along in here or something like that, right? So we'll get those
new centroids. There it is. And now we'll re-assign points. And what we'll see is this point
is now closer to the red star than it is to the fuchsia
star, because we've moved the red star. Whoops. That one. Said the wrong thing. They were red to start with. This one is now suddenly
closer to the purple, so-- and to the red. It will get recolored. We compute the new centroids. We're going to move
something again. We continue. Points will move around. This time we move two points. Here we go again. Notice, again, the
centroids don't correspond to actual examples. This one is close, but it's
not really one of them. Move two more. Recompute centroids,
and we're done. So here we've converged, and I
think it was five iterations, and nothing will move again. All right? Does that make
sense to everybody? So it's pretty simple. What are the downsides? Well, choosing k foolishly
can lead to strange results. So if I chose k
equal to 3, looking at this particular
arrangement of points, it's not obvious what "the
right answer" is, right? Maybe it's making all
of this one cluster. I don't know. But there are weird
k's and if you choose a k that is nonsensical
with respect to your data, then your clustering
will be nonsensical. So that's one problem
we have think about. How do we choose k? Another problem, and this is
one somebody raised last time, is that the results can depend
upon the initial centroids. Unlike hierarchical clustering,
k-means is non-deterministic. Depending upon what
random examples we choose, we can get a different
number of iterations. If we choose them poorly, it
could take longer to converge. More worrisome, you
get a different answer. You're running this
greedy algorithm, and you might actually
get to a different place, depending upon which
centroids you chose. So these are the
two issues we have to think about dealing with. So let's first think
about choosing k. What often happens
is people choose k using a priori knowledge
about the application. If I'm in medicine,
I actually know that there are only
five different kinds of bacteria in the world. That's true. I mean, there are subspecies,
but five large categories. And if I had a bunch of
bacterium I wanted to cluster, may just set k equal to 5. Maybe I believe there are
only two kinds of people in the world, those who are
at MIT and those who are not. And so I'll choose k equal to 2. Often, we know enough about the
application, we can choose k. As we'll see later, often we
can think we do, and we don't. A better approach is
to search for a good k. So you can try
different values of k and evaluate the
quality of the result. Assume you have some
metric, as to say yeah, I like this clustering, I
don't like this clustering. And we'll talk about
do that in detail. Or you can run hierarchical
clustering on a subset of data. I've got a million points. All right, what I'm going to
do is take a subset of 1,000 of them or 10,000. Run hierarchical clustering. From that, get a sense of the
structure underlying the data. Decide k should be 6, and then
run k-means with k equals 6. People often do this. They run hierarchical clustering
on a small subset of the data and then choose k. And we'll look-- but one we're
going to look at is that one. What about unlucky centroids? So here I got the same
points we started with. Different initial centroids. I've got a fuchsia
one, a black one, and then I've got red
and blue down here, which I happened to accidentally
choose close to one another. Well, if I start
with these centroids, certainly you
would expect things to take longer to converge. But in fact, what
happens is this-- I get this assignment of
blue, this assignment of red, and I'm done. It converges on this,
which probably is not what we wanted out of this. Maybe it is, but the
fact that I converged on some very
different place shows that it's a real weakness
of the algorithm, that it's sensitive to the
randomly-chosen initial conditions. Well, couple of things
you can do about that. You could be clever and try and
select good initial centroids. So people often will do that,
and what they'll do is try and just make sure that they're
distributed over the space. So they would look at
some picture like this and say, well, let's just put
my centroids at the corners or something like that so
that they're far apart. Another approach is
to try multiple sets of randomly-chosen
centroids, and then just select the best results. And that's what this little
algorithm on the screen does. So I'll say best is equal
to k-means of the points themselves, or
something, then for t in range number of trials, I'll
say C equals k-means of points, and I'll just keep track and
choose the one with the least dissimilarity. The thing I'm
trying to minimize. OK? The first one is got all
the points in one cluster. So it's very dissimilar. And then I'll just
keep generating for different k's
and I'll choose the k that seems to
be the best, that does the best job of minimizing
my objective function. And this is a very common
solution, by the way, for any randomized
greedy algorithm. And there are a lot of
randomized greedy algorithms that you just choose
multiple initial conditions, try them all out
and pick the best. All right, now I
want to show you a slightly more real example. So this is a file we've
got with medical patients, and we're going to try
and cluster them and see whether the clusters
tell us anything about the probability
of them dying of a heart attack in, say,
the next year or some period of time. So to simplify things,
and this is something I have done with research,
but we're looking at only four features here-- the heart rate in
beats per minute, the number of previous heart
attacks, the age, and something called ST elevation,
a binary attribute. So the first three are obvious. If you take an ECG of somebody's
heart, it looks like this. This is a normal one. They have the S, the
T, and then there's this region between the
S wave and the T wave. And if it's higher, hence
elevated, that's a bad thing. And so this is about
the first thing that they measure if someone
is having cardiac problems. Do they have ST elevation? And then with each
patient, we're going to have an outcome,
whether they died, and it's related
to the features, but it's probabilistic
not deterministic. So for example, an older person
with multiple heart attacks is at higher risk than
a young person who's never had a heart attack. That doesn't mean,
though, that the older person will die first. It's just more probable. We're going to take this data,
we're going to cluster it, and then we're going
to look at what's called the purity
of the clusters relative to the outcomes. So is the cluster, say,
enriched by people who died? If you have one cluster
and everyone in it died, then the clustering is
clearly finding some structure related to the outcome. So the file is in the
zip file I uploaded. It looks more or less like this. Right? So it's very straightforward. The outcomes are binary. 1 is a positive outcome. Strangely enough in
the medical jargon, a death is a positive outcome. I guess maybe if you're
responsible for the medical bills, it's positive. If you're the patient, it's hard
to think of it as a good thing. Nevertheless, that's
the way that they talk. And the others are
all there, right? Heart rate, other things. All right, let's
look at some code. So I've extracted some code. I'm not going to
show you all of it. There's quite a lot
of it, as you'll see. So we'll start-- one
of the files you've got is called cluster dot pi. I decided there
was enough code, I didn't want to put
it all in one file. I was getting confused. So I said, let me
create a file that has some of the code
and a different file that will then
import it and use it. Cluster has things
that are pretty much unrelated to this example, but
just useful for clustering. So an example here has
name, features, and label. And really, the only
interesting thing in it-- and it's not that
interesting-- is distance. And the fact that I'm
using Minkowski with 2 says we're using
Euclidean distance. Class cluster. It's a lot more
code to that one. So we start with a
non-empty list of examples. That's what init does. You can imagine what
the code looks like, or you can look at it. Update is interesting in that it
takes the cluster and examples and puts them in-- if you
think of k-means in the cluster closest to the
previous centroids and then returns the amount
the centroid has changed. So if the centroid
has changed by 0, then you don't have
anything, right? Creates the new cluster. And the most interesting
thing is computeCentroid. And if you look
at this code, you can see that I'm a slightly
unreconstructed Python 2 programmers. I just noticed this. I really shouldn't
have written 0.0. I should have just written
0, but in Python 2, you had to write that 0.0. Sorry about that. Thought I'd fixed these. Anyway, so how do we
compute the centroid? We start by creating
an array of all 0s. The dimensionality is the number
of features in the example. It's one of the methods from-- I didn't put up
on the PowerPoint. And then for e in
examples, I'm going to add to vals
e.getFeatures, and then I'm just going to divide vals by
the length of self.examples, the number of examples. So now you see why I made it a
pylab array, or a numpy array rather than a
list, so I could do nice things like divide the
whole thing in one expression. As you do math, any
kind of math things, you'll find these arrays
are incredibly convenient. Rather than having to
write recursive functions or do bunches of
iterations, the fact that you can do it in one
keystroke is incredibly nice. And then I'm going to
return the centroid. Variability is exactly
what we saw in the formula. And then just for fun,
so you could see this, I used an iterator here. I don't know that
any of you have used the yield statement in Python. I recommend it. It's very convenient. One of the nice
things about Python is almost anything
that's built in, you can make your
own version of it. And so once I've done
this, if c is a cluster, I can now write something
like for c in big C, and this will make it work just
like iterating over a list. Right, so this makes it
possible to iterate over it. If you haven't read
about yield, you probably should read the probably
about two paragraphs in the textbook
explaining how it works, but it's very convenient. Dissimilarity
we've already seen. All right, now we
get to patients. This is in the file lec
12, lecture 12 dot py. In addition to importing
the usual suspects of pylab and numpy, and probably it
should import random too, it imports cluster, the
one we just looked at. And so patient is a
sub-type of cluster.Example. Then I'm going to define
this interesting thing called scale attributes. So you might remember,
in the last lecture when Professor Grimson was
looking at these reptiles, he ran into this
problem about alligators looking like chickens
because they each have a large number of legs. And he said, well, what can
we do to get around this? Well, we can represent the
feature as a binary number. Has legs, doesn't have legs. 0 or 1. And the problem he
was dealing with is that when you
have a feature vector and the dynamic range
of some features is much greater than
the others, they tend to dominate because the
distances just look bigger when you get Euclidean distance. So for example, if we
wanted to cluster the people in this room, and I
had one feature that was, say, 1 for male
and 0 for female, and another feature that
was 1 for wears glasses, 0 for doesn't wear glasses,
and then a third feature which was weight, and
I clustered them, well, weight would
always completely dominate the Euclidean
distance, right? Because the dynamic range
of the weights in this room is much higher than
the dynamic range of 0 to 1. And so for the reptiles,
he said, well, OK, we'll just make it a binary variable. But maybe we don't
want to make weight a binary variable, because
maybe it is something we want to take into account. So what we do is we scale it. So this is a method
called z-scaling. More general than just
making things 0 or 1. It's a simple code. It takes in all of the
values of a specific feature and then performs some
simple calculations, and when it's done, the
resulting array it returns has a known mean and a
known standard deviation. So what's the mean going to be? It's always going to be
the same thing, independent of the initial values. Take a look at the code. Try and see if you
can figure it out. Anybody want to
take a guess at it? 0. Right? So the mean will always be 0. And the standard deviation,
a little harder to figure, but it will always be 1. OK? So it's done this scaling. This is a very common kind
of scaling called z-scaling. The other way people
scale is interpolate. They take the smallest value and
call it 0, the biggest value, they call it 1, and then they
do a linear interpolation of all the values
between 0 and 1. So the range is 0 to 1. That's also very common. So this is a general
way to get all of the features sort
of in the same ballpark so that we can compare them. And we'll look at what
happens when we scale and when we don't scale. And that's why my getData
function has this parameter to scale. It either creates a set of
examples with the attributes as initially or scaled. And then there's k-means. It's exactly the
algorithm I showed you with one little wrinkle,
which is this part. You don't want to end
up with empty clusters. If I tell you I
want four clusters, I don't mean I want
three with examples and one that's empty, right? Because then I really
don't have four clusters. And so this is one
of multiple ways to avoid having empty clusters. Basically what I
did here is say, well, I'm going to try a lot of
different initial conditions. If one of them is so unlucky
to give me an empty cluster, I'm just going to skip it
and go on to the next one by raising a value
error, empty cluster. And if you look at
the code, you'll see how this value
error is used. And then try k-means. We'll call k-means numTrial
times, each one getting a different set of
initial centroids, and return the result with
the lowest dissimilarity. Then I have various ways
to examine the results. Nothing very
interesting, and here's the key place where we're
going to run the whole thing. We'll get the data,
initially not scaling it, because remember,
it defaults to true. Then initially, I'm only going
to try one k. k equals 2. And we'll call testClustering
with the patients. The number of clusters, k. I put in seed as
a parameter here because I wanted to be
able to play with it and make sure I got different
things for 0 and 1 and 2 just as a testing thing. And five trials
it's defaulting to. And then we'll look
at testClustering is returning the fraction
of positive examples for each cluster. OK? So let's see what
happens when we run it. All right. So we got two clusters. Cluster of size 118 with
.3305, and a cluster of size 132 with a positive
fraction of point quadruple 3. Should we be happy? Does our clustering tell
us anything, somehow correspond to the expected
outcome for patients here? Probably not, right? Those numbers are pretty
much indistinguishable statistically. And you'd have to guess that
the fraction of positives in the whole population
is around .33, right? That about a third
of these people died of their heart attack. And I might as well have
signed them randomly to the two clusters, right? There's not much
difference between this and what you would get
with the random result. Well, why do we
think that's true? Because I didn't scale, right? And so one of the issues
we had to deal with is, well, age had a
big dynamic range, and, say, ST elevation, which I
told you was highly diagnostic, was either 0 or 1. And so probably
everything is getting swamped by age or
something else, right? All right, so we have
an easy way to fix that. We'll just scale the data. Now let's see what we get. All right. That's interesting. With casting rule? Good grief. That caught me by surprise. Good thing I have the answers
in PowerPoint to show you, because the code doesn't
seem to be working. Try it once more. No. All right, well, in
the interest of getting through this
lecture on schedule, we'll go look at the
results that we get-- I got last time I ran it. All right. When I scaled, what we see here
is that now there is a pretty dramatic difference, right? One of the clusters has
a much higher fraction of positive patients
than others, but it's still a
bit problematic. So this has pretty
good specificity, or positive predictive value,
but its sensitivity is lousy. Remember, a third of our
initial population more or less, was positive. 26 is way less than a
third, so in fact I've got a class, a cluster,
that is strongly enriched, but I'm still lumping most
of the positive patients into the other cluster. And in fact, there
are 83 positives. Wrote some code to do that. And so we see that
of the 83 positives, only this class,
which is 70% positive, only has 26 in it
to start with it. So I'm clearly missing
most of the positives. So why? Well, my hypothesis was
that different subgroups of positive patients have
different characteristics. And so we could test this
by trying other values of k to see with-- we would
get more clusters. So here, I said, let's
try k equals 2, 4, and 6. And here's what I
got when I ran that. So what you'll notice here, as
we get to, say, 4, that I have two clusters, this
one and this one, which are heavily enriched
with positive patients. 26 as before in the first
one, but 76 patients in the third one. So I'm now getting a much
higher fraction of patients in one of the "risky" clusters. And I can continue to do that,
but if I look at k equals 6, we now look at the
positive clusters. There were three of them
significantly positive. But I'm not really getting
a lot more patients total, so maybe 4 is the right answer. So what you see here is that
we have at least two parameters to play with, scaling and k. Even though I was only
wanted a structure that would separate the risk-- high-risk patients
from the lower-risk, which is why I started
with 2, I later discovered that, in fact,
there are multiple reasons for being high-risk. And so maybe one
of these clusters is heavily enriched
by old people. Maybe another one
is heavily enriched by people who have had three
heart attacks in the past, or ST elevation or
some combination. And when I had
only two clusters, I couldn't get that
fine gradation. So this is what data
scientists spend their time doing when
they're doing clustering, is they actually have
multiple parameters. They try different things out. They look at the
results, and that's why you actually have to think
to manipulate data rather than just push a button
and wait for the answer. All right. More of this general
topic on Wednesday when we're going to talk
about classification. Thank you.
