So how many of y'all
have been to the site, know a little bit
about Khan Academy? Oh, good, this is very good. That's probably why
you're here, actually. It's a silly question. So as most of y'all, I guess, do
know Khan Academy is most known for the videos. And we're going to
be doing more videos, and this is a super
exciting collaboration. But we're also doing a lot more,
and I'll talk more about that. But so we're all
on the same page, I'll show you a little montage
of some Khan Academy videos. Up the volume. [VIDEO PLAYBACK] -And the notation usually
is a capital sigma. -All of these
interactions are just through the gravity
over interstellar, almost you could call
it intergalactic. -So the right slot is i plus 1. -This animal's
fossils are only found in this area of South America,
a nice clean band here. -They create the Committee of
Public Safety, which sounds like a very nice committee. -Notice this is an aldehyde,
and it's an alcohol. -It's some type of an
infectious disease. -Exactly. -So the key is, when you
start to look at data, you have to look at
all aspects of it. --[INAUDIBLE] there 30 million,
plus the 20 million from the American manufacturer. -If this does not blow your
mind, then you have no emotion. [END PLAYBACK] [LAUGHTER] [APPLAUSE] And y'all are the first audience
that actually does appreciate why that should blow your mind. So it's exciting that we
could share that moment. So that is where we are now,
as Dean Waitz introduced. We've had over 90
million video views now. We've had over 100
million exercises done. It's actually about
$2 million a day now. And that's growing
at a fast pace. So we have about 3 and 1/2
million unique students a month. And that's growing at kind
of a surreal pace for us. And in the internet
world, you're so used to like million here,
million there, 10 million in Facebook. You lose sight of it. And that's why i did a
little back-of-the-envelope calculation. The number of unique
students we had last month is more than six times
the number of students that Harvard has
graduated in its history, just a little estimation
thing right over there. But that's where we are now. And I think we do have
a lot of time to do Q&A, and we can talk a lot
about where we're going. And I think a lot of
y'all know some of this, how this kind of
happened or started. And it's all been a
little bit happenstance and lucky circumstances and
the right things happening at the right time. But as a lot of y'all
know, it started in 2004. I was an analyst
at a hedge fund, actually at 9 Newbury Street. And I had just gotten
married, and some family was visiting me from
New Orleans, which is where I grew up. And my one cousin in particular,
Nadia, she was 12 years old, and I remember she
was super sharp. I hadn't seen her for
a couple of years. When you talked to her, you
felt like you were talking to someone your own age. It was actually 4th
of July weekend, and we were waiting for the
fireworks over the Charles River. And we kind of had
got our spot, and we had a couple hours to wait
for the fireworks to start. And so as I think as many of
y'all would do to kill time, I started giving my
family brain teasers. [LAUGHING] And I think, like
you've probably experience with most of your
family, they're probably like, Sal, stop, we don't
feel like thinking. This is a vacation. But Nadia said, no, no, no,
don't tell me the answer. I want to solve it. And I literally remember,
12 years old-- and these are the type of
stuff-- these are things you might see in a
computer science interview. But she was working
it out on the sand, and I was really impressed. She didn't shut her brain off. She wanted to
engage the problem. She was logical. She was analytical about it. And I told her, and I said,
Nadia, you should go to MIT. And when I said that,
I saw her mom kind of give her dad, my uncle,
a little bit of a look. I didn't know what that meant. And so the next morning, her
mom, [INAUDIBLE], told me, Sal, thank you so much for being
like a big brother figure to Nadia and her brothers, and
they really look up to you. And we love that you
think so much of them. But I actually want
to let you know that Nadia is actually having
trouble with math in school right now. And I said, oh, no,
that's impossible. I mean, the girl who was
doing those-- the stuff that she was tackling, I've
seen college students who can't do problems like that. And she was tackling them, and
she was engaging the problem. And on top of that, we share
a certain amount of DNA. [LAUGHTER] So and when Nadia
actually woke up, I said, what's going on, Nadia? I have trouble believing
what your mom just told me. And Nadia said that she's
having trouble with units, kilograms to decigrams,
or ounces to whatever. And I just told
her point blank, I was like, I
definitely understand how that can be confusing. I mean, even now, sometimes
I multiply, divide by. But, I told her, what
you were doing last night is 10 times deeper. And the conversations that
we've had about politics or philosophy or whatever
else, 10 times deeper than unit conversion. And she was kind of a
little bit-- she actually didn't believe. Is this like a pep
talk or whatever else? So I told her, I was like, look,
when you go to New Orleans, how about we do a
little bit of tutoring? We'll get on a phone. We'll figure out some way to
look at each other's writing or whatever else. And she agreed. I think more than
anything, she just liked the fact that someone was
taking interest in her life. So we started. And frankly, the first month
or so was pretty painful. Her confidence was
completely blown. Her brain was shut off. It was a completely
different Nadia when you asked her a
simple units question. Kilo means 1,000. Kilometer is how many meters? Long, awkward pause-- 1,000? Are you sure? And it was like, this
is the same girl who was doing these
complex logic problems, just because she was
completely disengaged. But eventually, we finally
got through that hump. And after too much,
actually-- once she got it, and she got it kind of all
together, she almost got angry. She was, I can't believe
that that was the reason that I was tracking
into the slower math class, that simple idea. So after that, I was like, OK,
I've worked with one cousin. This worked. And the whole time
I was on the phone, I could kind of hear her
two younger brothers wanting to be part of the scene. So I started tutoring them. And then the way our
tutoring sessions were going, I would kind
of give them a problem, and if they knew
how to do it, great. If they didn't know
how to do the problem, I'd give them a hint or a step. And we'd work the next one. And so I said,
well, maybe I could write some simple software. I was here working
at a hedge fund, and I hadn't been
using the coding side of my brain for a little while. So I was like, oh, this
will a fun little project. And so I started writing
this little JavaScript thing to generate these
problems for them, these little random numbers. And if they didn't
know the steps, it would give the
hints for those steps. And I started
telling Arman and Ali and Nadia, do these problems,
and we can talk about them the next day. And the next day, I'd say,
oh, how many did you do? Oh, I did 14. I did 15. How many did you get right? I think I got them all right. And so I didn't believe them. So I put a database behind it. [LAUGHTER] But it was just that
simple, very primitive thing that I was doing
part time, starting giving me all this information. Wow, they didn't know how
to do that type of problem. Boy, I've been really
gauging where they are in the completely wrong way. Or now they need the practice
so that they can really bone up on that concept. Or why is Ali, who
is eight years old, doing that problem at
3:00 in the morning? Which was actually
one of the things that I discovered when I
was doing little SQL queries on their little problem logs. [LAUGHTER] And so that was
there, and I started getting more and more
cousins involved, and I was tutoring
the, after work. And by November of 2006, I was
at a friend's dinner party, actually another MIT alum. It was [INAUDIBLE]. He did his PhD here. And I was showing off the
software that I was doing, my cousins are using it, and I
think it's really helping them, and all the rest. But my only complaint was I
was doing these tutorials, and it was getting
hard to scale. That connection that
I had with Nadia, the one-on-one
connection, even when you start having two,
three, four students, it started to become difficult. I tried to do conference
calls, and those were a mess. And he's the guy that told me. He's like, well, I don't
know if this will work, but why don't you make some
lectures and put them up on YouTube, and maybe you
can get some of the stuff out of the way for your tutorials. And I said, no, no, no, no, no. YouTube is for
cats playing piano. [LAUGHTER] It is not for
serious mathematics. And so then I went
home that weekend, and I kind of thought
about it a little bit. And I got over the idea
that it wasn't my idea. [LAUGHTER] And decided to give it a shot. And I put a couple of those
videos out there on YouTube. And I remember
that first screen, do you want to make
this public or private? And actually, I was tempted--
past because I was embarrassed. What if these aren't
good or whatever? And I said, well, maybe if
I could password protect it. But YouTube didn't have any
functionality like this. So I guess I'll make it public. But what are the
odds that anyone else is going to see this stuff? So I put it out
there, and I started pointing my cousins to it. I was like, hey,
before we do this, why don't you take
a look at these? This might be good review. Or here's some
stuff you can look at if you forgot the
last time we went over this idea or that idea. And I joke a lot about this,
but it's completely true. After a couple weeks, I had
about 30 videos out there. And they were very
simple, very primitive. And you could actually sort
on YouTube by upload data. And you can see those original
videos, very primitive. And I asked my cousins,
are these useful. What do you think of them? And they told me that they
preferred me on YouTube than in person. [LAUGHTER] And I haven't clarified whether
it only applies to math videos or is this is broader thing
that they don't just-- and it's completely unintuitive. It goes against
everything we assume. We always assume
that, yes, technology can give you productivity. Technology can give
you efficiency. But technology is something
that can always only approach the resource
intensive, the live version. But here, my cousins
were telling me that they preferred
the automated cousin to their cousin. And so it goes against
that intuitive gut sense of technology
in every other domain, but it made complete sense
when you put yourself in their shoes. Now Nadia didn't have
to feel embarrassed when she forgot how to do
something from fourth grade math. She didn't have to feel
like she was wasting my time if she wanted to
repeat a concept. She didn't have to
engage with the content when she wasn't ready to
engage with the content. I mean, there were days where
I agreed to do this with Nadia, but I had a tough day at work. I'd go home, I'd say,
oh, I told Nadia, I guess I better do this. So my energy level wasn't there. And I'm sure there were days
that the boy she had a crush on asked some other
girl to the prom and so she wasn't
completely there, either. But now she could get it
when she was ready for it, when she had the
question in her mind so she could pause and repeat. And so I got kind of
excited, and I kept doing it, just thinking that, at minimum,
this will help my cousins now. I didn't have kids at the time. I do now. I said, well, maybe this could
help my kids in the future. I could be a little family
legacy, a little thing that I could share with--
maybe my great grandkids could use this content. And I read a lot of
science fiction books, so it's kind of in
that vein of thinking. But there's no
reason why you can't. If Newton or Leibniz had
made videos on YouTube, we wouldn't have to. [LAUGHTER] There's reason to believe
Newton was a little eccentric. He wasn't mainstream like me. [LAUGHTER] So it might not have worked. So I just kept going. The other unintuitive
thing thing-- I'd go to dinner parties,
and I'd show people the little videos
that I was making, and I had my friends from
business school I'd show it to. And they're, how are you going
to make money off of this? [LAUGHTER] I was like, oh, no,
I mean, I have a job. I'm just doing this for
fun, and it's for my family, and I'm enjoying it. And I would get
these kind of looks like, what part of capitalism
do you not understand? [LAUGHTER] But then by this time
our fun had moved out to Silicon Valley. And so this will never
scale, never scale. You should make a platform
and get other people to upload content onto your platform. You should crowd source it. And I said, no, I
understand the principle. I have an MBA as well. [LAUGHTER] I understand the principle
of getting other people to do your work for you. But I'm enjoying this. And then I said, but
that's a good idea. Would you like to do
some videos yourself? And they would walk away. [LAUGHTER] But once again, it was
just because it was fun. And I just kept doing it. And then comments started
to come in on YouTube. And some of those comments
were just a simple thank you. And I don't know if y'all
spend a lot of time on YouTube, but it's not all
positive comments. [LAUGHTER] Not all G-rated comments. And actually, I was kind of
hardening the self-esteem of some of the video producers. Once we expose
ourselves to YouTube, there will be-- there are
times that I want to cry. People can say mean
things on the internet. [LAUGHTER] But for the most part-- and
this as actually important, and actually, this is important
to the genesis of the Khan Academy, because
I am someone who thrives off of positive
feedback and doesn't do well with criticism. First people said thank
you or this helped. And it was random people. It wasn't my cousins. It was people all
over the world. And then the comments
got more interesting. This is the reason why
I'm passing algebra. This is the reason why I'm
able to go back to college. This is the reason why,
after leaving the military, I haven't seen math in 15
years, I can now go back. This is the bridge that
gets me back into that. I got letters from a mother
of an autistic child. She said, this is the only
thing that works for my child. I got letters, multiple letters,
from random people saying that they're praying for me. And I was working
at a hedge fund. [LAUGHTER] [APPLAUSE] I'm sure some of my
investors had prayed for me, but for different reasons. So I mean, it was just
a powerful experience. So I kept going. And the site traffic
just kept growing. And you fast forward it to 2009. And actually, there were some
interesting events in 2008. 2008, during the financial
crisis, all bets were off. In our fund, we parked
all of the money in cash. And frankly, all of the
responsible fund managers starting reading the Federal
Reserve Act over and over and over again. And then I would go
to parties, and people would ask, hey,
what do you think about this financial crisis? I'd go, oh, this is the
difference between illiquidity and insolvency, and this
is why credit default swaps are called financial
weapons of mass destruction. And people were like, oh, uh--
and I said, well, why am I not making videos about
this stuff as well? So I started doing that. And it's funny. I got a call from two
major news networks saying that they were using
these videos before reporting on the financial crisis. [LAUGHTER] Actually, I got I got one email
from an investment banker-- I will not name the
bank-- who literally wrote a very short email, I now
know what I do for a living. [LAUGHTER] And it's heartwarming, better
late than never, right? I'm sure he was getting
compensated quite nicely, too, which is a little strange. So that happened. And actually that was first
national publicity we had. I was on the Rick
Sanchez Show talking about how we should start
new banks and all the rest. Poor guy-- things have
happened to him since. But in 2009, I had
this day job still. I was an analyst
at a hedge fund. But I had trouble
focusing on it. There's this site. People were writing me letters. The traffic was picking up. And were many other times, you
have a bad week at your job and you say you want
to quit the job. And you go to your wife, and
you look at your finances, you look at your
mortgage like, hm, and you look at
the different-- no, I'll stay a little bit longer. [LAUGHTER] But by fall of 2009, frankly,
this is all I thought about. It would actually have
been irresponsible for me to continue managing
other people's money. And frankly at
that same time, we got a little
external recognition. And I got this really powerful
letter from this student. We actually have
it on our website. The student, no
background in mathematics, used Khan Academy for summer,
went to the community college and got a perfect
score on the placement exam, which had never happened
at that community college. So that all happened
in a one-week period. So I sat with my wife. We looked at the finances. She was pregnant. We actually had our
first baby at that point. And so it wasn't
a light decision, but we said, look, we have
enough savings that I could take a year off and do it. And so I did. And like all things like this--
the site was doing just great. It was growing exponentially. But the funding-- we set
it up as a not-for-profit. Someone should realize
the social ROI here, that these videos don't just
teach millions of kids now, They can teach millions
of kids forever, billions of kids maybe one day. But they just weren't
getting traction. And like I was telling
a group earlier today, I would go to parties
and people ask you what you do for a living. And so, I make
videos on YouTube. [LAUGHTER] And I literally heard
this, some people were, well, it's good
that his wife is a doctor. [LAUGHTER] The meanness isn't
just on YouTube. But these things take longer. And I had a little Paypal
thing so people could donate. And we were getting
$25, $50 here and there. And that was powerful, that a
random student someplace would give $20 just out of goodwill. But then all of a sudden
in May of 2010 now, all of a sudden a
$10,000 donation comes. So I look at it. It was someone named Ann Doerr. And I emailed her. I said, this is the largest
donation we've ever gotten. If we were a
physical school, you would now have a
building named after you. [LAUGHTER] Which I think is
a very good deal. I don't know what the going
rate at MIT for a building is, but it's well in
excess of $10,000. But she actually
immediately emailed me back. She was local. She was in Palo Alto. And she said, well, have
we should have lunch. And so I met her for lunch. And she's like, where do
you want to take this? I said, well, a world-class
education for anyone in the world for free. And I showed her why. We can make exercises. That primitive stuff I
had done for my cousins, we could hire a competent team
to actually build that out. And we could eventually
connect kids with each other. I'd keep making videos,
the whole dream. And she nodded. I think it was
resonating with her. And then she was, well, if
I was the largest donation, how are you supporting yourself? And in kind of a proud way
as possible, I said, I'm not. [LAUGHTER] And so she kind of nodded. And I went home. And when I was in the driveway,
I got a text message from Ann. And she's like, well, you
should be supporting yourself. I've just wired you $100,000. And so that's was a good day. [LAUGHTER] [APPLAUSE] And actually Ann's
texts have started to become this like bearer
of good news since then. Then things started to get
really surreal and wacky. And even to this point, it's
just been a surreal rocket ship ride since that point. A few weeks later, some
folks at Google-- I say there are two major
institutions in Mountain View, Google and the Khan Academy. [LAUGHTER] I don't get as many
laughs at Google. [LAUGHTER] Some folks brought
me into Google. It was a big room. And they were very
complimentary. They're like, oh,
I don't know if you know that a lot of our
kids use Khan Academy. It's really helping them. And it was very flattering. These are some of the best
thinkers in the world, and their kids are
high performing students and all the rest. And they just point blank,
after I talked a little bit about what I was up
to, they said, well, what would you do
with $2 million? And I said, is this
an open question? [LAUGHTER] And they clarified it
was education related. [LAUGHTER] I've got needs, you know. [LAUGHTER] But so I told them the
same thing that I told Ann. We would build out
a virtual school. We would hire
engineers, designers. We would make this thing where
anyone anywhere could learn. And they kind of nodded. And by this point,
I actually had been through this cycle
of talking to foundations. And there wasn't a lot--
they liked the idea, but they're like, well, why
are we the first to fund you? And you kind of need
that initial validation, and I didn't have it. Although Ann donating--
she's the wife of John Doerr, famous venture capitalist. And so that was actually a very
powerful initial validation. But actually, I don't
think the Google guys knew about it at that point. But then it got even wackier. And then the Google part
has a good ending, too, but it got even wackier. Early July 2010, I was
running a little summer camp. Because I was just curious. I do a lot of this virtually. It's fun for me to work
with kids in a live setting and see what you can do if
you actually are interacting with each other. And so I had this
game where six kids were playing a game of Risk. And then the other 20
were trading securities based on the outcome
of the game of Risk. [LAUGHTER] [APPLAUSE] It's a good game. Frankly the summer
camp was all the games that I wanted to play
at my birthday parties until that point that I never
got any of my friends to play. But now people were
actually paying me to let me do it
with their kids. And so while it was literally a
trading floor of middle school students, I got another
text message from Ann. And she said, I'm at the
Aspen Ideas Festival. And I'm in a room of
hundreds of people. And Bill Gates is
on stage right now. And for the past 10 minutes,
he's been talking about you. [LAUGHTER] And it's like one of
those, pinch yourself. And I was like, is this a joke? Maybe Ann sen it to the
wrong-- she knows people that Bill Gates
would talk about, so maybe it was a mistake. Who knows? I didn't know. And then I looked
at the Twitter feed. I said, some people
have tweeted. This is happening someplace. This is in my
reality in some way. And so I went home. And it was the most
surreal experience. Because like this thing
happened, and I was like, what do I do now? Do I call him? [LAUGHTER] And so I don't know
if they realized what they were doing to me,
but the next two weeks were this strange-- I was
still in the closet, still just doing
the little videos, and there was this
little part of my brain, Bill Gates is-- what? And then I got a call
from Larry Cohen, who is Bill Gates' chief of staff. He said, well, you must
have heard Bill is a fan. He would like to meet
if you have time. [LAUGHTER] And I was looking at my
calendar at the moment, and it was completely blank. [LAUGHTER] And I didn't want-- so I
was like, maybe at 2:45 I can just fly in, fly out. We can work it out. And so we met. I guess I can call him Bill now. And he said, how
can I support you? And it was as weird. His chief of staff said that
Bill has three monitors, and he says one of the
monitors has Khan Academy up like all the time. And I immediately said, I need
to redo some of these videos. Like this is not a
child's game anymore. But Bill said, how
can we support you? And I kind of said
the same thing I told Ann, same
thing I told Google. And he kind of nodded. And when he nodded,
there were five people in the back of the room
who immediately-- you got that right there? And so October of 2010,
so this is like a little over a year ago, funding
from the Gates Foundation actually hit at
the exact same time Google decided to fund
us, too, $2 million for 10 projects that have the
potential to save the world. And so we were off and running. We were able to
get office space. I was able to hire another MIT
alum, the smartest guy I knew who was my freshman roommate. He joined. And we started to ramp up. And this is what we
started working on. And some of y'all are probably
somewhat familiar with this. This is a more
sophisticated version of really what I
started with my cousins. And the idea here is
each of those nodes-- the top one is basic addition. And I'll show you what
these look like in a second. But it will generate
as much questions as you need to master
basic addition. And once you've
mastered that, you move on to the next concept. And it's kind of the
most common sense way you would
implement anything, you would teach anything. It's the way video games work. It's the way you would
learn karate or learn a musical instrument. But the one thing
that we realized later, especially when we
tried to implement it even at a few summer camps and
things, as intuitive as it is, it's the complete
opposite of the system that we've all been
indoctrinated into. In the system that we've
all been raised in, what you have is you have
a fixed amount of time to learn a concept. And then the variable is
how well you learn it. So some of us get an A, some of
us get a B, some of us get a C. And it's used not as
an assessment to fix. It's used as an assessment to
place value judgment on you. You're smart. You're not so smart. You're hard working. You're not so hard working. And the real insanity of
it is, in those two weeks, we were supposed to
learn basic exponents. And we take an exam. And half the class
gets a C, some kids get Ds, 20% deficiencies,
30% deficiencies. Even the A students
have 5% deficiencies. We all then move
on to logarithms, somehow expecting magically that
we expect those same kids who didn't know 20%
of basic exponents to now all of a sudden
be able to understand the inverse function, to
understand what a logarithm is. But that's exactly
what we're doing. And the analogy is
imagine building a house. And the foundation,
you build it. You get the
inspector to come in. The inspector goes and says,
oh, it's an 80% foundation. Well, that's passing. Let's build a first floor. [LAUGHING] And then you build a
second floor, third floor, and then it collapses. And you're like, whoa,
what went wrong here? We have to do something. But that's what we're
doing in our system. So what we're
saying is, and it's the way you learn naturally,
instead of the fixed thing being the amount of time
and when you learn something and the variable how
well you learn it, let's make the fixed thing
how well you learn it. Let's make it
mastery for everyone, and then the variable
is how and when and how long you actually
have to learn the concept. And so this is what some
of these modules look like. This is actually
one that was just contributed by a volunteer. And I showed it earlier
today to some students. And this is an open source
project to build these modules. So any of y'all who are
eager and jumping in here-- and actually, John Resig's here,
who is the father of jQuery, built the framework for
making these modules. It's mostly JavaScript library. I mean, he should be
really giving this talk. But if any of y'all are
interested in participating, we would love it. The module you're
seeing there is actually contributed by a guy in DC. And it's one of
our best modules. And so what the
modules do is they'll give you an infinite
number of questions. And if you need help, there's
the related Khan Academy video. And the idea is you keep doing
it until you get mastery. And what's really
cool, what I really like about this
module in particular, is you're able to do
things that you can't even do with a traditional
textbook, that you can't even do with a traditional lecture. You change the slope of the
tangent line at each point, and by doing so,
you're essentially tracing out the derivative
of that function. And not all of them
are this, but this is what we want all the
modules eventually to approach. And eventually we
could do simulations, multi-player modules,
multi-player games, different things. We're experimenting with
goal-setting right now. Now this is another module. And this a more basic one. This is kind of what they looked
like when I was starting them for my cousins. This one we're still using. But basic idea, you keep doing
it until you understand it, show some level of proficiency. And it's an interesting
question of how do you measure proficiency, and
I'm happy to talk about that. But if you don't
know how to do it, you get the steps
and all the rest. I think that smiley
face is actually the only part of my original
code that's still in service. So when we started, it
was literally a year ago. [INAUDIBLE] had joined. We got a call from the
Los Altos school district. Is a district adjacent
to where I live. And they said, oh, we heard
you're doing some stuff. We'd like to meet you. And so we met them. And the school board
said, well, what would you do if you could do anything
you wanted with a fifth grade class. And we said, well, we'd
have every student working at their own pace, mastering
concepts before moving on. We think that would
free up time to do other interesting
projects in the classroom, build things, build robots,
whatever, paint pictures, whatever it might be. And the role of
the teacher changes from instead of being the
lecturer and the grader of problem sets,
the teacher will turn into someone who gets a
lot of data on how kids are performing, see where all
the kids are, and then do focus interventions on
students when they're stuck and mentor them on
projects and whatever else they might want to do. And so they nodded. They were very sympathetic. They're like, yes, this is
differentiated learning. This is the gold standard. But it's very radical. And so when [INAUDIBLE] and
I went to the parking lot, we're like, they're
very nice, but there's no way they're going
to be able to do this. But once again, this
is one of these very, I don't know, fated
circumstances. They emailed us in
two days, saying, can we start after Thanksgiving? And they wanted to start with
two fifth grade classes and two seventh grade classes. And so this right over here
is a dashboard from one of those fifth grade classes. And so the teacher
literally walks around. They would have this
dashboard and others. And I'll show you a few of them. But each row here is
one of the students. I've blocked them off. Each column there is one of
those nodes of that knowledge map you saw. And the model is if a student
is green, if that cell is green, the student is doing fine. They already mastered
or have shown proficiency in that concept. If it's blue, they're
working on it, but there's no need to worry. But if it says red
based on some heuristic, and we're always
working on that, it looks like the
student is stuck. They've done a lot of problems,
they've watched the video, they've looked at hints, but
it looks like they're still not getting it. And so the model is, well,
why doesn't the teacher just go sit down next to that
student and do a very focused, one-on-one tutorial? Or even better, why not get
one of the other students who have already mastered that
concept to sit down and be the first line of attack? And when you do that, and
we saw this happening, where the students were starting
to teach each other. And actually one of the
fifth grade teachers started to have a cadre of
fifth grade TAs who essentially the teacher spent a large
amount of time mentoring so that the teacher could scale. And then some of those TAs were
pretty impressive people-- you know, fifth graders, people. [LAUGHTER] And we realized something
very profound was happening. There's that one idea
of mastering concepts before you move
on, which is really just kind of correcting some
of the insanity of the system that we were indoctrinated into. But the other thing that we
realized that was happening is in the ed reform debate,
student-teacher ratio-- big deal. And obviously, at a
very superficial level, the lower, the better,
all else things equal. But we realized what was
even more important than student-to-teacher ratio is
student to valuable time with the teacher ratio or
student-to-engaged-time ratio. And here in a traditional
model, a teacher spends 90% of their time
lecturing or grading problems. That's maybe 5% of their
time, 10% of their time sitting next to
students and actually forming bonds with them. I mean, I sat in class
for years not ever having a real
conversation what one of my teachers in grade school. At MIT it wasn't the case. I had very deep-- [LAUGHS] [LAUGHING] There are several people--
I had you for 6002. Oh, really? [LAUGHTER] But all of a sudden, if
you take that 90%, and now instead of doing
a lecture, they're able to sit down next to
students and mentor them and guide them and form
connections with them, you're actually
increasing the humanity in the classroom, the
engagement in the classroom by an order of magnitude. So once again, a
very ironic thing. You're using technology,
something that people think is a very cold thing-- it's
efficient, it's productive, but it's not human-- but
you're using technology to actually make the
class more interactive, making it more human, making
it a more engaging experience than a traditional classroom. And we wanted to
arm the teachers with as much data as possible. And I want to make it clear. A lot of the specs for these
dashboards, these reports, came from the
teachers themselves. They're like, well, it
looks like it's working, but I want to see what
the students have been up to for the last whatever days. This says the light blue is
exercises, navy blue is videos, and there's a lot
of other data there. There's these game mechanics. You can see the achievements
achieved and things like that. This says what a student
has been focused on in any given time period. And the students get
that for themselves. And it's funny. Because in some of these,
you look at some of the state standards, and they
say, fifth graders must understand pie charts. In seventh grade,
you must understand multi-layer,
two-column bar graphs or whatever it is, right? And in a traditional
school system, you have these very unnatural
problem sets to teach that. But what we found is when
the data is about yourself, it's amazing how fast any age
kids know exactly what-- this is not a trivial diagram. But they're able
to figure it out. And so the inner circle is the
time spent on different videos. Outer circle is
different exercises. If you click on any
of the exercises, you get a problem-by-problem
breakdown. This is one of the
more recent problems. Height is how much time
spent on the problem. Blue is right. Red is wrong. That little video icon
means you watched the video. Question mark means
you used the hints. If you click on
any of those, you can see the problem that the
student had a problem with. We've recently added a feature
where you can kind of see the narrative of the problem. Eventually we would like to
actually see the student's work as they did it on the problem. We don't have that yet. So it's really
showing your work, and it's not just a
static, final product. It's actually then
doing the problem. This right here is
just another report that gives a teacher a
picture of everything that's going on the class. But pretty much every
class we've done is starting to tell a
pretty profound narrative. The horizontal axis
is just days working on the site for the class. Vertical is just a raw
count of modules completed. And each of those lines is one
of the students in the class. And what we're
seeing over and over and over again-- right
when you started, like you'd expect in any
class, there's a group of kids, probably a lot of
us were like this when we were in fifth grade,
who gets really into it. They're really confident
in mathematics. They race ahead, and in
the traditional mind, those are the gifted kids. Those are the advanced kids. Then there's a group that
are kind of in the middle. They're doing fine. Those are the average kids. And there's a group,
they're a little bit slower. They're working on some stuff. Those are the kids
that Day 6, they're still at those bottom
lines right over there. Oh, those kids are
a little bit slow. If we start separating
these, we maybe say advanced kids, medium
kids, remedial kids. But what we're seeing
over and over again is if you let all the students
work at their own pace, if you let them fill in all
those gaps that they built up, all those little 5%, 10%
gaps that they got borrowing numbers, multiplying decimals,
whatever it might be, and you allow them
to spend as much time as they need to really
internalize something, this is an important thing. We've learned to believe that
slower is somehow dumber, which is kind of crazy. Because maybe the slower person
wants to really internalize something. Maybe they really want to
understand it at a gut sense. They don't want to
just memorize a formula and get to the next concept. But that's what we've
been indoctrinated into. But now all of a sudden, if
you give that student a chance to internalize the information,
we're seeing over and over again that a student that
a few weeks ago you thought was remedial or a
slow student becomes the second-best
student in the class or the best student
in the class. And we're seeing it over
and over and over again that you really have
this constant flipping of the leadership of
who is the best student. And you stop even making
these type of judgments. And to some degree, we
were fortunate byproducts of the old system, where
you do the snapshot, and we just happen to be
one of those top lines at the snapshot. And it really does
make you think how many people might have
been lost at the bottom end. Now the-- [APPLAUSE] Everyone's like
questioning the system. Everyone's like,
oh, my God, maybe I could have-- [LAUGHS] It's
kind of an existential, oh, eh? Which it is. I mean frankly, it really is. Now I think I've
hinted at this already, but there's the assumption-- and
I don't think it's over here, but a lot of times, there's
a knee-jerk reaction, oh, computer-based learning,
they imagine some kind of Borg or Vulcan reality
of these things plugged into kids'
brains, and they're just doing this all day. And I just want to emphasize,
it's exact opposite. These classes are more
interactive than any math class. More time is freed up to
do creative activities than any math class
or any science class or any type of class
that we've seen. And just to capture a little
bit of the energy level, I'll show you a little
snippet of a news report that captured the energy level. They don't capture some of
the other interesting dynamics that happen in these classes,
but it gives a sense of things. Oh, volume. Volume? [VIDEO PLAYBACK] [CHEERING] -What makes fifth graders cheer? Would you believe math? -Yes. -I'm starting to
really like math now. -These kids are learning with
the help of Khan Academy, an online school. -You got it right. Good job. -Videos that are
interactive and fun, explaining difficult concepts
in a conversational way. [END PLAYBACK] And there's actually
a fun story from when that reporter visited. It was either that classroom
or the one right next to it. She saw a little fifth grader
doing trigonometry questions, 10th, 11th, 12th grade type
of-- maybe for us it actually was fifth grade. [LAUGHTER] But she asked the
little girl, do you think this is fifth grade math? And she got a very
mischievous grin on her face, and she goes, no. I think it's sixth grade. [LAUGHING] And so I think she's in
for a pleasant surprise. [LAUGHTER] But when we started this, the
energy level in the classroom was great. They were interacting
with each other. Kids were teaching each other. The teachers loved it. They felt like this is why
they went into the profession. They were forming
bonds with students. They were mentoring students. They were developing
their own teaching skills and the students'
teaching skills. They were context switching. They didn't know what
would happen whenever they walked into any classroom. Some of the kids actually
did not want to go to recess. Actually our biggest complaint
we got, and frankly it came from some of
the boys' parents, were some subset of the boys
actually got addicted to it, that they couldn't
get them off of it. It's not a horrible problem. But we still wanted
to at least see some-- we were little
scared to do this because it was so early. By this point, we were like
a four-person organization. And so we didn't want to
be measured too harshly. But it was like,
hey, if you're going to do assessment exams at
the end of the year, fine, but just don't
judge us too much, because we're still
in our early days, and that's why we were a little
wary of starting this pilot so early. But we did. The kids take the
assessment tests anyway, so we wanted to see the data. And the data told us kind
of an interesting story. At the fifth-grade level-- and
this is the seventh graders, but at the fifth-grade
level, 96% of the students were at grade level or
above, which sounds good, but actually this is a
high-performing school district, and it was
only slightly better than the other classrooms. But even that was
a mini victory, because these kids were
doing trigonometry. They're doing algebra. In my videos, I don't
use the exact same words as the state standard. And in the other
classrooms, they are to some degree
teaching to the exam. And so now at least these
students didn't-- and frankly, they definitely
performed no worse. So we weren't doing
any harm, and that's an important question to ask. But even more, and we're
going to do this this year, the fifth grade
assessment did not capture what that girl
knew about trigonometry. It did not capture the algebra. There was one student who
was working on calculus. And actually at the time, he
outstripped our exercises, and he just started watching
videos and things like that. But the seventh grade was a
kind of mind-opening narrative. And it kind of gels with what
we saw in that previous graph. And seventh there's
sometimes a misperception, because Los Altos-- I actually
cannot afford to live in Los Altos. So it is a very affluent
part of Northern California. But the seventh graders
were not an affluent group. This was actually an
algebra readiness group. It's a euphemism
for remedial math. And most of these students were
students from the other side of El Camino. Most of these were
English-as-a-second-language students. Some of these were
students with learning disabilities, some of them
severe learning disabilities. Or at least they were diagnosed
with learning disabilities. And going into the
year, that's the top bar right over there, 23%-- that's
the left side, the orange. 23% were at grade level. None were advanced. And even the ones that
were at grade level were barely at the
threshold for grade level. And you see 6% were
very below basic. Far below basic is worse than
random when you take an exam. So these were students who
were really having-- they just didn't answer the test. And after six months-- and
this one really spoke to us-- double the number of
students were at grade level. But this was the thing that
none of us had ever heard of and we didn't know
if it had ever happened before-- 6% of the
students were now advanced. This was a remedial math class. This was a class that's normally
like the academic graveyard. After you go there, you
get further and further behind everyone else. And 6% of those students
who were diagnosed with learning disabilities,
who were essentially told that you're never going
to be an engineer or physicist or whatever, they
had now leapfrogged ahead of the students in the
average class and were now this year having to be
placed in an advanced class. And when they saw that,
that's when the district said, well, we have to expand this. And so that's why
they are expanding it in different ways at the
fifth and sixth grade level and also at the seventh-
and eighth-grade level now. Now the last thing
I want to show, and I want to take questions and
have a conversation after this, I want to show a video that we
actually got, a thank you video that we got a couple
of months ago. And it's an example of an older
student, but it blew our minds. And it really helped
frame it for me, is that in education
reform, people talk about the achievement gap. Once again, that's important. People's hearts are
in the right place. But I think the
correct way of thinking about it is the potential gap. Because that generalizes it. It's not about ethnicity. It's not about economics. It's just about human potential. And I think this video
kind of speaks to that. [VIDEO PLAYBACK] -My name is Mark
[? Halperstat. ?] Growing up, I was really always a C student. I think I was really
pretty much always pretty pitiful in school. I don't think I've ever gotten
higher than a B+ in any math class ever, particularly. I pretty much thought
that the only thing I was good enough to do in
college was major in music. And I went off and I got a
music degree in saxophone. But I sort of almost felt that
it was more I was getting it because I was terrible
at everything else. I kind of worked as a saxophone
player for a few years. Really what I wanted to do
was do electrical engineering. And the last thing that I
remember completely not getting was trig identities. So I went to YouTube, and I did
a search for trig identities. And Khan Academy was the
first thing that popped up. I watched a bunch of
videos in the trig playlist to kind of get up to speed. I watched all the videos
in the calculus playlist. I watched all the videos
in the physics playlist. I watched a bunch of videos
on dividing decimals and even on subtraction by borrowing. I watched a lot of
videos on arithmetic. That was in 2007. I did that until
the fall of 2010. And in the fall of
2010, I took a leap and I decided to
go back to school and went to Temple University. I majored in
electrical engineering, getting a second Bachelors. And keep in mind, I don't think
I've ever gotten above a B+ in math classes, and I was
really a straight-C student growing up. And I just finished this year,
first year back in college, and I got a 4.0 GPA
for the entire year. I got perfect scores on
both of my calc final exams and also on my
chemistry final exam. I ended both calculus classes,
Calc I, II, and chemistry with an average
higher than 100%. There are some
Khan Academy videos that I probably listened
to the same concept over 20 or 30 times. And there is no
tutor in the world I could have paid to
have sat next to me and repeated the same
thing over 20 or 30 times without at least them getting
a little bit judgmental or at least them thinking
like, oh, well, this guy really is never going to
get this concept and he should just give up. Where the understanding
really happened was watching those
videos and also working through the Khan Academy
software and everything. The impact for me
in my life, I really see it growing exponentially
over the next 20 or 30 years. So from the bottom of
my heart, thank you. [END PLAYBACK] Yeah, well, thank you. [APPLAUSE] So I'd love to take questions
or comments or anything. Yeah, right over here. The future of assessment? The future of
assessment-- well, I think there's a lot of
interesting things here. I mean, what's fun
about what we're doing is what most
people associate as assessment right now
are these snapshot things. You get them, snapshot, A, B,
C, that's the assessment, or SAT or whatever it is. Now, every interaction
with this system is a form of a practice
and assessment. We're logging everything. But it's not it's not
high stakes in the sense that this is your one shot,
and if you don't know it, your future is ruined. It's you can keep trying
until you've mastered it like a video game, like
a musical instrument, like a martial art or
whatever it might be. And so I do imagine
a reality where instead of a
one-snapshot exam, you have a narrative and data
of someone over 12 years. And it's not just
a narrative of, do they know exponents
or do they know calculus? It's a narrative of, how
hard working were they? How much time did they put in? How consistent, how good
were when they failed? Did they get back
up and do it again? Eventually maybe
we can keep data on when the students
teach each other, are there statistically
significant outcomes when you tutor me
verses when you tutor me? And if that's the
case, maybe that should be part of
your assessment, that you're an effective
mentor, that you're an effective teacher. So just that by itself, I think
that could be a good direction. And there's even something
we've thought about, because how do you give
credentials or assessment in a way that's scalable
but also in a way that's not cheatable? And we've been
playing with the idea, and I don't know if
this will ever happen, but the gold standard
is really the oral exam, which you get if you're
doctoral qualifications and all of the rest. But it's resource-intensive. And you don't even get it
at the undergraduate level. But what if you crowd
sourced oral exams? So you have Five Ninjas
in the [INAUDIBLE]. That's a cheesy name, but
you have some title, Admirals or whatever. We'd have to think
of a better name. We're Jedis of Calculus. And someone else might
be a Junior Jedi. And in order for them
to become a Jedi, five current Jedis would
have to do a Skype oral exam and maybe you record it. They are now Jedi quality. And then they can get
to the next level. And obviously there's
incentives for you. And you could even keep stats. And maybe you're always
saying no to people, other people are more forgiving. And so you can weight
people's votes even. So I think there even
is a way to even do oral exams in a scalable
way where you actually get real good authentication. But it's an
interesting question. We don't know the answer. One thing I would like to
see in the whole learning credentialing world, which
we all jumble together and we call education, I'd
like to see the credential and the learning get decoupled. Because then what you
do is actually then institutions can
optimize for either. And also then the credential
becomes a better [? signal ?] [INAUDIBLE]. None of you guys-- you're
going to get an MIT degree. You're never going
to have trouble, you go to the next state, at
least getting the interview. They're going to say, oh, MIT. I know MIT. But that's not true
of the great majority of universities in the country. You could go to Louisiana
State University, or you could go to your
local community college, and maybe you learned
the material as good as the best students out
there, but when you go, your resume is still
going to be passed over for maybe the MIT who might
have not known it as well or the Harvard grad who
doesn't know it as well. And so the signaling
mechanism right now is broken. So if you actually
decouple it so you actually have strong signals that
anyone can take that exam, and if you take it, it's
as respected as going to a great university. Then all of a sudden,
you've kind of done something very disruptive
to the whole higher education, even the high school place. But it is something
interesting to think about. Yeah? I mean, isn't that snapshot,
though, of what we have right now with SATs and GREs? So it is a snapshot, except--
so SATs and GREs are not bad as long as we're
beyond the culture of, you only get to take
this when you're 17. I'd like to see a
world-- and I've actually given this advice to people
who are having trouble getting a job. Study for the SAT and
do really well on it. If you get 800 on your
math portion of the SAT and you do nothing else,
you will get an interview even if you're 50 years old. That will get you an interview
at whatever job you have. So I'd like to see a reality
where, yeah, anyone at any age, there's no stigma associated. It's a one-time, high-stakes, 16
or 17 years old, if I screw up, I'm never going to get
into a good university. It should be there are
these assessments-- and we can debate whether
that is a good assessment. You can always improve
on the assessment itself. But yeah, at any time, I
didn't do well the first time, but I'm going to make it my goal
to eventually do do well on it and then use that as a signal to
the world that I can think well and I'm logical and I'll
be able to contribute. But it's a fair question. We'd want to make
the assessments as good as possible. [INAUDIBLE] Oh, OK. OK, whoever has the
microphone wins. A lot of study is being
done to individualize the learning profiles, ie,
give the child the way he best learns certain content. How do you think about that? So a lot of the psychological
type studies or the-- so I have mixed
feelings about-- there's a large body of research. And we don't want to
seem arrogant about it, but the reality is,
it is really hard to determine, especially some
of these education studies, how well was the study run? I mean, it's almost impossible
to do a truly one-variable change and really pinpoint that
that was really the reason why. And I think because
sometimes when you focus too much, especially
in these areas where it is hard to make a definitive
conclusion about this was the cause why the students
did as well, I mean, there are theories. It's sometimes a mistake,
I think, to just take that and say, oh, I
shouldn't try that out. Or this is the best
way to do that. And so our approach, I
think the way we view it, there's actually two
approaches to it. There's the science approach,
and there's nothing wrong. I mean, we're trying to do that,
too, which is control study, change a variable,
very rigorous process. And some of that is going on. But what we want
to do culturally is I would call it
more of an engineering take on it, which is
put stuff out there, get a lot of traction, collect
as much good, rigorous data as you can, and then iterate it. And iterate fast. And do AB testing. Do control studies,
but in real time. And we have three experiments
running right now on our site where we change one
variable, two variables. And we're able to run
the studies ourselves. Now with that said, we
actually have the guy who's leading up our
analytics who was actually the head of high frequency
trading at Citadel, which is a large hedge fund. And so he also found
meaning and all the rest. [LAUGHING] Although I think
he found more money than I did before the meaning. [LAUGHING] But what he's also doing
besides right running these analytical experiments
and all of that is he is looking at the literature. That is something
that he enjoys doing. That's actually what he used
to do in the hedge fund world. He used to look at the finance
literature for something that could be a source of
alpha and that he would then implement in some
of their algorithms. And so he's now doing that. And in his mind, he thinks 2%
of the research is actionable. And so he's out there
digging for that 2%. But some of it is good. And so he's out
there digging for it, and we are going to
implement the stuff that we think is in a very
sound way proven to be good. Next question? Who has the microphone? Hi. I had a question
about-- I know there's a lot of research
to reform education systems in other countries,
especially in London, I think. Have you done any research
into other methods or plan on marrying the
different ways, I guess? It's kind of the same-- I
guess there's two parts. Well, I guess on the
research side, I mean, it's kind of a similar
answer, is that we are open. And I mean, if any of y'all
know really good studies that we should be paying attention,
really good ways to address this topic or that topic or help
students visualize fractions or whatever it is, we want
to implement that stuff. So point us to it. What we're resistant
to is making that where all of our focus
is, because then it becomes hard to build fast and
iterate and see what's working. I think another
part, I mean, you're kind of hinting at
the international. What do we do internationally? It's hard for us right now, and
it's a good problem to have, but we're getting pulled
in a lot of directions. We actually are getting
calls from the ministries of education from
countries and saying, oh, we want to run a pilot. We want to do this. And it's very tempting. It's like, wow, that
would be big deal. But the more we
think about it, any of those one-off
initiatives, especially if you're working
with the government, frankly, it could
overwhelm our-- I mean, we're a 20-person
organization now. We've just gotten to 20. I mean, we were one
person a year ago. And so we've made the decision
that in the very near term, we're going to be very focused
on English in the United States and try to be as
good as possible. And if we succeed
here, then we'll eventually be able to
do the rest of the world if we spread ourselves too thin. But one thing we
are doing, and this was with some of
the Google funding, we are translating the video
content into 10 languages. So that is going to be there. Spanish, Portuguese, and
Bengali are almost done. And then we have another
nine languages in the hopper after that. So just that I think
could be valuable. You could put then on
thumb drives, DVD players, and bring them out into
the field or whatever. It's not the full experience,
but even the full experience, the world isn't ready for
it because of bandwidth and all the rest. But hopefully those problems
might solve themselves. The microphone wins, I'm sorry. Do you have any thoughts about
things like proofs, engineering design problems, or even things
like compositional writing or public speaking or just
other areas in education? Yes, so we have all these
other areas of education. So how do you address them with
something like a knowledge map or things like that? And our mindset is education
is all of these things, starting with basic introduction
to new concepts and ideas. And that's what you
can do with the videos. Basic practice
problems, that one you can do with the exercises. And then you could have less
basic practice problems, more do proofs, write essays,
write computer programs. You keep going up that chain. And the way we
perceive ourselves is, right now the
education system has to do all of these things, or
should do all these things. And frankly, it's
focused right over here. Because it's even
failing at this stuff. And it never even gets
to some of the stuff that you just talked about. And so if we can make
this stuff really good and make it says
systematic and be thoughtful and logical about
making it better and better and iterating on it and
slowly creeping up that, eventually maybe we will
have ways for students to write essays and
they can peer review. And someone was telling
me there's a project here for chunk up essays and
another people edit it and things like that. I think there will be
ways over the next 5, 10, 20 years to address more
and more and more of that. Actually John is
working on a project right now for
computer science where it will be a hybrid of the
exercises and the videos where you can record a
video using his tool, and if someone pauses the
video, they can actually then edit the code at that
state right there. So it's actually an
interactive video where the state is changeable. So it's a bit of
a puzzle of how. It's a bit of magic to be able
to think how you could do that. But yeah, I think
that's our job. All of those things
are super important. And right now, the solution is
if we can take this part off of the traditional
teacher's plate, then they can hopefully
move to that part. That's an important thing
I like to emphasize. We think we're moving
teachers up the value chain, not replacing them or
anything like that. Microphone? Yeah? Thank you so much for what
you've done for education. I'm curious-- I've
heard studies saying that there are about
10 million scientists and engineers in America. If you could crowd source
to them to help be teachers, kind of like the way
you're democratizing teaching with MIT,
what would you call on them to help you with? And the second
question is, what are some important
strategic partnerships you might want to do in
the next year, two years, three years to take your
work to the next level? There's thousands,
millions of engineers and other people who could
be mentors, teachers. And even before
I started this, I would have loved to somehow,
while I had my day job-- I mean, that's what I was doing. I kind of had to work hard
to do it with my cousins. And there's actually
a project I heard of called Cloud Grandmother. It's literally like a
bunch of grandmothers who are there kind of on call. And if you need a grandmother,
you press a button, and there's a grandmother. And you can tell
them, talk to them, ask them what life was
like, where they grew up, and whatever else. [LAUGHTER] And just that I think is
a pretty powerful idea. But imagine if it was
cloud people who knew math. And you're at Khan Academy. You've watched the videos. You've done the exercises. We say, hey, this person still
seems to be having trouble. Do you need help? And all of you guys, maybe
you're at MIT or maybe once you're working,
you're like, hey, I have lunch from this
time to this time. I could be part of the
cloud engineering team. I'm having office hours. So if any student
needs help, I'm there. And if you have enough
of those students, it would be
perpetual, year round. At any given moment, you
would have 1,000 people waiting to help people. And you have to
think about things like safety and all that. But I think you can have
a reality eventually where a kid has a problem,
click a button, one of us shows up and says,
well, let's see. And you would have access
to all of their data, and you could be informed, and
you could actually help them and it would be recorded
or whatever else. We're not anywhere
near there yet. I mean, the technology
is already there. I mean, in theory that
could be built tomorrow. But I think something
like that could be pretty-- I think something
like that could be interesting for foreign language, too. I have a 2 and 1/2
year old, and I'm thinking about getting him
a Skype buddy in like France and Beijing. And so like they can do their
2 and 1/2 year old thing but in each other's languages. [LAUGHTER] And I think that
could be the closest thing to cheap
immersion type of thing. So, yeah, yeah. [LAUGHTER] Microphone? Oh, yeah, right over there. Hi. You talked about bringing that
into that one school near you. And I was wondering
if you have thought to bring it to other schools,
if other schools have like contacted you. And it just seems
like it would be a huge scale problem but
really awesome if you could. Yeah, so the question is we
did it with those two schools at one school district. How do we scale this thing? Is that even what
we should be about? And in our organization,
we've decided one thing. We have 3 and 1/2
million students a month. Most of them are not using
it in a formal setting. And I think one of
our secret sauce is that we've been
focused on the student. And so that's always
going to be our priority. But this school
pilots makes-- I mean, maybe more schools
should be doing this. But how do you scale it? How do you do that? And so this year we're doing
the whole district of Los Altos. It's not a huge
district, but it's on the order of
about 1,000 students in fifth and sixth grade. And then we are working
with other schools, a couple of charter schools,
Summit Prep, Kipp Academy, Eastside Prep in East Palo Alto. And we're seeing how it
works in those settings, and can we get similar
type of results? And we don't think
our role is going to be to hand-hold every school
in the country to do this. We would never be
able to do that. It would require thousands,
millions of people to do that. Our role is to do it
with this group right now and then see what works, what
doesn't, document what works and then kind of make it
self-service, put it out there so that other--
so right now, it looks like from our data
about 6,000 classrooms are doing it in some
way, shape, or form. We have no contact
with them, We don't know if they're
doing it well or not. But if we can make
it easier, maybe we can make that 60,000 or 70,000. And maybe-- I mean,
this is the fun thing about being a not-for-profit--
we have so many people who want to volunteer. And we're not leveraging
that right now. But maybe if you
guys are like, hey, I want to try this out
with a local school, I'm sure most of y'all in
this room based on this talk, y'all could go do it. Or you could visit
us out in California, visit some of the schools
there, see how that works. Then you could adopt
your own schools. I think things like that
will help the scale. And I don't know
if this will work, but the real thing
that will happen is-- and this is where I think
the rest of education reform maybe fails. It's been all top down. Let's pass a bill. Let's pass a law. Let's do this to union
contracts or whatever else. What we're saying is,
let's prove the case. And let's show the
world that it works. And when, frankly, the
mothers of the world see that there's a girl in
Los Altos doing trigonometry and their kid is not,
they will go to the PTA. And they will say, why
aren't you doing this? And it's free. And so there's no reason. Well, we just
don't feel like it. I mean, I think the
mothers of the world can be a very
powerful instrument. [LAUGHTER] One very serious problem
we have is that the science and, let's say,
engineering teachers in middle and high school don't
know what they're teaching. Have you thought about
engaging the teachers in your extraordinary program
and getting them involved? It's something that
comes up a lot. Do we do teacher training? And it was actually something
that came up actually very early. Even when I was just
doing the videos and it was just a
part-time thing, I got a letter from a very
large school district, an email, saying, can we use your
videos for teacher training? I said, well, why don't you
just use it for the students? Oh. no, no, we can't do that. We'd have to use it for--
and I guess philosophically, I mean, we don't mind it being
used for teacher training. But in our minds, I think
that's where a lot of ed reform efforts have failed, is
that they focus so much on the teachers, which is
not a bad thing to focus on, but it's usually at the
cost of the students. And so in ours--
especially because that's where most of our users are. We're going direct
to the student. We would teach teachers
if it's teaching teachers on how to become
better shepherds of their other students,
so if it's a new model. But it's not even clear that I
know how to-- it's not clear, I guess is the simple answer. But if it's going to
help the end student, we'd be open to it. But yeah, so that's
our core philosophy. Yeah, next question? Oh, right over there, yeah. So I've done a lot
of learning, and I feel like it's really productive
to learn through experience and first-hand learning by
actually doing something. You talk about actually being
a lab and testing things, for example. So I guess is that one of
the goals of Khan Academy is to actually get
to that point where we're teaching experiences
more than just skills? Or is that [INAUDIBLE] teachers? Yeah, so the question is, and
it's related to the question up there, is Khan Academy
all about teaching skills or is it eventually
going to be experiences? And once again, the
experiences are up here, these simulations and games
and all, whatever, projects. And so we just have to keep
working up that ladder. I mean, I told y'all about
that trading game for risk. When I did it with
the kids, they each had a piece of
construction paper that was one of the
colors on the board. And every trade, they
had to put a little slip on what the exchange price--
they would give it to me, and I would put it into
a little spreadsheet so that I would actually have
a chart later on to show them. But even then, I said, man,
we should have a little simple trading platform. And if we build it, then
everyone could do these. And then you don't
have to do it for risk. You could do stuff
like decision markets for who's going to win the
next presidential election. Who's going to win
the Academy Awards? Who's going to win
the Booker prize? And then all of a sudden you're
doing something analytical and fun and game-like,
but to win it you have to start
reading a lot of books. And you have to start doing
other interesting things. But tools like that
enable experiential things that right now are very hard. I mean, it's hard for a
teacher, especially one that didn't work at a hedge fund
to think about how a trading floor works or what
a market maker is and what a specialist is. And we had all of these
things with the middle school students. So yeah, we want to get there,
but we have a lot ahead of us. Hello. What would you think about
MIT's project OpenCourseWare? And do you have any
plans integrating with this great project as well? Yeah, and I will tell
you, frankly, in 2001-- I don't know how many of
y'all-- I don't know if it was '99 or 2001 that MIT
OpenCourseWare announced. 2001. But I don't know if y'all
remember the environment then. Almost every other
major university was exploring some type of like,
we're going to monetize this. We're going to create some kind
of distance learning thing. People are going to
charge money for it. And frankly, it was one of the
proudest moments of my life as an MIT graduate
when MIT just threw the gauntlet down and said,
no, this is for the world. This isn't about us trying
to make money off of it. And we're just going
to give it away. And so I think
that that was kind of a groundbreaking thing that
was done very early on at this. And it really changed a
lot of people's thinking and was a lot of leadership. I think the things
that it's faced is the content wasn't
purposed to this consumption. And so I'm actually probably
one of the biggest consumers of MIT OpenCourseWare. But it's like Lecture
9 of Thermodynamics. And you're like, OK, well, let's
see what happens in Lecture 9. [LAUGHTER] And you go, and I was
motivated to get to that nugget that I was trying to understand. So I think there's
things that MIT can do and maybe we can do together to
make it more special purpose. That's why I'm excited
about this project. Because it is coming from the
students, by the students, and the faculty,
but it is special purposed for this
type of reality. So I think it has a much better
chance of really becoming something that gets a lot of
traction, gets a lot of kids excited and learning
about different things. So I see no problem
with me in mixing the Khan and the MIT
brand-- once again, two major institutions. [LAUGHTER] Yeah,
right over there. So I think I recently read
that you won a $5 million grant from O'Sullivan's to
build a physical building or physical academy. And I was wondering what you
were trying to do with it or what your plans are. Yes, so we did get that grant. It's actually for
several things. One is to actually
expand our faculty. And we actually brought on two
art history professors already. I encourage y'all to
watch their videos. They're pretty interesting. And they had already
made a lot of videos. And no one had seen them before. And we were able to just really
give them the validation. So they're there. A part of that
grant, about a third, is actually to build a
platform so that other people can use the data, the
analytics, the platform for their own content. And maybe eventually we can see
the videos and the exercises that are really good, and we
can expose those to the world. But then another third is
to explore a physical thing, a physical experience. And we want to eventually get
to a real physical school. But we're going to start this
summer with a summer camp so that we get full ownership. And now we're going to be able
to do these things for real. We have real people. OK, what are the games? What are the
experiments we can do? And as we learn
more and more, we can then document
them or maybe even make software tools
for these experiences and then allow any math teacher
or science teacher or whatever teacher in the world
to actually do them. But we do want to do a school. Because I think when
you do a school, it's not about
incremental change. You can rethink everything. You can rethink the
whole model of what it means to be a school. Because as soon as students
are working at their own pace, why separate them by age group? If teachers aren't
giving lectures anymore, do you need to have one teacher
, 20 students, one teacher, 20 students, one
teacher, 20 students? Can't you have three
teachers, 60 students? Can't you have everyone
working together? Can't you have older
students mentoring younger? Do you have to separate calculus
from physics, from chemistry, from biology, from writing, from
logical, from computer science? Or can all of that be happening
in one big, epic room? And if a student gets inspired--
I mean, it's crazy right now. I was speaking to one of
the founders of Facebook, and it's crazy. If a student gets inspired right
now in high school and says, I am so blown away by genomics. I want to go do research
in that for a month. Too bad, no, no. You have essays due tomorrow. You have a problem set. You have all of this stuff. No, sorry, no creative
activity here. You have too many things. You're on the assembly line. But all of a sudden, if students
are working at their own pace, there's no such thing
as missing school. If your family wants
to take a trip to Latin America in October, you can. You're not going to miss school. And you can even do stuff there. Maybe you don't even
have one physical site. Maybe you have 20 physical sites
distributed around the world. And one site has a
couple of teachers that are experts in art history. One site has experts on physics. And so if I'm in one site and I
needed help with string theory, I can get on Skype
and have an experience with the teacher in London. But I still get the physical
experience and the community and the social experience
of the students at my site. So there's things that
we could completely rethink where we're getting
the best of both worlds. We're getting the
academic scaffold, and we're getting the
projects, the explorations and the startings of things. And we're actually
code naming the school we're starting Hogwarts. [LAUGHTER] No, and we'll see what happens. I have a 2 and 1/2 year old. So it's for selfish reasons
that I want this school to exist in the next few years. Sal, great talk. Thanks for coming. I think I speak for at least
a fraction of the students when I ask, are you hiring? I'm I hiring? Yes, and that's my
reason for coming. [LAUGHTER] Frankly, this whole
thing is I think predicated on getting the
best talent out there. And actually I was at
the business school earlier this morning, and they
have a case on Khan Academy. And the case was,
well, this is great. It's getting traction, but
how does this not-for-profit compete with Google
and Facebook? And actually that was a question
we were asking ourselves. But the one thing we found
is-- and I will tell you, we are paying upper
quartile of the Bay Area. And that's something
our board decided, that this isn't about
nickel and diming people who want to do good
for the world just because it's a not-for-profit. This is about creating
the single best content you can create
for the world to use maybe for a long, long, long time. So you don't want to short
change what that experience is. So we want to get the
absolute best talent. And frankly, the most reassuring
thing about this experience is people like John
Resig have joined. I mean, Johh-- he literally
wrote a cover letter that says, I wrote jQuery, the
most-used JavaScript library in the world. And I was like, yeah, I think
we'll interview this guy. [LAUGHTER] But he's the most
famous of-- I mean, you have more Twitter
followers than I do. But he's not the only one. I mean, he can testify
for the rest of the team, they are somewhat known
figures, or they should be known figures in their fields. I mean, [? Jace ?] who
came from Citadel, I mean, there's no better
analytics person. I mean, he could be
managing $10 billion today, and he could be making
$100 million a year. And he's deciding
to do this week. We just got-- I won't
tell you who it is, but there was
another gentleman who was very early at a very
prominent tech company that's a very prestigious tech company,
and he came, essentially we assumed as a philanthropist. Maybe he wanted to write a
check, support us in some way. And he brought his
foundation people with him, and we talked a little bit about
the projects we want to work on and what we could
use and all of that. And then his people kind
of walked out of the room, and he just kind of sat there. And we're like, oh, this is
weird, like, what is this? And he said, well, I
actually want to work here. And I'm like, wow, this guy
could like buy an island, build a roller coaster, and
pay people to ride it with him. [LAUGHTER] And he wants to like be near us. And so yes, I encourage
y'all all to join. And we're looking for
awesome, creative people who want to impact people
through videos and software and everything else. Oh, up there. Sorry, the microphone wins. Hi, over here. I was just wondering if
you could comment on what your thoughts on the
scaling education that's inherently experiential. So it could be like the trading
game or the chemistry lab or like learning how
to play an instrument. These are things that are
hard to drill down and be so reductionist in the approach. Yeah, well, like I said, part
of any experience, part of it is the more you get the skills,
you get the basic concepts, and then you eventually--
so right now at minimum, we're going to get that part
off the plate of the resource intensive, which is the human
interaction so that humans can focus on the more
experiential things. But then over time,
we can implement some of these trading games. We can implement
ways for students to connect with each other
and tutor each other. We even thought maybe
you have, after a video, someone can ask a question
on a video in a video format, and the next person
can answer the question in the video format. You have message
board threads based on people sharing
the same blackboard and answering each
other's questions. You could have like that
trading game I talked about. We could implement the
software so that any classroom can create decision markets. We did another game
in that summer camp where the kids had to
guess how many popsicle sticks where in a pile. And then they took
pictures of the popsicle sticks and put them
up on Mechanical Turk, and we paid people a penny
to vote on how many popsicle sticks there were to see
if the wisdom of the crowd really works. And we didn't know
what would happen. And so these are the type of
things that at minimum we can document and share with people. And even better,
maybe we can put it on the side in terms
of games and simulations and manipulatives
and whatever else so we can go further and
further up that chain. And I want to be clear. I don't think we'll ever be
able to get the ideal experience purely through software. But it allows a human
to take the experience that much further. Yup, right here. Hi. So I was wondering. I'm a really big
fan, and I'm seeing like the future would be
integrating this system into school systems
as it has already been in California schools. But one concern I
would think of is a school is not
just an environment for intellectual growth. It's also for emotional
and social growth. But Khan Academy is vary based
on individual-level learning. So I was wondering what
your thoughts were. Yeah, no, that's
a good question. And I think this is one
thing we hear a lot of. But there's one thing I
want to-- when I described that classroom-- so I agree. Education is actually
three things. It's learning,
it's credentialing, and socialization. And people say that a lot. In fact, a lot of
traditional educators, oh, it's about socialization. Don't want to involve computers. But how much
socialization do you get in a traditional classroom? I'm not saying all
classrooms are like this, but a lot of classrooms I sat
in, 30 kids, completely quiet, listening to a
lecture, taking notes. Absolute no social. In fact, it's anti--
its dehumanizing to be in a room with people and
not being able to talk to them, not be allowed to
talk to them, not to be able to express yourself. And so what we're
advocating, even if you're talk about the core
skills, we're advocating, no, we want kids to get up
and go help their peers. We want them to
mentor each other. In some of these
classrooms, when a kid is about to
achieve something epic, the rest of the kids
gather around them and start cheering them
on and then all the rest. And so even that is far
more social and interactive. And we actually want
the best achievements not to be that you
mastered 100 modules and you know calculus now. That's good. That's a good achievement. But it's even a
better achievement if your peers think
you're the best tutor. Your peers have that
respect for you. You're the best communicator. You empathize with your peers. And once again, the
spectrum of activities that are on the
learning side, there are the core skills,
understanding, one, learning how to do it,
but then understanding what it is, then applying
it, all of these things. If we could take this off
of the plate of class time, then class time can be more
for the games, the simulations, building robots, whatever
else it might be. So I think we can actually
be a catalyst for better socialization. I mean, my middle school
was Lord of the Flies. I mean, it was not-- [LAUGHTER] So I think there's a lot
of improvement, actually, we can do on socialization. Actually I think, frankly,
socialization right now is more broken in schools
than the actual academic part. Yeah? Oh, well, she's got one, too. OK, well, we'll go here next. So yeah, I actually didn't
know what Khan Academy was before I came here. You have like a halo. I feel like I'm speaking
to a prophet or something. [LAUGHTER] So yeah, I mean, I think
this is really cool-- I'm taking this
question very seriously. I'm sitting down here,
and I'm thinking, I want to go back
home, and I want to use this in public schools. Because if you go to a
public school back home in Puerto Rico, you're probably
not going to go to college. It's that bad. And I think this will help
my little brother who's having trouble in math, too. Something that I think is there
are going to be a lot of kids, actually, that don't like
math, don't like science. How do you get them
to finish the program? Where is the motivation? So there's a couple of things. And for a place
like Puerto Rico, we're actually translating a
lot of the content into Spanish. That will be the first
language that we expose and hopefully port,
eventually, the software. Although that's not something
we're able to do tomorrow. But the broader question is
how you get kids engaged. I mean, that's the fun
thing about the internet and this kind of
engineering mentality and iterative mentality
is that Amazon.com asks itself that same question. Well, not everyone is
going to buy a book. How do we get more
people to buy a book? Well, we can apply
that same methodology. OK, let's try this. Maybe if some people
watch the video of Mark, that will motivate them. Maybe if someone sees a
video from someone else or you give a talk or you
change the color to something, maybe that's what
does the trick. So I don't know if we'll
ever be able to get everyone into that motivated
bucket, but I suspect as we keep testing and iterating
and looking at the data and looking at the data, we'll
get a much larger spectrum of people. And we're already seeing that. I mean, even in the classrooms. That seventh grade
pilot, even day one what blew the
teacher's mind is this was a class where every
student was disengaged. This is why they
were in that class. And it was a video
game for them. And they got engaged. There was a healthy level
of competition and fun and camaraderie around things. And so I think the fun thing
is it seems like it's already moving in that direction. And then we can keep
iterating it and improving it so it gets even better. And I encourage some of
you guys to join us so that you can help us do that. And if you like
[? cited ?] like numbers, I would go play the
lottery right now. [LAUGHTER] Next? Oh yeah. right here. So traditionally, we
all learn from somebody who's older than us,
like even on MIT seal. MIT seal shows an older
figure and a younger student, a mentor-student relationship. So if your ideal world plays
out where the age won't matter anymore, do you
think there will be any problem, like
a social problem, with people accepting younger
students teaching them or younger people teaching them? And that's an interesting point. And I don't know 100%
if I know the answer, but my gut feeling right now is
right now we have that problem. Right now, because we
associate mathematical concepts or scientific
concepts with ages, there's huge stigmas, right? A logarithm is a more complex
thing than most things that people encounter
in their everyday life. But still, 40-year-olds
are embarrassed to admit that they don't understand
what a logarithm is. And that's because
it's associated with a seventh-grade
math concept. But they're not
embarrassed to go to karate class and
be a green belt, and there's a
little nine-year-old who is a black belt. And they're not opposed
to saying, hey, well, how do you do that? And so it's really about
removing the stigmas, complete decoupling the concept with
these arbitrary age ideas. And I think when you do
that, I mean, frankly, I think one of the
things that's a lot and I think we're helping
a little bit here, too, is parents are
afraid to admit what they don't know to their kids. They think it will somehow
undermine their authority. And so one, I mean,
Khan Academy can help them stay ahead a little bit. [LAUGHTER] Which we've learned
is happening. But even better,
it can decouple. We've had so many
requests, well, can you tell us on
that whole [INAUDIBLE], what's a seven-year-old
supposed to do? What's a 13-- we're like, no,
we're not going to do that. Because you're supposed to
do whatever you're ready for. You could be 50, but if you've
never seen math in your life, start right up there
and just keep working. And I think that will be--
I don't mind learning. I mean, it's weird. Academics sometimes makes
me feel weird learning from someone younger,
but anything else, I feel completely fine. And that's because
of these stigmas. Right up there. With so many K through
12 students, specifically middle and high school,
invested in social networks, what are your opinions on using
social networks in education? Yeah, I joke the Facebook
implies that it's a Facebook for a school. So we should use Khan Academy. But I think there's a lot
of things that we observe. The same way we can look
at Amazon and see, OK, Amazon uses the techniques
to-- look at Zynga. We could be a productive
Farmville in terms of-- [LAUGHTER] But it's true. They've perfected some
very-- people are paying them to water fictional crops. I mean, if that works,
imagine-- but at least they're blazing new territory. And these are directly
applicable to the type of stuff we're doing. So I think there's
a lot to be said. I think we're going
to explore it. And we'll just see. We do very minor Facebook
integration right now, but we do imagine a world
where you achieve things. You can share it
with your friends. You can do friendly games. You can do multiple people
together can achieve something with your social network. We're not doing it yet,
but it's definitely stuff that we would love to explore. Yup, right over here? Considering that students learn
so well from all your videos and that you've
made 2,800 of them, are you just a ninja on like
2,800 different K through 12-- Or a Jedi? Are you a Jedi? My wife will tell
you, and maybe I learned this in business
school, the ability to talk with
authority about things that I really don't
know much about. [LAUGHTER] [APPLAUSE] It's a skill. But I think I'm not too
different than many of you guys, is that I love learning. And my life is now-- I get to
learn things and be motivated and teach it, and
a lot of people can experience it
because of that. But the simple answer is, I
mean, 2,800 sounds like a lot, but you do four or
five videos a day, it does add up
very, very quickly. And yeah, I suspect many
of the people in the room could do what I'm doing. And I think it's just about
mindset and willing to just go put yourself out there and
just do them and put them out there and see what people
say and keep iterating and keep doing it. I mean, a lot of
people wanted me to do organic chemistry videos. And that was the first-- I
was like, I've got to do this, but I took 512. And I did all right, but
I didn't feel like I truly understood organic
chemistry the way I might have physics or math. But I was like, no, if
I'm going to do this, I have to understand
it at that level. So I immersed myself in
chemistry and organic chemistry for a month, read
everything I could find, went to the used bookstore,
old Russian books I was trying to decipher. And I kind of started to get it. Oh, there is a kind
of a theme here. And there is an
intuition to this. And then I started
making the videos. But so yeah, I think
what I bring to the table is that I love to
jump into something and immerse my brain in it, and
eventually it kind of clicks. And frankly, that's why I
liked the hedge fund world. I talked to a trucking company--
y'all should work at Khan Academy, not a hedge fund. I don't want to make
this a-- but you work at a trucking company,
you context switch, and you learn about all
these different industries, and you learn about
how the world works. And so it's kind of the
same type of mental fun, so to speak. Yup, other question? Eventually we'll get the
microphone to this gentleman here. This kind of goes back to
the issue you had with Nadia where she was good
at solving puzzles but kind of got stuck
in a particular concept. One of the biggest problems
in current math education tends to be people learn the
concepts of math very well and learn how to apply
it, but they don't really learn the beauty of it. And once they get
to a certain stage, they can't really
think outside the box or learn proofs and do
[? original ?] research. And what do you
think about that? And how do you think Khan
Academy can adjust that? No, I think that's
the problem with some of the top-down
curricular development, is that at no point on
the Common Core standards that students should be in awe. At no point does it
say, blow students minds with Euler identity. It's not there. Even though that is
what it's all about. I mean, that's the
stuff that gives you-- and it's funny, because so many
people are fixated in like, oh, well, no one is
going to like math unless you show applications. And look, if you can
show applications, show applications. There is something
valuable about that. But what I always point
out is in math class when people say, when
am I going to use this, they're saying that
out of frustration. And the reason why I say
that, no one ever says that in a philosophy class. When am I going to use this? When are we going
to have to care whether there's life after
death or whether there's some platonic cave or whatever? No one says in PE,
when am I going to have to put an
orange ball in a net? What is that good for? No one says that. [LAUGHTER] [APPLAUSE] Not one says that. And that's because
they're stimulated. They're motivated. They see the fun. There's something visceral. There's something human
about, in their minds, those experiences that they're
not getting in math class, even though math class
is the purest philosophy. It's not just word games. It's actually going to
get you to something about the universe that's
being told, something deep. And it's being lost right now. I talked to a couple of students
who made some of the videos, and one of the
students brought up, we have to give the students
the why of doing this. And the superficial
interpretation of that is, oh, we have
to show that it's useful in banking or
accounting or if you have to build a building. But the why is actually, I
think, an implicit message that's given in the
awe of the giver. And if you give a
lesson like this, even if you give a
million applications, by the tone of
your voice, you're telling the student there's
no reason to learn this. You shouldn't care. But if you really are in
awe of the subject material, if there really is something
mystical and powerful about it to you, the student
will get the why. They'll say, wow, this is
the truth about the universe. This is telling me something
about how everything all fits together. And I think yes, if any
of y'all make the videos or if y'all want to volunteer
making exercise modules, those are the winning videos. Those are the ones
that will-- they all don't have to be there. Not all of my videos are these
epic, mind-blowing experiences. But if you can, I mean,
that changes everything. Yeah? Do you think that
a textbook could be made to blow students' minds? How do I feel about textbooks? Yes. And what was the second part? Do you feel like
a textbook could be made to do all the
Khan Academy things, blow students'
minds in this way. Or is there something in the
interactivity that's necessary? I think good books can. I mean, I've read
books that blew my mind, that are
saying something, the idea there is profound. Actually, science fiction
books, many times I'm, wow, that is a thought-provoking
idea, or not even just science fiction,
any type of book. Textbooks don't do that. I mean, sometimes they
have a little orange box, like, oh, so-and-so
works as a contractor. And it's just like,
it's not mind blowing. And I think
textbooks have value. I have a bunch of them. They are good references. But I think they have
to introspect on where that industry is going. Because I think a textbook for
most people does two things or is supposed to do two things. It's supposed to
teach with some text. And I think, frankly, what
differentiates people who can go to MIT versus
people-- are people capable of reading that text? I think most of us are
actually decent book readers, but we're actually
a small minority. Most students cannot. They've never had
a good experience being able to learn from that. And then the practice is
just a couple problems at the back with every
other question in the back. And the publisher
doesn't know who's reading what,
what's adding value. They can't iterate on it. And so it is, I think,
an interesting question for a large industry. Let me make that
the last question. Oh, yes. [INAUDIBLE] Sal's been going at this
turtle's pace all day. [LAUGHS] So let me first
give you a last couple words to the group. Yeah, thank you for having
me, the dean and everybody. For me, it's so exciting
just to-- I don't know. When I come to MIT, I
really feel like I'm with family on some level. And I get teary-eyed
and [INAUDIBLE]. So I'm really
honored to be here. Thank you. [APPLAUSE] In this epic room, you are
definitely the Education Jedi. So it's been great
to have you here. It has been great. Oh, thank you. [APPLAUSE]
