[MUSIC PLAYING] MARK GOLDSTEIN: Hey,
welcome everybody. Welcome to "Talks at Google." We're coming to you live
from the Googleplex in Tunis, and everybody out there
in livestream land. We're going to be talking
about Claude Shannon today, someone that many of
you may know, and many of you may not know about. Machines-- machines that
play chess, machines that juggle, machines that
are flame-throwing trumpets, and yeah, how about the
curation of information theory? It's all about Claude Shannon. And he's one of the
most important pioneers of computer science in our
digital age that most of us barely know about. But no longer, because
Jimmy Soni and Rob Goodman have written this book,
the first full biography of Claude Shannon. And they're here today
to talk to us about it. So come on and
welcome Jimmy and Rob. [APPLAUSE] [INAUDIBLE] my seat. OK, so let's start briefly. Can you describe why Claude
Shannon is so important, why we should care
about him, and what were his major contributions? JIMMY SONI: Yeah, take it away. ROB GOODMAN: Wow. So I think Claude Shannon
was important for a couple of reasons, and
obviously we don't need to tell this crowd why
he made such a big difference. But we wrote the
book, in a sense, to do something
about the gratitude that we have to someone
like Claude Shannon, who is responsible for so
much of the technology we rely on every day. Scientifically, we think that
his two big accomplishments were his 1937 Master's thesis
where he does an important job of laying the foundations
of digital computing out of 0's and 1's. And a big part of
that accomplishment was showing the connection
between Boolean logic and the relays,
or switches, that were part of computing systems. And Shannon got
his start working on giant analog
computing systems, but he really did a lot
to uncover the potential of digital systems. Chris Dixon wrote a great piece
about this in the "Atlantic" that we read a
couple of days ago. And the way he put it, I
thought, was really evocative, and I wish we had stolen
this for our book. He says that Shannon thought
about how to map logic on the physical
world, and I think that's a really succinct
way of putting it. The other big contribution--
by the way, he was just 21. So it makes me feel like
crap, because I'm already 33. I'm not going to
live up to that. But anyway, at 32-- which is still younger than me-- Shannon releases what a
lot of people consider his masterpiece-- which, again, this
crowd probably knows very well about, it's his mathematical
theory of communication-- in 1948, which does
a number of things like exploring the properties
that communications systems have in common, coming
up with the concept of a bit to objectively quantify
information, and then, of course, coming up
with those digital codes to compress information, and
also to accurately transmit information in a noisy channel. And the reaction to this
was just remarkable. John Pierce, who we were
talking a little bit about before we went on,
said it came as a bomb. And that's what we called
the chapter on Shannon's communications, his
Information Theory Paper, The Bomb, because that's the
kind of impact, of course, it had in the science and
engineering community. And we were setting
out to explore why this matters
for our lives now and why people, even without a
science or math or engineering background, ought to know
what information theory is and what information
theory makes possible. And do a little justice
to Shannon, who deserves a little more name recognition. MARK GOLDSTEIN: Thanks. Now, neither of you are computer
scientists or mathematicians. ROB GOODMAN: Oh, you got
us, much to our chagrin. MARK GOLDSTEIN: Yeah, so
can you tell us a little bit about your backgrounds? How did you learn about Shannon
and come to write about him? JIMMY SONI: So I had been
gifted a book called "The Idea Factory" by a
friend, and the book is by a guy named
Jon Gertner, who became a close advisor to us
as we're doing this project. And "The Idea Factory" is a
narrative history of Bell Labs. The way we joke about it-- and
this is not meant as an offense to anybody in the
room-- but we said if you could envision a freak
merger of Google, Apple, and Facebook, that was Bell
Labs in the 20th century. It was the second-largest
employer in the country after the federal government. It had, basically, a
government-backed monopoly ownership of the
telephone system. So it had enormous resources. And in a way, it embraced a lot
of what the people in this room embrace, in the sense that it
gave people remarkable freedom and flexibility to do
different kinds of work. Shannon was actually
a part of something called the Mathematical Research
Group, where basically it was a group for all the
misfits within Bell Labs. They didn't know where
else to put them, so they would assign them
to a man named Thornton Fry. And he built what
was essentially an in-house consulting
organization, but it was a
consulting organization where they got to
pick the projects they worked on and didn't
have to work on anything they didn't want to work on. And so he was a part of this
incredible group of PhDs. And what it gave Bell Labs was
this just unbelievable source of talent. And it gave them
six Nobel Prizes, credit for
co-inventing the laser, inventing the fax
machine, sending the first long-distance
phone call, sending the first long-distance
television transmission. Oh, the transistor
came out of Bell Labs. So this book is an
extraordinary look at that place and the people within it. And one of the people who is
profiled throughout the book is Claude Shannon. And you get him in
bits and pieces, because it's not his biography. It's a story of the company, and
so you get his story weaved in with others. And I went looking for
a biography of Shannon, and I didn't really
find one that covered his life end-to-end. And I said to myself, it
just seems crazy to me that this guy's
fingerprints are all over the modern world, that
people in this room 50, 60 years later work on things
that he worked on or pioneered, and that much of
the public doesn't have any sense of who he is. Not only that, even
the people like you who might know who he
is don't necessarily know the details of his personal
life or of his early life or of his education. Rob and I are old
friends from Duke, and we've done a lot
of writing together and a lot of writing separately. And our first book was a book-- well, it's a look at that
ancient Roman senator, Cato. And we were both kicking
around for another project. And when I came to this,
I just said to Rob, we could do something here. You and I are not engineers. We're not computer scientists. We're not mathematicians,
but that's exactly the reason that we
ought to do this, because if we can make
this accessible to people, in order to do that,
we're going have to learn all this
stuff from scratch and try to make
people understand why he was important. And we pitched the project
to Simon & Schuster, to an editor named Alice Mayhew,
whose name you don't know. But you know the name of
the people she works with, Walter Isaacson,
Sylvia Nasar-- who's the author of "A
Beautiful Mind." And so she understood right away
what this book was, which is, it's in part the story
of Claude Shannon's life from beginning to end. And it's in part a
story of discovery. What does it take for a mind
to produce the kinds of things that Claude Shannon produced? She understood
that, because that is what "A Beautiful Mind" was. That is what Walter Isaacson's
book on Einstein was. There's a really wonderful
book called "Tuxedo Park," which few people in this
room have probably looked at. But another look at
someone, Alfred Lee Loomis, who was this
character who funded a lot of wartime physics. But she just knew the
model for the book. We approached her. Simon & Schuster liked the idea,
and we were off to the races. MARK GOLDSTEIN: It's quite a
leap from Cato to computers. JIMMY SONI: Well, it was nice
working with a figure who was more recent. We could talk to his
friends and his loved ones. Cato was a bit more
impenetrable in that way. MARK GOLDSTEIN: Yeah, you are
both trained as journalists. Just briefly, what are
your current positions now? JIMMY SONI: Sure,
so I'm an author. I'm also an editor at
the "New York Observer." ROB GOODMAN: And I'm a
PhD candidate at Columbia. My day job is in
political theory. MARK GOLDSTEIN: Great. So let's talk a little
about Shannon's life. What about his early
years-- where he grew up, his parents, his teachers,
other influences-- before he left home. ROB GOODMAN: Yeah,
so Shannon grows up in this tiny town called
Gaylord, Michigan-- 2,000 to 3,000 people. And one of my personal
favorite research bits we did for this book was
looking at the Gaylord newspaper archives from the 1910s,
the 1920s that were online. They actually digitized
their old newspapers, so you can read the whole thing. And it's the kind of town where
the headlines were things like, "Verne Mass Loses
Finger," "Meeting Called to Discuss Artichokes,"
and "Girl Kills Wolf with Broomstick." So that's actually
pretty awesome. And she needs a book. But that was the kind
of town that it was. So the cool thing about
it was that Shannon didn't have any kind of a
tortured or tormented childhood like some geniuses did,
but he had a childhood where he could play with things
and make things and build things. And he started this tradition. And then Dick Gertner
also talks about, in Bell Labs, these great
Midwestern inventor, tinkerer, scientist, engineers
who come out of the early 20th
century, and often many small towns in the Midwest. And one thing that we were
able to talk about in the book and to research in
a little more detail were the projects that
Shannon tinkered on as a boy. One of the coolest was, he
made a barbed wire telephone network that ran between
his house and his best friend's house. And this was not
unique to Shannon. There were lots of
off-the-grid places that didn't have major
telephone coverage, and farmers would
speak to one another by ringing up electric signals
to run over the fences that were already there. So Shannon didn't invent
this, but he was 10 or 12. It was pretty impressive for a
kid of that age to rig it up. He rigged up, with
his friend Rodney, a barn elevator that
went from floor one to floor two in the barn. And it said that Shannon
was the mastermind. Rodney was the guinea pig. JIMMY SONI: And
Rodney survived, so. ROB GOODMAN: Yeah, he lived. Well, we saw an interview
with his sister. So also, he had a pretty
academically-inclined family. His dad was one of those kind
of jack-of-all-trades that you'd seen in a
small town like this. So he was the furniture
salesman, the town undertaker, and the judge of probate. He was much older than
Shannon, so they didn't really have a very strong connection,
because there was just a big age difference. His mother was a
teacher, and occasionally the high school principal. And his sister Catherine
Claude always said, was better than him at math. And she was part of the reason-- I think she was three
or four years older-- that he got into the
field in the first place as sort of a sibling
rivalry thing. And she would always give
him math puzzles as a boy. My other favorite thing
about Shannon's childhood is he mentioned that his
favorite story growing up was the Edgar Allan
Poe's "The Gold-Bug." Which I don't know if anyone's
read it, but I looked into it, and it's the only
Edgar Allen Poe story that ends with this
lecture on cryptanalysis about this giant block of
text that's a buried treasure code from a pirate. And this guy, the
narrator's friend, goes into this
interminable detail about how you decipher
it using redundancies in the text to decode what
symbols mean and so on. It goes on for 10 pages. And Shannon just ate this
up, and it's pretty fitting that he goes on
to actually become someone working in cryptography
as part of World War II. We'll probably get into that. But that's the background
he comes out of, a pretty happy, mathematically
inclined, tinkering inclined, sort of kid that finds a
way to put that to good use. MARK GOLDSTEIN: Anything
special about his schooling at that time? Dig anything up on
teachers, principals? JIMMY SONI: Not
particularly, he actually-- and this will be
comfort for anybody who struggles with
this-- but he got some B's on his report card. We learned this by poring
over the "Gaylord Herald Times," which at that
time would publish who was the top of the class
and that sort of thing. So he got some B's. He remembers later. He writes a letter
to a teacher who he remembers very fondly and
talks about the students he remembered from his class. There was nothing
within the records that we could find that
suggested that there was anything that
happened at school that was of particular
importance to him. I think, if I were to take a
step back, in a way, the fact that he didn't have that kind
of intense upbringing that like a Beethoven did or
a John Stuart Mill where their parents are drilling
them in Latin and Greek and trying to turn
them into prodigies, Claude Shannon's parents
seemed to embrace what I think we'd call
now free range parenting, like Claude Shannon was playing
with things that he probably should have been playing with. But it gave him this ability
to work with his hands, to be very practical. I'm assuming that a lot
of the people in this room played with early
desktop computers and been built and
rebuilt things. And I did too, and so
I kind of identified with this part of
Shannon's life, because I remember staying
up super late at night trying to reconstruct computers
and build computers and swap out parts and waiting
for the latest video card and all the rest. And I was writing all
of this on a laptop that, if I took it apart, I
would have violated the terms and conditions of the laptop. And I just had this moment of
thinking, wow, I wonder if-- we both had kids during the
time of writing this book-- I wonder if my daughter is
going to ever have the ability to take things apart the
way that Claude Shannon did, or even in the novice
way that I did. But this was somebody who
played with broken radios and would go around collecting
parts and assembling things. The education he had was,
I think, a more informal education. He was very smart. People recognized it right away. High school only took
three years for him, and he gets A's in the
subjects that matter to him and B's in the
subjects that don't. But otherwise, it was a
fairly ordinary childhood. MARK GOLDSTEIN:
You get the feeling from reading Bell Labs book
that this entire generation of brilliant early
electrical engineers that came out of the Midwest
that didn't have computers to tinker with. You get the feeling they could
take a tractor apart and put it together with their eyes closed. ROB GOODMAN: That
sounds kind of accurate. JIMMY SONI: And not only that,
they would have wanted to. That was their first instinct. When Claude Shannon
is in a nursing home at the end of his life,
in a very sad period where he is fighting and struggling
with Alzheimer's, he takes apart his walker
to try to figure out if he can reconstruct it
to make it work better. So this is something that
is just threaded into him and who he is. And it's one of the pieces of
the story that is endearing, and I assume is endearing
to a lot of people in this room who probably
have similar inclinations. MARK GOLDSTEIN: Apropos
of your daughter being able to take
her computer apart, in 1984, Stanford, the
Macintosh has come out. Andy Hertzfeld comes to
give a talk about it. And he goes, look at this box. Isn't it beautiful? You can't take it apart. You need a special tool. You can't get inside. People are going boo, boo. Because, well, it's
like a toaster. Who'd want to take
your toaster apart? Voice from the back
of the room, "me." So these are the
times we live in. So out of high school, he goes
to University of Michigan. I actually work for someone
from University of Michigan here at Google who says, yay. Let's talk about Shannon in
Michigan and then on to MIT. So what happened in his college
and later academic year? ROB GOODMAN: So one more,
and our favorite other piece of information from
this period is we dug up his college application. It's amazing the papers
you can get on dead people, but we found his
college application. And in those days, you could
cross out your spelling errors in pencil and still send it in. It was like three pages
of fill in the blank. So it was not the most-- it was a great
public university. And if you were a Michigan high
school student that graduated, you could go to
Michigan University. So Shannon double majors
in engineering and math. And again, and
other people did it, but he was sort
of unique in both having that sort of
abstract background in mathematical logic, and
especially Boole's theory, and in the more
practical engineering side of electrical engineering,
which we can get into. But anyway, he sees a
job application one day from Vannevar Bush
at MIT to apply to MIT as a graduate student
and with special emphasis on working on Bush's
differential analyzer, which is just a fascinating
machine that we learned about in the process of
working on this book, which is, of course, one of the
great early room-size analog computers. And one of the things that
I learned of the process is that when we say
"analog," we don't think of the derivation of that word. But it's very literally a
machine that makes analogies. Bush has a great quote where he
says that, if for instance, we want to study the effects of
the differential equations that operate on a bridge
swinging in the wind to see if the
bridge will stay up, we design this analog computer
so that it will, in a sense, obey the same equations that
are affecting the bridge. So we set it up to make
an analogy for what the bridge undergoes, and then
this applies for engineering problems, and Bush is
using it for questions like the stability
of the phone network and the electrical grid, but
also for more advanced physics problems. Later on, it was
applied to things like studying cosmic radiation
or studying atomic structure. So it's the best, most advanced
calculating device of the day. So Shannon comes
to work on this, and Bush really has
an eye for talent. He's one of these great
scientific networkers and organizers. And as he puts Shannon
to work specifically on working on the
relays that are attached to a later version of
the differential analyzer that helps reconfigure the
system on the fly, in a sense, so you don't have to break
it down or rebuild it for each new problem. And then Shannon gets to
thinking about the connections and analogies between
the relays, the kind that we saw in the Theseus
video and a Boolean logic. The idea that Shannon
comes up with, essentially, that you can use switches as the
units and symbols and operators in acting out Boolean logic,
that ones and zeros can represent yeses and noes,
that a series and sequence can represent ANDs and ORs, that
whether or not an outcome happens, like a
light turns on, could represent the IFs of Boolean
logic and the outcome. So Shannon writes his
great master's thesis that explains how to use these
principles of Boolean logic to change circuit design so
that rather than being a trial and error process that
you can have a knack for, Shannon actually shows
how you can dramatically simplify it and do everything
that's required to construct these circuits on paper. And he actually does a
couple of examples of this. He says, here's a circuit
for a basic adding machine. Here's a circuit for
a combination lock. And he draws them. And for the first time, someone
is simplifying the process and designing these
basic computer circuits with any kind of yes
or no gate entirely on paper. And of course, this is a
blockbuster for its time, and it wins him all
sorts of awards. It wins him the Noble Prize,
which is like the Nobel Prize with the letters
transposed, which is an award for the best young
engineer, a young engineering paper. But it's recognized,
as people go on to say, as probably the most influential
master's thesis maybe ever. We both wrote master theses. We're not quite in that league. But Shannon's certainly
was in a league of its own, and that's what what launches
him to a degree of prominence. It puts him in these upper
echelons of the science engineering world. MARK GOLDSTEIN: And
The difference engine is kind of one of the
first reifications of Charles Babbage's,
the difference engine. Or William Gibson's, if you
like the book, the novel. Was Turing building Enigma
machine at the same time? And was that machine
at all related? Was that an analog machine? JIMMY SONI: So it was. Turing and Shannon are working
on a lot of the same concepts at the same time. And so I think the Turing
machine paper was published-- what was it? ROB GOODMAN: The Turing
machine paper was the same-- JIMMY SONI: A year, yeah. ROB GOODMAN: He was
working at Enigma until the war, a little like
five or six years later on. JIMMY SONI: But one of the
more interesting elements of the story is that they-- so they're separated
by an ocean, but thinking and working
on many of the same things. And they're also very similar
in personality and temperament. Neither of them, they're
not the life of the party. They are pretty quiet. They keep to themselves. And this is a bit probably a
leap forward in the chronology, but it's worth doing. But there's a period where
the British government is suspicious about
whether or not the Americans are going to have
the technological wherewithal to send messages that are
actually encrypted and actually protected. There's a lot of
suspicion on both sides, but particularly the British. So the Brits send Alan
Turing to the United States to do a tour of
different facilities in New York and Washington DC. And the idea is he needs
to essentially stress test these devices that are
used to, for example, for Franklin Roosevelt
to communicate with Winston Churchill. And so Turing is sent
to the United States. Funny enough, there's
enough suspicion that his paperwork is
actually not cleared, and he gets held up in
immigrations and customs for a little while. He ends up at Bell Labs, where
he spends a good chunk of time. And in what is one of the
most incredible moments, I think, in computing history,
Alan Turing and Claude Shannon have tea basically
every day while they're at Bell Laboratories. And these are not guys who
are going to go chat up-- they're not unfriendly
people, but they're certainly not going to go make
new friends very easily. So they have tea every day. And interestingly
enough, the secrecy of the work that they're
doing prevents either of them from talking about code
breaking or cryptography, and so it frees them up to talk
about artificial intelligence. They are sitting around
asking questions like, could you build a computer that
could work as well as a brain? And this is in the 1940s. And so you have these two
giants sitting together at tea, and we always like to wonder
what it must have been like in that particular moment. And so there's a couple
other pieces to the story. One is that Turing
actually visits Shannon at his home in
New York's West Village. And we think maybe a
half dozen, if that, people visited his home. And so it says something
that Turing was there. But they end up striking
up a friendship, and so that's the one
connection between them. Intellectually, they
do stay in touch, and later on, after the war
is over, Shannon and his wife go and visit Turing. And like old
ballplayers, I guess, they go down to
Turing's basement and start playing with a
computer that he's built. And they just keep up. They just pick up exactly
where they left off. And the story, obviously,
has a tragic ending. Turing, by some
accounts, it's suicide. And by all accounts,
it seems to be suicide. But there are still some
discrepancies about that. That happened shortly
after Shannon's visit, but we always of
think that there's something appropriate
in the fact that these two wartime code
breakers and code makers were able to reunite
after the war. And without missing a
beat, they go and play with this device in
Turing's basement that actually-- he's trying to
make a computer that will talk, and that's what he's working on. MARK GOLDSTEIN: Interesting,
because Shannon goes on to Bell Labs, which is
the pinnacle of research in making people talk. (ACCENT) "We have ways
to make you talk." Anyways, so let's
talk about Bell Labs. And not just in the context
of Shannon and Turing, but Bell Labs in general. And when we talk
about Bell Labs, "The Idea Factory" traces its
100 year history, more or less. It started in New York
downtown, and then by the time the war came, they'd
moved to New Jersey. And Shannon was there through
that entire transition, so there's a lot to say
about it his tenure at Bell. ROB GOODMAN: Right. For those of you that go to the
Google New York office, Bell Labs, you can still see the
building near the high line now. And you can walk
across the high line and look to what was
the original Bell Labs building that even
had one of the ports for cranes to come
in and out of before, when it was an industrial area. But anyway, and we've written
a little bit about Bell Labs in separate pieces
about the book. And we've said that
Bell Labs probably isn't a good example
of how to make a successful company
because, well, they had so many external advantages. They had the patents
going back to Bell. They had become a monopoly. So they never really had
to worry about money. But at the same
time, we also say that if successful
companies in the economy can behave like Bell Labs,
we're all better off. Bell Labs took that money
and put it towards-- poured it into basic
research, and poured it into the work of
people like Shannon, a little bit earlier before
him, Clinton Davisson, who got a Nobel Prize for
working on atomic structure. But poured it into people who
didn't do work that immediately paid off in a small
window, but was enormously important for the development
of technology more broadly. And there's a great
quote, I think it was from Henry Pollock who
overlapped with Shannon at Bell Labs, who said there was this
attitude there that we may not be doing work that is going
to pay off in 10 or 20 years, but hey, we're
the phone company. We're going to be there anyway. So there was this
confidence of being able to have people like
Shannon who could noodle around on whatever. So that's not Shannon's
entire career there. When he comes on board at
Bell Labs, it's wartime. And he has a lot of
assignments that he's not especially happy about. And those include
fire control, which is sort of using statistical
modeling to control anti-aircraft guns of the
kind we also saw in the video. He takes some of those
insights especially working with Norbert Wiener,
who had written a kind of famously intimidating
text on statistical modeling, and some of those insights
go into information theory. And also, as Jimmy
mentioned, Shannon is working on cryptography,
unscrambled speech systems, and writes a paper
that doesn't come out until after the war is over. But it's about the
theoretical conditions for unbreakable code, about
proving the unbreakability of a one-time pad code, even
though, practically, it's the kind of code you
wouldn't be able to implement in actual wartime conditions. But Shannon takes these
insights from fire control, from cryptography, and
from his other interests, and he pours them into this
project on information theory that he's working on a lot of
in his spare time, and later on, during his day job
at Bell Labs when he gets a little more freedom
after the war winds down. And we have these quotes
from his girlfriend at the time who lived
in Shannon's building. She said that some
days, he just didn't want to go to work because
he was stressed out by the crowding in the office,
by the wartime conditions, by the amount of time he
had to be in the office. But he was always, at
home, wherever he was, he was scribbling on napkins. He was talking over
ideas to himself. He would stare off into
space, and he was working on something that was big. And after he had a
little more freedom to do this full
time at Bell Labs, this is when he
develops his great work on information theory. And the neat thing is that
Shannon is, of course, not the only one, especially in the
math group, but more broadly at Bell Labs, who's doing
this long term, very long horizon basic research. And of course, information
theory is Shannon's paper, he's immediately of
theoretical interest. It immediately provokes
so much interest in scientific literature. It doesn't really start
to have practical payoffs for many, many
decades down the road, but that's the neat thing
about what Bell Labs enabled Shannon to do. They were willing to keep him on
and pay his salary as he worked on this thing that had enormous
payoff decades down the line. And kind of as a final nice
gesture when Shannon decides to take a job at MIT
and leave Bell Labs, Bell Labs decides to
keep him on the payroll. And they keep an office for him
just as a gesture of respect that he was one of the giants. We went there. We went to their new building
at Murray Hill, New Jersey. They have a bust
of Claude Shannon, and one of the buildings
has been named after him. But he's still one of the
people, one of the few people, along with the inventors of
the transistor, that is just still revered in that place. JIMMY SONI: Just to
add to that a bit, I noticed that there were
some chess boards outside. And it called to
mind for me, just because we've been steeped
in this, Bell Labs was a vibrant place in that way too,
that there were chess games, that there were competitions. Shannon builds at one point-- I think it's a rock,
paper, scissors robot, and they have a contest against
some other guy who built a rock, paper, scissors robot. That might not be
the exact story, but it's something like that. It was a company that gave
people remarkable flexibility. This is to hazard a guess. I'm not sure there is
a mathematical theory of communication without
Bell Laboratories, both because of
the content of what Claude Shannon was working on. There was sometimes
when he was working on very practical things. One of his first pieces
of work at Bell Labs is actually looking at
the coloration of wires within the phone system and
whether the coloration of wires could be improved upon. And he actually does. He writes a paper that improves
upon the coloration of wires. So some of it got to that
very practical level. But remember, he publishes
his mathematical theory of communication in the
"Bell Systems Technical Journal," which is an
academic journal run by Bell Labs for the better
part of the 20th century. It's like, corporate blog,
eat your heart, right? This is serious stuff. This is a lengthy 77 page paper. And Bell kept this
publication going for most of the 20th
century, and you can read the archives online. There is something
about a company. Again, they had a certain
blessing in the resources that they had. But the ways that they chose
to use those resources, we think there's still a
lot left to learn from them. And again, "The Idea Factory"
is a wonderful way to start, but there are obvious echoes
of Bell Labs around here. And I do think that
there's something about the kind of freedom
they gave people that led to some remarkable things. This is a private
sector company that won six Nobel Prizes over the
course of the 20th century. ROB GOODMAN: And
not even Shannon. JIMMY SONI: Yeah,
not even Shannon. ROB GOODMAN: He should
have won one too. JIMMY SONI: And Shannon,
and information theory, and so you really
have to step back, and say to yourself,
what was in the water? And we do get into some
of that in the book. MARK GOLDSTEIN: And they
built telephone poles that still last for 50
years because that was part of their design aesthetic. JIMMY SONI: Even
better, the math group that Shannon joins, one
of the rites of passage is they climb telephone poles. And you have to
actually go and be a telephone engineer on the
ground for a little while. It's sort of this-- I wouldn't quite call it a
ritual, because I'm not sure that everybody was
required to do it, but it was the sort of thing
that the math group would just dive in and do that in order
to understand problems. And they would attach
these mathematicians to physicists and to
engineers and to others, and say, just go. Go help them for a little while. Figure something out. And it was a pretty
amazing thing. MARK GOLDSTEIN: And
we should mention that the pull quote there is
that the paper in the "Bell Technical Journal,"
a couple of years later was dubbed in
"Scientific American" the Magna Carta of
information theory. ROB GOODMAN: Yeah. When people ask us to explain
why this paper mattered, that's the pull quote
we use, because it's this founding document. This is the thing. MARK GOLDSTEIN: And not only
was it the founding document, but he basically produced
all of the follow on theorems and proofs as well. He left almost
nothing to be done. ROB GOODMAN: Just really
obnoxious of him, yeah. MARK GOLDSTEIN:
Complete, complete. JIMMY SONI: The other
interesting thing about that is, when he first publishes it--
and this speaks to the other nice quality of Shannon is
that he's a very modest guy-- he calls it "A Mathematical
Theory of Communication." "A Mathematical Theory
of Communication." When it's republished a year
later as a book by people who take it-- and some of that book
is him, but a lot of it is not, it's the work of other people-- they put the title,
"The Mathematical Theory of Communication,"
which just goes to show how highly
it was regarded by people in the field
within no time at all. MARK GOLDSTEIN: While
we're on the theory, I was going to get
to it a little later. JIMMY SONI: Perfect. MARK GOLDSTEIN: But
it's so important. Is there an elegant
statement of it that's as concise as e equals mc
squared or Newton's postulates? Or is it really, really
hard to understand? ROB GOODMAN: I think if
there's one thing to pull out, and this is actually not
our choice to pull it out, but it's Shannon's formula
for bits in terms of H as a measure of probabilities. And interestingly, Shannon's
formula for bit content is on his bust at Bell Labs
and several other places where they have his a bust. And it's also on the
back of his tombstone. And Jimmy went there. I wasn't able to make that trip. Jimmy visited Shannon's grave
in the Mount Auburn Cemetery. And if you walk
around to the back and push aside a
couple of bushes, you can see it
engraved on the back. When we spoke to
his family, I think there was some talk of
putting it on the front. But I think-- was it his wife? JIMMY SONI: I think his wife. So his kids wanted the
equation on the front, and his wife thought it
more appropriate to have it on the back. And now, it's covered by a bush. And so if you go there,
it's a famous cemetery, Supreme Court justices,
the presidents of Harvard, et cetera. They're all buried there. And you have to
push the bush aside to see the equation
engraved on the back. ROB GOODMAN: And
I would say that I think one of the
reasons it does sum up, why it's made e equals
mc squared level maybe, is that it does
sort of encode a lot of information you can unpack. And one of the things that
we like doing in the book is tracing the intellectual
history of information, of trying to pin it down as a
physical, objective quantity. And We trace the different
fumbling attempts to get to it,
especially beginning with Lord Kelvin back
in the 19th century working on transatlantic
telegraphy. But also getting
into the predecessors that Shannon himself
cites, Harry Nyquist and Ralph Hartley,
both his predecessors at Bell Labs, who also
have their own formulations of information content, and
Shannon's addition to this is the idea that
information content has to do with the size
of the symbol vocabulary. It'd already been there in
Nyquist's work on telegraphs and then Hartley's
work more generally. But what Shannon comes and adds,
and Norbert Wiener was also working in a similar direction,
what he comes and adds is a probabilistic
element that it's not just the size of the
symbol vocabulary you're choosing from. It's the fact that you have
to measure in the probability of getting a certain choice. And the analogy he uses for
basic understanding of what a bit is think of a coin, a
fair coin, as storing one bit, because there's a 50/50 chance
it can land on heads or tails. And as you weight the
coin more and more in one direction or the
other, the information content that it stores could
be set to decrease because the choice becomes
a little more predictable. But it was Shannon who add
this probabilistic twist. And of course, as modest
as he was, he said, I didn't think that
was a big deal. I guess we think
it's a big deal. MARK GOLDSTEIN: It
is a very big deal. And searching around on the web
for some pithy Shannon quotes, I came up with a couple
not quite, maybe not quite, as elegant as his tombstone,
but maybe more understandable to a lay audience. And I came up with--
these are Shannon quotes. "Information is the
negative reciprocal value of probability,"
or "information is the resolution of uncertainty." And thinking about
that, it struck me, if you look at the two polls
of his whose most famous work, you've got Boolean algebra. You've got basically, forgive
me, Bayesian statistics on the other side. You've got the certainty and
the structure of a binary number system contrasted with
uncertainty, probability. It's a nice dipole, I
think, to reflect his work. ROB GOODMAN: Yeah, that's
a great way of putting it. We didn't think about
it in those terms. JIMMY SONI: We didn't
think about that. ROB GOODMAN: I'm going to
use that as a pull quote for the paperback. But that's a really
good way of putting it. JIMMY SONI: For a particular
audience, it's a-- MARK GOLDSTEIN: Yeah. We'll get probably deeper into
the philosophy in information versus knowledge. I actually have a great
Frank Zappa quote. JIMMY SONI: Can I just
say one other thing that we didn't really get into,
because this is something that speaks to people more
generally who maybe aren't deep in the theory, but want to
think about how a theory like that or how something that
consequential can be developed? It's incredibly important
to remember two things. One, that he started thinking
about this in 1938, roughly, 1937, 1938. It takes 10 years for
this to crystallize into this paper that now has
an entire field devoted to it, a field of study, and
people read it every year. It takes 10 years. It takes a long time
for this to crystallize. During that time, he's
working on cryptography. He's working on how to shoot
things down from the sky. He's playing with the telephone
system at an enormous scale. The other thing to
remember is he's not working on it full time. It's what we affectionately in
a recent piece called a side hustle. I hope I have a side
hustle as good as this, a mathematical theory
of communication. But I think those two
things are important, because it's really easy to
look at somebody like Shannon, and say, well, that's just full
time working on this, 10 years, it's a given. It really wasn't. He was, I wouldn't
say distracted, but he was occupied with many
other things during the time. And in a way, I think
that actually makes what he did all the more impressive. But those two facts, I
think people forget it. And you can read it. You can see his name
next to this paper and gloss over the fact that
this is a decade long journey to get to some of the defining
principles in this field. MARK GOLDSTEIN: From the
sublime to the ridiculous, let's talk a little
bit about Shannon the trickster and the tinkerer,
and maybe even the bon vivant, although you say he wasn't
that much of a social guy. But I read some stories. Tell us about him. ROB GOODMAN: So the great
thing is that Shannon was a born tinkerer. From the very earliest
records we have of him, this is what he did. But at the same time, once he
becomes the information theory guy, he had a little more
freedom to just show this off. And we do think it's nice
that Shannon could have made a play towards
becoming a much more public figure, a public
intellectual, a pontificator on science. Just he's not
interested in that. He's interested in just
playing and making things with his hands. So when he moves into his
house in Massachusetts, he had a two-story workshop
addition added to the house that some people
called it the toy room. I don't think Shannon
himself called it that. It was called dad's workshop. But he built things in there,
like the flame throwing trumpet that we mentioned. He had a fleet of
customized unicycles. So he had off-balanced
unicycles that made unicycling and
juggling simultaneously even harder, because apparently,
juggling and unicycling was too easy for him. So he gets really into the
scientific study of juggling. He writes one of
the first papers that we know on juggling
physics where he comes up with an equation correlating a
number of balls, time in air, and so on. He asks one of his colleagues
if he can measure his juggling as he holds him upside
down on the theory that one would be
able to combine the elegance of toss juggling
with the physical efficiency of bounce juggling
where you bounce the balls off the ground. So if gravity is helping
you juggle upside down, theoretically, it
would be a much more-- it didn't work out. Because as we say in
the book, how well does anyone do
anything upside down? He does that. He makes the ultimate
machine, which is one of our favorites,
which is a box, and we've seen a copy of it. A lot of people know this. It's the box. You press a button on
the box, and a hand comes out of the box. Press the button. Turns itself off, and then
tracks back into the box. JIMMY SONI: You can see YouTube
videos of this, by the way. It's a famous device. MARK GOLDSTEIN: The
ultimate machine. JIMMY SONI: Yeah, it's
the ultimate machine. It's only purpose is
to turn itself off. ROB GOODMAN: So this is
what Shannon does for fun. Oh, there's the Roman
numeral calculator, which is called THROWBACK, which
is an acronym for, I think, Thrifty Backwards Looking Roman
Numeral Calculus, something like that. But it's all caps
THROWBACK is it's name. And one of the first
wearables, which is a project he worked
on with Ed Thorp to beat the house at roulette. So the idea was that it
divides the roulette table up into eighths, they got into it. And if you can calculate
that with some knowledge, if you know when
the ball goes down, and you start the
computer, it can signal which eighth
of the roulette wheel the ball is likely to land in. So Thorp and him win some
money from the casinos in Vegas using this. Oh, the other cool thing
is that its output is via kind of primitive earbud. So based on the frequency-- I think they had to memorize
these frequencies-- you could tell based on frequency
where the ball's going to land. One time, Thorp, the
ear bud falls out, and someone screams,
because they think it's an insect because no
one really had seen ear buds. So Thorpe gets in trouble. He has to run to the
bathroom and put it back in. So they call this
off because they were afraid of
getting roughed up by the mafia, which controlled
the casinos in Vegas at the time. So they don't really take it as
far as, say, the MIT blackjack team. But maybe they could have. But anyway, this is what
Shannon does with the freedom that he gets to be a tinkerer
or a prankster or a jokester. Oh, and he also builds
an early chess playing computer called Endgame
that can handle six pieces. So he's all over the place in
terms of his hobbies, projects. We can go on and
on with this list, but that's what makes
him such a fun character. And we think, obviously,
you speculate a little bit when you write a biography,
but you think that there's this playful spirit. That even when he's
not unicycling down the hallway of Bell Labs,
it's in his work too, because he was the
kind of guy who just asked the silly questions,
and got so much out of them. And it was that courage to ask
the ridiculous questions that enabled him to do what he did. JIMMY SONI: It's worth also
just emphasizing, at least using one example with the
roulette wearable device. So the origin story of
this is Ed Thorp is at MIT, and he tries to get an
audience with Claude Shannon. He does, and the
secretary warns him. Claude Shannon doesn't really
have a lot of patience, so just keep your stuff brief. And he goes in, and he's asking
him about a blackjack paper that he had written. And Claude Shannon
advises him that if he wants to get it published,
he needs to just change the name slightly. And then Claude Shannon
asked him, so what else are you working on? And then Thorp sort
of intimates that he's got this idea for a
wearable device that could give you a
slight advantage against the house in roulette. They spend the next eight months
working to make this a reality. Claude Shannon buys a
regulation roulette table from some warehouse
or something. They install it at the
house, and Ed Thorp, for a period of time, moves in. And so they just
work on this thing. These are two MIT professors. And Claude Shannon's like
a giant within the field by this point. They're spending eight months
playing with a roulette ball and trying to figure out,
thinking of the mechanics. Then, as if that
weren't enough, they take the device to Las
Vegas to actually test it. I think that one of
the things that I'm most inspired by
about Claude Shannon is just how far he
took these curiosities. I mean, how many
of us have an idea, and it sort of goes
into the trash heap? ROB GOODMAN: That'd be cool. JIMMY SONI: Yeah, it's
like, that's neat. OK, I'm not going to
devote eight months to trying to do this. But this student was a perfect
stranger to Claude Shannon. He didn't know him. This was the first
time they'd ever met. And that leads to an
eight month collaboration, which just goes to show the
sort of thing that happens when an idea entered Claude
Shannon's head, what could emerge from it. And we have physical
representations of all of those things. A lot of these devices are
still stored at the MIT Museum. They went on tour
last year to Germany, and they were exhibited. I think he's one
of the only people that I've ever read of who
has both papers published in academic journals and things
that have been in museums. But this is who he was. It's a big part of the
latter part of his life is this kind of tinkering. And people like to say that
the latter part of his life wasn't as productive, or
that he didn't achieve what he did in 1948, and he
kind of went downhill, but if you think about what
you're comparing it to, the 1948 paper
was extraordinary. People have referred to it
as Einstein or Newton level thinking. To say that what he did after
that didn't quite match up, I think is, A, you're setting
an extraordinarily high bar. But B, it misses all of
these things that he did do, the devices that he built. He continued writing
papers that took some of the theoretical
work from 1948 and showed engineers how they
could practically apply it. It's a period in which
he's mentoring and working with some of the younger
generation of information theorists, people like Robert
Fanno and Bob Gallagher, who worked with him on
papers about pulse code modulation and other things. And so there's this period in
the latter part of his life, though, which has a lot more
freedom and flexibility, and a lot of his
most inventive things come out of that
phase of his life. ROB GOODMAN: And one more
thing I wanted to add, we did a Reddit AMA
a couple of days ago. And in the middle of
it, a guy emailed me, and said, oh, you have a picture
in your book of Shannon's juggling clown diorama. And he had a couple of these. Some that juggled in real life,
some that just looked like it. I helped him build that. I was a high school
student at the time, and Shannon was such
a nice guy, and it was one of the funnest summers
of my life working with him in his workshop on this. And he sent us a picture
as proof of old Shannon, in the, I think this must have
been the '70s, early '80s, and this high school kid. And he got in the AMA,
and it was awesome. But it was just when he latched
onto a project or someone that was interested in the
same things, he did it. He made it. JIMMY SONI: He built an RV. So they bought an old school
bus and retrofitted it as an RV. And there are some
accounts that suggest that it's the world's first RV. ROB GOODMAN: It makes sense. JIMMY SONI: But the Shannons,
and Betty in particular-- and we should actually
talk about her too, because she's a huge
part of this story. His wife, Betty, is
his equal in intellect but also and tinkering. So Theseus the mouse, the video
we watched, the wiring for that was actually Betty. Betty did the wiring. That's a story we learned later. Shannon gets the
credit, and deservedly so, he was sort of
thinking about it. But Betty made sure
the wiring worked, and that the thing
would work as it needed. They were full partners
in a lot of what they did. We wrote a piece about this
in "Scientific American" that debuted a couple of days ago,
and have heard back from people who knew Shannon to say,
thank goodness Betty finally gets the credit that she
deserves for a lot this. She pushed him to try
things like getting interested in the
stock market, which he becomes obsessed
with for a brief period. He had this quality
of being able to ask this sort of absurd
question, but then try to build the answer to
the absurd question. And so there are
just robots he builds to perform functions that you
sort of think to yourself, why? Why would you do that? But that's just a little
bit of a digression. MARK GOLDSTEIN: Not at all. I was going to go into, what
about the women in this story? But that could be
a whole other hour, I think, and I want to give
the people here a chance to ask some questions. So it's your turn. Do you have some questions? AUDIENCE: How would Claude
Shannon describe himself? ROB GOODMAN: How would Claude
Shannon describe himself? JIMMY SONI: He wouldn't want to. ROB GOODMAN: Yeah,
that's a good one. That's a good point is that,
in interviews, he was just sort of an evasive, humble guy. He said, well, that just seemed
like a pretty simple idea to me at the time. It just came to me. But I guess he would
describe himself as someone who liked
puzzles, who liked to think about the way things worked. And I think that kind of
undersells who he was, but I think he would
have described himself in pretty minimalist terms. There are a couple of
interesting moments where Claude Shannon gets a
little more autobiographical. One thing that we dug up from
the archives of his papers was a 1952 talk he gives on
the topic of creative thinking to the Bell Labs engineers. Oh, and there was one quote
that we really liked from it. I think we made it
the chapter title. He said that creative
thinkers have this productive dissatisfaction. They just see when things don't
fit right or don't work right or are puzzling, and
they just stick with it. And they follow it through. So I think he described
himself as someone who was sort of "usefully
irritated" in a way sometimes. He also, much later on, when
he wins the Kyoto Prize, which was set up as a rival to the
Nobel Prize in some fields the Nobel didn't
cover, including math, which Claude Shannon
gets much later in his life, he gives an overview
of his life, talking about the progression
of computers from slide rules to the Apple IIe, which he had. And he talks about how much
he'd gotten intellectually out of pursuing his hobbies, that
he just described himself as a hobbyist in a
lot of ways, and how this was such an integral
part of his work. So we gave a big block quote
of that speech in the book, just because it hadn't
been widely published, and there aren't very many
times that Shannon would open up about himself. He was actually very nervous
representing himself in public. He gave the inaugural
Claude Shannon Lecture. I think it was in Tel
Aviv at the Technion. JIMMY SONI: Ashkelon. ROB GOODMAN: Ashkelon,
yeah, and someone said, he was a nervous wreck. It was named after him,
and he was being honored. And I think he was
at the bar pounding a couple of Old Fashioneds
before he went up there, just because he did not like
to represent himself in public. So when he did, it
was worth pulling out. That's a bit of a
long-winded answer. JIMMY SONI: There's another
piece to that too, which is in the creative
thinking lecture that he gives, one of the things
that nearly every person who we spoke to about him, and we
spoke to a lot of people who knew him, he was
incredibly, incredibly modest. In a way, almost
modest to a fault. He did not go out
and seek adulation. Envelopes would come with
awards or with invitations to speak at
prestigious lectures, he would put them,
many of them, in a bin called Letters
I've Procrastinated on Sending Back for Too Long. Awards, honors, recognitions,
it just had no effect on him. There was even a
reluctance in him, I remember, hearing from
Peggy about they found out that their dad had won the
National Medal of Science. And that it was a lift
to get them to go to DC, do the whole thing,
There's a famous photo of him shaking Lyndon
Johnson's hand. There were times
when he wins awards, and he just stashes them away. When an interviewer is
asking him about an award-- I think this is from an
unpublished interview we uncovered-- he says, well, I
have a couple dozen of these things
in the other room. But let's get back to
the matter at hand, which is problem solving and
understanding how things work. He had this great
line that we love that's "if I can find a solution
to a mathematical theory, I get a big bang out of it!" And he says it with
an exclamation point. And I think that this
self-effacing, modest quality of Shannon is connected
to this not being affected by awards and honors. And what that
allows him to do is be free to explore
whatever he wants. He never feels a trace
of self-consciousness being the guy who founded
the mathematical theory of communication and the guy
who juggles and unicycles and plays with children's toys. This is just not something
that affects him. Meanwhile, many of his
colleagues do wonder. They're like, he's a
named chair at MIT. He's supposed to
act a certain way. We find some
inspiration in the fact that he didn't act that
way, that he just pursued what interested him most. So in a way, if
he was going to be asked how to describe
himself, I think if he was forced to do it-- and he would be forced. He would say he was
a problem solver, that he enjoyed solving problems
and figuring things out. But what he happened
to figure out were things and problems
that affect us still. MARK GOLDSTEIN: You
mentioned earlier that I was going to ask about. These were people who
influenced Shannon, but Shannon influenced
other people. We know Claude Levi-Strauss
was a neighbor, and he had a serious
effect on him. I know that Leonard Meyer,
the philosopher on music, considers the implications of
information theory on hearing music. But you also mentioned that
you didn't put in the book that there's a story about
an encounter with Steve Jobs. "The Scene at the Opera"
just premiered this weekend in Santa Fe. It might be timely to hear
about Shannon and Jobs before we have to close. JIMMY SONI: Sure. So this is a great story. This is the 1980s. Both Jobs and Shannon are
being given honorary degrees at the University
of Pennsylvania. And again, we think about this. For Shannon, he had so
many honorary degrees that he actually rigged up
a rotating tie rack which he put all the hoods on. This is how insignificant he
thought these things were. For the rest of us, if I
win an honorary degree, I want people to
see that the moment I walk through the front door. ROB GOODMAN: I'm wearing
the thing for a week. JIMMY SONI: Right. I want to wear the
hood permanently. For Shannon, it's just like, eh. Oh, another honorary
degree, here we go. So they go to the
University of Pennsylvania. And at this point,
Jobs is well-known because of Apple, but not
quite the canonical figure, iconic figure, he is now. And so after the
ceremony is over, everybody's milling
about the quad. And Shannon has
a group of people around him, because Shannon
is Dr. Claude Shannon, and the people in that
space know who he is. He's got this throng
of people around him, and Steve Jobs really doesn't. He knows who Dr. Shannon is. So he tries to elbow his way
into an audience with Claude Shannon, and he does. And he gets up to
him, and the story, according to Shannon's
daughter, is roughly that Steve Jobs says, Dr.
Shannon, it's such an honor to meet you. My name is Steve Jobs. I work at Apple. And Claude Shannon says, well,
Steve, it's great to meet you. What do you do at Apple? And the end of the encounter,
as best as we can tell, is that Steve Jobs actually
sends the Shannons an Apple IIe that he himself assembled. And so they have this in a
private place in their home. But it was all of that,
the recognition and fame and celebrity, it was
really lost on Shannon. It wasn't what he was after. In fact, in a way, some of the
commercialization of things was sort of lost on Shannon too. It never struck him that
he ought to build a company and take it public. He was always just
interested in pursuing his own private curiosities. ROB GOODMAN: Though he did
love investing other people in public companies. JIMMY SONI: He did. He did. And he was wealthy by
the end of his life. He got on the ground floor of
a number of early companies, including one that was
acquired by Hewlett-Packard. I believe he actually
knew Hewlett and Packard, if I remember correctly. But in any case, that's the
famous Steve Jobs and Dr. Shannon moment, which is that
at that moment in history, Steve Jobs had to explain to
Claude Shannon who he was, which is pretty cool. MARK GOLDSTEIN: Well, thank
you so much for coming, and thank you for
writing the book. ROB GOODMAN: Thank you so much. JIMMY SONI: Thank you all. MARK GOLDSTEIN: I'm glad we now
know a lot more about Shannon. [APPLAUSE] ROB GOODMAN: Thank you so much. JIMMY SONI: Thank you very much.
