Blink: The Power of Thinking Without Thinking

each year Microsoft Research hosts hundreds of influential speakers from around the world including leading scientists renowned experts in technology book authors and leading academics and makes videos of these lectures freely available just waiting for you I'm so glad you could all join us this afternoon I'm not Mike am i Mike now did you just see that usage of technology very proud of myself my name is Tamra pasok this is the visiting speaker series our guest today is Malcolm Gladwell I've promised to keep this very short he is the author of the amazingly popular book the tipping point and is also the author of the soon-to-be incredibly popular book blink which is all about making decisions in an incredibly fast method and he explains it in such a wonderful left-brain way that you might actually agree with him and I'm really excited to hear what he has to say and to hear the discussion afterwards please help me welcome the brilliant genius Malcolm Gladwell thank you very much I I'm neither really a genius but I'll let that slide hi it's great to be back here I was here several years ago for the tipping point although someone just told me someone on him ominously that my appearance back then had been taped and archived and I always feel slightly nervous when I heard that because I just think well what if I said something several years ago that is utterly incompatible with what I'm gonna say today which is entirely possible because my line changes constantly so you can't hold me to anything I said before that's my only request or we go anywhere my new book blink is um it's uh it's about rapid cognition and by that I mean the kind of thinking that takes place in the blink of an eye as opposed to the kind of thinking that we do consciously and deliberately over some period of time and I use the word thinking because I think that what goes on in that the kind of decisions we make in the blink of an eye represent thinking it's rational it's just that it's operating below the level of awareness and operating by slightly different rules then the other kind of thinking which were very familiar with and the book is a call to kind of understand and appreciate the enormous power both for good and ill of that kind of thinking and I tell all kinds of different stories and rather than try and summarize the book what I thought I would do is just tell you one story from the book which I think might be of interest to many of you here and also illustrate some of the things that I'm worrying about thinking about in this book and it's a story about chairs in particular about a guy named Bill stump and Bill stump is a very in a chair world bill stump is sort of the Michael Jordan you would if you were all in the chair business you would all know who he is he's this legendary figure who worked a lot with Herman Miller and over the years in the 1980s he designed for them to really break through chairs called the organ and the equi and they were these sort of new ideas in office chairs and in the late eighties Herman Miller went to him and said we'd like you to design a truly amazed like the greatest office chair of ever and just start with a blank sheet of paper so Stumpf went back to his lab and he lives in Minneapolis and started from the beginning he started over with chairs instead of look the chairs and realize that there's all kinds of there weren't that point all kinds of crazy things about office chairs it didn't make sense for example they had one-piece backs right and you would adjust if they had any kind of setting to adjust for the back you would adjust the whole thing at once which makes no sense because you have an upper back and a lower back and your upper back and lower back have very very different needs and requirements and there's no single way of having a single setting which will make your entire back happy so he decided he wanted to have a two-piece back to the gym and then he realized as well that the way the chairs were built the seat pan and the seat back were connected in this very kind of linear fashion so if you tilt it back on a chair the seat pan would move out right which is insane why would you want to lose the support of the seat pan when you lean back in the chair they should move independently because and that will failure to make them move independently what's causing a great deal of lower back strain so he fixed that and then he realized all kinds of things like the biggest problem with office chair like the ones you're in right now is if I kept you here for four hours you would wish I won't do you would get incredibly hot of course right you it would get sticky that that material you're sitting on is doesn't breathe very well so he needed material to breathe and it came up with a special mesh which had the ability to stay cool even if you sat in it and all day but of course mesh is difficult because it's mesh if you stretch it and then you get up it'll pinch right so we had to make non pinching mesh which I don't know how many of your engineers is an engineering task that is actually exceedingly difficult certainly he did all these things and millions of dollars were spent in development and he presented Herman Miller with this prototype and they said it's fabulous before we take it to market we have to do what people do in the chera world not just the chair world but in fact all of the corporate world we're gonna market research it right we've got to see what people think of it now in the chair world you do things you do two tests of things you give people you make people rate chairs on a scale of one to ten for comfort where one is hideous or one is uncomfortable and ten is perfect and you make them one to ten on aesthetics where one is hideous and ten is beautiful and I took this chair out and the comfort scores were everything you would expect sevens eights nines even which is unheard of in the chair world and they were very very happy about that but then they did the aesthetic scores and they got a slap in the face the people came back and said gave it to Zoar threes which is you can't go to market with a two or three people were saying this thing is so ugly there is no way I would Avram it looks like a piece of lawn furniture looks like a one that I saw the results of one of the first focus groups it was described as the exoskeleton of a giant prehistoric insect it was described as something off the set of Robocop which is big movie back then and it was people hated it and you can't the rule of thumb in the chair world is you really need to score above seven on both those dimensions to be able to sell something and they didn't have it so what do they do now this kind of dilemma faces companies all the time they have an idea they think is really good and they go and try to validate their own intuition by asking consumers and consumers sometimes deliver a verdict that makes you very unhappy and the question is what do you do with that verdict well Herman Miller decided to hell with the consumer verdict they like the chair and they were gonna bring it to market so they did just that they brought it out and in the beginning no one predictably enough like this chair didn't sell but slowly began to pick up some support from here and there and really kind of edgy avant-garde designers kind of took to it because it was so kind of weird and then it won some you know prize from France for new design and then he started to show up in in television commercials and then it began to pick up real traction in Silicon Valley and then suddenly it just took off and the sales were mounting by twenty and thirty percent a month until it became not just the best-selling chair in the history of Herman Miller but the best-selling office chair of all time I'm talking of course about the Aeron chair and I'm sure that many of you even work in an Aeron chair or work in something yet has been directly inspired by the Aeron chair the Aeron chair changed the face of chairs as we know it it's probably the most influential office chair of the last 35 years but that story I'm quite sure from you presents a problem right here was this product which was absolutely revolutionary which made zillions of dollars for its company and made heroes out of its designers and yet by the normal rules of corporate operation it should never have come to market right the very mechanism that its company had set up to judge whether it was worthy of coming to market do now pronounced it absolutely unworthy it's a paradox right it's a problem or than that it's okay what do you do of what usefulness in other words is a mechanism for measuring people's reactions to something if those reactions aren't ultimately predictive of how they feel and sure enough just to test this Herman Miller two years after the Year Miller the Aeron chair came out went back out and did another round of focus groups with people and asked them what they thought of these chairs and people said well we think it's an eight on comfort and by the way we also think it's an eight on aesthetics they once thought it was ugly and now they thought it was beautiful so what usefulness furthermore is that is a test of measuring people's reactions if one year it measures and says it's a two and two years later they come back and says it's an eight so is the chair beautiful or is it ugly now I'm very interested in that story in this book because my book is a book about snap judgments about the kind of thinking we do like this and when you take a chair out to focus groups you're getting people to make a judgement like they look at it do you like it right now my question that I asked in this book is how seriously do we take that judgment what does it mean we understand if someone took the chair home and live with it for a year and sat it in every day and gave us a judgement we understand that kind of judgment but we're not doing that we're having a focus group we have the chair in the center of the room when we say what do you think if people say bla bla bla bla well what does that mean what do we know about the accuracy the the strength the that whatever of that particular judgment and so the book is an attempt to come up with some kind of principles for understanding when that judgment is trustworthy and when it's not so let's just walk through that story and see if we could extract some principles principle number one is that our preferences as expressed in a snap are extraordinarily unstable that's a very important point because it is the assumption of many people in many aspects of our lives but let's just talk about market researcher the assumption of a lot of market researchers is that when they measure a preference for something they're measuring some kind of inherent position on that issue or product or thing right but in fact that's wrong in fact when you measure something in a by a snap judgement what you're measuring is unbelievably volatile let me give you an example Coke and Pepsi remember the Pepsi challenge like two cups once got Coke one's got Pepsi you give it to coke drinkers and you say take a sip of each which one do you prefer and lo and behold coke drinkers always prefer Pepsi right that's what caused coke to do the whole new coke thing twenty years ago because they're getting killed in this seeming extraordinarily objective measure of what people preferred there's all this internal correspondence in Coke at the time when they were like oh my god maybe is we're not the best take is tasting coke right maybe American American tastes of change and colas not what it once used to be maybe we got to be more like Pepsi after all we're asking not just five or ten people but you know tens of thousands of people were given the Pepsi challenge and the result that's an extraordinary robot statistical margin and they it's something like 60 or 65% of coke drinkers said in the sip test that they preferred Pepsi doesn't that seem to suggest that Pepsi is better tasting than Coke right she seems like a very stable way of measuring preferences no doesn't suggest that at all because the whole test is in fact a kind of fraud it's a sip test when you're sipping any drink you will always prefer the sweeter one right if on the other hand you drink the whole can you the sweeter the sweetness will start to become cloying and your preference will shift back to the thing that's not so sweet Pepsi is sweeter than Coke Pepsi will always win a sip test if you make people drink the whole can they'll prefer coke right well which of those two scenarios is more like the real world well unless people are buying Cola taking one sip and throwing it out you're actually far more interested in what they think after tasting the whole can there is a case where the preference the expression of what people think in a snap judgment is completely volatile it is entirely dependent on how you choose to measure it and whether you choose to make them sip it drink the whole can I'll give you another example of sort of need props for this one I don't have them another example of how I can demonstration of unstable preferences suppose that I gave you a little we did a Pepsi challenge but I ask a slightly different question which is one glasses coke one glasses Pepsi I just want you to taste them both and tell me which is which which is Coke which is Pepsi right not that hard I suspect if you did this right now in this room probably 75% of you will be able to distinguish between the two of them coke is sweeter and Pepsi or coke is not a sweet Pepsi is sweeter a little more lemony coke is more of a kind of molasses he kind of right that's easily done all right now so now you're all confident that you can successfully make discriminations among the colas all right you can't if I change that test in a very subtle way I will completely obliterate your ability to make that very simple judgment and if test the changes as follows I give you three glasses one is one of those colors the other two or the other and this time I'm gonna make it easier for you I'm gonna say I don't want to know which is coke and which is Pepsi I just want to know which one is not like the other two right Sesame Street question sound easy totally difficult if I did this in this room right now the success rate on what's called the triangle test would be at 1/3 it would be chance you can't do it what people do when you do I've done this so many times now that I what they do is this they taste the first one they go yeah there's a second one in they go yeah except third one oh they go then I go second one and then you know you've got them because now what have I done I've shaken you but you used to be able to have a very clear way of establishing a preference you were just comparing these two things but by introducing a third element I've so clouded the picture that you don't know what you're you can't make a decision anymore right you've lost it your hold on this question of what is Coke and what is Pepsi which you thought was so strong and profound is incredibly fragile I can shake it just by introducing a third element and that's the first lesson of these things when you look at wha how somebody can go from thinking the Aeron chair is a 2-1 year to an 8 the next year that shouldn't be surprising because when you measure the two you're not measuring anything robust right it's a preference preferences could change like that okay second lesson second lesson has to do with what's called the storytelling the storytelling problem is psychologists like to call it and that is that when you ask people to explain their snap judgments more often than not they simply make up stories there's a huge literature on this in psychology and it is it is so utterly daunting for the for the UM for the cause of explanation gathering but basically it says that when we have a snap judgment that is a product of our unconscious and our unconscious is a locked-room right it's uncommon SAR unconscious it's outside of our awareness we have no idea and we cannot have any idea about what's going on inside that room and so when we're asked to cope with explanations we just make stuff up very very famous study that demonstrates this which is we could put all these things we could have done them today with in great fun if you had several hours but you hang two ropes from the ceiling and you have people come in and you say there are four possible ways of connecting these two ropes right there the ropes are far enough apart that if you grab one rope and you pull it over like this you can't reach the second one so the question is come up with four strategies of tying these two ropes together there's all kinds of objects around the room that you can use as props first thing people do is they see a chair right pull the chair out they tie one rope they've went to the chair reach over grab the other rope right put them together that's one then they see there's an extension cord in the corner and they go over Taj extension cord to one end after ups then they see there's like a whole thing in the corner they grab the hull they pull one rope towards them grab the other one together and then they're stuck thinking okay I got three by notice before I'm sitting there they're thinking and thinking in ten minutes past they're starting to think am i stupid I mean I started to doubt themselves and at that point the experimenter walks across the room to get a cup of coffee and as he does he brushes one of the ropes with his shoulder and it starts to swing back and forth at which point everyone always goes like this oh of course and they get up and they say they take one of the ropes and they swing it back and forth like a pendulum then grab the other one and when it swings over they connect them and then the experimenter asked them where did you get that idea for the and they'll always say you know I had this image of 8th grade physics when we were studying I'll say I saw this movie that don't Tarzan Tarzan swinging on the divine no one ever says well because you brushed it with your shoulder and I realize that now why don't they say that are they lying are they embarrassed no they're not lying or they're embarrassed it's because that was a nun cue that was picked up by their unconscious they don't know where it came from all they know is the outcome which is their unconscious telling them pendulum right that's all they know well in that kind of situation what do we do when we're asked for an explanation we make up a story and that story has nothing to do with the actual reality of the situation I had a long discussion when I'd write my book up with his tennis coach Vik Braden who was telling me about um he's obsessed with this issue because he's a coach he's trying to explain ways of playing tennis better right so he spends lots of time talking to tennis pros you know Pete Sampras about their tennis and what he says is the explanations that pros give for how they play tennis are never useful and he gave me the following example you talked to Pete Sampras it's used it was our whipping boy how does he hit up a topspin forehand this is what he'll say he'll say at the moment of impact with the ball you roll your wrist to produce the topspin right okay so pick Braden videotaped Pete Sampras hitting topspin four hands and slowed the video tape down - so he's looking at slices of a hundredth of a second and he showed that Pete Sampras never ever rolled his wrist in fact he cannot find a single professional tennis player who rolls his or her wrist when they had a topspin forehead you roll your wrist after the ball is gone if you roll your wrist when the moment of impact you sprain your wrist right now here are people who know more about topspin forehand anyone else alive they hit the topspin forehand better than anyone in human history yet they do not know why they do it or how herby they don't know how they do it and to ask them to provide an explanation is to come up with an explanation that is more than in this case useless to cook it is to come up with an explanation that is actively harmful that's what we do as human beings when we're asked to account for a react or snap reactions we make stuff up and yet society as far as I can tell is rife with people who have this naive faith that somehow as individuals we're capable of actively representing the reasons for those decisions that are made in a moment left point third lesson from all this and that is that it is not just the case that when we ask someone to explain themselves in the moment they come up with nonsense it is also the case that the act of asking someone to explain themselves actually changes under certain circumstances what people believe there's a large body of literature on this but the simplest explanation is something is this famous experiment called the poster test and the poster test is suppose I divide this room in half right down here right and I have crossed this wall all these posters tons of posters and I say to this half of the room you guys can have any post do you want take it take it home it's yours right pretty simple the best click the one you like the most to this half of the room I say same thing pick the post you like best take it home but before by the way you take it home could you just write out a couple of paragraphs of explanation of why you liked that poster right okay then six months ago I call you all up I say that poster remember that poster I didn't like your poster this half loves the poster love it it's up in the corner in my bedroom love it what a great poster this half is like I hate it threw it out and then I look at your poster choices and I find that this half the ones who were required to choose posters chose different posters they didn't choose the same one is your name I'm guessing that this half randomly speaking is roughly the same as as this half shows version and then I look at the kind of posters you chose and I realize it as if there's a there is a pattern in the difference that the posters on the back wall were of two varieties some were impressionist paintings and some were really cutesy ones up like kittens on you know balance bars you guys all chose the kittens you chose the impressionist posters now why is that because the act of asking you to explain your preference biased you towards things that you could explain I'm guessing that only a small number of people here feel confident in their ability to write meaningfully and intelligently about impressionist art so as a result what did you do you went with the kitten because you said my unconscious level well I can talk about why I like kittens I had a kitten when I was young the other night right so the act of making you explain your preference not just led you away from the thing that made you most happy from your true preference but it biased you in favor of the thing that was most conservative right it biased you in favor of the safe choice and against the thing that was strange and unusual and that to me is the single most scariest and most dangerous part of approaching snap judgments in this naive way and that is by asking for explanations we create a situation where people turn away from what is radical and a little bit strange and towards the things that are safe so think of this in the context of the Aeron chair people are faced with this chair and they're asked to explain what they think about it and they don't have the language to describe this thing it's totally unlike any chair they've seen before right so what are they left to do they have to prop with some kind of explanation and the only words they can find our words that have all kinds of negative connotations I hate it right but what do they really mean they didn't hate it at all they loved that chair or they had within their heart the capacity to love that chair once they got used to it right what they really meant was not that I hate that chair but that chair is unusual I don't know what I think of that chair yet I've never seen a chair like that that chair is different from all the chairs I've had in my life now that reaction is profoundly different from the reaction that I hate that chair in fact it's on this side of the tech spectrum whereas Haynes on that end of the spectrum and yet by using this simple mechanism of requiring people to explain their preferences we have come up with a system that is incapable of distinguishing between objects that people truly hate and objects that people actually will end up loving and have the capacity to transform a marketplace that is the danger of taking a snap judgment at face value not understanding that when we leap to a conclusion like this there is a whole world of complexity below the surface and that this particular conclusion that we reached to is not robust it's fragile and it can be it can be corrupted or or biased in all kinds of complicated ways unless we understand those sources of corruption and bias then we're helpless in the face of making sense of this very critical part of human decision-making so that's a little pricey of one of the many many arguments in blink and I'd be happy to ask answer any questions it and then standing up will do suggest what do I suggest instead yes well I I don't hate all market research I hate market research that I feel is that is naive and literal so in the case of say the Aeron chair for example suppose instead of asking people what we thought of it we just gave them the chair and we just observe them for a couple months that's fine right just have them and or suppose that we waited six months to get their reaction why what's the hurry why try to you know this is a chair that you spend millions of dollars because you know it's truly unusual it's gonna take some getting used to if people have to sort of incorporate its aesthetic into their worldview that's gonna take time and actually Herman Miller understood all of this intuitively that's why they went ahead with the project even the face of such overwhelming so I don't mind observational stuff stuff that has some kind of you know if people interpret what people say as opposed to taking it at face value if there is some kind of intermediary step then I think it's fine but too often there's no interpretation it's that what we heard was XY and Z without realizing that that can mean many different things sure so yeah if that falls within the snap judgment range over there for B you know you know you're the same person two days now if you make a different thing the fact that by forcing someone to make a decision about the interview which microsoft does famously you're actually biasing people away from the more radical but possibly better personnel this is I have very very strong feelings about interviews although I for reasons that I am now kicking myself I don't actually have I have a little bit on interviews in the book but not a lot I should have perhaps put in more you're quite right you know if you talk to psychologists they will tell you that there is no more flawed methodology in American life then the job interview it's an absurd process which nearly everybody botches and nearly everyone believes by the way they're not watching as they are watching there were a very limited number of things you can learn about somebody in a face-to-face encounter they are real right but they're limited so you can learn that somebody is you can learn how energetic they are you can learn how extroverted they are you can learn you could have some measure of their self-confidence in that encounter so if you're hiring someone to do lots of job interviews you can learn something about how they approach it's not be very very useful you can also learn whether you're attracted to that person and whether that person is attracted to you another potentially interesting you know important thing if your main motivation in hiring is to you know come up with employees who you can later hit on there are all kinds of other things which you think you can find out which you can't write you cannot learn from an hour of conversation with someone how open-minded that person is can't can't learn how conscientious they are how hard-working they are can't know anything about their ability to function in very different contexts right I was talking to an academically group recently and I was you know in the academic world typically before they hire you you give a job talk right insanity what is the job talk it's a case where I take somebody and I force them very early in their career to stand up before a group of incredibly discriminating critics all of whom were seen into them and with their entire career in the balance gave a successful lecture now none of the traits that you were testing in that moment are at all relevant to the life of an academic in fact I'm not condemning someone who chose academia in large part because they aren't very good at performing in high-pressure situations and don't want to and have no interest in so doing right it's crazy it's like if you were the general manager of like the Seattle Seahawks and you made everyone who is trying out for the offensive line do croche I think because well so it's okay crochet I don't know if we can why would you be interested in that right why test for things you don't care about yet we do this kind of constantly the job interview for if you're hiring someone to work the perfume counter at Saks then by all means do a job interview but a classic face to face job interview but if you're not then I think you sort of have to come up with better strategies okay make a distinction between experts and non-experts and specifically that we can rely on experts to be able to articulate and hopefully some sort of an accurate way the reasons why something is good versus something is bad and we've seen through the talk you've given today that there are experts you can trust an expert you can the sports star for example versus the executives at Herman Miller is is there some sort of a way that we can tell externally whether somebody else is an expert without being an expert ourselves experts can step backwards Rolland most experts yes you're quite right that one or the right talked a lot about this in the book one of the fruits of experience and expertise is that you do get better at knowing how to explain where your snap judgments come from or at least at least or at least you're less susceptible to the kind of biases and corruption that I've been talking about now I gave the example of tennis stars not being Madhu's but that's actually a specific case because a tennis star who's 25 years old and playing is actually actively uninterested in that version of their expertise right they don't think about they're experts in playing tennis not in teaching tennis so that it makes sense that they wouldn't have mastered yet the ability to understand precisely what they're doing if they're doing it in a kind of unconscious mode but I am very I do think there are striking differences between the kind of between between those who have specific knowledge in a field and have up it snap judgment and those who don't and you know there's no perfect way to measure that of course what we've good but we've moved this whole project is to move decision-making from the false security of the kind of deliberate were a conscious world into something that is by definition a lot fuzzier and I think we just have to learn to live with the fuzziness and that means we don't always know right away whether somebody is his expert Herman Miller trusted bill Stumpf because Bill stump had a history with them that's why they were going to say you know he did the ergonomy equi they made us a lot of money he's probably right this time too so there is a kind of we're getting into as opposed to you know this area of human judgment which is a necessity something that cannot be beautifully calibrated and described and in the same vein our experts dangerous because they do maybe know more and that Richard changes the way they think about the issue versus the average man well this gets into I don't know did Jim Zaraki come through here with the wisdom of crowds he's my colleague at The New Yorker who's written his book which is sharply critical of a lot of expert judgment and saying that aggregated decision-making by groups of non experts can be superior to experts in many forms I'm actually profoundly sympathetic to a lot of his arguments and so my I'm I should be careful to kind of specify what both Jim and I are trying to do is to say that the kind of standard model the notion that an expert working rationally and deliberately and logically consciously through a series of choices is always the best way to resolve a problem we're both we both have problems with that we think that that works for some things but it's you know it's crazy to assume that that model actually is the best for all manner of decisions and he could takes it from this side to say actually groups are better than individuals in many situations and I could take from the other side to say there were moments when the kind of this kind of unconscious decision-making has real advantages if that sort of answers your question I'm wondering in your studies and you notice any power and sort of learn something yes well the common pattern rather than focusing on individual and differences in individuals I like to focus on differences in environments so to give you an example I have a chapter in the book that talks about the classical music world and for years and years and years classical orchestras were entirely male if you asked people in the business maestro's why they only hired men they would say well men are better musicians or women they are fundamentally inherently better and you say well why do you think that nate say well we have auditions open auditions and women come and they play alongside men and the men always play better so you know what could be a more transparent and objective way of measuring musical ability and if men are always winning you know women only ever win the competition for the harp and so you know who there's a problem with women here well then for a variety of reasons in the 80s they started to put up screens in auditions so you could only hear you couldn't see and immediately they start to hire women and now in the last 25 years orbiters have gone from being about five percent female to being roughly fifty fifty which says that since the imposition of the screen women had disproportionately won more auditions than men which would suggest actually that if anyone is inherently better musician it's a woman but put that aside what that says is that in ways that they did not realize the evidence of their eyes and their unconscious feelings about women were prejudicing the snap judgment because an audition is a snap judgment the snap judgement that kind of maestro's were making about people playing music that they were they had their judgment was corrupted by this enormous e apparently powerful thing called you know sexism and when we put the screen up what we do is we took the maestro who was a very bad snap decision maker and we made him a good one again right now that says to me that people make good snap decisions in environments where some attempt has been made to to clean up sources of contamination and what's interesting about that story of course as well is that it's a case where we made someone a better decision maker by taking away information that's another key very interesting thing that snack decisions are better when we've gotten rid of sources of contamination and also where we have simplified the kind of decision making palette I talked a lot about that in the book as well so that is where I would like to see rather than try and identify individuals who have this special ability I much prefer to think of it in terms of environments where people who have the ability to make successful snap judgments are allowed to exercise that ability freely and openly and that's it to me the key a lot of usability testing difficulty and they try a feature in our lab and sometimes buggy you are it's the demo and so it's not the real circumstance are we wasting our time no I mean it depends what you it's like you know think about I'm from New York so this is a topic Manhattan it's a topic of course that's near and dear to my heart may be of no cultural relevance to any of us at all any of you at all you write your shrinks you know I'm gonna catch and you say I hate my mother now what does the shrink do in response to that statement they have one responses to say Malcolm hates his mother another response is to say Malcolm at the moment is saying he hates his mother but he actually means something else and we're gonna spend the next ten years now so a usability study it depends right depends how you're interpreting that right if you use if you're looking at what their people are doing and saying and you're interpreting it like a shrink does it's very useful if you're not it's not useful at all now that may seem really obvious point but you would be amazed at how few people do the kind of shrink approach when I was doing this book I was talking to all these guys in Hollywood who do you know user studies of movies like do pre market screening of movies and science I had an actual conversation with a guy who said he said you know one of the weird things about testing comedies is that you'll watch the audience while they're watching a comedy and they'll be laughing their guts out and then you ask them afterwards did you like that movie you think it was funny and they'll say now isn't funny at all and I said to that Oh so which of those two versions do you believe and the guy said well what'd they say of course is that not backwards like I just have looked at the guy I just said you know I don't know what to make of that like that's just really deeply puzzling this is Hollywood they're not dumb but they're just not interested in to him it was a kind of the actual laughter that you would experience in the moment was a kind of was noise in the system tickets not based on the laughing people in the theatre but the buzz they get afterwards as to whether that person tells other people that yeah but it's a funny movie or not and my actual question is we need focus groups with multiple people in it is there value to that because they knew each other polluting issues out of room one strong personality one compelling personality to basically turn the whole room you think maybe one on one or kind of more focused focus groups with individuals is more valuable than throwing a group of people but then again that goes against the wisdom of crowds yeah no I think any kind of I think any step away from the kind of standard focus group methodology is probably a good one so I'm short of throwing about altogether which would be my dream yeah I'm sure like mixing it up and doing all kinds of viewing people in the people's experiences in the greatest possible range of situations is probably all good on your movie point it's very interesting you say that that is where you go that's what they would say they would say I actually care more what they say about the movie than what they did because I want I wanted this movie talked up but then that assumes that what the way they described the movie to a friend is similar as the way to the way they describe it to you and that's wrong the problem is that the the form of asking someone formally after the movie is a powerfully distorting form it's not the same as over beers with you know Jim telling Jim I've been looking for studies retrospectively that look at how well the firm to predict what actually happens so do you and how well the studies actually predict outcomes and I cannot find anything yeah so maybe you come across something in your in your research that you can you know there's precious little data because if you were in the focused with business would you produce such data but but I will say this that the bias the problem is the further you move away from things that are in the mainstream the more probable you have so imagine that you have just produced a movie die-hard five right and you want to know is diehard five as good as diehard four four can you predict that find that out from a focus group absolutely why because your focus group will be full of people who saw diehards one through four they're completely familiar with the genre they've no difficulty explaining their feelings about die hard how hard can it be right so and also die hard 5 is an utterly known quantity that's not the problem the problem is you know you get something like on the family which had terrible focus group you should have never made it to air only made the air by a kind of chance yeah and all these signs that all these things that do that's something that's a radical departure from the mainstream that's where the focus group is going to have increasing a little validity now so much that comes on to the marketplace is purely derivative that you could probably pop up your numbers on focus groups if that's all you're testing and if you know so and for people who traffic only in derivative products then knots about but I'm assuming that right here your interest run beyond purely derivative products right you actually occasionally more than occasionally come out with products that challenge people that require them to reorganize their mental universe you can't focus group that right that's where it would really fall down patience of snap judgments yeah where are people in the voting booth making a snap judgment or have they been taking in months and months of information yeah I get that question every time I talk in the northwest yeah I I always had the same answer which is that I am a Canadian and I have absolutely no understanding of the American political process and each year that passes in this country I find myself having less and less understanding so I have no clue I don't think it doesn't the notion there is a notion in the political world that I find out early puzzling which is that it is a snap judgment and that there is some how large percentage of voters up for grabs right up to November you know second how could you be up for I don't I don't notice the notion if I decided is so weird to me that you and that you would it would actually you actually be in the booth you'd be like whoa you know oh so no it's an all that I got I just you know so I steer clear with the exception of telling the story of Warren Harding in my book who I think is a wonderfully emblematic figure I don't say I really have nothing to say about politics I've been biased against this side a similar thing that you do have in Canada I think like juries yeah does I mean there's any of your research sort of relate to well if I don't have any stuff on juries because we're just getting there like that the core of psychological research was I was interested in this book there's a bunch of people who are examining things like our levels of on what pata tubes do we carry with us unconsciously and what impact does we have in our behavior and what we've gotten to the point now where we understand that there is a clear distinction between our unconscious and our conscious attitudes they have nothing in common with each other that you can consciously be for example an extraordinary liberal person and yet simultaneously Harbor quite retrograde views on those same subjects and that those unconscious beliefs will affect things like how you spontaneously behavior ransom so let's say that I'm someone who has who's consciously very liberal and unconsciously not so I might be perfectly open to taking applications and seriously considering African Americans for a position at my company but when I actually meet them my spontaneous unplanned unconscious behavior might be quite hostile and I would be totally unaware of that but it would he would significantly distort the outcome of the interview so that's where we are right now we understand these things exist and now we're trying to figure out our we psychologists are trying to figure out precisely what does that what difference does that make in settings like Jimi trunks the presumption so far appears to be it's got to make a big difference and I would love to see you couldn't do this of course because no no profession is more deeply resistant to the findings of science or the findings even of reason than the legal profession but you know let's just do a simple series of experiments where we put a screen up where you can neither see nor listen to the accent of the person charged right let's just have let's put the defendant give him a screen let him let the defendant email with the jury write email with his responses or her responses to the to the Jersey Boys yeah a prosecutor let's see if that makes a difference I would be very surprised if it didn't and I would think that if we were trying to construct a sort of more just society we would start to seriously consider those kind of considerations the court so it was the best anything about your book is this idea denta we have to restrict the information that we have available to us and what did that also could apply at a society break up recently there's been people talking ihara you may have mentioned it's fairly well-established if there's differences between men and women on certain kinds of activities but maybe it's a society we're trying to say we don't want to know it's actively the student making a bad decision that we don't want to make so let's just is it possible some things are just better off the side of not knowing about because we know it's gonna leave it to us making wrong decisions ha boy yeah I would the question is a very interesting one and that is about the benefits of information restriction and I would make a distinction between in the book I talk about specific kinds of decisions that are snap decisions that are markedly improved by deliberate attempts to restrict the quantity of information that's on the table and that's pretty clear that as human beings one of the things we do is systematically overestimate the value of marginal pieces of information and time and time again you can see that people making predictions the classic study of this is you take a group of psychologists and you say we would like to you to make a diagnosis about a patient and we're gonna see how accurate your diagnosis is right so you start out and you give them the psychologist a paragraph about the patient then you come back and you give them and you see what they say and then you say well how confident are you in your diagnosis you measure confidence and you measure what the diagnosis is then you give them a page then you give them five pages and when you give them 10 pages of information and what you find is that regardless of how much information you give them the accuracy of the diagnosis remains exactly the same what changes is their confidence in the diagnosis with a paragraph they're very they're very little conference in what they're doing with 10 pages of information they think they're absolutely right as it turns out their success rate their accuracy rate is very very low in all cases but so our attraction to more data is not about the improving the attract the accuracy of our conclusion it's about just improving our psychological state right about our confidence in the decision that we're making so it's a very important case on a on a case by case basis there are instances where we may want to clear the playing field but to go I was I can't resist responding to the Pacific comment about about about mr. summers at Harvard and his comment about women there what he's making is a kind of classic I even go so far to say to use the ad hominem a classic economists there he's just it's his assumption is that there is no difference there's no there's no denying that there are differences substantial differences between men and women but the question is are the differences between men and women greater than the differences within each gender and the answer is they're not they are on the scale of things that separate individuals relatively trivial so it's not that you can't say there are differences in men and women it's just that they're just not that important and it was it's not likely to be the principal difference in explaining why women might succeed at a certain being if he's mentioning that you actually mentioned your book that you know what women are more likely to do worse than a test amassed if they're if they've mentioned the fact that women get worse yes as the heart of administrator it's like it's a particular burden even if it were actually true in a substantial way not to be mentioning it so he doesn't disadvantage those people who are you know who for whom within the group they can be exactly what are two more questions imagine sending email extra information yeah all night and saying that's good information and yeah yeah the question is what's the floor at which point does limiting information start to do more harm than good and there absolutely is a floor but I will only say that I think in many of the cases that I'm that people are interested in reforming we're a long way from the floor I don't think that even if you restricted in a standard criminal jury trial even if you restricted access to visual or aural information about the defendant you would be somehow lacking for information you're still flooded with stuff about it this person it's just that it's all confined to a kind of relatively narrow emotional band and sensory band I don't have any problem with what I'm losing because like I said you're not the mistake many patients many juries make is they end up deciding whether they like the person or not and we are manifestly uninterested in whether you like the defendant the issue is only whether the defendant is guilty or innocent and so I'm quite happy with what I take off the table in that instance but what a nice question sure so could you comment on the effect of time because one of the things that like in snap did I mean the whole notion of a snap judgment is that you're forced to make it immediately having both done and is part of a user test some things like back in definitely situations or someone's like okay what did you think of that and it's very it was kind of kind of overwhelming and had you had a day or something you might respond for it and I wonder if anybody's tried to do studies where they're trying to see you there yeah with the chairs time this is a the question is about the effect of time on snap judgments and it's the Seamus it's the last question because this is actually a really really really huge issue and I touch on it a number of places in the book just to briefly there's a chapter in the book I'll just talk about one aspect of this which is there's a chapter in a book about cops please work but the shooting of a Madhu D Allen and which I talk a lot about the kind of decisions that police officers make and the question is cops have to make snap judgments all the time so what distinguishes a good police officers snap judgment from a bad one and there they that it really does seem to be the case that to the extent that you can slow down the situation they're in they'll make a better and better snap judgment now their definition of slow is different mere definition of slow so classic example is the shooting of President Reagan in 1981 or three I forgotten three I think it was you remember this that John eats outside the Washington Hilton he come Reagan's giving a speech he's coming out John Hinckley's in the crowd they're pressing around Reagan in Cleese arm reaches out he's got a 22 he fires I think five shots the bodyguards grab him wrestling to the ground now how long do you think how much time do you think elapsed between the first moment at which John Hinckley was spotted at which the the attack became evident and when John paint all the shots were fired and John Hinckley was lying prone on the ground first time one point eight seconds now in 1.8 seconds this is look this is this is answers the question of why Reagan got shot in 1.8 seconds how good a snap judgment can a bodyguard make the answer is not a good one at all you can't do it one point seconds as a bodyguard all you can do is this they for the guns now if you talk to bodyguards they will say that's one of the biggest errors in the body this is a big argument in the bodyguard business they're going to shoot bodyguards have guns right now we would all say well of course but many women say actually no because attacks take place in such a short period of time you have a limited number of time of things you can do and decisions you can make right if all you if it's one point in seconds and you go like this you've squandered your 1.8 seconds right you're going for your gun to do what the guy's firing shots right now the damage has been done right now your first instinct should rather be to put yourself between the shooter and the target or to take Reagan and to pull him down to the ground instead all these guys go like this and Reagan's get shot right so take away the gun that's what they always say in any case that's a separate but the whole game there then if you want to make bodyguards or cops have make better decisions you've got to find some way to inject some time into that process so that's why now quite unlike in Reagan's time if you're a bodyguard and guarding the president they'll be what they call whitespace they'll be you know 15 feet between the president and any unknown person at any given time and that's because 15 beat feet equals yeah a second or 0.5 of a second and that's all the difference in the world and the same thing with cops that there's an enormous amount of research and work and training in recent years in trying to train cops about how to inject time into dangerous situations so don't leave your vehicle stay in their vehicle you know radio for help never a cop shoot well this gets into a whole nother thing but whole thing about cops the number of situations where they ever chase people should be severely limited because you don't want to ever be rushing after someone because that creates this kind of but that's a it's a bit it's a really really big issue but there is a kind of balance that you need to snap judgment paradox if we need some time to properly germinate anyway well thank you all very much

Channel: Microsoft Research

Views: 98,702

Rating: 4.8894134 out of 5

Keywords: microsoft research

Id: FeegLvAkbY0

Length: 56min 11sec (3371 seconds)

Published: Mon Sep 05 2016