Book Talk: "Noise - A Flaw in Human Judgment" (Kahneman, Sibony, Sunstein, Brockman) | DLD All Stars

[Music] [Music] so i'm so happy that you're all here for this new kind of dld i can't hug you i can't talk to you personally but i'm so happy that you're here and for now before we kick off i have a special surprise for you a wonderful surprise a dld pre-event surprise the dld all-star book event you know we are very flexible from dlg we are flexible and we are spontaneous and when our good friend the great connector john brockman called me last week to say that he had a wonderful panel brought together um i couldn't resist i i said yes immediately yes john we have to do this what is it three superstars of behavioral economics went together wrote the book this book will be out only in may and now it's february and we have the chance to present it to you to you my dld community i really couldn't say no so we get created the dod pre-event john invited daniel kahneman kes sanstin and olivier zebroni to speak about this book and this is the dld pre-event cass is the robert wormsley professor at harvard and director of the program on behavioral economic economics and public policy at harvard law school olivier zibuni is professor of strategy at hec paris and an associate fellow at the business school of oxford university and daniel kahneman needs no introduction everyone reads his book everyone knows his studies his research he's the eugene higgins professor of psychology at princeton and the winner of the 2002 noble prize but the two others are also wonderful they are great stars it's so it's a superstar session it's like it's like eric clapton stevie winwood and jack bruce have a jam session together and john brockman the the publisher and editor of edge.org form helps to shift who who the guy who helps to shift this discourse from politics to science and technology in our times who brought all these famous scientists together on earth edge to talk about what's next john you brought it together you're the moderator over to you so we're here to talk about noise uh which has a lot of different meanings for our purposes uh we're talking about a revolutionary exploration of why people make bad judgments how to control both noise and cognitive bias um so for starters i'd like to ask each of you uh what is noise i'm sure you'll have different approaches to this um so why don't we start with uh olivier and go around clockwise hi john it's ironic that we have echo when we're about to talk about noise but that's how it is so now is the one that we are talking about is unwanted variability in professional judgments quite simply and i'll let denny elaborate and give us some examples well the concept of noise is a complement to the very much used and possibly overused notion of bias there are really two types of two families of errors of judgment as we'll see in a minute and bias is one of them and that's the one that typically we talk about all the time and noise is another that actually very few people talk about and that would be the subject of of our book and of today's conversation okay great so if you get on a scale and it knows you as a little heavier every day than you actually are that's bias that's my scale if you go on the problem of noise is pervasive and not sufficiently attended to so if you have a judge who decides that someone who did something gets a very long prison sentence and another judge who has exactly the same person decides that that person gets a short sentence that's noise if you have a doctor who on a monday morning says you're going to have to go into the hospital you have a really bad case of covid and says in the afternoon maybe or in the evening to exactly the same patient you know you have a very we're seeing a little unwanted variability in my communication given technology that problem of variability all over the place is part of human life and it's productive of very serious errors [Music] question uh danny you've been talking about bias uh since i met you in 2007 and a long a long time before that um uh why did it take so long to come up with noise and and uh and ed uh it was hiding in plain sight uh and is noise science and if so you know how do you talk about it scientifically or is it your intuition uh okay and you you should feel free to talk among yourselves uh it's above my pay grade uh i think this is the time to show the target slides olivia are you the one showing it i think our friends in the tech room are the ones who are going to show this slide that we consume because it's very abstract about it here it is this picture actually has a long history as so far as we know it was certainly used by daniel bernoulli in 1778 and what it shows is noise and bias so when you look at the top left you have five shooters shooting at the same target and they're accurate on the right you have a noisy display they're not biased they're just scattered they're all over the place bottom left is biased and bottom right is both biased and noisy and you can see that these the noisy and the bias are two very different kinds of error but they're both bad i mean obviously something is wrong with the noisy group and something is wrong with a bias group and it's not the same thing that is wrong with the two and interestingly enough when you think about how we talk about error in general we talk about error almost exclusively in terms of biases and that is because uh when we think of a particular shot which belongs to the bias group we have a sense that we understand it it's the same force that's operating on all five shots when we look at a particular shot on the on the top right no single shot is noisy noise is a characteristic of the whole setup of the whole ensemble and that's a very big difference with it turns out it's much easier for us to think about bias than to think about movies so what will uh this work on noise mean for the field of psychology well i think that you know what we hope it will mean is that people will realize that actually there are in organizations and wherever there is error that there are actually two sets of problems and that we've been focused on one and not on the other i should add that we have a pretty good idea of the relative importance of noise and bias and we have a way of measuring it which goes back 200 years and and it goes back to carl friedrich gauss and to and in that equation that describes error noise and bias have really equivalent roles and that is very difficult to appreciate but it's actually elementary algebra and it turns out that noise as a source of error is really quite important and yet we don't pay attention to it maybe we should give an example yeah it probably helps at this point to take an example danny doesn't it and by the way i think we've seen the slide now we don't need to to keep it on display forever perhaps perhaps the best example although that that may not be you know that may not seem to be interesting but i think it is is the example from the insurance company that we described in the book where if you are asking an insurance company for the policy someone called an underwriter will actually have to quote you a price depending on an assessment of how severe risk the company is ensuring and depending on who happens to be available in a particular agency of the company one or another underwriter will answer the request and we'll provide a quote now if you ask the people in that insurance company the senior executives of that insurance company is there going to be a big difference depending on whether you know the quote happens to be done by cass or by danny who are too qualified underwriter uh who are working today they would usually say no not a big difference of course this is a human decision so we expect that there is going to be some variability but when you ask them how much variability in general and this is actually quite a popular answer everywhere when you ask the sim the same question in general they say about 10 10 is probably what we expect there will be when you measure how much difference there is between two judgments of the same case when you actually measure it which is what we call a noise audit you find that the answer is not 10 the median difference is 55 it's five times larger than people think it is and that is the story of noise everybody understands that there is noise everybody understands that when it's a matter of judgment people are not going to have exactly the same judgment when you ask a judge you know if you have to sentence this defendant do you think that another judge would give exactly the same sentence the judge will say no of course not exactly the same sentence but the judge expects that another reasonable qualified unbiased judge would pretty much give the same sentence and every time we measure that variability it is vastly greater than people expect it to be that is why we think noise is a serious problem is it also an opportunity it's like completely an opportunity so um if if you have a firm that is suffering let's say from error resulting from noise where customers are treated really differently or plans are treated really differently depending on whether it's sunny or snowy or depending on which particular person happens to be in charge there are things you could do we call it decision hygiene one simple idea is to have a lot of independent judgments and then to aggregate them that is a recipe for decision hygiene and it dramatically increases quiet and as a result of doing that it can save a lot of money and increase fairness so you have a book uh which is just about to be published danny's last book thinking fast and slow is sold millions and millions of copies sales go up uh every year uh as college students come in uh everybody has to buy it um when the book came out i remember stephen pinker at a conference said danny's ideas about human rationality in the book might be the most important ideas ever uttered by a human being so uh how does that play out in in this realm of noise which seems very focused on business on insurance on public policy it has to affect all of us because we all make decisions but can you give examples just for the average person or are people interested in ideas and and not specifically in public policy feel free to talk among yourselves i'm not here as an interviewer okay yes why don't you go ahead with some examples from medicine because that's the topic that everybody can relate to okay so suppose you are uh feeling some pain in your chest and you go to a doctor and the doctor says i think you have a heart issue we're gonna have to do a lot of tests and then suppose you don't like the you shouldn't have tests because it's not a heart issue it's an issue that you're under stress that's an example of noise in the medical profession you can also have noise with respect to your cas can you hear us it looks like your connection is bad yeah we can't hear you yeah we can't hear you so here here's another example uh which i think everybody will be able to relate to a job interview i mean i think everybody has had a job interview and many people have been on the on the receiving end of the job interview you know of course that not everyone is going to make the same judgment about a candidate in a job interview and that's why usually there are several interviewers but the extent again the extent to which people disagree on the candidates that they meet and therefore the the amount of randomness there is in the quality of their predictions about who is going to be a successful uh candidate if hired is absolutely staggering we review some of that evidence in noise and it's very surprising in fact every time you ask people how well do you think you are able to make a decision of this kind in a job interview and you ask them to measure with some degree of confidence how well they're able to do that they vastly overestimate how well they're able to predict the performance of a candidate because it is just very hard to realize how much variability and how much randomness there is in that decision that's another example danny what's another one well i'd like to add it's not only an example but a direction most of our book as john was saying is really concerned with what we call system noise so that there are systems like an emergency room or an insurance company that make decisions and we look at the variability of those decisions unwanted variability that shouldn't exist and we have some ways of dealing with this we have a proposal what we call decision hygiene a set of proposals of how to deal with this but what we consider really quite important is that when you have an individual and that could be an individual making an important decision for herself or it could be a national leader making a decision that is actually looks and feels unique that unique decision could be improved if you apply decision hygiene to it and one of my favorite quotes from the book which is our friend olivier came up with it is that a unique event is a repeated event that happens only once and so the scope is not restricted to system nodes the scope of the surgeon hygiene and what you can do about noise includes singular decisions and that vastly increases the scope of the noise problem and so john to to to to continue to answer your question what we are hoping comes out of this book for the people who read it is a sense that whenever you're making a decision whether it's a recurring decision like interviewing hundreds of candidates and trying to pick the best ones or a singular decision like the fateful decisions of political leaders or maybe the once-in-a-lifetime business decisions of industry leaders there is a way to make that decision that reduces this unseen variability this unwanted but unseen variability we call noise if you do not apply these methods which we call decision hygiene you run the risk of being influenced without knowing it by all kinds of forces that introduce some randomness in your decision as if a chance device was intervening to change the output of your thinking you wouldn't want that to happen you would want to stop that from happening if that was the case that's what you need to do with noise you need to start to introduce decision hygiene in your decisions because just like hygiene protects you against germs you don't see and you don't know and you will never know what germs you avoided by washing your hands decision hygiene will prevent unseen forces from introducing variability in your decisions do you have examples from different fields you could each of you talk about yes give a few now that my technology is working a bit better so suppose people are being interviewed for jobs you might have an office that's deciding whom to hire those are fantastically noisy where an interviewer might think this candidate is great and another interviewer might think this candidate isn't great and it's noise or it might be that an interviewer is in a really good mood because the sports team that she likes best won the other day and says i really like this candidate whereas if the sports team had lost the candidate would look much less good and interview decisions are of course crucial to operations in many places the same approach the same understandings apply to admissions decisions for many universities and to evaluations of candidates who are already there who might be promoted or might not be promoted might get raises or not and as i hope these examples suggest it happens in our own daily lives when we're evaluating our next decision it might be about whom to marry it might be about what to buy and these are noisy decisions it turns out which in the case of marriage sometimes produces error well the idea of the the main idea i think of decision hygiene is how to deal with problems and the suggestion the probably the most important suggestion that we have is to break up problems into units and deal with those units independently and this principle of independence of different parts of the solution is an essential principle of decision hygiene it's difficult to implement it can be implemented in all sorts of ways from the independence of different people who participate in making a decision to the independence of distinct judgments and that that you have to make on your way to making a decision for example different aspects of a business that you're considering for an acquisition uh we are recommending making judgments as independent as possible and and this is key delaying your intuitive global evaluation of problems until the end so we're not proposing to dispense with intuition which people would reject and refuse we're proposing to delay intuition and that i think is a fundamental principle of of decision hygiene as we presented here's an example that for which a lot of people are focusing right now which is what should let's say germany or india do about climate change you might have a general intuition that the economy is doing very badly in one or another country and doing a lot about climate change isn't a great idea or you might have an intuition that climate change is an existential threat and doing a lot is a really good idea but once we do as danny suggests a disaggregation of the component parts suppose we're thinking what should germany's plan be for climate change then we will be able to make a lot of progress in reducing bias and also in reducing noise because the pieces once aggregated will often lead to a judgment that if not universally shared at least produces a lot less noise than if we just consulted broad intuition about climate change is a point that we haven't mentioned yet about the difference between noise and bias that i think should be mentioned and it is that bias is actually quite difficult to measure in order to measure bias and judgments you have to know the truth it turns out that noise is in in most situations in many situations much easier to measure and it's easier to measure because you can measure the variability of judgments without knowing which judgment is the accurate or the correct one and we call that a noise audit and there are possibilities with noise audits that simply do not exist when you're trying to minimize bias and so there are several features that this highlights bias we think in in many situations if there is a big bias for example in the insurance company if there had been a bias that is equivalent to noise in the judgments of underwriters the company would have detected it and if it hadn't detected it it would have been run out of business so bias is detectable much more easily than noise and noise it's detectable and therefore controlled probably to a extend the noises noise can be measured and once it's measured you can take steps to control it but there is another difference that we should point out as as we talk about measuring it which is that we we don't usually work up the same outrage about noise as we do about bias and i think that's a mistake and we should try to remedy it i remember a conversation with a judge an actual juvenile judge in fact who to whom i was explaining the idea of noise and saying noise is basically the fact that if a juvenile is tried by you the sentence will be different than it will be if the same juvenile in the same case is tried by the judge next door to which this judge replied well that's a fact of life there's nothing we can do about it is there and then i said well if the juvenile got a different sentence because one was white and the other was black would that be acceptable and she said no of course not and so morally we think that a randomness and unfairness and injustice that is caused by chance is somehow more acceptable than one that is caused by bias we think it's not we think it's a problem as well and we should do something about it but we need to work up a little bit of energy and outrage about it so that we actually start the effort of doing noise audits as danny was describing and doing something about decision hygiene one way to think about this is part of what we've identified as lotteries everywhere a lot of circumstances in which people don't think they're entering a lottery so if you're suffering from let's say some sort of psychological issue that's making you scared or sad and you go to a psychiatrist you're actually entering a lottery we find in the sense that what the psychiatrist says you're anxious you're depressed you're having a bad week those are three very different responses and a psychiatrist who's trained might actually offer one of the three that's a lot of noise and since mental health is one of the most serious problems facing uh humanity these days the idea that there's a lot of noise in the system that should be outrageous and it's it's a scandal but it's also a great opportunity there are strategies that can be chosen that can uh specify with more accuracy the what the actual truth is and as danny says even when there isn't clarity on truth there are strategies that can reduce the noise all right questions since science is uh is a methodology um and and you do experiments and you replicate them how do you as a psychologist what does this mean for the field of psychology how psychology is done and and how do you do experiments uh regarding noise that are replicable well uh in other words you won a nobel prize because you did experiments and uh they were repeated successfully where's the science here well the science potentially is a science of studying the variability of judgment instead of focusing on the mean focus on the variability and what are the conditions under which variability increases and what are the conditions under which it is reduced so this in principle is is a problem that science can deal with and psychology can deal with it's just a really fairly dramatic change of focus from focusing on the mean to focusing on the standard deviation so how will that play out in the field like the way you train psychologists the way psychology is taught it's a big field and it's very varied i'm not sure that this needs a profound change in the way psychology is taught but i'm expecting that if the book is successful this will create a large research enterprise because it's a topic that really has not been studied it's been largely neglected and it's going to be important within psychology to see what are the determinants of noise it's going to be important i think in comparing organizations so there could be entire areas of research that do not exist at the moment of comparative noise studies and i expect that that could really develop if you know our book is accepted as we hope it will be but will it have an impact outside of a study of large organizations like like ptas or teachers union teachers groups or the kind of stuff we're all looking at today i mean among the decision hygiene ideas their ideas about how to conduct meetings there are ideas about how to prepare checklists when you're trying to deal with problems so those are applicable not to log not only to large decisions and large organizations they're actually applied to how to run a pta meeting so and there are better and worse ways to run meetings and and the impact on noise is very clear that there are some ways to run meetings that favor noise and there are some ways that control or to take another example john something that we do not even think of as a judgment is actually a judgment sometimes take forensic science take the examiners who look at fingerprints left at a crime scene and say i compare this with the fingerprints of a suspect and yes this is a match or no this is not a match we have been trained to think of this as an objective determination as we discovered mostly thanks to the research of a gentleman called etiel droid who works at ucl in london it is actually not an objective determination it is a human judgment made by an examiner and that judgment is susceptible to a lot of noise noise between examiners so different people can actually disagree about fingerprints more often than we think not very often by the way but more often than we think and even the same examiner can when shown the same pair of prints at different times disagree with himself so if we don't put in place the kinds of measures that brought in this case proposes to introduce some hygiene in those processes there are going to be mistakes there are today mistakes when where those procedures are not respected so the book comes out uh in may uh what do you want your book to do what happens after someone reads it what would what would what's your desire for a result of the effect we would want researchers to drop whatever they're doing and start studying noise they don't have to drop what they're doing but we would want to switch in that direction we would want organizational leaders to ask themselves if their noise in my organization are there noise audits that i can conduct in order to establish and measure the amount of noise and and we would want practitioners and consultants to be conscious of noise as a problem so this could we hope uh induce changes in many areas so we go ahead castle sorry so we'd like a quieter world so let's say i'll just give one example that uh restaurants and other establishments are being inspected for conformity to requirements if it turns out that the inspections are noisy so that everything depends on who's the inspector on the or whether the day is cold or warm that's a real problem so that similar restaurants would be treated very differently and we know enough to know that really happens we want more quiet there one reason is it will introduce more fairness so that similarly situated people aren't treated differently another reason is that it should produce as we show in establishing the often greater contribution of noise than bias to error it should reduce more accuracy and in the case of food inspections it might mean less disease and less covet 19. and so the thought is there's a big agenda for improving decisions it also as john you're suggesting it bears on individual lives each of us is noisy and since i've started working on the book i'm alert to noise in my own family and in my own decision making now that's not our focus but the book certainly can quiet that form of noise as well i'd add to that that in order for noise to be addressed and recognized we first need to start to listen to it we first need to stop ignoring it and to stop sweeping it under the rug and what organizations are very good at doing which is why we're having this conversation today and you know people don't talk about noise all the time organizations are very good at suppressing noise at having procedures and ways of working that hide the presence of noise if you are one of the underwriters i was talking about earlier or if you are a doctor in a hospital that cass was mentioning or if you are one of the judges whose verdicts can be so variable you never actually get the chance to compare your judgment to the judgment of someone else and so you can live with the illusion that if asked your colleagues would agree with you when there are in fact situations in which you disagree organizations do a very good job of explaining them away they find reasons they point the finger at the corporate who made a bad decision and they exclude him so that they never actually realize how much disagreement there is between people who are making those pronouncements about the outlier decisions so we need to start recognizing those accepting it as a normal fact of life as a normal feature of human judgment even in fact sometimes as something that we can work with to reduce noise in the organization because if we aggregate the independent judgments of multiple individuals we will have better decisions that's one of the decision hygiene techniques but for that we need to face the music and to recognize the presence of noise it's interesting noise is something people really weren't talking about uh before danny's proposed this book uh and now it's uh it's something we're all dealing with in different levels in this regard um how is noise impacting your lives personally you know like let's forget the organizations for a minute and just people walk around today and uh the whole idea of bias has become a meme everybody has bias stories uh people don't have noise stories um in my own life i am don't have an organization i have a i have a phone and it's me i don't have to interact with lots of people in a formal way uh what's my takeaway for this was to take away a view individually like you go home you see your kids you deal with your parents uh you have relationships uh you know how do you feel just a personal personal take i don't think that people will become aware of noise and their own decisions it doesn't work that way you have no opportunity to discover the counter factuals that you could have decided differently so if there is any impact on the lives of individuals it's going to be through decision hygiene it's going to be through procedures that may become more popular about the way to think about decisions that face you and about judgment that you have to make i don't believe that people will become aware of noise because almost by definition you see one thing you do not see what it could have been you do not see the counter factuals and it's those counterfactuals it's it's the cloud of unrealized possibilities that constitute noise and our book is not going to make that visible it really is going to stay invisible we're going to have to deal with it without in many cases being able to see it you can give two examples one of which suggests some optimism on the individual side i'm much more alert in my own decision making to the possibility that i would think differently on another day in a in a different mood and i'm also much more alert than i was before i started on the book to the possibility that someone exactly like me might reach a different decision which means consultation of people who have different perspectives and maybe aggregating also to danny's point about groups it's just the truth and i think we all can recognize this that sometimes one person in a group will happen to feel very assertive let's say at the beginning of the meeting and can move the group in a direction on which the group will eventually have enthusiastic consensus whereas an otherwise identical group with that person quieting himself let's say at the beginning might end up in a completely different place and that's one reason why groups can be really noisy across groups and that suggests a few ideas about to handle let's say the risk that the individual confident person in minute one can drive the group in a particular direction and we have a a bunch of things to say about group management to increase accuracy and reduce noise within and across groups i think we have about three minutes left so i'd like each of you to wrap up with a minute starting with olivier noise is an unseen phenomenon it's a very big problem for two reasons that we've talked about at various points in in this conversation one is that it makes us make errors it's a code it's a source of error that is equivalent to bias that plays the same role as bias and that we shouldn't neglect the other is that it's a source of unfairness and injustice and for both reasons we should be serious about it uh the motto of a book is the following it's wherever there is judgment there is noise and there is more of it than you think it tends to be invisible and we tend to underestimate its impact noise is an unexplored country it's one of the few on earth that we really haven't spent a lot of time in and spending a little time there can pay tremendous dividends in terms of improving human decision making and reducing suffering and what are the downsides of studying noise you might think oh my gosh the world is a lot less orderly and consistent than i had anticipated or hoped okay i think all you have to lose are illusions about your own ability to about your own omniscience danny you get the last word oh well uh i think an interesting conversation that is going to take place is going to link judgment with algorithms and that is the development that awaits us because one of the characteristics of algorithms is that they are noise free and at the same time people want the autonomy of making their own judgments and i expect a lot of conversation about noise in the context of the interaction between human judgment and and the rules and algorithms that are beginning to impact our lives and i think this is a good example of discussion getting underway thank you all very much thank you so much listening to you for guys i was fascinated and you made me so curious of this new concept of decision hygiene i think we all all the audience learned so much and it's only the beginning probably because what you said on one hand the organizational types of decision making the psychological types of decision making also the um coding decision making support is one side i think what you just introduced is a new chapter of the conditional humane it's a new philosophy of what we as humans do and how we do it i'm very impressed i'm so curious about your book and i'm looking forward to have many more discussions like this i hope it's only a beginning thank you for being bringing this together john thank you for being at the dld all-stars event it was already a highlight i'm i'm whatever i'm i'm so happy thank you thank you thank you you

Channel: DLD Conference

Views: 21,605

Keywords: DLD, Munich, Burda, Websession, 2021, All, Stars, Allstars, Conference, Digital, Tech, kahneman, sibony, Sunstein, Brockman, BookTalk

Id: V9-VkegRtjA

Length: 44min 22sec (2662 seconds)

Published: Tue Feb 23 2021