Malcom Gladwell and Adam Grant: Keynote Conversation | 2017 Wharton People Analytics Conference

[Music] thank you for that very nice introduction I was pleased to be described as part of the not very taxing portion of the conference the that's actually very true you know I imagine this is going to be a conference to do tomorrow full of very very complex complicated analytics so I thought it'd be a useful counterpoint to all that intellectual sophistication for me to make up just a dumb obvious point many people believe that I that dumb obvious points are my strength and I'm always happy to oblige my dumb obvious point is that a analytics are of no value if you don't have a conversation beforehand about why you want to use particular analytic like I said is a kind of an obvious point but I worried that we don't have those kinds of conversations so what do I mean well I'm in it what I want to I want to start by just by playing you a little bit of a clip it's from an interview that was on the podcast long-form last week with a woman named Sheila kohake co hot car and she's a UH she's a writer she just came out with a big book on Steve Cohen the hedge fund trader at si si and she's talking in this clip about a job that she had as an analyst at a hedge fund some somewhere where many of you some of you may end up in New York City and just play the clip we can get a feeling for her I mean it was really interesting crash course in so many things for me just sort of gender politics and unfairness of trying to be a woman in a very male industry and high finance and Wall Street and money in the economy and all the sudden I really didn't know anything I'd no idea what I was doing I was trying to learn it on the fly which I did really enjoy in a way it was a sort of challenge but I kind of knew the whole time I didn't want to stay in that business and I was trying to figure out what I wanted to do but the money was really good so I would kind of you know I said I'm gonna quit and gonna leave and do this that and I'd you know get a bonus in January and I think uh so stupid to walk away from this I should pay but you know it was it was hard I wasn't very happy I wasn't very good at the job even though I found it sort of interesting to learn about it it was um you know it required making quick decisions about what to do with other people's money and I am a very cautious careful thoughtful person the thought of making a mistake actually keeps me awake at night like it's really upsetting thought and so having to just sort of look at the market and look at the news or see something some development then have to make a snap decision about revise resell I found that incredibly stressful and in fact that is that does not make you a good trader I mean the people who do well at those jobs or people who have a huge tolerance for risk and who are not emotional about what they're doing when you have to be very dispassionate and kind of cool under question I was just like a basket case so you know after nine eleven things on Wall Street got pretty bad and I had moved to a slightly larger fund at that point and they just started laying people off and I was one okay so on the basis of that clip what kind of person is she like Oaxaca she describes herself as being very cautious incapable of snap judgments and emotional in a way that many of the people around her were not if we were to analyze her using the Big Five personality scale you know which is extraversion neuroticism conscientiousness openness and disagreeableness she's clearly really high on conscientiousness she's super conscientious that's why she feels so frustrated in a fast moving environment and she's neurotic she worries a lot she gets stressed out if things aren't perfect right she can't sleep at night if she can't do her job that's why she's so unhappy doing that particular job if you read her book though you begin to understand how youthful those traits are she goes after Stevie Cohen who is the secretive man in the hedge fund world and she does an extraordinary job of bringing his world to light and she its a prodigious feat of reporting I'm a reporter by trade as well and I know how hard it is she is several orders of magnitude better reporting than I could ever be she's unbelievably dogged she goes back four or five six times to talk to people she bangs on doors and doesn't give up she's incredibly dogged if I had to sum her up using a word that's not in the big five I would say that she's really slow by which I mean she does a job as thoroughly as possible regardless of the clock that's ticking on the wall she's a tortoise right she's in fact a neurotic tortoise she covers every inch of the ground and I think we would all agree that there are times and places where being neurotic tortoise is really really valuable it's what you want right you want a neurotic tortoise so here's my concern if we really want notic tortoises for certain kinds of jobs particularly jobs that require mastering incredibly complicated worlds and being doing a very thorough job and not making mistakes which i think would would be highly suited for the present workplace if we want neurotic tortoises so much why do we design selection and evaluation systems that are biased against neurotic tortoises now I could go on and on about this since I've believed or not at the moment I'm obsessed with the state of neurotic tortoises but Adam omen gave me 15 minutes so I'm going to be very brief I want to give you one example of how we make life difficult for neurotic tortoises and that's the L SAT now I could just as easily have talked about the SAT or the GRE the GMAT I picked the L SAT because I figured there would be very few lawyers in this room so I could safely make fun of it without offending anybody and I should also make the confession that I've never taken a standardized test before my life have no idea but they're like but I was doing working on this talk in the library this week and this student sees me that I have papers on the LX et open on the computer and he gave me a sample test because he was studying for his health ed and I tried it and I got every question wrong so it's really hard it turns out the LSAT and it's really important you know you know this if you just rank all of the law schools in the country on the basis of their average of the average LSAT score of their incoming applicant an incoming class then you get the you get the US News ranking there's no difference essentially when we rank them that's what we're ranking so they're really really really important and they're really really hard so what is the outset well to answer that question I'm gonna have to do a little detour into the theory of test giving there's a distinction and I'm aware as I make this point that Adam Rose off stage knows it's probably ten times more about this than I do and so I am setting myself up for some major problems maybe when we sit down and talk but nonetheless I'm gonna go ahead there's a big distinction between speed and power in test giving so a speed test is a test where all the questions are easy anyone can be expected to answer them but what we're interested in is how quickly you complete it so most children's card games are speed games right snacks a snack as a kid that's a speed game right the underlying task is not difficult what we want to know is how quickly do you do it video games a lot of video games are speed games they're not the underlying task is not difficult on the other hand power tests are tests where the underlying question is very difficult difficult and what we're interested in is not how quickly do you answer it but how actively do you answer it so Scrabble is a power game at least in the way that it's normally played in my family it's speed games and a cryptic crossword is a power game not a speaking we don't care how quickly you do it or at least so most the time and the reason we separate out power and speed is that they are separate variables there's a little bit of overlap but you can be very good at speed a lousy at power and very very good at power and lousy at speed right and a really good example of this is chess so chess is a power game it's not a speaking to target but we play chess in a variety of ways where we speed it we place a time constraint so there's classic chess which is I think 90 minutes for the first 40 moves then 30 minutes for the balance with 30 seconds per move on in that's where we've it's relatively unseated then there's rapid chess which is 10 minutes and then there's blitz chess which is like 3 minutes or however fast you want to do it and when you do it that way when you compare the rankings for classic rapid and blitz you see real differences so Wesley so is the great up-and-coming American chess player right now on classic chess he's almost as good as Magnus Carlsen he's pretty much the best player in the world he's terrible if it's chess was a terrible he's not even close to being in the top ten right he really really on the other hand there's a 24-year old Chinese Grandmaster named Aaron Ling Darren Ling is not in the top ten in classic chess he's 12 I think I'm probably never going to be world champion however a blitz looks Jess he's fantastic he was the blitz champion in 2016 he's almost as good as magnus carlsen at blitz so if you are when you combine speed to a power test we say that the test is speeded and the point is that when you add speed to a power test you are not improving the accuracy of the test right you're changing what you're measuring you're mixing two different variables and when you mix those variables you're going to get different outcomes so who is the best chess player in the world well there is no such thing as the best chair that's chess player in the world there's only the best chess player at the at the particular mix of speed and power that we choose to play in international competition if we made international competition three minutes Darren Ling is going to be the household name as it stands because we don't do that you've never heard of Darren Lang right so there's a are built in our butt very element to the way in which we evaluate people under power conditions which is to what extent are we speeding the test so what is CLS what is the LSAT well the LSAT is obviously a power test the questions are really hard I couldn't answer any of them that I nearly tore my hair out when I tried to do the practice test in the old sannyasa but it's also speeded right when you take the outset you have three hours to do it you can't take all the time you want there's a finite amount of time that you're given to answer all 101 questions and three hours is not long enough for most people who take the LSAT most people run out of time and they end up guessing on the balance of the test so on the LSAT we have chosen to evaluate candidates for law school on the basis of power test given with a speed constraint so this is my question why is the LSAT speeded spring that to me you've got the test is divided up into five parts each part has 35 questions I'm sorry each big part is get you up 35 minutes to complete each complete each of five multiple-choice sections why is there a 35 minute minute where did that come from I can understand why we have time constraints on check if there was no chess clock the games would last two weeks you couldn't watch a chess match it would be you need to have to quit your job we got to be incarcerated to be a chess fan it will go on for so long totally make sense to me that you'd want to impose a time constraint on a power game when it comes to a spectator sport right the LSAT is not a spectator sport why on earth is it speeded right I don't get it what is it about the legal profession that suggests that a speedy constraint on a power test is a good idea so let's go back to Sheila koha car are neurotic tortoise and I want to imagine her taking the LSAT so I have very bad a PowerPoint let's see if this works there we go okay so here we have a hare and a tortoise Sheila is on the right neurotic tortoise and hare is some super quick bright character so hair start with her hair attempts all 101 questions he answers 82 correctly so he has an accuracy rate of 80 1.7 he doesn't get any of them because he manages to finish in time because he's super speedy his raw score is 82 his LSAT is one sixty five 65 he finishes in the 94th percentile he gets into pen he gets a cushy job at Skadden Arps and no one sees him again until he makes partner at the age of 50 okay neurotic tortoise Sheela comes along Sheela answers 80 questions correctly because she's terrified of making a mistake and she's getting really stressed out I'm sorry she answers a 1080 she answers 78 correctly she is an accuracy rate of ninety seven point five percent but she ends up guessing on the last twenty one because she runs out of time and she gets a couple of those right her raw scores also 82 her LSAT score is 165 she's in the 94th percentile she too gets into pen and ends up at at Skadden Arps now for the from the standpoint of the LSAT the neurotic tortoise and the hare are absolutely identical they had the same score right but that's nonsense these were these two candidates could not be more different I mean think about for example if we unseated this particular test it's quite possible that Sheila are neurotic tortoise could answer I don't know maybe if you gave her enough time she would have answered 98 questions correctly maybe 97 the hare on the other hand he's already run through all the questions he cannot do he's hit his ceiling at 82 he'd done right he's not smart enough to handle the remaining 19 questions but Sheila I don't know give her an extra couple of hours maybe she turns out to be some kind of LSAT genius so the effect of speeding up our test is not just to make an apple and an orange both look like apples it's also to potentially obscure our knowledge our understanding of the neurotic tortoise who could be really really really really good at the particular tasks of being a lawyer let me go further the same bias exists in law school so in law school you have your grade consists of three things in general you're an in-class test a take-home test and paper write a paper in class tests are highly speeded power tests take-home tests are moderately speeded power tests exam research papers are pure power so here we have hare tortoise an ordinary Joe and you can see that hair is going to do really good on the in-class test it's going to be okay on the take-home and he's going to be terrible at I think at the SI tortoise is exactly the opposite Sheela struggles in in class she gets freaked out she's really neurotic take home she's okay and the paper of course she nails it poor she nails it give her a maximum amount of time to do a really good job in a situation where we rank rate all of those three things equally in class take on paper hare and tortoise have exactly the same GPA but if we're at a law school that chooses to rate in class greater 50% of the greatest in class hair does better than tortoise and if you flip it though and you have a situation where I'm sorry and if you're in a school which takes in class tests really seriously it makes it eighty percent of the grade hair looks like a winner and total looks like a complete loser now I come back to my question who is actually the better lawyer based on those two proxies who's a better candidate for the legal profession well I don't know based on those two things their LSAT score or their GPA because I need to know to what extent you have rated power versus speed in your evaluation right and until we have a conversation about what we want from the legal profession and what we want a lawyer to look like we don't know how to accurately rate those three things when it comes to evaluating legal students in their in their when they're in law school now why haven't we had this particular conversation why haven't why hasn't the legal profession sat down and said are we more interested in tortoises or hares right should we be speeding or unspeaking the way that we evaluate students I don't know the answer to that question it puzzles me for for I've been trying to get some of the legal profession to tell me why they speed the LSAT and why they wait in class exams as much as they do because everything that I understand about the legal profession tells me that I think I would want an erotic tortoise when it comes to doing my legal work the best answer I can I have managed to come up with in talking to people about why we why they else that is speeded is that's the way they've always done it at some point in the 19th century some guy decided well let's give them three hours have we stuck with that ever since which suggests to me that this is it may be all very well and good for us to have an analytics conference here in Philadelphia but I would be happier if a week before we had an analytics conference we at a conference where we sat down and had a conversation about the kind of analytics that were interested [Applause] we are so low tech that we're going to record right here from tripod so Malcolm great to have you here at the Wharton people analytics conference thank you I notice you have less hair than last time you came with that deliberate I have well I was talking about relative to you or relative to my my previous belt is my previous hair state yeah okay I'll have some data questions about that later but you just told us that we should be thinking about slowing down standardized tests like the LSAT or the MCAT or the GRE or any of it or if we don't we should at least explain why we're not slowing them down okay give me a give me a reason why you want to speed up our tests before speeding up our tests I have some reasons you want to hear them yeah okay disclaimer first though one of the things I love most about your work is how you push all of us to question our own assumptions and so I feel that the most respectful way to have a conversation with you is to challenge our assumptions okay are we good with that yes okay so with that in mind first of all the reason that I want the standardized test to be short whether I'm looking at students or job applicants is because we already have long power tests they're called grades and so if a student gets to spend four years accumulating those grades why in the world now do we need to get another tortoise contest well there's an additional reason so we have L sets and SATs and GREs are not knowledge tests classically speaking they're cognitive evaluations right so we're trying to get at something that is different from grades even if we unseat them so in that sense I guess what I'm interested in as a pure cognitive evaluation why are we biasing in favor of the hair I'm interested in the kind of measuring the kind of pure cognitive strength that that that shows its face under pure power conditions okay so we for pure power professions cool I like that however in psychology we have this distinction between typical and maximum performance typical performance is basically how well do you do on your usual day maximum is how good are you at your best and those two measures tend to correlate pretty highly so that the prior your maximum performance also the higher your typical performance which leads me to think that although there might be some really really brilliant tortoises and also some less brilliant hares that most of the time the two go hand-in-hand and that processing speed is also a proxy for you know the complexity of information you can handle so if that's the case do we really need to use the two apart for the rare people who fall in one of the off diagonals I was surprised when I saw you as I pointed out in my talk you know 10 times more than this and I do but I was surprised when I was reading up on this recently the extent to which psychometricians insists that speed and power are separate quantities now it is additionally the case that you don't when you move from a - in is when you move from a speeded power test to a pure power test you don't necessarily change the shape of the curve but you do jumble the the rankings of people on the curve and is that jumbling that interests me so maybe the jumbling if you look at for example on the chess if you compare classic versus splits chess rankings top 20 players in the world there isn't a huge amount of difference but there are these cases so in of those 20 players there's maybe four who have dramatically different classic rankings as blitz rankings Darrin Lane being one of them Wesley so being another and then there's weird cases like Magnus Carlsen the greatest player in the world who is marginally the best at classic chess and so far in a way the more you speed it up the more he becomes dominant so there's two things going on here one is that when I arbitrarily add in a speed component I start to lose at the margins I may have a general sense of what's going on but I'm missing people and I'm obsessed with missing people and the second thing is here is my understanding of what makes someone good so you learn a lot about Magnus Carlsen when you look at his performance under different speeding conditions you understand what is brilliant about him as a chess player is that he doesn't make mistakes even when you when the game is going like that right that's a really interesting observation about it yeah it is and so I can see the rationale for that I guess it just seems like in most complex jobs it's not quite as independent right so like in your old running days the fastest sprinter is never going to be the best marathon or vice versa but I think in general the more expert somebody is is a job I read something once about 10,000 hours which we'll talk about too and you know the more the more expert you are right the less you have to rely on sort of slow system to thinking the more that you're fast intuitive visceral heuristics are accurate which I also read about in another book that you might have blinked at once or twice and with all that in mind shouldn't we just assume most of the time that the experts are going to be there but but come on I chose the legal profession for a reason you did and I chose it for a reason because I think this is one example where that that relationship between speed and performance starts to break down I absolutely do not want a speed reading lawyer so the the kind of person who when I was and I played that Sheila kohat are a clip for reason as well that mindset up there is a woman who is incapable of being fast so she has a she has a personality constraint on speeding up she can't sleep at night she worries she can only be dogged and thorough in her chosen profession of being a investigative reporter that is absolutely central you cannot be a speed-reading investigative reporter right then work but if you're someone who doesn't want to go back to fifth time you're never going to get that kind of story so there are specific moments in other words when we have to understand that the cognitive profile of the profession is different my father was the mathematician there's no upside for him being fast he might publish a great mathematician my publish you know ten great papers in their lifetime why does it matter whether they why would we want to reward a mathematician who wrote his paper in six months as opposed to two years I think you answered your own question in an article you wrote a while ago which which actually said that the more output you produce also the better your shot at stumbling on to greatness you actually said that the more bad ideas you have yeah the better you will be and so don't we actually want a reward speed to get the quality on a lawyer not the lawyer right so now you're you're you you're I know the game you're playing and I I'm trying to play a speed game come on but you are you are you're not listening to me Adam I'm being very specific about lawyers the lawyer cannot Deloitte we do not want the high output loss and error lawyer I'm sorry what do you think just go on I wasn't listening for real now I'm listening okay okay so why do we want lawyers can you imagine the lawyer who came to you instead so here's the contract take a look if it turns out it's not the right thing we can just go back and do another version later are you kidding me that's a that's a disaster I'm reminded you know that you're that story in the financial crisis where someone puts the comma in the wrong place and they end up playing you know I forgot $20 a share for laymen and not $2 a share who was the person who read that document 2:00 in the morning the hair it wasn't Sheila kohat car she would be the one who'd waited five times why because should be petrified that she put the comma in the wrong place there are specific you know or the person who is in any kind of high-stakes job where the penalty for error is high you can't afford to have hairs so it's what I'm objecting to is the very thing that you're talking about which is you're trying to make a general set of principles about selection and I think you can't make a general I think you have to be much more specific in saying not only by the way there are parts of the law where I might want to hair so what I want the legal profession to say for this kind of law and this kind of law in this kind of law I want the neurotic tortoise for this kind of law I want the hare and I want them to say okay so let's create a safe space for the neurotic tortoise as opposed to penalizing them at the point of entry I like that a lot for the tortoise where I wonder about it is what are the consequences for the people around the tortoise and for the organization and you you may not care yeah I think that's reasonable right to care more about making sure that people don't get missed then about like whether a law firm does well but I think that their insistence dynamics terms we can think about equi finality right multiple paths to the same end and being neurotic is one path to that right like I can relate to that I remember like being afraid when I was studying for tests like I would do so badly that I would not only failed but my professors would take away points on my previous exams because no way I could have earned what I'd gotten before and that anxiety was really motivating for those of us who are defensive pessimists yeah but it's only one route to that thoroughness right so you could be really emotionally stable and also incredibly conscientious and you could be fast and then you could have that motivation to wanna double check and triple check and quadruple check and so I guess I wonder do we need could it just be a really conscientious hare who's fast to execute and then is also careful on this thing this thing saying can't we have all basketball players who resemble Michael Jordan you know you you know we can't argue for the perfect form because the perfect form happens once in a generation I think that if you're going to if you want highly conscientious highly neurotic people they're going to be tortoises by and large remember as well that I'm not saying about removing all speed constraints so on the breakdown of take-home tests exam in class exam take them test SAT in class exam I'm not saying throw out the in class exam I'm just saying be honest and open about why you're waiting the test the way you are and be clear about who you dis is disadvantaging in each instant I could I could get on board with that I then wonder so if we tie this into deliberate practice and I do want to give you a chance to to set the record straight on it widely misunderstood set of ideas but as I think about that I think okay you know you take your bill gates like argument for example one of his real advantages was that he was able to accumulate that practice faster or earlier and in turn you know the hope is the people who do that then also become extremely thorough right and I'm wondering how you think about that so the people who do rack up more practice and therefore more expertise are probably going to be mostly speed people and what do you do to even the score there after selection and admissions yeah well so computer programming I know it just a little tiny bit about that and the little tiny bit I know suggests to me that people who are the 90th percentile computer programmer is not just better as say writes better code but she is faster and makes fewer errors so there's a case where I don't think there's a neurotic tortoise component I think it appears to be that the hare is what you want or at least the if you're good you're going to be a hair you can be a bad hair but if you're good you're going to be being being fast goes hand in hand with being accurate and creative and so that might be a different effect the more it's funny because I just started digging into this thing and then I'm impressed more and more with how different disciplines have how different their kind of ideal profiles are and I mean this is part of a kind of larger argument for us being much more accepting of difference when it comes to selection for a certain source of of domains one of the things I worry about there is as we okay so let's say we could profile every job which there are people in the room who have been working on this very problem then we end up in a situation where we're really good at selecting for individual job performance and we're terrible at selecting for the qualities that would make for a high-performing culture or a team with diverse skills and backgrounds what extra layers do you want to add invent put the second thing for high-performing teams and yeah as we think about not just optimizing my own individual contributions but also you know what's what's the what's the sum of the parts well why is it uh why isn't it good enough to say that if I have a condition that allow such an environment that allows individuals to maximize their potential that will ultimately be for the best of the but take this could take an example I made up you know University faculty such as you belong to as a writer my principal observation about why other writers fail is that they are in too much of a hurry I don't think you can write a good book in two years you may disagree you have done that I think but you're an anomaly mostly most of us can't write books that quickly and we need to be a little bit more tortoise II and a little less here'sh the problem is that the world wants you to be a hare your publisher says I want it now you're under pressure to do this XY and Z you have a one year sabbatical where you're trying to cram it and finish you've got teaching load instead of zero zero zero zero but in fact even something is simple one thing that almost all of the professional writers I know so not people who have a day job like yourself do the very best of them do is that they write drafts and then they put it put the book in a drawer for six months then they come back to it they build in they turn themselves into tortoises forced themselves to slow down and go back now does that that in a sense harms assistant and that the amount of output is lowered but I don't think the problem with writing in America right now is a failure of output I think it's a failure of quality right so there's a case where I think the overall system could use maybe a little lower level of production and some higher production values and I think that having individual writers who write better books makes us all better now that doesn't really answer your question because you're talking I think about much more coherent organizations but it's in New Yorker a better organization if writers slow down and write fewer articles in the but those articles are very memorable I think if you did a systematic analysis of the financial health of the New Yorker you would learn that if it's the New Yorker is a hit driven enterprise that probably eight articles your account for 90 percent of people's interest in the in the product and so that to the extent you can encourage people to write fewer hits you're better off yeah I think the academic sheet on this is by the time you write a book you've already been working on the topic for five or ten years and so a lot of the Tor network happened upfront right yes it's a research part of it yeah which doesn't guarantee that the book --look be sensible or even understandable but I do I do wonder then so give us a chance to update the thinking on ok so you know 10,000 hours probably it might be your most widely discussed bit of writing I think arguably the most misunderstood as well what would you actually like us to conclude about expertise and deliberate practice well as I have explained many times and no one is interested in listing I was only I was interested in that because I was interested in the idea that if it takes you a long time to master something longer than you would imagine then that must mean that you need a lot of help a and B that you must be in a situation that's patient that's what interested me was the context that if if we're all Naturals then the context in which we perform what we do is irrelevant right if you're born being able to to to be a scratch golfer then why do we need you know to spend money developing young golfers it's all there right but once you understand actually not only does it take a long time to get good even if you're really incredibly talented to begin with but it takes place at incredibly long time then you understand oh not even Roger Federer could be a great tennis player without a coach without a place to go and play tennis without parents who drive them there without people who remember Roger Federer for years and years news was known for having a terrible temper and it's when you go back so we get a little roger federer riff here but the great at the beginning of his career he was thought to be someone who would never amount to true greatness because he didn't have the right person requisite personality he would have these meltdowns they would throw his racket he would storm off the chord they were like ah another one of these people is going to squander his talent but that's just because we were observing him in the middle of his necessary period of Tantus tennis apprenticeship and he once he had completed that he turned into the tennis pro we know someone who is whose control of his emotions is perhaps as good as anyone who's played the game right so even Roger Federer required a patient ecosystem to become truly great that's all I was trying to get at i 10,000 hours I mean at the number that has been thrown around by a number of people who were looking at musicians which I just thought was intriguing but it was never meant to be a kind of definitive and nor was it meant to be a statement that talent didn't matter it was a talent look how requires a lot of time to be you know I can spend 10,000 hours in any number of things and I will never be any in more than mediocre yeah and I guess in turn the role that luck an opportunity played in making that possible yeah is a huge part of the story so we have audience questions which I would love to throw some of them your way one of them is your your harsh on standardized test because of what they're missing and maybe arbitrary or artificial performance standards that are missing key skills that might be relevant for a job or personality traits for that matter what else is being rewarded that we shouldn't be measuring like can you afford to take an LSAT prep class and how do we get that out of a system well a really good bit of good system would be to dump standardized tests entirely huh you know there the contribution for all the fuss at this country when I say this country a lot of other countries don't have standardized tests I mean it's not it is not a given that human beings need to conduct their entry to elite institutions this way I don't understand why people are so obsessed with them given the fact that there actual predictive usefulness is small I mean grades are away better used SAT scores give you a little bit of bump as grades but there are actually all kinds of other tests that do a better job of this than the SAT I mean the mythology around this test it's almost it's not a it's some of that there's a kind of fetish for these things in American society which has an outsider I find incredibly puzzling but sure yeah you're not even a system where people are hiring coaches at enormous costs in order to improve their score on a test it doesn't really matter all that much in the end I mean it we are now at a level of absurdity with this particular game why don't just call a halt to it I'm sick of nother ways of trying to fix the system I think it's time just to dump the system to say why this is just a it's a quate if we're starting from the question we are asked ourselves if we were starting the American educational system from scratch tomorrow would we have the SAT and the answer is of course we would so why are we persisting in this charade so I've a thesis or at least a hypothesis which is I think we're in the charade because it creates an illusion of certainty and it allows those of us who make selection decisions to believe that there are more deserving and less deserving or more qualified and less qualified candidates which i think is largely an illusion but if we throw the system out all together we're still going to be looking for sources of certainty how do you tackle the more fundamental problem of people having this sit and admit that what they do everyday in sorting and selecting and betting on applicants is basically throwing darts and that you know there's there's basically a lottery running there whether they let us run the lottery well our mutual friend Barry Schwartz suggested we should do I totally agree with him I mean I think that's this morning the smartest things have a cut off say that five minutes I might you know I'm ten I'm interested in order to apply for pen you must be in the top 10% of your class and you must do one interesting thing on the side and then then we're going to throw all those names in the Hat and pull them out I mean I can tell you with 100% certainty the freshman class pan under the circumstances would be infinitely more interesting than it is now I'm glad we have enough no freshmen in the room I guess you know what the surface is for me is a lot of things the one that I want to follow up on though is there's still going to be some arbitrariness in where you draw the cutoff and you know you're never going to end up at a complete lottery with what you called elite institutions if you were going to design your own selection system what would you put in that you think is less arbitrary than the alternatives on the table today I would I thought about this actually that to me to this actually is a good deal of relevance for recruiting new organizations as well the conversation is too one-sided so if you read the literature on what makes for a meaningful college experience almost all about literature stresses the the role of the the way the student interacts with their institution that is when I show up on campus on day one how do I behave do I seek out the most interesting professors to me and take their classes do I join if I'm interested in music do I join the band do I go out for cross-country do I do I willingly throw myself into the experience or do I smoke dope in my room right that the variable is you not the institution and we have somehow lost that fact so if I'm an institution what I'm really interested in she ought to be in is what does the individual want from me so I would say when you write a instead of writing an essay that talks about what happened in your own life and the institution says oh I like that and then flip it the essay should be what do I want from my college education who would i if your MIT the cream of the cream right or your Harvard or whatever you should say someone who's applying to your school should be able to say with a certain degree of specificity which professor they would like to study with and why and if you if you're not at that stage intellectual development then don't schools because that's what schools are for right go to a place where if you want and if you want to join a frat and party you should say my principal interest is joining a frat and partying and as a result I would like to go to Duke but I mean the point is you can that institutions ought to have clear personalities and not to recruit those people who are interested in that kind of thing right if I'm Carnegie Mellon and I have maybe the greatest robotics faculty in the world what I want to know is if you're applying at Carnegie Mellon if you're into robotics why what would you do if you came here comment did you come last year and sit on a robotics class and what did you think right how do you see yourself you know have you read stuff by any of the professors at Carnegie Mellon and if so what did you think of the stuff you read that relevant what are you going to do when you get here is the question I'm interested in do you want companies to do the same I do I think they do do that too so I'm excited to do a better job of colleges as doing that if this is one of many areas where I think the public where where education can learn from the private sector but I feel like there are situations where maybe we ask that question in a very big way well how do you see yourself in five years which is too far in the future I think generally but um but I think yeah moat that we should read we could do more of that we should if I'm an employer and I really want to take your argument seriously and I want to make sure that I'm not privileges to beat over power where do I start an employer so that's interesting I guess I would like to see I am the millionth person to say this that the ultimate version of speed versus value is not a perfect fit here but the apprenticeships or try out trial periods are a version of power over speed right I'm removing the time constrain making my evaluation and I'm saying if you would like to come why don't you just come and why don't we all just see how things go over a period of weeks or months I always my brother who is an elementary school principal in Canada like every and all effective principals try to do this which is when they hire a teacher they try to get a teacher on a contract before they offer them tenure because they think they think that they understand the best way to about whether a teacher is a good teacher is to actually watch them teach for a while the longer the better my brother never offers a tenure to anyone that he has an observed first and luckily he's in a system it allows them to do that right so I mean to accept that we can that should be it shouldn't be weird for you to go and spend six months in an institution at the end of the day you an institution to say it's not it's not working that shouldn't be a black mark on your resume at all that should just be that should actually be a positive mark it says that you should you're brave enough to experiment to go out in the world and try stuff and that's actually a spirit we like here so I like the idea of stretching out the time horizon for work samples it does raise a question though of we have good evidence that people who job hop more often tend to be less loyal that's committed less likely to be good citizens and so if that's now a sign that you're willing to experiment but this is a job I think this is a different category than job hopping in fact the end goal of this kind of these kinds of experiments is I would think to to end job hunting that's a if I can do a better job of fitting you with the organization you're less likely to lead me not more so it's not a sign of someone's underlying happy feet if they do three experiments in a row it's a sign of their desire to be the opposite of someone with happy feet to try to find a new home you know I can see that I like it I also there's an interesting question here about about engineers and you've actually written recently about how engineers think you I did that piece because you told me you turned me on to this brilli an article from Danny from Danny Goya joy joy joy joy and realize that was my fault your fault by the way parenthetically we appear someone who's a writer the best best thing you can do is just have lunch for that I'm like once every couple months penny just and take a pen paper just write down everything it says that's what I do I've done it on so many occasions fantastic it record yeah seriously I've never had that experience but I will I will say that you uncovered some really interesting insights about how engineers process information and we're trying to make organizations more evidence-based we're trying to make them more data-driven that's what engineers do for a living so anything we can learn from how engineers think as we think about sort of making HR and the world of people more data-driven I don't know I mean do you one of the the most remarkable one of the most salient facts about thinking and learning about the culture of engineers and I would say parenthetically I grew up in that culture since my father simply civil engineer is that if you're not an engineer I'm not sure you want to spend a lot of time with engineers my father accepted their you know it's a very very particular culture I don't know whether you want to make the world resemble engineering culture I sort of think what you want to do in that case is to find better ways for these two very different cultures to speak to each other I mean I think we absolutely need engineers to think like engineers but we absolutely don't want everyone to think that way and nor do we want the non engineers to shut down the engineers we want to have both healthily at the table I worry a little bit about you know about uh about the hiring process becoming I'm more worried about it becoming too dependent on analytics than I am about it not being dependent on analytics enough I wish there was a little more humility about what can and can't be measured you know the Union I follow this most mostly in sports and you can't follow the analytics revolution and say basketball and not and not simultaneously be thrilled at what we can know and deeply humble about what we absolutely can't know I mean there was a and I've forgotten his name there was at the center for there were two European players playing in the Denver Nuggets earlier this year neither we're playing very well and the consensus was that maybe they were going to be you they were going to one or both them was going to wash out Denver traded one of them either nourish or the other guy to Portland it is now the case that both are playing unbelievably well all that was necessary in one case was for the other guy to leave town and in the second case was for the other guy just to go to a different team to Portland as opposed to Denver and all of a sudden now the talking about him is one of the best centers in the league if you can find any analytics that helped you predict that outcome be my guest it was an intangible it was they weren't happy together and apart they're fantastic and that when I look at the example of the basketball fans in this room will know nuran chin who's the other guy yeah the that that just tells you that there's an awful lot that we can't easily understand about ELA performance I completely agree with that assessment I think we could probably reduce our false positive and false negative rates yeah more than we currently do what I like though about bringing engineering discipline to the table is you're not a big fan of the myers-briggs as one example engineers look at that and say who made this up and why can't we do better in half a century of actual social science and it's much easier than to update broken systems with you know with better ideas do you see more of that to come in the world of analytics and how do we do that well maintaining humility Wow do I see more of that well I'm not in you know my if you have a historical perspective on the use of analytics you tend to be somewhat pessimistic about their intelligent use because it strikes me that there's just so much overwhelming laziness in the way we use metrics to to analyze success some of that laziness is being confronted with the current generation but but I'm you know is that just a case where in every new generation makes up its new met new metric and then goes to sleep and accepts that unquestionably until the next generation comes I mean I don't know it's like I know numbers I'm not terribly impressed with it with the ability of human beings to be consistently self-critical about the value of the analytics they use to make sense of the world I mean we are still clinging to growth to GDP as a useful measure of how well we're doing as a country if you poke into that that is bananas I mean it's about as lame a measure that you could possibly use to tell how well we're doing right so like but yet it's the first thing that's that's quoted when it comes to when we have a when we use an analytic to assess the the how good our performance of the country is it's crazy so I mean I'm not you know either forgive me if I'm not into if I'm not a optimistic about how well this will go in the future I think it's an important sobering note and it also levels the playing field a little bit so that it's not just lawyers and dukey's who feel a little bit hurt by commentary everybody has a reason to question themselves with that aside Malcolm it's been a real treat to have you here there are tons of people in this room myself included who got into this field in large part because of your work and we're all really grateful that you continue to do it so we're going to head to the cocktail reception now thank you thank you you

Channel: Wharton School

Views: 45,633

Rating: 4.7962465 out of 5

Keywords: Wharton, The Wharton School, adam grant, malcolm gladwell, conversation

Id: 88t_Tfg3Ecw

Length: 55min 9sec (3309 seconds)

Published: Thu May 18 2017