Using behavioural science to improve the way we work - Laszlo Bock and Katy Milkman

[Music] good afternoon everyone I'm Robbie tillyard and I lead behavioural insights teams UK practice focused on making work better getting people into good work and improving the labour market in their classic work on subjective well-being Kahneman Krueger and colleagues found we are happiest when we are engaged in intimate relations or spending time with friends this seems a simple if exhausting prescription to follow on the other hand we are least happy spending time at work and commuting and would probably rather be alone than spend time with our boss we are in other words living in countries filled with David Brent's we think the workplace needs to do better and given the productivity challenge faced by many developed countries improvements in the workplace are a vital lever to increase economic growth a bit we've worked with partners applying behavioral insights to increase the adoption of flexible working improve levels of inclusion help people return to work after illness lower burnout for some of the most stressful occupations and help the most vulnerable find work we're really pleased to be hosting this session with the support of the UK's Economic and Social Research Council the ESRC is undergoing a major research programme investigating the impact of management practices and other aspects on productivity and they're very interested in hearing about how they can support behavioural research so see Paul Nightingale who can maybe wave his hand if you'd like any money much of this work has been I think in Silicon Valley is enough amount of money for you lászló much of this has been inspired by the work of our next two speakers Katie milkman is a professor at the Wharton School at the University of Pennsylvania her research in workplaces helps us think about how to keep our commitments improved diversity and inclusion change our habits and create behavior change at scale if you like what you hear today we can also endorse her podcast tautology meanwhile Laszlo Bock is the former senior vice president for people operations at Google the author of a book anyone interested in organizational behavior should have on their bookshelf work rules and now the CEO of humu a company making work better through science machine learning and a little bit of love today they are going to discuss lessons from science and on-the-ground experiences using behavioral insights to make the workplace more engaged diverse inclusive and productive after a conversation they'll also take some questions from the audience could you please give them a warm welcome so this is an incredibly fun opportunity for me I should say very selfishly I'm so pumped that I get to do this with Laszlo I've known him for roughly a decade and have the honor of being a humu advisor but we actually don't get to talk that often and haven't caught up in a few months on the company so really really excited that publicly I get to hear all the latest so I actually just wanted to start Laszlo by asking you the obvious question on everyone's mind which is can you tell us a little bit about you know both why you started who moon exactly what you've been up to for the last two years since leaving Google where you were leading teams to be better and and individuals to be happier Thanks so one of the insights I had at Google along with my co-founders who happen to be from Google as well although the companies now we're only about 10 or 15 percent former Google people was that once you get big and big is defined as more than a few hundred people it's really difficult to drive any kind of change in organizations you can give people papers to read sometimes they do sometimes they don't you can set them to training and often that has an interim booster effect but no long-term effect you can hire consultants to tell you what to do and that's a very expensive thing to do or you have leaders say here's the direction we're going in and it's really hard to activate everyone in organization and align them and it's part of the reason why there's sort of this you know engagement hasn't changed and engagement is kind of a BS construct but for lack of a better one it's part of the reason why I come any performance roots to mean and what somebody pointed out was one of the things we've been doing while I was at Google was rather than trying to drive like any traditional change mechanism we were using choice architecture and nudges tiny targeted interventions to drive massive behavioral change so the Epiphany we had was that most places are not like Google most places bad days at work don't mean they're out of free lobster they actually you know they're actually much worse than that and so the idea was can we take some of this insight about what drives human behavior and find a way to apply it generally across organizations across job types across cultures and use the power of nudges and choice architecture to drive lasting meaningful behavioural changing companies so that's what we do in here mo we identify what behaviors people should change to have the biggest impact on productivity retention eudaimonic happiness innovation and inclusion and then we use nudges to change those behaviors and it turns out it works really really well far better than we had hoped and not in all cases and not always better than we hoped but but quite well so far so we've been really fortunate but one of the things KT started up along with her colleague Angela Duckworth at about the time I was leaving Google was an initiative at Wharton called the behavioural change for good which was all about actually a similar idea about how do you actually drive mass scale behavioural change and while we focus on companies you have a much broader focus so I think it'd be great in the interest of conversation for field also here like what is that and how's it going and and what's cool and new about that so you're actually witnessing our ketchup cuz I'm an adviser to whom ooh and Laszlo as an advisor to our initiative at Wharton so there's like a little bit of a hidden agenda and we only advise each other on stage at conference exactly yeah it's a twice a year commitment no stage exactly it's always a nice cities in the world so the behavior change for a good initiative we're more focused on the basic science and the application but we're pretty applied and what we realized is that one of the missing pieces of science we felt in behavioral science and particularly sister with Angela Duckworth was figuring out how to create change that sticks so we know a whole lot about how to change choice architecture how to default people into a better decision and when that's a decision you can default them into forever like retirement savings oh one switch and now you're doing it that's great but unfortunately most behaviors aren't that frictionless and a single switch doesn't change them in an enduring way so we really wanted to advance that science so we founded an organization that's focused on doing that and the approach we've been taking is to do what we call mega trials so we partner with a really big company that has data on actual decisions made by people and is trying to drive behavior change in a positive way and we run massive experi with many different hypotheses tested at once all designed by different brilliant behavioral scientists who are on our scientific team and by doing that at scale we can really advance science much faster instead of one study being run every five years roughly on this topic we see you know twenty run in parallel and a two-year time period and we can make apples-to-apples comparisons about what works best so that that's what I'm up to not that different actually than what you're up to how do you we're now off script that was it you're not supposed to ask me any wait how do you how do you get a big company to enlist for that like how do you get them beside well if anyone here no actually you know we've been incredibly lucky in that because we have funding from generous donors and foundations and we're bringing scientists of course they they do this pro bono yeah we've had a lot of luck when we sort of go to organizations and say hey do you want to sign up for us to give you a bunch of ideas all you have to do is share data and let us play we've gotten a lot of yeses that doesn't mean that there aren't you know hassles but but we've been so our first partner was 24 hour fitness large gym chain I am not allowed to name our next partner quite yet but probably in a couple weeks I will big medication adherence studies gonna be next and we've done some work with large school systems and who is that next partner I'm a stone well I'm supposed to ask you question sorry and this is actually very helpful for us because one of the things that we're working on is generating insights that can be scaled outside of our you know little hub at a university and taken into organization so I'm curious about some of the most exciting things you've seen come out of academia and be implemented and practice and be useful to organizations like whom oh so we when we started to move so a lot of I want to be careful cuz there's a lot of practitioners in the room it's easy when you work for a company actually you're at this conference so I'm not gonna offend anybody if it were a different setting I'd probably offend people it's easy if you're in a company to not have academic quality rigor behind the work you do you you know read an article read Dan Pink's book and go like oh here's the solution and Dan's a friend of mine he's wonderful but you know reading a book and then saying okay now we're gonna be Purpose Driven everybody read the book mission accomplished that that's like how most companies approach science in their organizations and similarly if you're a startup I've now seen this and we looked at startups a lot in my old job it's easy to just kind of lie about stuff basically and again say here was my experience I'm gonna read two papers I'm gonna say you know we unbiased recruiting because you know we train on your employees and therefore and we screen out the bias and that actually tends not to work very well Amazon was in the press recently because they built an algorithm to better predict who they should hire and it's screened out all the women because they trained on their existing population which has very few women and so the way they fixed it was they said let's remove all the words that are tied to gender but if you're a machine learning person you actually know that won't fix the algorithm because the algorithm says here's the inputs here's the outputs let's find all the ways to do that and it's okay so if you remove the obviously gender driven words it will find other signals and continue to exclude women and when I talked to him his ons head of HR she said oh don't worry we only had that for two years you know they hire 50,000 people a year right and there's a lot of turnover like that's a lot of qualified women who are not hired so we started from academia so we've done all this work at who at Google but then we started from scratch and said what does the literature say about what things have a causal what things that are controllable by individuals have a causal impact on the outcomes organizations care about and on eudaimonic happiness and then we started building models and then we piloted on 20,000 people using Mechanical Turk and only then did we start working with companies and initially it was trial and test and pilot and now two and a half years in you know we can say with statistical certainty with academic quality rigor that you know we can drive a 2 to 13 percent productivity improvement for companies the mediating factor that does that is increasing you know monic happiness and we can kind of talk through all that so it hasn't been a single thing it's instead been building a model of what are the most things most controllable things for and the people around them to do differently how do you have a better diagnostic around what those things are and then how do you use nudges delivered at the right time in the right combination of people and that's critical as well and we can talk about that to have the biggest change so it's uh it turns out it's a really hard problem but it's gone it's gone quite well we've been very lucky it's great you can just get rid of mic off yeah sorry I already wore than Laszlo this is my four week Koff now you're all experiencing it too and exposed to it yes I'm spewing it inhale everyone the exit is that way um so one so I want to dig in a little bit too so you said you've built it on academic research and you're trying to do academic live level rigor when you do your evaluations but I want to dig in a little to some of the insights that you found in academic research that you were most excited to take and apply and and ideally that you even have some proof worked where there are things particulars that you said this is really useful to what we're doing and you were able to take those and incorporate um and I'll let you think while I tell you what my follow-up question is going to be which is gonna be anything that I may forget their first question and so this is no I got yeah and anything also that was a discipline I guess one of the one of the coolest things well one thing that we found was Amy resents Keys a professor at Yale University she does work on or part of our work is on something called job crafting and she also looks at do people find meaning in work and when you combine her work with Adam grant your colleague Adam grant award ins work Adam grant demonstrated that when people find more purpose and work they're more productive and if you kind of look across the studies kind of on average and this isn't this part isn't super scientific but across the studies if you take a simple average you get about a 20 percent lift 21 percent lift and productivity if you can move people from sort of median in terms of finding meaning in their work to 90th percentile that's not in any particular study that's me like reading all of them and then kind of when writing the book squinting so that's that's the least scientific thing I'll say today but it's roughly correct I'll challenge you on that I bet you will say something even less because that sounds like it was like at least a rough estimate based on numbers grapes are the best fruit a scientific Ami's work on job amy initially found that only about a third of people find meaning in their work but that's true across professions so whether you're a doctor or lawyer and nurse clergyman whatever and so one of the things that can drive finding meaning and work is something that Amy developed called job crafting and at one extreme it's actually restructuring the job but it turns out there's positive effect just from taking the time to do the exercise to figure out what job components are satisfying and not and even if your job doesn't change at all her work shows that you're actually happier and you find more meaning in the work so we take all that part of our construct about what drives eudaimonic happiness is meaning of course and there's a nudge you can send to people that is simply telling people where where we first identify that meaning is the thing that's going to have the highest impact on productivity and so on the nudges basically take five minutes this morning and write about the parts of your job you like and the parts you don't like and then a couple weeks later think about those parts a couple weeks after that how are you spending your time but it's not sufficient to just nudge the individual because if I try to change by myself it's very difficult if they context an environment around me changes it's much easier so you also we also send out just a co-worker saying ask this person about their job or do a parallel exercise and they come together as a group we might nudge the manager to convene a group of people to talk about these things so it's the combination of taking this academic insight which is which is truth right in the Platonic sense she's discovered something true about the human experience and then finding a way to instantiate it in a corporate environment but also reinforce it by having a collection of people work together in digitally which seems to be a really important like everything in scale everything that you can deliver in a digital way so that you don't have to worry about training you know hordes of people to go in and deliver yeah exactly right so large companies we talked about this a little in the group at lunch large companies globally spend 371 billion dollars a year on training or at least in 2015-2016 that was the number and to give you a sense of scale global ad spend was 650 billion that same year so you know all the stuff that supports Google and Facebook and Instagram and whatsapp and you know all the ads you see on TV and the billboards by the side of the road all that spend is only twice as much as what all our companies spend on training and most of that is wasted money because it doesn't drive systemic lasting behavioral change so it can be repurposed and the way you do it is contextual customized complementary nudges target event IDI what is gonna be most important at that moment in time okay so not now I have to call you know you like is that scientific so because I said I don't know what the great statement because but you're saying all the money we're spending on training is wasted certainly as a right I'm a professor and believe in education it can't be the case that all of this money is being wasted and that we should only nudge and we shouldn't teach people how to fare that so that's why almost all of that money is wasted there's a one of our one of the companies we work with they spend over a hundred million dollars a year on training and they're had the people analytics did a bunch of analytics on them and came back and said the ROI is neutral and there see HRO said well if it's not causing any damage let's keep doing it and so that model is like a training as an entertainment form yeah exactly well it's like a PR move so this is why I asked about like how you got companies to get on board with this because I don't know if it's if it's a variation of what's it called the not the endowment effect when you give somebody something and they they've come overly attached to it nailed it yes a plus thank you from a college I was rejected from that feels really good know you've never told me that before I'm so ashamed look at some anyway I want tell you about the people we took part of the reason all this training stuff yeah exactly it's not a causal the part of what happens is somebody establishes a training program and somebody made the point in the session earlier today that like you know you have all these people who gentleman from Novartis made and I presume he was not speaking about his own company but about all of your companies but that a lot of training programs stem from a leader has a belief that this is important we should put everyone through it or I was at GE and you had Croton Ville and everybody had to go to Croton ville and it was this religious thing but most training efficacy is measured by did people like it and sometimes sometimes sometimes did behavior change but very rarely almost never do other people report that your behavior changed as a result of the training and almost never is there a business outcome that is measurable tied to the specific intervention and part of that is it's hard to control for all the variables so that's why I say most of it is wasted or might be most of it might be waste most of it most of it now no because if you want to change human behavior the problem with like take take manager courses send somebody to manager training you become a new manager you're gonna go to train or anytime a new employee is hired you said you know you have some onboarding process the proportion that content that they retain is very low and the proportion of the content that they retained that they then actually put into practice is even lower and the reason by the way I'm super sympathetic to this but I have to be okay I mean so so for example Sheryl Sandberg is a friend of mine we work together she left for Facebook our kids went to birthday parties we're not particularly close I don't oversell it these days but we were for a while like life is good I know axe to grind however just a small X lean in was I thought a well-written thoughtful book and had a lot of important things to say there were a lot of fair criticisms and ten years or however many years later still only about thirty percent of women are people in executive positions are women the work environment has not been transformed and part of the reason is that book spoke to women and it didn't tell me and people like me white male executives what to do differently and if you want to drive change you can't just tell this person you got to be different and not change the environment around them there was a gentleman again in the lunch session economist from the University Chicago his first thing was demo and I can't remember his last name what was his last name Jones Jones Damone Jones and he made the point that interventions are typically targeted at underrepresented individuals like in terms of employment skills training how to interview job fairs things like that they're targeted at people who are as much as anything those weren't his words exactly but he roughly said who are the victims of the system and why should the people who are the he didn't say victims but the people my language the victims of the system be charged with having to change he did say the intervention should be directed at the people who are running the system because they're the ones who have the problem not the people who are on the receiving end so coming back to training and nudges and what we do trainings fine it's a lot of fun I love playing with like a little beanbags on the tables and you know it's it's fun and people enjoy it and learning environments are often great and pleasant if you want to drive real material change you should do it scientifically which means you run controlled experiments which means you have to isolate the intervention and to go back to 24-hour Fitness most companies say we really care about our people look how much training dollars we spend and that's the equivalent of saying I'm in super good shape because I spend ten thousand dollars a month at the gym you actually need to measure the outcome and change behavior and that requires contextual transformation that's local and systemic and nudges are the best vehicle I've found in 15 years of trying to figure this out but I'm open to other solutions so again you're preaching to the choir below let me push you to give us some counter examples so you've told us we've told us about meaning and purpose and how well that worked and it translated from the academic literature I warned you I wanted to hear about a failure yes well and you know just as not all educational programs add value I think it's fair to say that not all nyjah's work I've tested plenty myself that we're unsuccessful along with many that work you know you want to weed out the good ones keep keep using those and then but what are there any things that you've found disappointing they were hopeful for that we can learn from so two quick examples one was and then I want to ask you another question because I'm gonna get into a topic that's near and dear to both of us one was at Google we did this work on something called project Aristotle which is around team effectiveness and it found the foundational factor that drove team performance at Google was psychological safety and you know that's not novel but within the Google context it was and so the intervention was initially we would run the diagnostic and then we would go to really just in case there's a couple people who don't know all Vimy i mentions real-time psychological safety we do a quick you know promo for what that means I can do I will defer to the professor but I might now I might botch it and then I look even worse okay all right psychological safety I want to you know I got to go to Harvard and learn about it from Amy Edmondson who came up with this idea when she was studying hospitals it's about the idea of creating a space where it's safe to to speak if there speak out of there errors where you feel comfortable talking about mistakes that are made rather than letting them propagate so I think her most famous work on it was in hospital teams she was sure that sort of the tougher the team manager the better the team would perform in terms of errors horrible catastrophic errors because you gotta you gotta give him tough love did you found just the opposite those teams where there was a more sort of I will say nuanced appreciation that we need to be open to hearing that you goofed and you gave them at the medicine to the wrong patient they caught errors really quickly those environments where you felt safe admitting your mistakes we're much more successful and then this sort of spawned a whole field of research thank you that was maybe long see this is what happened you have to go faster that was good I gave like the whole this is there will be a quiz my my husband's much better at explaining things than I am because he gives this short version I don't know the so so you we identify psychological safety as project project Aristotle and the initial intervention was we went to teams that were low in psychological safety and pointed out to them they were low on psychological safety which did not make the environment any more safe in fact we discovered in our pioneering way that it made it far less safe and we had to do a lot of work to kind of repair it so information by itself does not always actually lead to resolution on the whose side so our whole thing is right diagnostic identify themes at the team individual company levels nudges redo it six months later show all kinds of improvement we worked with a tech company with a famously awful culture so that does not narrow it down that much and so at t1 we run the diagnostic we start nudging and there's like thousands of nudges in an automated way being driven to everybody in combination it's all very clever six months later we rerun the diagnostic and on two of the three major themes the indices indices dropped and they were indices you wanted to yeah these are things where you know it was like I don't remember the exact ones but it was like you know one was around transparency one was around procedural just you wanted more of it you got yes yeah things got worse and this you know again because of their cultural dynamic the CEO was not gentle in expressing his disappointment nor was the Board of Directors and and so we dug into the data and because we're this was this is one of our first customers and we're like oh man this if this doesn't work this bodes poorly for our future and we actually found two things were going on one was people who were not in the initial sample came into it and they had been employees but they didn't believe the company carrier was sincere about changing and the people who came into the sample at t2 were more disaffected intended to be from underrepresented groups that experience the company in a worse way than average so that drove the scores down the second thing that happened that we didn't expect was people said they literally wrote this in the comments they said I lied last time and the reason they lied was because and this has since been I've I've actually this has been validated by like Nick Reid who's the CEO Vodafone Mike Corbitt who's the CEO of Citigroup whole vast narasimham who runs Novartis that's the most unscientific thing you've said so far okay I'm just I'm just keeping count thank you if it's verified by CEOs it doesn't make it true well here's the thing what they sit yeah I'm a CEO so don't trust me either what they what they said was that typically when you ask people what they think in a corporate environment they're not gonna tell you honestly so the classic example is is your pay fair you know it's like an IQ test you never say my parents a pay is fair right because your thing in the back of my your head I'm gonna say it's a little low so that maybe HR gets the message and I get a raise so what people in this company were actually doing was they decided they could trust the company more both of these populations because they conflated the behavioral change they saw locally as a result of these interventions with the company actually changing its conduct and behavior and so so I guess I'm dodging your question a little so what ended up happening was it's not really a bad example we we thought it was a disaster but it was actually people being honest and then at e3 another six months later everything was up again and we now see that about in ten percent of the companies we work with things get much worse than YCJA lee and there aren't common denominators across the companies where things get worse because sometimes there's companies where things are going really well and people just been reporting saying things are great and it takes a while for them to believe the company cares because the most important thing with when you're doing employee diagnostics is you actually have to show that there's change so it sounds like the lessons I'm taking away from that example actually are that one sample selection bias is a huge deal you can't have like you know some people you're looking at at time one and different people you're looking at time - because they're not comfortable so you have to have comparable populations to run experiments and do science properly and then the second issue is it's really nice to have behavioral outcome measures instead of attitudinal ones because people can't lie with their behavior correct or I haven't seen them do it yet yeah let me know when you figured that one it well but that brings me back to your work because psychological stated one of the other things we've seen is it's a precursor to seeing business benefit from diversity inclusion efforts and logically it makes sense because although we didn't realize this when I was early in days at Google if people don't feel safe you're gonna have the same outputs as if the team was homogeneous right the underrepresented people are not gonna take that chance by and large but if you have high psychological safety then you unlock all kinds of benefit but you shared with me a paper that you'd done around the efficacy of inclusion training and DNI training which had some really interesting results on this theme in in a corporate setting about what worked and what didn't yeah I felt a little bit like a hypocrite when I was saying hate some training must work so so this is an experiment I ran with a large and amazing team of people led by Edward Chang Adam grant Angela Duckworth were on the team Cade Massey all sorts of fabulous people and and this group of people came together to work with a really big company that was ready to try to roll out diversity training which we've all seen I don't even know of a company that we could name that doesn't have it yeah um and it has never been scientifically evaluated in a randomized controlled trial that has been published with behavioral outcome measures except in like you know a university lab but not at a company and we said we need we need to do this this is important and we wanted to make it work we all believe in education we work at an educational institution we said let's build the best darn training we can built on science and measure whether it changes attitudes and behavior so we ran a randomized control trial with 3,000 people we gave them a digital training it lasted 60 minutes built on all the best principles on you know how to persuade people there's we teach the lessons based in research we show them how to use them we you know had an acronym to remember all the tools you use all that fun stuff there's video there's audio it was really great people loved it it did change attitudes so that's some good news when we measured that right after you take in the stream it's a randomized controlled trial so one group gets the training one group doesn't the group that didn't actually got the training on psychological safety perfect which is funny and and then we measure their attitudes afterwards so the attitudes move although interestingly attitudes maybe not surprising the attitudes move most among certain populations and those populations in our group were men more than women international populations more than this was a study in we were publishing in the United States so we called international anyone who's not in the United States there's a large global workforce so those attitudes move most even though actually a baseline those groups attitudes were slightly less progressive towards women and minorities okay so that's some good news maybe you believe in training out we measured a bunch of behaviors and basically we got zilch the one thing that happened and this is actually a surprise is that a little bit of a lean in effect so I'm gonna you know give lean in its day in the in the Sun after we've knocked it down a little bit women who got the training we're actually more likely to seek out mentoring now we had built a mentoring program where we're measuring who you're nominating and say you'll take out for a coffee date and we actually we designed it as a program where people were invited to do it would take someone junior to them out what we found is if you got the training you actually invited a senior colleague out and and this happens when you're if you're a woman who got the training specifically so it seems to it actually minorities had the same reaction so it seemed to create some sort of anxiety about oh no things are not so great in my workplace let me look out for myself but it didn't change biased behaviors that had really no effect whatsoever on the groups who were most targeting as you mentioned the the groups that are the largest population men in these organizations and also not international employee so the very people whose attitudes changed most change their behaviors the least well what I what I thought was cool about and we should do questions whenever but the one of there were two really fascinating things one was that part of the study was an elf but you still publish that and there's all this pressure to you know only publish the cool things and not share notes so kudos to you and the team for the the integrity behind doing that and the second was the choice of B hate the tying into actual behavioral outcomes was very clever because the study was done and then three weeks later there was a nominate people and then three weeks later there was another nomination thing for awards and then 20 weeks later I can't remember there was something else but it was timed at such a point that all these things were gonna happen naturally afterwards that actually gave you a measure of behavioral change based on programs the company was already gonna run so in the interest of making what we're talking about practical I'd never seen that done before it never occurred to me I emailed the team and I was like we got a we got to now look for these moments because every organization has these natural moments where you can see evidence of a behavioral change and for bias it's are you nominating people are reaching out every company has awards programs you nominate people that's a natural moment to have an outcome variable to test your experimental design it was brilliant no thank you um well it was many other brilliant people the team we can blame on that for but anyway I'm glad you like the studies there are no results which do support your claim that education is a misspent we're we're miss spending on education certainly was the main message of our paper there may be other benefits right to this training like it may meet we meet we can't just UC Starbucks have an incident like it had our Sephora in the US there's a lot of conversation these organizations have to respond somehow and they don't want to just say like we're sorry you know they want to say they're doing something so it's an action you can take and there may be PR benefits but to claim that that solves the problem is really it's problematic so to turn it back on you what are the brilliant ideas you guys have come up with it who move it can take it a step further on inclusion and diversity efforts because training does not seem to be doing it have you guys come up with other solutions that you're excited about it yeah so and in our current product suite the fundamental construct is better diagnostic better insights nudges and we've actually built a basically a again an academic model with seven components about what drives inclusion and organizations and there you know some of the things you'd expect like belonging some of them relate to team conduct which is a little harder to measure and then the mechanism for changing behaviors again nudging so for example one of the ways you increase psychological safety is you want people to express voice more so we might nudge somebody going into a meeting speak up in the first 10 minutes because it's easier but we had also nudge another person in the meeting to say if they're a real leader say ask questions instead of make statements or you might get a nudge saying don't say anything at all in this meeting and we would nudge some colleagues some peers in that meeting to say something like you know if somebody is quiet ask them what they think out of curiosity are these phenotypes so are you like telling the women to speak out more because we have stereotypes that they might speak up less and the men to speak up less or are you giving everyone the same message and hoping that by doing so you'll hear more voices in general so it's tied to so again our construct has these seven elements and underneath their sub elements so it's tied to what that specific team does but in our algorithm we actually do kind of moderate it so that we're not telling you know the underrepresented groups to take on the tacks of doing all this kind of work because that would make the problem worse I mean kind of the blessing and curse is in most environments you have like white Manda's majority and as a result they're gonna get most of the nudges to take action even if we didn't do that I don't find it works better if you do yeah maybe we should open up we I could I could I know I so forever and I forget that you're even there um so I might have said some inappropriate things as a result of like calling him out on that wasn't scientific enough but we would love to take some questions from you guys if you have it and there's a couple roaming microphones so feel free did oh oh he also looked it up we didn't tell you why don't we start there and then the gentle on the front was bad choice architecture and so what are the sort of practical challenges you face when you're running randomized control trials in the workplace given that these things like contamination the spillovers and things like that yeah so so the math is hard is the biggest challenge so when we deploy nudges for example we always do a 20% hold back as part of it we look for homogeneous groups where we can compare we look for areas where there's productivity data available so it tends to be call center sales organizations operations things like that then we actually have to do a lot of scrubbing of that data because if you actually want to attribute causality in a rigorous way there's not only things like like companies typically look at productivity they don't often consider seasonality they certainly don't consider the fact that each individual's performance has random variation over time I have a good day of a bad day so there's a fairly substantial piece of work we do around isolating that but it's actually essential because in good conscience otherwise we can't claim that we're the driver of it if you're inside the company the bar is typically much lower if you think something's going to work and you have science on your side the people who are making sort of build buy decisions and come you know your customers as you if you will basically just want to see that stuff works so you don't need to have that extra layer of rigor where you scrub all the productivity and it's kind of sufficient to to just kind of have groups that looked roughly similar and try it in one place but not the other but it's it's a non-trivial task to do it really correctly it's just the burden of proof is if you're an internal agent probably lower than if you're an external agent gentleman up here had a question if we're having problems doing persistent and continuous change does that imply we're not focusing enough on changing the systems and the context people are in and more controversially does that also imply that nudges aren't enough that we need something beyond our just to do that kind of deep change so it depends on context again Damone's point was at some way you have to get into policy change right and structural change and so that absolutely should be part of it inside most organizations though what we found is there's so much low-hanging fruit like you know nobody ever loves performance management and promotions they always think the systems are unfair you know one extreme is dramatically revamp all of those and make them as fair and robust as you can but often you can get tremendous benefit by just nudging people to communicate more so for example you know nudge managers and employees that two months from now we're doing performance reviews so manager tell your employee and talk about performance standards employee ask your manager the quest in the middle of the review process nudge the executive to make sure you're running adverse impact analyses because sometimes it's done at the end it's too late at the end then you have to go back and unwind it with everybody so you the executive nudge is in the middle of the process do the adverse impact analysis and then afterwards and nudge the manager go and explain to everybody what it would take for them to move ahead next time and by the way nudge them to go to the people who just missed getting promoted because often the most disaffected people are the ones who just missed it and all their friends got promoted and there's random variation who gets advanced or not right like you got an easy project or not you clicked with the manager or not and no company on this planet goes to the people who just missed it and says don't worry about it six months from now you'll be fine it all works out in the end so a combination of those nudges actually gives you a tremendous amount of benefit before you even have to get into how often do we do it is a real time or not and all these things that are actually much more difficult from a change management perspective I would just add on to that really briefly that so I definitely agree with the spirit of the point like nudges aren't everything we have to do more we want to change culture sometimes even but I would also say that sometimes nudges can be a way to change things like identity and culture even though that's not how we typically think of them the sort of canonical nudge of I default you into retirement savings doesn't change the way you're thinking you didn't even notice it but there are a lot of there's a much broader category of nyjah's they they've been called Ralph Hartwig calls them boosts things that actually educate you change your mind and shape not only your behavior but your thinking about things and so therefore those can lead to shifts on identity and shifts and culture have you identified any unintended consequences of nudges in the experiments that you run for example you increase productivity with a given nudge but life satisfaction probably goes down because you're yeah great question you want to go first yeah that's a it's a great question like when you nudge one thing is are there crowd out effects yeah it's a huge concern in the academic literature and of course the results like anything are mixed so sometimes it seems like there can be crowd out there was a study that David Lieb said did recently were a nudge on savings and one domain seemed to just basically crowd out spend it crap you know you pay down this credit card debt but then you take on more credit card debt but then we've seen other examples even in the savings domain where where that doesn't happen and in in other setting so it really does seem to be case-by-case which I think means we just have to be very vigilant about this risk and and actually try to test for it but but at least the settings where I've seen issue the spillover isn't like I made you more productive at work but then you go home in you know you abuse your spouse it's more likely to be a spillover in a directly relevant sort of naturally linked area like I get you to save more in one account but then that just offsets savings and another so I do think when we think about spillover it makes sense to think about it in sort of concentric circles rather than worrying if I get you exercising more all of a sudden you're going to start you know you're not going to take any more vacations it's it really you might eat more but we have to think about what's the logic what are the logical spillover risks yeah it's saying in our experience we have generally not seen issues like that and the reason is each time we rerun our diagnostic we identify what will have the highest marginal benefit so it may be hidden in there because the behaviors that have the highest marginal benefit shift quite a bit over time so that may be because something was like number two and then there's a spillover effect and so it sinks to number five and now it pops up to number one because we've depressed it but the overall effect is greater you demonic happiness and organizations and greater measures on the other outcome variables so even if we're causing we don't seem to be but even if we're causing it it's more than lost if we're causing a little harm it's more than swamped by the overall uplift and benefit we're getting hi I work in a big government agency and as you can imagine it's quite hierarchical and we talked about individual level variation versus other kinds of variation where would you say the most if you're going to deconstruct that variance where would mostly be found and if you're going to nudge at different levels of an organization where would you get most return for your lunch so our fundamental unit of analysis even though we start with the individual we aggregate it up to the team level and we focus on sort of the lowest organizational level and then we do the math where we sort of look at that versus all the other teams and then roll that up across the company but we focus on fundamentally the team not the individual because our belief I'll say belief is that that's where the change happens right it's like the Framingham Heart Study in Massachusetts found that right if you want to quit smoking hang out with different people if you want to lose weight hang out with different people so that's where we start we actually though to the second part of your question we note simultaneously at every level of the organization because the different levels you have different amounts of control like you have more policy level control at the executive level and you have more sort of narrative symbolic power at that level so we find that the nudges we write for people at the executive levels are actually quite different than the ones on the frontline level and it's the combination of how they interact with each other that's that's incredibly complicated but incredibly powerful as well we just got the time cut off so on that note I will say thank you guys for being a great audience and letting us do what we we sometimes doing the phone instead of on stage and I hope you enjoyed the rest of this conference thanks for being here you

Channel: The Behavioural Insights Team

Views: 10,278

Keywords: behavioural economics, behavioral economics, behavioural science, humu, katy milkman, laszlo bock, bx2019, work rules, work, commuting, HR, human resources

Id: shdCZvSmMvU

Length: 45min 49sec (2749 seconds)

Published: Tue Oct 01 2019