David Spiegelhalter: Communicating statistics in the time of COVID | The Royal Society

good evening i am carlos frank the auction professor of fundamental physics at durham university and i am also the chair of the raw society's public engagement committee thanks so much for joining us this evening here on the royal society's youtube channel because we're here tonight to celebrate the 2020 michael faraday prize which has been awarded to professor david spiegelholter this prize is given annually to a scientist mathematician or engineer who is not only outstanding in the discipline but also able to communicate scientific ideas to a public audience in a clear and engaging way the word of course is named after michael frederick faraday fell of the ross society the very influential 19th century scientist who discovered amongst other things electromagnetism not only was faraday a really brilliant scientist but he was also an enthusiastic and committed public communicator of science so much so that in the 1820s he founded the christmas lectures at the royal institution which were aimed at young people and of course these lectures continue to the present day 200 years later father is famous for many things that amongst them a reply he gave to the then chancellor of the exchequer william gladstone who once um came up to him and asked him about mr faraday but what is the use of this phenomenon that you have discovered electricity and faraday is reputed to have replied well i don't know sir but one day you may tax it now david can count himself amongst the list of exceptional scientists and communicators including martin pollakov the winner lazier daniele george martin rhys david attenborough brian cox and josephine berber now now whilst um normally we usually would be celebrating this evening in london at the home of the royal society perhaps um with a glass of something sparkling in hand tonight we're at least able to come together virtually as it were from all over the world to join david as he discusses the absolute vital importance of clear and reliable communication of numbers and statistics in these strange times of the covet 19 pandemic i very much hope you will enjoy the discussion and i will now hand over to our wonderful host of team hartford broadcaster journalist and presenter of one of my absolutely favorite radio programs more or less time good evening everyone and welcome to the virtual royal society i think we have a wonderful hour in prospect uh timely discussion of the statistics surrounding kobit and more importantly a chance to enjoy the wit and wisdom of professor david spiegelholter who i will introduce properly in a moment but before i do i should tell you who i am my name's tim harford i'm a financial times columnist the presenter of the bbc radio show more or less and the writer of a new book called how to make the world add up and i it is my pleasure not only to ask questions of sir david but also to give you all this bump so excuse me a second i'm going to read you the bug because it's important first if you want to send questions we can take questions via slido i am going to do my very best to understand how slido works uh if you um the slider link is in the video description the access code for this evening is hashtag z999 which feels like it should be the name of doctor who's dog and somehow isn't hashtag zedd999 um we're probably not going to be able to cover all the questions we'll do our best but if your question is is pithy and insightful and brief um that will maximize the chance that i actually get to put it to uh to sir david and also important there is live captioning this evening and if you would like to see the live captions uh click the subtitles or the closed captions button uh in the bar at the bottom of this video and if you want to tweet during this event i actually discouraged tweeting while listening to david i think you should just soak up all the wisdom but you can tweet if you want and the hashtag is covid science so look i should say um say a few words about uh david spiegelter he is the the uh the chair of the winton center at cambridge university he is uh an amazing communicator about risk and about statistics a long time friend of more or less regular guests and i think the nation's favorite statistician and he is i mean he's a worthy winner of the faraday prize i've said i actually realized i never quite asked why he won the faraday prize but which is given for science communication because it's just so obvious that of course he would win it uh apparently it's kind of it's just basically for being so awesome for the last uh the last several decades um i should say uh david told me i shouldn't say this but i'm going to say anyway he's written an amazing book which is called the art of statistics uh terrific book you should get it as a christmas present for all of your friends and on the back of the book none other than some bloke called tim harper said david spiegel-halter is probably the greatest living statistical communicator um it must be true because i said it on a book i still believe it um it's my pleasure to introduce to david spiegel hello david welcome how are you hi hi tim um great pleasure to be here so just give us the potted history of how you got to be david spiegelholter how you got to be this this voice on our our television screens explaining how good risk what's the history of all of that okay well before going into that i i should just say um you know about getting this faraday price because it is extraordinary um honor and privilege to get this when i you know look at the people who've won it before uh david attenborough and brian cox and jamal clearly and marcus lisa toy and loads of others i'm not going to mention but they're all the people that i really look up to when it comes to science communication so i think it's it's really nice to i'm really delighted to get it i'm very sorry i can't be there you know and have a reception all that sort of thing royal society did send me a nice little hamper little you know red riding hood hamper which i haven't opened yet i want to see what's in there so it's really delightful to be here and in exchange for plugging my book can i mention that after you've bought my book perhaps for christmas you might want to get tim hartford's how to make the world adder honestly this is so incestuous it's untrue because i've got i've written on the back of his book as well saying how wonderful it is so we've got that over with now i'm glad um okay so how did i get get here um i started off doing maths at university and then i did pure maths and it got too difficult so i changed to statistics and i found i liked it and and i've carried on doing statistics as a sort of academic way for quite a long time but getting more and more and more involved in really practical problems like the public inquiry into the bristol babies that died and also public inquiry to harald shipman and all those sorts of things just gave me more and more of an interest of how data is used in society and then in 2007 i was incredibly lucky that david harding of winton capital management you endowed a chair at cambridge university the professor for the public understanding of risk and i got that and um i retired at the beginning of last year but but since then all that time i've been philanthropically funded by uh by this um endowment uh to to work on communication and um and the the um people like the bbc have been fantastically good at having bbc four documentaries that are fronted by academics rather than celebrities so i've i've done those and um and yeah and it's been a delight to be able to change career you know even at my advanced age into essentially communication almost as a full-time job although you know i do uh you know i still do some research work but most of my research now is with psychologists because david harding again funded a whole winton center for risk and evidence communication and which now i work with in in cambridge and that's mainly psychologists and computer communication professionals fantastic group of people who have taught me so much and that statistics is not the only thing in the world amazing and there is a podcast that you present um called risky talk called it's very good but you it's you've got quite quiet recently it's almost a day you've been quite busy um i encourage everyone to subscribe to risky talks i'm we've got some new ones coming out elaine goodman's produces it and does a brilliant job and risky talk it's about talking about risk but it's also how you talk about risk because that's what we sort of work on um you know is just the the risk communication in all different areas so um yeah i do recommend subscribe to risky talk yeah i seem to remember it was basically uh climate science covid covert kobe goodness up goodness me um yes there's a whole variety of things that we will once you know eventually we'll be able to talk about when this is all over um but i mean we should talk about the the statistics and the communication of statistics during this crisis um tell us about what you've been doing yeah okay well can i can i share my screen show some slides about what i've been up to um so i i'm gonna do that so uh that is um okay so i'm supposed to be giving a sort of lecture um so there's my lecture um and uh so that's me and uh let's see if i can um oh yeah sorry i did you want to show my colleagues at the winton center this is from a little while ago so the colleagues have changed a bit but as i said this is a great group of people that i work with now um one of the things i in my book i start by saying all the time and i just repeat this all the time data does not speak for itself and this is a lovely quote from nate silva's book the signal of the noise the numbers have no way of speaking for themselves we speak for them we imbue them with meaning and that has been so true within the covid crisis that there's been you know masses of numbers around but what we make of them depends on the story that we're telling the narrative in which they're embedded and we can make the it look dangerous we can make it look safe we can make the numbers look big we can make the numbers look small depending on how we treat them okay so um i've been trying to work my way through this and it's been a very busy time of course for statisticians and i i as i'm going to say i think the statisticians working for the big organizations such as the office of national statistics and phe mainly done an amazing job in getting the stats out um but there has been huge demand from the media for comments from experts a lot of the experts are working so hard supporting sage and working for these organizations and are reluctant or just don't want to um or can't get out and talk to the media so it's been dumped on me and a number of others to to do a lot of media work and i've been doing weekly press briefings during the peak of the epidemic with the science media center and i should say most journalists have really acted rather well partly because they have to because they've got to be nice to us because they need us so much um something i do want to say is it is quite tricky to remain non-aligned because people always wonder are you for locked down or against lockdown so i'm not i'm not going to say i'm not going to say you know i don't i don't think i should be recommending policies at all that's not my job and there's a constant need to say is not my job not what i know about i have refined my skills in just stopping interviewers and saying i don't i don't talk about i don't do i don't know about model epidemic models i don't talk about what policies should be should be enacted so um that has been something i've had to learn um i haven't too much attention sometimes i wrote this column um some time ago about in the guardian about saying we should be very careful about doing some eurovision comparisons between european countries and then this is back in may boris johnson mentioned me in parliamentary questions saying we shouldn't make any international comparisons and um urged by some colleagues i i tried to repudiate that in a tweet to say polite request to the prime minister don't say i didn't mean we can't make any comparisons i i what i was talking about is we shouldn't be talking about who's first second and third of course we should try to learn from other countries that have done a lot better and that was my one viral tweet with ten thousand retweets and then of course i got mentioned by kierstermer parliamentary questions you know against boris johnson at this point i thought it's time to start having a lower profile so um this is not the sort of the situation i want to find myself in as a statistician part of the communication that i have been doing is to try to translate the kind of risks of covid into you know to try to embed it in a story that might be easier to grasp it's incredibly difficult to grasp i think the risks of coping and so i wrote this paper in the british medical journal on trying to use the normal risks we face as as comparators and and one of the first things you have to do is to distinguish what are the risks you've got the risk of catching it you got the risks of dying if you catch it and then you've got the risks of both together of both catching it and then dying and those that those are the ones that are really quite easy to estimate because we just look at how many people in each part of the population how many different people in age group have died of covid over the peak of the epidemic and when we do that i mean i'm sorry about this rather complicated graph for those who like grass and this is a bit like one of those um like like one of those um press briefing graphs where you put it up and then whip it off immediately very quickly but i'm going to try to do it a bit better than that and the crucial thing if we look on this right hand side it shows the risk of dying from covert over the peak of the epidemic with age just rose massively exponential increase in the risk of dying from covid and roughly speaking for every year that you get older the risk was increasing by about 12 to 13 percent age is the overwhelmingly strongest risk factor for coving even stronger than ethnicity which is important stronger than sex which is important stronger than most most diseases as well it's age and i'm i'm always amazed that it's not talked about more that people should be you know the younger people should be really trying to protect anybody like my age i'm 67. you know keep away from me i'm i'm at higher risk and you know i'm about 100 times risk the risk of somebody who's 27. so just keep away from me and keep away from your granny if you can so um and the risk is therefore doubling every five to six years you know massive grading if we look at it on a log scale we can see that you know people in their 90s had about a two percent chance of dying two percent of all people in their 90s died over the over the 16 weeks of the peak of the epidemic but down here for these kids is one in a million or so so this is a a massive less than one in a million charts you know a massive difference in age um and i was trying to think of a way to translate this and i i thought well people over 50 very roughly speaking over the peak of the epidemic the 16 weeks of the epidemic it was like we experienced about five weeks extra normal risk because this is the normal risk we face of dying there over the year and this was less than this so it's about five weeks extra it's almost like over the living through the peak of the epidemic only one over 50 instead of experiencing 12 months of risk during this year they experienced about 13 months of risk i thought this was a good analogy it has failed dismally nobody likes it nobody's picked it up okay so what about another analogy that i was using back in march about the chance of dying if you catch covid and i was on more or less back in march you know describing this analysis this is the first estimates made by the imperial group not based on based on chinese data and diamond princess data on the risk of dying if you catch code and um varying this is out of 10 000 varying you know over 10 up here for the elderly down to a tiny figure one in 5 000 or so 1 in 50 000 for for children and what i noticed is that that is very similar to the annual risk of dying anyway of some something else of something else so i i was on more or less describing this and i said um the risk of if you caught covered or similar to the risk of dying this year from other causes another terrible failure in communication because this was hugely misinterpreted the bbc very nicely reproduced this graph and then katie hopkins and here's the bbc's reproduction of my graph showing that the risk of dying from covid was very similar to the risk of dying from something else this year and dear katie hopkins before she was banned from twitter tweeted out to her million twitter followers fact-based tweet corona deaths are not outpacing what is normal or usual and blaming me for this so she was interpreting this as meaning that the coronavirus if you caught it didn't add at all to your normal risk of dying this year i thought no they're the same magnitude it doesn't mean they're the same thing you know in fact what i should have said of course is if you catch the coronavirus it doubles your risk of dying this year and i should have been clear about that right from the beginning because what happened the sun had the headline your risk of dying is no different this year despite coronavirus pandemics as expert as me i didn't say that it's the last thing i think so that's what happened and of course i complained to the sun and i i i i've many many times the last one was just last week i complained to newspapers about their their headlines and they change them i mean i recommend anyone else who gets the bad headline complain and complain immediately they do change them online anyway um and they changed it to something slightly bitter better your risk of dying from current is roughly the same as your random risks as expert so and that was slightly better so um what's interesting i think my final slide on this first part is to we might ask well what's happened what about this infection fatality right now that's what that's the risk of dying if you catch it what is the latest estimate the latest estimate from the same group that's the latest estimate i mean it's quite extraordinary it goes right through the middle so according to them after a hundred this is based on many countries so this is hundreds of thousands of covert deaths analyzing those you come to exactly the same conclusion that the average risk of dying if you catch covet is the average risk of dying this year of something else and so i think this is a i think it's it was a very good one thing i have got right out of everything else i've got wrong over this um pandemic was this claim back in march of course actually this is in the first wave the risk of dying if you get it now in the second wave is reduced i i'm not going to say probably between 30 and reduction to 50 reduction of what it was before so that line has moved downwards a bit so with that i'm going to stop i'm going to stop showing my screen and go back to chatting to tip thank you well that's what i've been up to that's what i've been up to yeah well you've been up to many of things um and thanks for sharing some of them with us actually it's quite struck by that last pair of graphs um this isn't really a risk communication point but um it's striking how little the verity at al uh estimate changed that basically they seem to get it right first time out it's impressive it it is extraordinary that and that was based on very you know thin data essentially from china and from as i said from cruise ships and everything like that it was it was a it was a wonderful piece of work everyone it was in that report from march from imperial college that's received such uh a lot of um criticism and comment and that was in there and it's fantastic with the work yeah i mean a lot of people criticize that report i'm not sure how many people have read it um for my money it's it's quite striking they didn't get everything right but it is quite striking how much they did get right uh but a very early uh you know very early cuts of the data and and they were very available you can talk a bit about uncertainty later but they were very clear about the the fact that they were making projections with a lot of uncertainty and under a lot of different scenarios but um maybe we could get back later to this whole business of projections and scenarios and their communication absolutely well i had all kinds of questions for you but i i made the mistake of looking at slido and i now have to throw all my questions uh out because actually the questions that are coming in from people online are i think very uh pertinent uh there's a good one about uncertainty but which we'll come to but um let me let me start with one from uh from superzario who asks after the awful visual aids which accompanied the presentation by the prime minister and his advisors on the 31st of october what's the best piece of advice you can offer to presenters of statistical information when using visuals or maybe we're not using visuals oh that's interesting well i'm going to come i'll come back to that i've got some more slides about what others have been doing um and i'll come back and criticize those or say how good they were later on i mean i think yeah well so let's i want to let you do that let me ask you a different question yeah okay we could come back to that but i mean i think that i'm not an expert on visual design and lots of people have written extremely good books on on visualizations and and i think you know you have to follow the advice that's been given i could say alberto cairo i think is is the guy i always would go to for to look at visualizations but but one of the crucial things of course is to test them and i you know i failed in my mind that visualization drawing the dots and showing they they run along the line i thought was brilliant i thought i was very impressed by it but it completely failed in its communication message most of the time so i all you know all the risk communication work we do in the visualization or the communication work we do we say there's no absolutely correct way to do anything the crucial thing is to work out what you're trying to communicate and then test it but what about how do you cope with the fact that i mean there's a lot of motivated reasoning out there a lot of people who are very keen to reach a certain conclusion i don't know why katie hopkins yeah misinterpreted what you said maybe she didn't understand maybe she was strongly motivated to to not understand i don't know well one of the people working hard to to reach a particular conclusion and to twist the data in a certain way so how do you deal with that well as i say i started off by saying the numbers don't speak for themselves and most graphics don't speak for themselves either so i've learned the trick that other people have recommended is that above the graphic when you put it you slap on a big headline saying what the main store the main message from it is so you make sure and you make sure that's an integral part of the graphic that people can't take out later on so it's actually part of the graphic not just in the legend or the text or something around it so you um actually really help people in the interpretation i think i think it's only fair to do that for general audiences clearly these things can be can be manipulated the the the graphic can be can be rebased or re-titled but you'd have to then work harder to do that it's harder to accidentally yeah yeah i think i think i mean you've got to be you know you're going to be quite confident that this is the message and of course a good visualization you know allows people to learn more from it than you're just headline figure i mean and take more from it you should always be trying to provide allow people almost to make up their own mind that this is a convincing um claim that you're making on the basis of the visualization and i always go back to anora neal and she says the business about transparent communication you know intelligent transparency is that okay i'm going to go through the checklist that i always do sorry um if so nora neil great philosopher of um you know and expert on trust i'll come back to her later sure he says if you're trying to communicate about data um then you've got to do four things you've got to make it accessible people have got to be able to get at it and people have actually that's got very good during this crisis you can download you know a lot of data from the office for national statistics and public health england and even you know the stuff that's delivered in the briefings um you know the the spreadsheets with that data generally are available either at the time or very soon afterwards it's got to be accessible and it's got to be comprehensible people are going to be understanding you've got to be able to test it it's going to be usable in the sense that it answers people's concerns but her final one is it's got to be accessible so people should be able to check your working if they want not everybody will do that most people will take things on trust but if you may if you show some data and you make a claim you've got to be able to back it up people have to be able to check your work if they want with this i think puts uh you know asks a lot of of the ordinary citizen or potentially asks a lot of the ordinary citizen um and i've got a question here from mark moffat who says how can the average citizen access the statistical skill necessary to make rational decisions i've got yes what else can i say no i think this is a a a big issue and um i'm on you know go on and on about the need for increased data literacy in in the population and i think everyone agrees this covert crisis has demonstrated the enormous need for citizenry um to be uh well informed um because in the end all this people have to have um feel that the policies are being enacted are justified by by the evidence and so i i do think that um this is a vital concern and the the ways in which that's done are norms i mean i think it should be a much more integral part of schools education um it it should be part of um you know i think a lot of science communication you know that can help programs like more or less do a fantastic job at providing that um and uh and we should be just helping people understand work all the time and we'll come on to that what i mean by trustworthy communication a bit a bit later but i do think this crisis has really illustrated the importance of this oh and another fantastic bit of research that my colleagues have been doing i've been testing people's resilience against misinformation about covid and the biggest predictor of people's resilience to misinformation is numeracy now i can't being a good statistician i can't claim causation there i can't claim that if people numero numeracy skills were were bigger that would make them more resilient i'd be a bit much but i'd love to think that that but it is a strong association and it is so i think that again you know we're going to see we're in an age of misinformation we're going to see even more when the vaccines start being rolled out and we have to do what we can to to buffer people from that i was quite struck by that study actually i noticed that to be honest it surprised me it challenged my preconceptions because my feeling has been that a lot of the conclusions we reach are they're political they're emotional their what our friends tell us they're you know what we want to think because we're trying to win an argument all of these other things that govern um whether we fall prey to misinformation or not but it's quite interesting that actually the story your colleagues were telling us maybe it is as simple as well i'd add up you'll you'll do better i i do find it very encouraging i mean there's obviously a whole in the whole area of tribalism in in the sense the way people are i mean i don't want to start talking about the full scepter the full range of skeptics about covid and things like that but you know they are there is a strong voice we hear we hear i think there's a lot of misinformation out there um and clearly there's a tribe that is attracted around that that that kind of voice um and especially when because it there is some media that support that voice and and so i i i do think i find it encouraging that it's not just pure tribalism yeah no it is it is encouraging david you said that you know it was important that people were were well served and were given information that they could trust and that they could understand and you'd be you've been promising to show us more stuff sooner yeah can i show you how does everybody who isn't david spiegel has been doing it yeah can i show you some stuff i like and i don't like sure just okay so i'm going to share my my screen again and hope this works um and uh so okay so here we go again yeah how have others been communicating and of course more or less has done a fantastic job in all of this and and you know of course they have um so um but i'd like to just point to the work of the office for national statistics and i might also um obviously admit to a conflict of interest here is that halfway through the epidemic um i was made a a non-executive director of the uk statistics authority um which oversees the work of the office for national statistics so of course i would say they're doing great work wouldn't i um but this is the data that just came out on tuesday um showing um the the progress of the epidemic nice clear illustration of of the covert deaths uh the non-covert deaths the five-year average this is registered death so it dips up and down according to bank holidays and if you just blow up the last few weeks it shows very nicely in fact that you know deaths have been going along you know um non-covert deaths are roughly going along at the five-year average slightly below and so essentially all the excess the growing excess now is from covert deaths but all is not normal the the next figure i think is terribly important and this is the one i've been banging on about this for months and months and i still get very upset about it that not enough attention is being paid to where people are dying um the if you look at this if you look at this is the number of excess deaths a big spike in in hospitals and then a deficit in excess deaths um during the um during the summer um big spike in excess deaths at care homes but crucially look at what's happening in people in homes now normally about you know 3 000 uh two thousand and something people uh die um in in their home each each week um but this has been running at about six seven hundred now nearly a thousand extra deaths occurring at home every week and this happened throughout the summer and shows no sign of changing at all this just seems to be a new way of life that about 35 extra home deaths are happening so one in three more a third more people are dying at home than normal very few of them are copied they're crucial what's the quality of these deaths what's the end of life care that these people are receiving if it's good then maybe that's fine you know people prefer to die at home rather than in hospital or or in an institution um but maybe it's not and maybe some of these people might have lived longer have they gone to hospital i think this is such an important issue that has not received sufficient attention but a nice graphic again from ons and i go to the public health england dashboard which was awful when it started months ago absolutely terrible and now it's got really good um i like the fact that when they look at plotlines this is from the data from just an hour ago and when they plot the the cases the new cases they do it by specimen date not how many are reported each day i get god i get so fed up with the fact that the media all they do is report how many new cases were reported each day which goes up and down depending it's always terribly low on sundays and mondays it's a big spike on tuesdays it's it's you know it's just ridiculous but if we look at the actual day of specimens we can see you know basically a leveling off you know a slow increase still between 20 and 25 000 positive cases a day that represents about 50 000 50 to 55 000 new um cases each day because less than half of people are actually tested um testing positive patients admitted to hospital again leveling off at about 1500 a day by half the peak but the crucial thing is leveling off um and what would what might happen to that over the whole of the winter and if we look at deaths and the the top line of this shows the fact that you know what is a code of death there's at least four definitions from a covered death so i could go through the public health england now focus on death within 28 days of positive deaths after their disastrous use of not having a time limit at all which meant that once somebody tested positive for covert if they ever died of anything ever in the future that would count as the cove of death now it's 28 days which actually reduces it a bit too much because some people might still live beyond 28 days and then die but in the best of all is the deaths with covet 19 on the death certificate which comes from the office for national statistics and that's although people have been going on about oh we've passed five fifty thousand deaths well you know no we haven't we went past that months ago months ago well you know where it's nearly 62 000 coded deaths on the death certificate and quite possibly more than there'll be more than that um deaths again going up to about 300 a day now now leveling off okay so i think that's quite good presentation the other good source i find is public health england weekly surveillance report this is today's it came out this afternoon and i this is extraordinary look at this is the um the number of hospital admissions with covid in the different age group and this is 85 plus and they are showing a steep and increasing number in hospital admissions for covert fortunately um you know leveling off for the younger groups and this is hospital emissions for flu anything oh this is quite low compared with that look at the scale that goes up to two a hundred thousand so this the whole graph fits down here there are essentially no flu you know admissions for flu at the moment now it is quite early on in the flu season but i'm gonna i don't like making predictions at all but i would like to predict there's going to be almost no serious flu this winter and we don't i don't think we have to have any mysterious reasons for that yeah the the measures that have been taken out taken even before lockdown at tier one the distancing that's taking place just which we've all got used to um i think is sufficient to cut down much of the transmission of flu so we're not going to say we're not seeing any flu which is good um you know because we wouldn't want to have both of those at the same time okay not everything is good i i mean this is a total disaster and this was shown at the briefing last saturday which was brought forward because this graph was leaked by somebody personal persons unknown to the media so it appeared in the bbc on thursday on on the friday so then it was used in the briefing on saturday and no matter how many times they can say these are scenarios and not predictions or forecasts everyone focused on this this prediction this prediction this scenario which ended up to 4 000 deaths a day it's a massive total and the point is these were hopelessly out of date even by saturday last saturday or no saturday before last when it was when they were shown on october the 31st the cambridge group that produced that had produced it a lot you know a month earlier um and uh based on just one scenario and these are worst case scenarios this is assuming we don't do anything we just let it oh we just just let the virus go we're not going to do any changes in policy or behavior so these are worst case scenarios under assumptions which were shown to be untrue even by the time this was being shown cambridge had revised its model twice and reduced this right down so why this was shown i just think it is not not good um at all and just allows all the skeptics to pick on that um anura neil again my hero said that she she's an expert on trust her wreath lectures on trust were still deserve re-reading even though they're 20 years old and she says that we shouldn't try to be trusted he says we should try to demonstrate trustworthiness trustworthiness trustworthy communication statistics is a vital component the code of practice for statistics which the government statistical service runs under puts trustworthiness as its number one pillar i i it's such a good thing you have to ask all the time is this trustworthy communication and um you may have seen this before i'm sorry if i'm repeating this but this is my little diatribe i did back in may about the briefings and um i'm sorry if you've seen this before they're now very familiar with these these daily press briefings how well are they working do you think well i i watched yesterday's and frankly i found it completely embarrassing um we get told lots of big numbers in precise numbers of tests being done 96 1878 well that's not how many were done yesterday that includes people that were posted out test tests that are posted out we're told 31 587 people have died no they haven't it's far more than that so i think this is you know actually not trustworthy communication of statistics and it's such a missed opportunity you know there's a public out there who are broadly very supportive of the measures they're hungry for details for facts for genuine information and yet they get fed this you know what i call number theater which seems to be coordinated really much more by you know a number 10 communications team rather than genuinely trying to inform people about what's going on i just wish that the data was being brought together and presented by people who really knew its strengths and limitations and could treat the audience with some um respect they're now very familiar with these daily press briefings okay so um that actually got me on gogglebox you know for heaven's sakes so um that that was my one real diatribe i then tried to avoid um doing that again um but part of demonstrating trustworthiness as we've already mentioned is admitting uncertainty about what we know and i'm obsessed like many statisticians are with acknowledging uncertainty i mean and that could just be be putting a range about a number or it could be admitting that we just don't understand what's going on our evidence isn't that good and this second one is something i'd really like to emphasize um let's look at some of the communication that's been done again rather openness about estimating r the magic number the average number of people that each person might in fact will in fact now there's eight different groups around the country estimating art which i think is extremely good to have multiple competing models producing art and so here we see uh some a little while ago eight different estimates of r with intervals around them with their with their with their um you know uncertainty intervals around them and interestingly these are estimating the same quantity using essentially the same data they've got access to the same data some of these are very precise intervals but they don't overlap it's a lot of no they cannot all be right so this is precision rather than accuracy actually they're all broadly said giving the same picture so from a practical perspective they're not that different but what's absolutely crucial is that on the whole they're far too confident and that's because the estimates of precision that these models are delivering are based on assuming the model is correct but we know the model is wrong all statistical models are wrong generally because they're too simple then we just don't know what's going how this process is actually working in society so we go back to the classic phrase all models are wrong but some are useful so i'd be very very cautious it's just as well as just one group estimating this so very very cautious about taking margins of error on face value because they are generally um you know contingent on assuming your assumptions are correct and the one thing we know is your assumptions aren't correct um many people i know have been arguing for is almost sort of star ratings on quality or strength of evidence for effectiveness of policies this is already established um by the uk work centers um which i think the envy of the world i think they're brilliant for the education endowment foundation for example reviews all the evidence about um the effectiveness of interventions in education and they produce a kind of toolkit like that isn't this lovely and so and they look at they give it's like trip advisor they'll give star ratings for how much it costs um how what its impact might be and how good the evidence is so if you look at aspiration interventions the second one down um we find it look it's fairly expensive it doesn't seem to work zero impact and the evidence is rubbish anyway so maybe we shouldn't do that one but if we look down the bottom at collaborative learning it's fairly cheap there's quite good impact up to five months of extra you know educational gain and the evidence is pretty good for padlocks and we found in our research our team found in our research that people audiences general public audiences really notice quality of evidence they take notice of this um now sage have been doing this in their recommendations there's a great document where they summarize the potential impact of different interventions on r and so for example they say uh the record using wearing face coverings outdoors um very low impact on community transmission high confidence in other words there's no point in wearing face masks outdoors and and there's good air you know and they're confident about that conclusion closure of places of work worship community centers well it might reduce yeah by up to 0.1 difficult to estimate only moderate confidence so they give a they give a sort of range between norton point 1 of the effectiveness but they're not even really confident about about that either so i think this is extremely good communication you can give a range and also your confidence in your analysis so i deeply recommend this these ideas um so i'm going to finish off now um my this is my diatribe i think data literacy is a vital skill in modern life covert crisis has made this even clearer and statisticians have shown themselves to be deeply wonderful people but of course i would say that wouldn't i thank you david i i want to hold on that question of uncertainty for a moment um partly because i find it fascinating partly because our our most popular question on slido uh is on the subject it's from david fox um i'm going to ask his question but i'm going to i'm going to tweak it myself so he asks is it more important for a statistician to communicate to the conclusions or to communicate the degree of uncertainty associated with conclusions it's a great question i could have thank you for who sorry who is that what a fantastic david fox okay they they're they're equally important they they they go together and it's bit like i said about the graphic they shouldn't be separable in that you know the the um in a way the the claim and your confidence in the claim should be absolutely strongly interlinked and uh and the way to do that i believe is as i'm trying to demonstrate is first of all if you're trying to estimate a quantity give a range and our again our research has shown that if you do give a range there's no reduction in trust in the source you do not have to give a single number you the source is trusted just as much if you can give a range unapologetic uncertainty is what we call it and and actually giving a range without even giving a central number i think it's even nicer because it stops the media fixating on oh the number because the media the media otherwise would ditch the range and only use the single number so i think just giving a range is a good idea but then we've seen these ranges are not necessarily very reliable so you have to you should also state your confidence in your whole analysis your whole understanding of the process your statistical modeling and many people this has been being done in medicine with the grade scale for for years and it's deeply established and so i i don't believe it's not a choice of one or the other they should be absolutely linked together but i worry that there's a risk that uncertainty can be weaponized like something that i've that i've looked at myself and others have looked at goes back to the tobacco industry who were very clever when the the evidence on smoking and cancer came out they would go well you know it's not certain it's complicated scientists disagree all of which is true the same thing then gets said about climate science yeah so how do you avoid when you're expressing uncertainty how do you avoid that being seized upon by people who go well look you know they these dolphins they don't agree they don't they don't know what they're talking about they're always changing their mind yeah so this is the merchants of doubt our argument that that's how the tobacco industry um you know countered the claims about lung cancer whatever was was not by directly saying they're wrong just by constantly introducing uncertainty and building on that and now it's a very important point because it's happening in climate science we say and it has happened in cobit as well so and the first thing is that that is no excuse not to acknowledge uncertainty i think we have an ethical duty to say what we know and what we don't know um but you the and i think you should be unapologetic about your uncertainty you also should be unapologetic about saying what you do know what there is consensus about it what there is general agreement about in the science and and i'm sure people have got bored with me saying this is about what john krebs when he always you know oh i don't think he's out yet the podcast can't remember i think it's in the podcast we've done on them on on where we featured john krebs um when he was head of the food standards agency and he was faced with disaster after disaster sort of you know scrapey and sheep and um and you know in every problem that you could ever imagine um and he just had a policy for you know when you're communicating you say what you what you know you absolutely started we know this we are confident about this and then you say what you don't know you have that humility to say what you're doing and then you say what you're doing about it what you're doing to find out and reduce you know improve your knowledge and then you say what people can do you give them self-efficacy and then you say and this is god this has been um chris whittie has tried to do this but others haven't then you say you will come back to people and the advice will change it's not a u-turn you say as we learn more we will be changing what we say and you say right from the start that we will be changing what we say and and chris whittie when he started off back in march standing up there with boris and patrick i was ticking them off chris was great he was just going through these like you know very well indeed but not everybody is so good at establishing that i think credibility and that trustworthy communication there always will be people who try to weaponize uncertainty um but i i do think that if it's accompanied by um you know absolute confidence about what we do know and what is important then we we and actually accompanied by just data and facts and that's why i put up those statistics that coming out of ons and public health england which i value far more in terms of convincing arguments than i do uh projections and scenarios and worst case scenarios and things like that because actually you can't argue with them that's it now got several good questions about you know uh interpreting the statistics and about communicating statistics but i actually wanted to slightly change gear and ask you a question about gathering and using distance and this is from rachel williams and she says how can real world health data recorded in gp practices and hospitals be better utilized for public health purposes oh my god that's a big question oh yeah i know absolutely as well yeah it's such a good question but and i'm not really the right person to to answer that except to say yes obviously um we live in the age of data there's a huge amount of data now being collected but you know health um you know information systems are notoriously have been notoriously poor appallingly joined up you know the fact that the hospitals and the gp practices can't even speak to each other most of the time um and it has been a disaster essentially and yet the nhs should be this fantastic resource for learning from data but it's getting better it's improving all the time there's some wonderful um uh analyses that have been done during the covert crisis by two different teams the open safely team and then julia hibbersley cox team in in oxford analyzing gp practice data involving sort of 11 million patients in one group 17 million patients and another and analyzing what's happened to them with in with kobe extraordinary power that we've got in this country that that is the it should is the envy of the world and we can make even more of it but it is a problem um you know not enough has been done but uh all i can say is we're moving in the right direction and um there is some again i would refer particularly to ben goldacre's open safety team in oxford who are really pushing this very well yeah it is remarkable stuff and this is something that economists are wrestling with as well you start with these very well behaved data sets that are a specific specifically gathered by statisticians for statisticians and that's all great but then to go hang on a minute we've got all this data that's being just sort of you know scraped together because people are just trying to run the nhs or they're you know they're running the visa card network and you know there's a lot we can learn yeah and but again i would say this wouldn't i but the office for national statistics has got a huge program now in accessing that kind of routine data making rapid use of it and they are doing that already they've been doing it throughout the crisis and in background for economic statistics and that is a complete project that's going to grow and grow to make use of this routine data but you're right it's a mess quite often it is messy it's not not nice little um rectangular you know spreadsheets that we would so that we know and love but it is necessary to get that get that data on the desks as attractive dashboards for for decision makers everywhere and to feed it back to the public um again we're the public health england dashboard being updated every day and so many people looking at that i i think just demonstrates what can be done um in in in a crisis and that this is you know sets a wonderful example for the kind of transparency we should be expecting from now on mike hill asks what are the key factors that we should consider when when we the public are being presented statistics during the covert crisis oh so tim you should be asking this you've got a whole book on this about what questions to ask but i mean you've got a nice thing about you know look at the emotional impact and you know your first your first thing you've got in your book is to say the first thing to do is how does it make me feel which i think is brilliant i my own personal responsibility absolutely i sniff it you know you've got to sniff the data is it fishy is it a bit smelly that's what's being claimed and that requires some skill and background but what it essentially means i think the first thing is look at what's the source of the information you know why am i hearing this what is is this people who are telling me what why are they choosing to let me know this data and what is the what they want what do they want me to feel are they manipulating me or are they genuinely informing me so i can make a better decision for myself or are they trying to persuade me of something so the very first thing i look for is the motivation of the data provider even before i look at the data and um i think it's incredibly important again something that my our group in cambridge spent a lot of time our whole center it's got our little tagline is to to inform and not persuade you know it sounds a bit pompous but actually i think it's a very important principle when so much information is given out in order to manipulate you to try to change your behavior or your feelings or whatever so i would look my very first question is look at the motivation of the data provider yeah sorry i wasn't speaking of motivations i wasn't looking to give you an excuse to mention no no it was it was actually it was the the number one question that people had voted up so uh yeah okay so um just just to just to clarify there's no accomplishment but i could i could then you know get like you doing your way you know i've got in my books you know a list of other you know subsidiary things you can look at about that you start looking at the the quality of the design and all that kind of stuff but i i really go back to this idea of trustworthiness you know is the claim the final claim that's being made if a claim is being made how trustworthy is that you know is it is it actually justified by the data that's being presented but again that one of the most difficult things of all that i have to ask myself all the time when i'm seeing a scientific paper or a claim or reading skeptics blog or something what am i not being told what what data has been cherry-picked to show me in order to produce his argument what am i not seeing because that is one of the most difficult things to decide it's like what the dog that didn't bark in the night because you're not criticizing what's in front of you you're criticizing what isn't in front of you and uh this is a real challenge although it often is possible to tell whether someone is trying to persuade you of something or not and that's i mean when you're doing that when you're sniffing the data yeah yeah is someone is everything pointing in the same direction or is it is it is it messier because if it's messier then i'm immediately thinking well this is this is slightly more trustworthy yeah um so look we have we've all had a sort of crash course in statistics over the over the last years we've been one of one of very few joys that 2020 has presented us uh do do you think that um do you think the public's familiarity with graphs and percentages and case counts and is that going to continue when this is all behind us oh i do hope so as it might be you know the one you know benefit from uh from this whole crisis is people realizing the need to to understand what's being communicated and the need to be able to critique that and the and to demand that trustworthy and transparent and balanced information is provided to them because as i said you know this is you know we're in a state where we all you know general cooperation and is absolutely essential in in the context of the way we're living at the moment and so um i i hope that this increases people's um you know feeling of of what should be given to them um and i hope it would improve their um you know uh sense of just wanting to know more you know maybe they should try to understand what exponential growth means well all these things maybe this could be important i mean i i don't know i mean we we've both sold a lot of books during this crisis i don't know how related that is um but um you know again i think that does demonstrate a lot of um interest in this in this area so you know if one good thing comes out of it maybe that is yeah i was sold with more books if you didn't keep closing the bookshops but you know there is i suppose they have their reasons for doing that um look one final question david so so there is this engagement we want to continue to continue and so what you know what should statisticians like you nerds like me what should we be doing to try to keep that engagement going oh oh i i again i yeah that's tricky i it shouldn't be you know just us getting out there and talking to the public i think you know to my profession i say um and we're really trying to do our to do this as much as possible that you know more statisticians have to be involved in engaged with public communication whether it's working with people who are doing the communication you know or whether it's doing it themselves and you know you have to do what suits your personality and your interest but i think it's absolutely essential that essentially non-aligned groups who don't want to argue who aren't advocates for a particular policy but just want people to understand better and should have a you know should push themselves up a bit more above the parapet i mean it's you know can be can be unpleasant at times but i think it is a worthwhile a very worthwhile thing to do and it's very unfortunate that so many scientists either during the crisis either don't want to do it or can't do it because of the the nature of the work they're doing i i have argued that you know when you've got a group of researchers working they someone should be nominated and trained to get up there and get out there on more or less and today program and jeremy vine and i love jeremy vine it's my favorite gig going on jeremy vine they should be out there describing their work and and and that's their role in the team to that extent and i i really think that is an important um feature and of course i would say this would like office for national statistics has been doing that very well you know they have been out there on jeremy vine and putting people up there not just the head but other people at different levels actually talking about statistics and then one of the things that that happens david isn't when you get a name the media can always come calling for you and i know that you have been very actively trying to to mentor other scientists and younger statisticians and so thank you for that and thank you everyone for listening um the more information about other royal society events is available uh in the show notes on the website um there's an evaluation survey you can tell us how wonderful you thought uh david was in the youtube description please do that because you know loyal society administrators have worked incredibly hard to put this bent on or you know they'll be out on the streets before christmas if you don't fill in that form so please do that and most importantly thanks for listening thanks for giving us your attention and thank you and congratulations to the very wonderful professor sir david spiegel-halter thanks a lot and good night thank you and good night thanks tim thank you for the questions

Channel: The Royal Society

Views: 18,247

Rating: 4.8832116 out of 5

Keywords:

Id: JW9plVfanjo

Length: 61min 24sec (3684 seconds)

Published: Thu Nov 12 2020