July 13 Steven Levitt Making Sense of the Real World

a pleasure to start off this summit and to kick off uh this new virtual uh summit that we're doing here with dr stephen levitt who is the best-selling author of the freakonomics series and i am so excited to hear from him today because so much has been changing uh in the country so again you can ask discussion questions and stuff we'll be doing some q a there too at the end um but i think one of the things that i just want to say about him is that she's a professor um at the university of chicago's economic department and one of the things that i thought was very impressive was to receive the highly esteemed john bates clark medal for being the most influential economist in america under the age of 40. so and that's pretty much equivalent to like a noble um prize there so it's wonderful i'm sure we've all heard about him we've seen his books and if you've traveled in airports they're always there and they're around so with no further ado i'm going to actually turn it over to stephen and thank you all for joining and i look forward to coming back on stage a little later thank you kelly so um there i am hello everyone we're not exactly in orlando but uh it's good to be here virtually and i want to start there'll be plenty of time for questions later let me start by telling a quick story so um back in the day i used to do consulting with big companies this must be about 10 years ago i was at one of the biggest retailers in the united states and um and it was my first day with them and so i was being introduced to all the top people and the top data people and the thing that was really um they were really excited about was uh that they were going to introduce like a frequent flyer card a frequent purchaser card and they had invested 300 million dollars in this program and they were convinced it was going to be the salvation of what was at that time of struggling retailer so i was just in the mode of asking questions and i i said it well so tell me a little bit more about this program i said well people will have a car they have to sign up for the card and then when they do that every time they make the purchase they'll get the equivalent of one percent uh back on any future purchase and i kind of thought being an economist i thought well that doesn't exactly sound like one percent back doesn't sound like it's the kind of thing that can save the company and i said um and uh why do you think that uh well what effect do you think will happen i said uh we believe this is going to incl increase our revenues by 10 and i said really 10 uh that seems pretty extreme i said um you know what um why do you think this is better than just giving lowing prices by one percent because lowering prices by one percent they don't have no impact at all this is a cash back problem this is going to change everything and i said but but i'm really confused why why do you think this is going to have such a big impact because like economically theory-wise you wouldn't think it should be any better probably worse than giving people cash back because they have to wait a long time to buy it again and they said well because we ran a randomized experiment and my eyes got wide open because it was a firm that didn't seem that good for my first initial interaction with them and i was really surprised that they had run a randomized experiment i said so you ran a randomized experiment where you gave people um this frequent buyer card and revenues overall firm revenues jumped by ten percent i said yeah that's what the experiment showed and um so it just didn't seem possible so i just kept on asking questions i said so just let me let me ask you a little bit um so how did you find the people i guess you you have a credit card right now that people have so you must maybe send letters to people in the mail who had your credit card they said yep and i said i guess it's a randomized experiment so you divide it into a treatment group and a control group 50 50. i said okay and i'm guessing what happened next is that a bunch of people who got that letter inviting them to be in the treatment group a bunch of them never got back to you and you know maybe i don't know maybe um you know 15 percent of the people said they wanted to be part of um you know to get this this cash back card they said about 10 10 and then what you did is you waited maybe three months and then you added up all of the sales of the people that 10 of the people who would sign up for the car for the freaking fire car and then you compared how much they bought at your store to the people who were in the control group they said exactly and they bought 10 percent more yes and we're so excited and that's why we invest 300 million dollars in this and our roi is gonna be like five to one given our results and i said um how hard would it be do you have access to those data now these were the top data people the company and sure i said could i just like is there someone here who could go and run one little number for me and um they said yeah i said could someone just go and and just find the 90 of the people who got invited to be in the treatment group and who didn't want to be in the treatment group and i actually have a prediction and the prediction is that those people bought about one percent less than the people in the control group like why would you say that they'd like they're the same as a controller i said well maybe they're not and um so if someone go run that that'd be great so about 15 minutes later uh people can one person came back in the room and said okay so i ran those numbers and um and um you're exactly right the the people who were invited to be in the treatment group but didn't get a card their sales were one percent lower than the people in the control group and i said so do you understand that that means that your your card didn't work at all that ten percent of the people increased they sell their purchases by ten percent and ninety percent reduced it by one percent and so actually on net it had no effect like what are you talking about that they said to me what are you talking about we did a randomized experiment and the people who got the card the revenues went up by 10 i said yeah but that's not how randomized experiments work they said yes yes it is randomized anyway so [Music] the day um you know came to uh an end and we didn't resolve that issue but they did hire me to work for them and so i told my team hey look we've really got to explain to these this company they're they're wasting 300 million dollars on this roll out of this car when the only reason that they think it works is they ran a completely flawed randomized experiment they didn't think about the fact that when you run a randomized experiment and people don't want to and and you have a treatment group and a bunch of people opt out that you got to take the whole treatment troop against the whole control ground like the easiest simplest thing in the world so let's put together a five page powerpoint presentation that just explains it there's obviously some confusion so we put together that powerpoint presentation and uh we sent it to them and they said look i don't know what's wrong with you guys you're like supposed to be good academics but you don't understand we did a randomized experiment like i don't you guys don't understand so i was frustrated by the look we got it this is super important there's 300 million dollars that they're about to waste on an ever doesn't campaign we have to get so so my team put together i said look we put together a five-point powerpoint presentation let's just put together a 50-page one let's just walk through everything you could possibly not understand about this thing and we did it and we sent it to them and they said why you know we're paying all this money why you keep on sending us these stupid powerpoint presentations that are totally wrong and we would argue back for them and then just to show you how crazy business is um it was finally decided since we believed that they were totally misinterpreting their experiment and they believe they're interpreting it wrong right they they were german right that we would have it decided by the ceo with no data science background at all he would be the judge of the jury to decide whether who was right and so we literally went and we sat down in front of him and they gave their side of the case we gave our side of the case and everyone looked at them to determine whether or not the results had shown that indeed this this card had been a miracle or not and um he very hesitantly kind of mumbled a little bit and then he said well i kind of think um i'm with um professor levitt on this one okay which was amazing because it was going to save them 300 million dollars in what was going to be a completely failed uh rollout of a card to which immediately the people on the other side said well the experiment the randomized experiment had nothing to do with why we were pushing this um this project anyway we have a bunch of other reasons that are much more important so despite the fact that all of the evidence they had suggested this card would have no impact and despite the fact in the end the ceo had voted on our side and acknowledged that it would have no effect they went ahead and they spent 300 million dollars on the rollout and just as expected they saw no impact on their sales just in indesignable increase in their sales and this retailer is if not currently bankrupt uh really really close to bankrupt uh one of the formerly great companies of this country uh so why don't i tell a story i tell this story because i think it is so indicative of when i experience the world and i go out in business or just in life i think the level of data proficiency and fluency in our country is abysmal it's uh really an embarrassment and it is my belief that there's virtually nothing more important we could teach children our current generation children then to be fluent with data understand data to know the difference between correlation and causality to um understand randomized experiments and the the pitfalls of randomized experiments and the difference between a randomized experiment and a an epidemic epidemiological study where you when you observe people to frankly just to be able to to ask good questions and then to assemble a pile of data and understand the data is dirty and has to be cleaned and to do simple analysis and visualization to try to understand the world around to me that is a baseline skill for functioning in a modern economy and it's one that very few people very few people have so that's my belief that that we have an obligation to our children to be teaching data science i would argue that in reality we are essentially not even trying to teach data science that um we have something like um sorry my bad completely if okay we are not even trying to teach our kids data science uh we when we talk about data science um so we generate some data so on our free economics twitter feed we put out a survey to people this is obviously not a random sample these are the kind of nerdy people who number one uh listening to freakonomics radio and number two are going to take some time to do our survey but we asked them about the kinds of skills and things that they use and what kind of training they had gotten in school and virtually none of them had received any kind of data science training in school um and then we went ahead and we looked at um on what kind of map stuff was my world being taught in that and it's virtually no specifics even is being taught now i'm not really talking about statistics uh when i say data science because i think that most of statistics is as taught now is actually kind of an older version of statistics which is very theoretical very abstract not really about data so much but about almost what you do when you don't have data right when you have distributions and you want to make you know asymptotic kind of assumptions but even among general students the level of taking statistics in in them as in part of the math course is something like seven percent in our data and then we went to the ufc and the applicants the university of chicago and we looked at the very elite how many of them are being exposed to things like statistics and and there again the amount of time even among these elites the number of years they were doing in data related subjects was um you know maybe 10 of of um of the math they were doing uh if i try to do a rough calculation based on all this of the amount of the day that is spent by our current high school students doing data related activities at school my guess is it's about five minutes a day um and that which that which is being done is being done almost exclusively through the science track it's interesting that although to me it seems more logical that we would be integrating data science into the math track that in practice it has really been subjects like biology as an example ap biology was completely redone maybe three or four years ago with the new emphasis on data so it's not impossible to um to bring the data but i think the facts are i think we can all appreciate that really data science data analysis hardly being taught hardly being taught in our current schools and kind of the irony of it is it's being tested so um the sat has put a lot of data analysis into it even in the reading section now um ten percent of the questions are data related the act science test is essentially call data analysis and and and what's interesting is in general economists have found that teachers are very quick to teach to the test that when a test test a particular kind of material that the curriculum very quickly follows but for whatever reason in in terms of data it has and i've seen almost no changes in curriculum in response to either the sap changing or the act changes towards increased data time so i think it's interesting to ask why why is it that we don't really teach data science well i think the first answer is is pretty obvious well um let me go back let me say one thing so why i'm sure somebody's so important why don't it's important i think it's important for a number of reasons okay number one in terms of jobs if you look at what the department of labor puts out as forecasts of growing jobs and in indeed good well-paying growing jobs they overwhelmingly are stem of course but even within stem they are intensively data oriented um the future in the future workplace is going to be dominated by data if you think about it um not just jobs that are strictly data related but the job of a manager has more and more become a data related job that good managers have to be able even good sales force people are often much more highly reliant on interpretation of data and the way they do the selling blue collar jobs think about how the auto mechanics job has changed over time the modern car is is a computer in many ways and what auto mechanics used to do is change dramatically as cars have changed the future there's no doubt the future should belong to jobs that involve data but it's not just about jobs i think it's also about living in modern society health care decisions health care choices with the availability of data and the complexity often of the environment which you're asked to evaluate health care plans financial literacy i think is important um civics being able to interpret what politicians are saying and what's in the newspaper increasingly rely on data and i think uh the the populist movement that we've had that has elected a very anti-science uh anti-intellectual president i think all of that can be in part trades back to our inability or ineffectiveness and teaching people about data all right so why why don't we teach people data okay so i think there are a bunch of obvious reasons why we don't teach people currently very much about data the first one is the data are new we used to live in a world that had a sparseness of data and just you know the statistics are amazing that in the last two years we've produced as much data as mankind produced in all of all of history prior to that and um if you think about going to the moon the kind of computers we went to the moon and how we did that essentially almost without data it's just um it's been a revolution it's one of the greatest changes that we've had is the uh the rise of data okay and curriculum moves slowly and the curriculum that we currently teach more or less existed 40 or 50 years ago hasn't caught up with uh with data a second a second reason that we don't teach data is that it doesn't fit easily as a centerpiece of any one current subject okay so um it it's adjacent to it touches with it improves upon virtually every subject from biology to social sciences to statistics to physical education i don't even think about any history and you can bring data into any kind of subject but but the people who teach those subjects are often not trained in data and it really isn't the centerpiece it's a it's a tool and so i think we've had a hard time thinking about where to put data um and i think the third reason we struggle with data and teaching data is that nobody really agrees what we should be teaching or how to teach it what data science is and i really want to kind of super important i want to contrast that with computer science so um unlike data science which i think is any push to really teach data science has been has failed i think computer science has been reasonably successful at integrating into the curriculum and part of the index because i think almost everyone can agree what we mean when we say computer program and i think we also have a fair amount of agreement about how to compute a program how to teach computer programming uh well but we're data science i think there's both less agreement on what the subject is what we should be teaching how we should be teaching it and that creates an obvious and obvious difficulty okay so all that being said what is my recommendation what do i believe we should try to do so i believe we should try to integrate data science and uh so i put together with joe bowler who's uh education professor at stanford we put together a big conference to bring together many leading mathematicians high school math teachers college level mathers trying to understand how to um how to think about integrating data science into its existing curriculum and um it was a passionate group and we we met for two days but i've rarely been less depressed at a conference than i was at this conference okay not because we didn't have amazing people and not because we didn't have great passion and have an important project but because the nobody agreed on whatever literally every person in the room had a different idea and what struck me was that it was really these ideas were not you know well well good-hearted had no real chance of happening okay so what i did learn which i was surprised is that many of the people who are the strongest proponents of data science are have a deep deep hatred for calculus they think that calculus somehow in the high school curriculum has ruined everything and they see data science as a way to knock down the preeminence of calculus it was not my feeling at all like i i i would if i had my brother's student more of all stem right more science more more engineering more and more everything okay so i'm not thinking necessarily that that we want to like kill calculus and plus it's completely implausible to try to kill calculus because the universities rely heavily on calculus and the elite universities demand capitalists and you can't unilaterally get rid of calculus unless you get the universe to say they want to get rid of it and i've talked to presidents at universities and admissions directors universities they're not willing they're not interested in getting rid of calculus and i don't think they should be because texas is important for certain certain paths all right so i think an idea that you replace calculus with data science is a total non-star the second common thread that uh i heard at the conference is that we should have separate tracks we should have a data track and a calculus track and we should let students diverge and um and have choices about that okay and i actually think that's a really bad idea for two reasons number one i don't see any reason to believe why students or their parents will make good choices at the age of 14 or 15 or 16 about whether they should learn a lot about data or the way they should be learning about catholics i mean it's not that well informed not that obvious um i'm not sure what i would have done for instance secondly i think that is just a recipe for turning data data analysis data science into a remedial program a program that is only given to students who um who aren't aren't thriving in a current math program and that would be a start but i think it's a wrong start because i think data science is important to everyone literally so when when we did when we did our study our survey of the piconet group we asked people these were you know highly technical people um many of them scam people and we asked them how often they use different kinds of math and about eleven percent of people said they used algebra some kind um four percent they used said they used calculus even less used trigonometry jumping but twenty three percent of the data people said on a regular basis they did data visualization something like 40 percent said that they analyze data sets and databases um 70 roughly used excel people are actually using these tools and these are tools we should be teaching and these are talented people you know successful people use these tools and to think that we should make data science a remedial remedial path track just didn't make sense okay the third common theme that arose when people were talking about well how we should change freedom was that we should have integrated math okay which probably is a great idea the idea that you shouldn't teach geometry separately than algebra separately than data something encountered we should try to put it all together and make it question focused which is fine which is great it's just the difficulty implementation was of overhauling an entire math sequence and trying to integrate just seemed to me totally hopeless so my team and i went back after this conference and we just thought is there another path uh and and i think we settled on something found something which is really simple and straightforward and uh has a lot to say for itself so here's the idea right now almost every student in america high school student will take geometry a year of geometry something like that in a year something like algebra 2. and um there's a lot to like in both of those classes but if you look carefully at them i think even the biggest proponents of those classes would say that there's a lot of archaic material in there that there is a lot of stuff that is being completely outdated by computers that that much of what we do in algebra 2 is is about calculating calculations that you're never going to do in your entire life because with a calculator in your hand or computer around you're never going to need to do these old style calculations so it's like just a contrast that if you've seen that movie hidden figures there once was a time when people needed to do really hard calculations that time ended 50 years ago but we still teach students like it's important okay so um the incredibly simple solution that we're proposing that we're pushing is that we should take the year of geometry and the year of algebra two and let's just take the high points let's let's take all the really good stuff in those two courses and let's just combine it down into one year okay and i think we would still give students an excellent education in geometry and algebra too uh maybe better because we wouldn't have so much extraneous stuff and that would just simply free up a year for the high achieving students a year on the way to campus where we could start from scratch we could do whatever we want in my own personal preferences i would uh i would probably put in a bunch of data science a little bit of statistics i would do some computer programming and i would do some financial literacy and who knows what i think you know with a year there'd be an enormous amount of space okay but other people might have very different preferences and and i think whether we whether we actually make this data scientist or or um or more programming computer programming or whatever it will be straight statistics i think it's got to be better than we do right now it's got to be that the best part of computer science and data science and financial literacy are more valuable to students than the worst topics in algebra 2 right now and in geography and to make this work would be incredibly simple we've already got algebra two textbooks and geometry we just have to cut chapters out only teach someone uh the hardest part is creating a new curriculum that incorporates whatever we would try to teach data wise and uh there are some curriculums a really nice curriculum put together called ids that was put together by um folks at ucla with the help of the national science foundation uh i think we do others i think if you wanted to indicate financial literacy there'd be some work to do but but i think it's worth some effort uh and some collaboration with publishers and um some some work obviously with the uh the test makers because uh testing stuff people won't teach it but all that i think um goes back to the basic point that uh the future the future belongs to data and uh and the longer we wait the greater risk we are uh put ourselves at for our our national competitiveness and really for the health and well-being of the students that we're trying to do up okay let me stop there and um take some questions which i think that kelly has been collecting here for me see what we got um so maybe i'm wrong kelly maybe you didn't collect any questions for me uh can i discuss computational thinking so i'm not sure if i could discuss computational thinking or not i'm trying i'm always really nervous about talking about stuff i know less about than anyone else in the room and this very well may may fall into that category but um i was with conrad wolfram for mathematica not too long ago he was at our stanford conference and he presented us a curriculum he's put together that i think you would say is based in computational thinking and i have to say it was really really interesting um so i don't know how to i don't know exactly how to characterize uh so computational thinking i guess i don't know what people mean by but i think what they mean is that you've got computers so don't teach people things compute don't teach humans to be bad computers teach humans to do the things that computers are not good at and to think about how to use computers effectively so um so so i like i think the idea is solid i think the idea that we shouldn't be spending our time teaching people to be bad computers that's very sensible now again i i'm also very pragmatic in the sense that um i think about what i use every day and i think it's helpful to be like some people think you don't really need to be able to multiply and divide and take fractions i disagree with that because um uh because i do think that in everyday life i use stuff like that simple things like that being able to take square roots all the time but uh so computational thinking as as conrad wolfram at least envisioned it involved um a fair amount of heavy programming uh integrated into questions with amazing case studies and and i think he did if you can i'm not sure how you can find his curriculum um it's through mathematica obviously which he's part of and requires mathematica it's been adopted in estonia interestingly um but so this is obviously a rambling answer but to me i don't think computational thinking isn't how i would organize data science um or maybe i don't know the definition of it but um but i think it's an int like i take that if somebody put that in there i'd be happy to have it in it but it isn't the way i myself would organize it okay let's take another uh question here because how do we connect the dots from all the subjects so we discussed data science and we realize the importance of it with the real world most teaching occurs in silos so how do we as educators work up uh work to include it in science social studies etc more than just a chart of media like so that is the 64 million dollar question the billion dollar question how do we do that um because i do think it's exactly right that that data science lives and thrives and breathes as a tool within uh different disciplines so the way i use data and analyze data in economics is very different than the way someone might do that use it in biology or even in psychology so you know in economics we typically can't run randomized experiments so we built a bunch of tools like what we call natural experiments that are that are attempting to kind of tease causality out of data that weren't designed to generate causality okay so here are my thoughts so the first thought is i think without some core knowledge and teaching about how to analyze data i think it will be a challenge to bring data into these adjacent fields okay so that's why i believe that we actually need something called data science where we where we teach the basics the basic features of um how to um analyze data okay once you've done that okay then i think the challenge is twofold one is teacher training so most of the teachers in the subjects which are exactly data are um are not well versed in using data themselves um and so uh i think we need to provide teachers with the tools to do that and do that well okay but i also think that's not so hard i think that um the at least the evidence that i've looked at with ap bio okay was that by ap bio features at first when the college board introduced lots of data into the new curriculum uh we're unhappy and challenged but within the year there was an enormous groundswell of materials and peer support and training to help do it the other thing we need is we need curriculum we need examples we need we need the textbooks to be building and data and or we need outside sources we need repositories where we have good data and um i think that that's um that's really the job um so so one thing i i haven't talked about is if you think about how computer science has i think done a really good job of integrating beginning to integrate into the high school curriculum i think they've done that in part through national leadership so groups like cs for all which uh have provided a lot of the thought leadership and materials that that that's needed and i would love to see something like that emerge for data science where we actually had a group that was taking the kinds of questions you're asking today and thinking about the right answers to them and bringing the right groups of people together to try to agree with that but i do think it's hard like i'm sympathetic to the fact that i'm a science teacher i'm in high school i'm trying to teach my kids data how do i do it how do i do it when my students don't know about data so i have to say the beginning and when the materials are out there to make it easy for me to do so all right next question how would you consider melding mathematical modeling with statistical reasoning as a poor learning experience for all students so i do believe that all students should be closed and let me go back i didn't say so so maybe one of the criticisms that people um who are listening would launch say look you're acting like the only reason that we teach people stuff in school is because it's useful like you keep on saying people use data science all the time but but the purpose of high school isn't necessary to teach people a set of skills like excel that they can use on the job it's to give them a worldview a way of thinking it's achieved something you know we teach we teach algebra and geometry because it's beautiful because it changes the way you view the world it inspires people okay so um let me say i don't disagree with that that much of what we teach in school whether it's history or algebra geometry you might never use again but it has can have other purposes in terms of brain development in terms of inspiration but i also believe that just because we use data doesn't mean that it can't be as brain developing and as inspirational as other more arcane topics okay so i do believe every student deserves a shot to learn about data and i do believe that we can connect better with many students through data than not through data uh one of the examples that that i did a free economics radio podcast and one of the examples of a professor that um i interviewed who had um taught eighth grade math at a low-income school and she tried to find any data in various ways and um one of the things she did is she told the story about one of her students probably the worst student in the class so this student whose only engagement with her was the student would repeatedly ask if she couldn't give um this teacher a makeover because she was like so poorly put together and she wanted to help her look better so they did a david a data practice where every student got to choose a topic and this student did chose her topic we wanted to analyze the quality of various i think it was eye shadows and so she in a very meticulous data oriented way uh went through a range of eye shadows and evaluated their quality and it was the one time that the student engaged with uh with coursework the entire year so i do believe that um that data are good entree into getting people engaged okay so um how to engage mathematical modeling and statistical reasoning again i don't know maybe not the question that i can give you that i'm not the best one to give you the answer to i think there are different approaches to it so um the the approach that i use with my student is um and european university level students is i like to first talk about theory it's not about a theory my own viewers i like to put all of the theories on the table okay so i have an absolutely different approach to mine and many people many people follow reasonably like the scientific method which is to propose the theory and then they try to suppose a hypothesis and they try to test that possible data and then you can't obviously prove it there but you can disprove it i actually i actually teach a different way with my students i like to try and come up with every possible plausible theory that a reasoned person could hold and then i like to think about the predictions of each of those theories and try to understand um then how do i distinguish between competing peers because in economics unlike much in science it's usually not there's like a single theory or a theory that we have and then a theory that will beat it it's often you know a bunch of theories about how the world works and in economics what matters might call like a partial identification is the words we use for it but um but i think in that world um it's a different modeling approach than you would take in in another setting where you want to do something more like the scientific method so what i do with my students is i then try to give them a pile of data and i say among these five theories what do you think the data support the best okay and and what almost always happens in messy social science data is that there'll be hopefully four five six predictions across different theories and uh uh and you know three of the theories might all agree so like um you know almost every economic theory is going to say that the man slumps downward um and so that won't distinguish between theories but it is a prediction of all the theories and so uh what they usually find in the data is that well you know one theory maybe four of the six predictions is borne out in another theory only two of the six predictions are going on okay and so the kind of inference you draw in a messy world is very different than the kind of inference you would draw in a world in which uh you can you know in chemistry have predictions that um you know the ratio of molecules should be exactly you know uh two hydrogens for every oxygen there i would take a different approach i don't think there's one size fits all but i do agree with the principle that um the integration of a modeling approach of thinking about how the world works from a theoretical perspective and then wedding that to actual data and how the data fit is is an incredibly important endeavor all right so uh next question why wait until algebra two why not start at grade nine when students are beginning high school studies so we reach all students i mean so you're totally right and let me go even further why start at grade nine uh we actually have put together a a list of um going all the way to kindergarten of incremental steps you can take we introduce students the data all the way k through 12. so um i i certainly think that's right now um just to be clear by melding geometry and algorithm two it doesn't mean that you wouldn't teach data until um say junior year so you could you bought yourself a year it could be in a year um so yeah absolutely um i think that you could um you could bring data science in and i think it's a really good idea but my own belief is the earlier my own belief is if you let me bizarre educations are you know dictator whatever i would bring data in from the beginning and i'd do it intensively and i would make it the focus of um many things okay in terms of realities of how we do it i think the analytical geometry is a good realistic way to get it done okay but i didn't make clear you could then make data times like a ninth grade math class and then um do you know two and um geometry in 10th grade or 11th grade it depends how you're going to do precalculus okay but absolutely look the more the better the earlier and look i'm happy and um i'm happy to let people say that i'm wrong and that we should be introducing data science some other way uh i certainly don't think i haven't even have the right way to introduce data science uh what are the current policy next question what are the current policies that you implement and suggest to engage students to take stem courses what's your policy recommendation uh adapted by your country was accessible or so um so i think i think incentives matter i'm an economist and um obviously economists think that um you know independents matter and when i think about um policies that um so sorry i've been distracted i mean i have i don't know if other people have i had no explanation i don't know if there's anything you can do about that um but uh so how do you engage to take stem courses you need to have incentives okay and right now we have very weak incentives um because we hardly test stem courses right so um to get into college the emphasis is on reading and math um and a particular kind of math and uh i think that that the the more we would make high stakes testing you think they're going to rely on high sex testing if you have high stakes testing um for a broader stem classes you would have greater incentives to do it um so in terms of um how our positive recommendations so we've made policy recommendations and i think we're actually we're having a certain amount of you know traction not very much but a little bit of traction until coveted i think coveted has been a real detriment to most change in education so i think i think a lot of good changes will come um from the reaction to covet because i think reorganizing the classroom and and learning what we can about remote learning and and reorganizing how we do things even when we're back in person using the lessons from kobit will be important but in terms of other kinds of changes to curriculum i think the bandwidth and the tolerance of school leadership and government to take on more changes mean that it's really really until it is gone or at least you know under control i think there will be a lot of difficulty in getting many changes at all to to what we're trying to do in terms of stem um another question should we move towards creating a separate data science school subject or integrated for example in subjects such as physics so i take whatever we can get anyone who can figure out how to get this in data science in i would take it my own view is that um it's hard to integrate because it's just i don't see the obvious place to integrate it would be my statistics is the obvious place to integrate it but it doesn't really do any good to integrate with statistics because nobody takes statistics at the high school level anyway um so among the subjects we teach so for instance but i you know my feeling is that physics is not the right place it just doesn't feel like the right place to be because physics at least as i see my high school students being taught it is completely and totally theoretical um there's a zero uh data component doesn't seem natural at all and it doesn't i you know i think physics teachers might not be the most natural ways to attach it um but look if there was a school district that said hey i think physics is a great way to attach data science let's take half a physics or a third of physics and make it a data science class i think they'll be fine okay again i think it's not ideal because physics comes so late in the career um comes in in almost hours in in the 12th grade um you know in many ways i honestly think it wouldn't be completely crazy to rethink how we do the social sciences um social studies history and just think about attaching data science there um you know i think i think just like math has not kept up with developments in the world i think social science teaching in high school has not kept up uh the fact that we have you know such a strong emphasis on history as opposed to on economics or even political science or um or or various data related um social sciences i think that would be a crazy place to to try to um to integrate data science as well next question do you think that is necessary for data science to be included in the curriculum of high school education will that enable our teachers and learners oh high school will will that enable our teachers and learners to understand better the different sciences um so i absolutely do um i i just i just feel like there's very little else as important in day-to-day life managing day-to-day life as not to be a data scientist i'm not saying everyone's gonna be a data scientist i'm just watching people around me like i have a nanny and when when i'm like quizzing my my you know 16 year old kids in my nanny about data like simple things about interpreting what we're seeing around us um and i see them struggle and i just wonder how they can make good decisions in life when they don't understand the basics i mean call people to understand the basics but it just really strikes me it's fundamental it is fundamental to me in the modern days of reading and being able to do you know multiplication tables is to be able to have some ability to make sense of data some ability to make a make a graph or to to understand the difference between correlation and causality um so let me um let me stop there on on what i was complaining let me end with a story so i feel like somehow i've um i've cast a very negative net first with my example um about this retailer who didn't know how to do data and and um and i keep on saying oh it's so hard to implement them but let me at least let me be aspirational uh as an end to this presentation and let me tell you a story um where i've seen the power of data science and of stem more generally okay so it actually started this first starts with the conference not too different than this it's same kind of people showed up to the conference it was live it was live in chicago at the peninsula hotel and in the q a session somebody asked me what i was working on and this is 10 years ago 15 years ago and it turned out i just started a project collecting data on street prostitutes in the city of chicago and so um along with a sociologist we put um trackers out with clipboards out on the street corners where the prosecutors were working and we paid them a little bit of money and they would just each time they did a trick we would record a bunch of information okay but that's actually the point of the story there's an interesting study um for instance we learned that chicago police officers were more likely to have sex with an on duty uh prostitutes are more likely to have sex with an on-duty police officer than they were to be arrested by an on-duty police officer okay but that's not the point of story the story is that um one of the people in the audience that day uh was in from out of town and he wanted to experience some of what chicago had to offer and so he had um hired a call girl and when he showed up it turns out i found out later when he showed up at her apartment where she did her tricks my book freakonomics was sitting out on the coffee table and so to make conversations and oh my god i just met the author of that book today and um and he's doing a studies collecting data on on prostitution chicago so much to my amazement the next day i got an email and it came from this call girl and she said oh i understand you're you're doing this this project on um collecting data on prostitution and i'm a high-priced call girl in the city of chicago and i've collected an enormous amount of data on my palm pilot on all the clients that i've ever served are those the kind of data that you would hope to elect and i wrote her back immediately and i said absolutely that's a god i'd love to have your phone by the tip so we met for brunch and we sat down and here's where the story gets even more interesting it turns out that prior to being a call girl um this woman had um studied computer science in university and she had actually worked as computer programmer in the us military on the star wars defense system she went to work at a fortune 50 company as a computer programmer and she was making about 80 000 a year doing that and she just decided that being a prostitute would be a better life and so she took her stem uh training and she put it to use she built her own uh web page it's 15 years ago when it wasn't that easy to do and people didn't do it that much and she um started her business and when i met her she just could not have been happier she was making about two to three hundred thousand dollars a year working something like 15 to 20 hours a week as a call girl and um she was just effusive about uh her life change so um i began to um you know here we were you know we talked about getting her data and it actually was pretty simple to figure out how to de-identify a date or whatnot it only took about 15 minutes but here we were sitting at brunch and i i'm kind of awkward in general i didn't know what i was going to talk to her about so so i just started asking her all the questions that i was asking ceos like i asked this year's video just like you know i learned to kill time by asking people lots of questions and plus i really wanted to learn more about prostitution i didn't know that much about it and um and she really told me honestly gave me better answers um to most of my questions about the economics of the business and did most of the cos i talked to but i finally tripped her up when i asked her how she set her prices and she said um well i didn't really know what to charge i just kind of went on the internet i looked at what other women were charging and some were charging a hundred dollars and then we're charging 500 so i just decided i'd charge in the middle three hundred dollars and that's really so agitating to economist because like the one thing that economists think we're good at is um is knowing how to set prices and and we have simple formulas called the inverse elasticity rule where if you know the um you know if you know your marginal cost you know the less they say demand you know exactly what to charge it's a pretty easy problem to go and try to experiment to figure that out but but nobody does almost nobody does that even in business people don't do that um so then i was thinking about you know could i give her advice about her price and she told me she had this dedicated phone line that um only her her um her clients called her on and so i asked her well when the um when the phone rings with your client calling you how does it feel well you know sometimes i'm excited but a lot of times you know i don't even pick it up you know i'm just not in the mood really and um and um you know so i'm kind of different i'm like my god you know i'm you're you're a local monopolist um if you're optimally pricing uh you've got to download something to bankrupt so you should be you have a huge markup you should be so excited when the phone rings like the best sound in the world should be your phone ring she looked at me confusing you know confused and i i didn't i'm sure i didn't explain it very well at the time the logic like i didn't explain it very well this time either but um then i just like came back to my sense and i really looked i'm not i'm not here to maximize her profits i'm just trying to get access to her paw pilot so we parted ways i didn't think i'd see her again but um but uh but but i teach a course on economics and crime at the university of chicago and um and because i was studying prostitution i decided to add a new lecture and and as i tried to write that lecture i just realized it was really hard i was like really struggling in writing that lecture um and suddenly i had an epiphany i said well why don't i call my new prostitute friend and have her come guest lecture to the university student's forum so i called her and said hey um professor 11 how would you feel about coming in and teaching my my lecture for me down at the ufc oh no i i couldn't do that i'm a very private person and i'm a terrible public speaker i really couldn't do that and um the thing is i know enough about prostitutes and economics that we have something in common which is that in the end it's really all about price right it's like like for the right price we'll do anything so i said well what if i charged you what if i paid you your hourly wage to come down and teach my lecture said oh oh i misunderstood i thought for my hourly rate i'd love to come to class just tell me whenever it is i'll be glad to do it so um we arranged it and she came down and she gave the most amazing lecture to my undergrads she was thoughtful she was analytical and people loved it literally a third of all the students in my class told me later that it was the single best lecture that they had received in their entire four years at the university of chicago which is a pretty sad statement about what me and my cloud colleagues are doing in the classroom but um after she gave her a lecture with the q a and one of the students raised his hand and said um oh how much do you charge and she said well i charge 400 an hour okay and i became so irate because i knew she charged 300 an hour and i hired her for two hours and i like very carefully put six you know crisp 100 bills into an envelope with a nice thank you card for and then she told the class that she charges four hundred dollars it's like lies to the class and says she charges 400 an hour i'm furious because it's like not like i can use grant money to to to do it's like i couldn't go to the national science foundation and say hey i'm too lazy to teach my own lectures so i hired a prosecutor to do it can you give me 800 and literally coming out of my pocket is 800 and it was just so aggravating too that i felt like we had this relationship of trust and then she completely violates my trust and lies to the students and but i still felt like i had if she said she earned 400 an hour and i told her i'd pay her hourly wage i have to give her 800 and so anyway i'm so angry fuming in the side of the room that she's like ripping me off like that and then the next student raised her hand and said well how did you um how did you decide how much to charge and she looks at me and she has this big beaming smile on her face and she says um well you know i used to charge 300 an hour and then the very first time i was with professor levitt he convinced me my services were far more valuable than the 300 i was charging he suggested i should raise my prices and so i did and i ran a little experiment and i raised my prices and uh it turned out that um my revenue went way up when i raised my prices that my demand was extremely inelastic and uh so i tell you um meeting professor levitt was the most important thing that's ever happened to me in terms of business so look i don't know if uh what's the what's the message of that the message is everybody deserves to be taught the power of stem the power of data and if you can use those skills to increase your profits in prostitution you should be able to use those skills and everything thank you so much have a great conference uh let's all work together to make stem uh central to the lives of everyone in america in the world take care well wonderful thank you so much steven for that i mean it was just great to hear dr levitt's perspective um so i just want to welcome everyone to um to the summit and i just want to thank you for your participation and joining us so i wanted to give a little bit of history because for many of you this might be something new here and i will have to tell you right off the bat this is new for us right we never thought that we would be in this global pandemic and dealing with the situation so we're going to make this engaging there's a lot of learning here but i first wanted to start with um a little bit of the history about the stem leadership alliance summit and i think this context is really important because a group of visionaries actually came together to really think about what is stem and when we started to engage in this conversation we felt that stem was an acronym it was this word that was being thrown around that what did it really mean and so as we started to do a deeper deeper dive and engage in a conversation the importance of integrating stem across all subjects areas was really important stem should not just lie within the sciences science certainly should be the hook but really what it should be about is having those connections where science brings in mathematics and not just as this histograms or measurement but something so much more and so much more meaningful and that technology should not just be a tool or a device but technology should really have a purpose to thinking about community problems community issues how do i think about what i need to solve for and then when we think about engineering it shouldn't just be a project engineering should really be it should be solving those problems it should be pulling all of this together but the real magic happens with stem when you start connecting that to literacy you connect that to history you connect that to art into other subjects there so that it becomes a holistic approach and that students aren't saying well why am i learning this there's no relationship there but really starting to think um there's these connections so that's what i want to really emphasize is that the whole goal and purpose of what we're trying to do here today is to break down those silos you're going to be hearing from leaders across the board around stem education and they're going to bring a perspective that's unique to their mindset to their their area of of passion and commitment that they have but the magic happens is when you take all of this learning you create those connections and you bring it back to your environment so you need to think about how do i take the learnings from this virtual setting and start to apply this share this with my colleagues and really start to bring this to the next level here so in a few moments you're going to be hearing from steve barbados and steve barbados is going to talk about he's the executive director from ipea and he's going to talk about those three principles so we're going to be engaging in this conversation and you're going to start to see these connections happening where we bring in david barnes is going to talk about the mathematics we're going to hear from one of our science group gurus christina royce and then we're also going to have the opportunity to hear from asce geraldine gooding and her colleagues as they talk about the engineering and bringing it to this higher ed experience but here's what i want to say is when all of these people come together and they start to focus on integrated stem it becomes the magic and starts to turn around from being just science technology engineering and math and being conducted in a silo approach but rather that we're starting to see the synergy happening together the other part that i really want to share is that what we're trying to do is we're trying to think of this in terms of phenomena based learning and taking that because that can lead to project based and then it gets students to figure out the problems that are happening there so when we talk about this we need to think about how do we get students to ooh and ah and wonder and get excited about the world around them and the only way that we can actually do that is when we start to connect the learning so that's the challenge there that's what we're really trying to push for all of you is to engage in a deeper level of thinking that it's no longer just about starting in the middle school area or the high school area this is about starting with all students at an early onset we start from pre-k we move it into elementary middle school high school and then on to college so thinking about this across the board here and the other part too that we have to challenge ourselves when we're listening to the presenters today is this blended learning model we don't know what the school system is going to look like so here in the us when school starts back in the next month or two we don't know what it's going to look like in many countries around the world we're facing the same problem here the audience we have here today we have over 45 u.s states represented we have over 15 countries represented yet what makes us so unique is that we're all facing similar problems here so there's a few things that i actually want to highlight that you're able to do so there is this whole place that is called the lounge that you will see and there's a various amount of tables that you'll see sitting there now throughout the day we have provided some networking opportunities and we want to encourage you to go to the lounge to grab a seat at one of the tables and engage with some of your colleagues because you could actually have a conversation about how are you doing this remote learning or blended learning what worked for you what didn't work for you because the articles that came out about remote learning really showcased that we weren't doing enough so the lounge is a place where you could have that kind of conversation you could have a conversation around stem you could have a conversation around what are you doing in terms of mathematics data science as we just heard but it starts to engage you in a level of conversations there i also want to encourage you to very visit the various booths the booths are our sponsors our partners our learning partners our corporate partners you're going to be able to hear and see and meet with them as well and i really want to thank all of our presenters and we have over 60 presenters 55 sessions it's an amazing undertaking and this would not have been possible without three other very special people who you are not necessarily seen but they are behind the scenes they're backstage and they're really helping to run the show and that's first of all i want to give a shout out to sal fernandez for just being a thought partner on this and to helping to organize everything i also want to thank roy harris and rosa laganna i really could not have done that without all of you um and again i want to thank all of our presenters and mostly i want to thank all of you so the audience we have over 500 people that have registered so far for this summit the messages keep coming in and again we want to continue to encourage this not to just be a one-time summit that's occurring this week but really making this where it's an engaging session that continues through the entire week so i want to also back up for a moment so i mentioned the lounge i mentioned the booths we also have the lobby the floor plan you can kind of see what's going on so you're going to be able to join the sessions you're going to be able to participate in some of the tables that i talked about but now i want to move to the right side so on the right side of the screen you'll see discuss and it occurs with three little bubbles there so discusser chat i want to encourage you to use that button to be um to be able to um making sure that you're asking questions that you're making some comments there because we're going to be sharing this information and maybe you have some great resources that you know about and we want to be able to share that so i encourage you to always visit the stem leadership alliance dot org website because we're going to take a lot of that information that you're sharing and we're going to put it on our website to share as well so again we want to continue to make sure that we're having those conversations there so i want to share just a little bit more about what's happened in this last year so last year when the stem leadership alliance summit met we actually met in orlando florida and that was the kickoff of something that really started to transform the conversation in the dialogue what happened was that you had major leaders in stem education from around the world that came together and started to say you know we shouldn't be doing this in silos as i mentioned early on and they actually had some magic that happened they started to connect the dots and created these integrated lessons and they were able to share them with the participants and all of a sudden we started to see the light bulb come on so we're educators were able to participate in things like making a roller coaster for example a roller coaster that all of a sudden connected science technology engineering and mathematics but it also went one step further why not have someone who designs roller coasters be part of that conversation why not bring the career pathways in so again we want to encourage everyone to think about it not just within school but the out of school component too and you know this is something you can also do in a remote learning platform it's not where it just has to be the educator up front having that conversation there's ways to bring in other partners to bring in other resources that could be extremely valuable for your students so we want to think about that so as we moved away from the summit we actually and i need to give a shout out to our philippines affiliate there they did an incredible job with the guidance from the unilab foundation with lilly beth and then you're going to hear from one of the participants and leaders i should say of the summit the organizers of the summit later on today i want to give them a shout out because we moved the summit into the philippines and so it's hard to believe not so long ago that we were in cebu and enjoying the summit there and we had over 13 asian countries that were participating in our asia summit and to watch that stem integration concept shift not just in the u.s but now starting to move in a global manner there and so again they were able to take it we were able to hear from leaders from all over the world that talked about that stem needs to be more than just this acronym but it needs to be this level of engagement there and so again we're we're going to push this conversation we're going to challenge our thinking we're going to have some little breaks here and there for the networking and we're going to take the summit that we've done now and we're going to move the needle because all of you i'm putting a challenge out i want you to take the work from here i want you to think about how you're going to put it into practice and i want to hear from you our entire team as i mentioned with sal and rosa and roy we want to hear back from you and we want to be able to use this as an exemplar if you're doing some great things in terms of of remote learning and related to stem let's hear from you so i want to thank you and i want to thank the entire team um for helping this the 10 times team has been remarkable there so as you have questions remember to use the chat but visit the lounge visit the lobby we're going to have a lot of network connections there and i just want to take a couple moments here um to share just a couple brief videos there's some great work that's happening out there and i really want to be able to share that with all of you so first just bear with me here because as we all are we're learning on how to use all these new technology platforms but here we go i'm going to share and there we go maintaining an operating infrastructure that brings people warmth comfort and light [Music] our company is pseg which is public service enterprise group what we do is we provide electricity and natural gas to millions of new jerseys [Music] i like to think of us as a company that is responsible for building and maintaining and operating infrastructure that brings people warmth comfort and light but the end game is always the same it's basically to allow people to live the lifestyle they want to is reliable calolo partners is efficient [Music] we are producing approximately 20 of oahu's electricity annually we are burning the refinery's waste oil and we can also burn biofuels and gas if made available to us our efficiency is approximately forty four percent versus conventional plants that are in the low thirty percent range kalaelo partners is here to stay bear with me one moment all right i want to thank everyone and we're going to um give you a few moments here that to become a little bit more familiar uh with kind of what's going on around us here so i want to encourage everyone to test out the lounge uh this is a great networking opportunity and so you can start to meet some of the other uh participants there i also want you to just kind of get used to the floor a little bit so this is um it's a little bit challenging here but we want to try to make this a really real experience and i think you know we've we came into this whole covet crisis here and this whole pandemic of what's been going on we were a little bit unsure and i think this is a way that we still can virtually connect network and meet other people so i'm going to give you a little bit of time to go right ahead so all right oh i'm sorry there's there we go that was the backstage and that was my fault there see we're all learning together so again we're gonna just take a couple minute break here and i want you to encourage to you know if you need to get a cup of coffee or maybe it's the end of the day for you someplace else and you might get a cup of tea and but we're going to still show a couple videos but i want to encourage you to also use the lounge and try some of these other new features and we're going to be getting started back here at 9 30 on the dot um and you do not want to miss these presentations and i'm telling you here we have an all-star lineup you certainly it's going to be hard so we're trying to factor in some breaks and so you can and even have a lunch there too or a dinner but i encourage you to just participate as much as you possibly can i think you're going to really get a lot out of this so with that i want to thank you and i'm going to go ahead and just share another video here eyes ears nose mouth hands experience drives the imagination to places wonderful joyful and mysterious by seeing feeling and touching what is we picture what isn't yet and then questions flash years turn pencils fly imaginations explode [Music] because we're inspired to figure out to invent to create and maybe just maybe to make the world a better place [Music] museum of science there you have it all right so let's just take about a 10 minute break everyone and we're going to get started back shortly here but um let's go to the lounge and let's see who we can network and some of the other folks that we can actually meet this is great there's already some people sitting over there so you can actually grab a seat table one uh table three is actually full there um but hop on over and let's just see what we can find or who we can find i should say you

Channel: STEM Leadership Alliance

Views: 279

Rating: 5 out of 5

Keywords:

Id: fdZL7-OH8mQ

Length: 86min 58sec (5218 seconds)

Published: Sun Aug 30 2020