The Age of Surveillance Capitalism | Shoshana Zuboff | IAI

[Music] so I'm really happy to be here with you sitting on the floor or standing in the back I want to start off by asking you a question it's a beautiful evening it's a Saturday night if I've got that right I'm a little mixed up but Saturday night right so there are lots of things we could be doing and of all of those things you've come here to talk about this I'm curious to know what are the concerns issues feelings that bring you here to this discussion and I would like you to reflect on that for just a moment and see if you can boil that down to one word and then I want you to shout out the word and Jasper is gonna write down these words and then he's gonna pass that piece of paper over to me sound good all right so what what concerns feelings issues bring you here tonight give me a word okay account weight all right what what poor Jasper okay a accountability fear privacy control utopia dystopia I was wondering about that manipulation intrusion interest is that recent interest in what exploitation control democracy misinformation data mining mistrust personalization personalization paranoia like are we paranoid or or are we realistic self-examination commodification okay how are we doing there Jasper I think we've got enough okay now here's what I want to talk about you know this book was published in January and I left home for what was supposed to be three weeks on the road and I'm still moving a lot of people are interested in this and so I've been to a few countries and I've been to a lot of cities and I began the very first event back in January asking this question of that audience which was in Boston since then I've been around the world still going I've asked this question every single night afternoon whenever the event was I want you to hear what my other Jasper's around the world I'm just gonna read you a few lists from different cities okay listen fear anxiety manipulation control identity freedom resistance power democracy law agency rebellion another city fear privacy paranoia responsibility kind of like accountability slavery empowerment defense democracy another city collaboration understanding parasite resistance revolution political prisoners another city sovereignty determinism anti capitalism fear profiling resistance inequality exploitation solutions rev lucien democracy dignity autonomy manipulation sound familiar okay now here's what really interests me about this different people different cities different countries they the surveillance capitalists have one word for us they call us users we are experiencing a range of nuanced feelings concerns sources of anxiety and also things that motivate us and drive us democracy that represent our interests these words are a way of articulating our budding sense of our unique social political economic and indeed psychological interests that we share and we share them not only with the people in this room but with the people in Brussels and Paris and Boston in New York and Los Angeles and Madrid and a whole bunch of other places now I want to draw a parallel here in 1833 there was a lady who lived not far from here her name was Harriet Martineau she was described as a social observer if she lived today she'd be a famous sociologist and she wrote about British society and one of the things that she wrote about was admonishing the aristocracy for the following fact because she said that what was so stupid about the aristocrats view of British society was that they had one term for everybody who wasn't an aristocrat that was called the lower classes and she said inside the lower classes were included the manufacturers the tradesmen the artisans the laborers and the poppers everybody who wasn't one of us was part of some big anonymous category kind of like users so that was 1833 and as you know the great struggle of the 19th century with the emergence and institutionalization of industrial capitalism was how those folks in the so-called lower classes came to understand their political and economic and social and psychological interests and it was out of that the the idea of workers and the working class emerged and it was out of that that the great Titanic struggles of the 20th century were born which were between capital and labor but all of that began earlier in a situation much like the one we are in now I am NOT suggesting that we are about to replicate the 19th or 20th centuries I am suggesting that we are at a moment in history when we are poised to discover our 21st century version of our collective interests and our 21st century version of the kind of the forms of collective action that we will create that will be relevant to us in our time in our moment in history to move this dialogue forward in politics and in society so that's my framing now having said that I want to talk a little bit about what is surveillance capitalism and in order to do that I want to begin with an important distinction the surveillance capitalists have been at this for twenty years which is to say only twenty years that's a good thing it's very young so we shouldn't feel too beleaguered too much you know to resigned too helpless this is still a young thing they have had 20 years of essentially a lawless run unimpeded by law and they have had many strategies that have helped keep law and regulation at bay and also that have helped keep public resistance at bay one of those is a very specific rhetorical strategy and I want to illustrate it before I name it okay so let's dial back to the year 2000 before surveillance capitalism was invented and about four years before any trace of it leaked into the public in the year 2000 in the United States you know we have a great Technical University called Georgia Tech that's you know science engineering and so forth a group of data scientists and engineers from Georgia Tech put out their plan for what today we call a smart home they called it the aware home and they had all the goals for the aware home that we have today like older folks can age in place and be connected to their families we can be have energy efficiency we can have security we can get feedback that might improve our our our lifestyle or health and so forth when they did the schematics for the aware home they picture devices in the wall computerized devices in the wall that would send data but the schematic they drew was of a simple closed loop the data would go only to the occupants of the home and it was for the occupants to know the data to decide who else was gonna see the data if it was going to be shared to figure out what the data means and if the and if any consequences of that meaning we're gonna be implemented in some way it was all down to the occupants and the scientists and engineers wrote in their report obviously this is very intimate data privacy is critical so it can only go to the occupants of the home fast forward to the year to 2017 17 years later University of London - very cool legal scholars decide to do an analysis of one smart home device the nest thermostat nest as you may know was first purchased by Google now it has been completely dissolved and absorbed into Google they decided that any moderately vigilant consumer who installs a nest thermostat must review a minimum of 1,000 quote unquote privacy policies 1000 why is that because the nest thermostat agglomerates data in your home and it sends that to nest aka Google but it also sends it to other third parties which send it to other third parties which send it to other third parties which send it to other third parties almost in an infinite regress no one of those parties takes any responsibility for any of the other parties so your data is out there becoming part of large AI training sets and you don't know what's happening to it that's a whole other story we don't have time to go into that but here's the thing what's the difference between 2000 and 2017 there's one difference it's digital technology digital devices it's the same goals of a smart home what's the difference the difference is surveillance capitalism folks right same technology different economics so my big point here is that the surveillance capitalists have tried to persuade us that what we are experiencing in their territory because they now own and operate the Internet what we are experiencing are the inevitable consequences of digital technology they want us to believe that what they do is the only way we can have a digital future and my whole point is that surveillance capitalism is an economic logic which was an invented in a specific moment in time actually in Google during the years of financial emergency of the dot-com bust 2000 2001 2002 it they figured it out they invented it they elaborated it it took root it flourished it made a ton of money between 2000 and 2004 and 2013 sure capitalists were threatening to pull out between 2000 and 2004 nobody knew what they were doing but applying the new logic of surveillance capitalism their revenue line increased are you ready by three thousand five hundred and ninety percent they discovered the breakthrough to rapid monetization created the surveillance dividend raising the bar for investment in the tech sector so that this model spread through the tech sector and is now spreading across the normal economy in virtually every domain ok so what is it it's an economic logic how does it work I'm gonna be super crash course about this but let me just give you the top line capitalism has always worked somebody said commodification earlier who said commodification okay who said commodification there you go okay come on a vacation that's an important word here capitalism has always worked by claiming things that live outside the marketplace like innocent forests and mountain sides and rivers and fields and dragging them into the market dynamic turn them turning them into commodities that can be sold and purchased and in that sense surveillance capitalism emulates a sort of centuries-old pattern of how capitalism grows but with a very dark twist it turns out we're kind of out of territories to commodify Google discovered what may be the last virgin wood and that turns out to be our private experience Google claimed our private experience as free raw material to be translated into behavioral data these data would be collected from every domain it began online as you all know but it now reaches far beyond online who's got an Alexa in their kitchen who's got a TV that listens who's walking down the street not only passing the CCTV cameras which are public sector but the tech companies have thought for the right to be able to take our faces from any public space without our ever knowing forget about consent forget about the right to combat so now these data are being amassed they are valued for their rich predictive signals they are shunted into complex supply chains of data flows they are conveyed picture those old-fashioned conveyor belts conveying them right into the factory but what is this Factory it's a computational Factory it's called a artificial intelligence but like any factory it makes products in this case it makes prediction products what's computed our predictions of our behavior turns out there are a lot of businesses who have a keen interest in knowing what we will do soon and later first to sell is targeted ads but it doesn't stop there there are all kinds of reasons to know what we will do in the future what price to sell us their service or their product whether or not to give us an insurance policy and at what level of cost whether or not to give us a mortgage or a bank loan there are lots and lots of reasons they want to be able to predict our behavior how are we going to drive that you know that can influence what our auto insurance look like there are many many issues and they arise in every single economic sector all right so these prediction products are sold to business customers in a new kind of marketplace strictly interested in what we will do in the future zoom out a tiny bit these are markets that trade exclusively in human futures that's what they do surveillance capitalists are selling certainty not to us but to business customers based on deep deep data and broad broad data that's about us but that's not for us it's about us but it's not intended to sell art to solve our problems but to meet the goals of its business customers all right now what we've seen here is that we've got markets that trade in human futures you look at the economic imperatives that define those markets if you're gonna compete on predictions what's the first thing you need you need a lot of data you need a lot of data for good predictions because as everybody knows AI likes to chew up a lot of data then it turns out competition keeps grinding no only do you need a lot of data you need varieties of data let's get them out of that out of you know off the laptop let's get them out of the house let's get them in the city let's get them in the car let's get them on the maps let's get them walking around their home let's get their voice let's get their posture you know are their shoulders stoop let's get all of that let's get their faces we like their faces because we like to analyze the muscles in the face there are hundreds of tiny muscles in the face they produce gestures effective computing analyzes those facial gestures predicts your emotions emotions turn out to be highly predictive of your future behavior all of this is very very valuable the market capitalization of Microsoft Google Facebook and now increasingly Amazon depend on the revenues that throw flow through these markets a leaked Facebook document in 2018 revealed what's going on in its AI hub Facebook describing its own operation we don't get that many glimpses into this what did it say our AI ingests I'm using their words ingest trillions of data points every day it trains millions of models AI models predictive models every week and is able to produce six million their words not mine I'm going to put them in air quotes 6 million predictions of human behavior per second per second that's what's going on in the factory okay surveillance capitalism puts us on a collision course with democracy they understood that they needed scale they understood that they needed scope in the way they collect data ultimately they understood that the most predictive insights come from actually intervening in our behavior and learning how to nudge coax tune and herd our behavior in the direction that satisfy their guaranteed outcomes now this is a whole new experimental zone I call it economies of action and we've been lucky enough to see some of these experiments hiding in plain sight for example Facebook conducted what it called massive scale contagion experiments were used subliminal queues on Facebook pages to see if it could alter real-world behavior and real-world emotion it did this in 2012 and 2013 the results were written up in prestigious academic journals in both cases the researchers emphasized two conclusions one we've learned that we can using subliminal cues online affect real-world behavior and real-world emotion number two we've learned that we can do this in ways that are undetectable to users we can do this by passing user awareness they never know it's happening all right so now here's a crash course on the collision with democracy everything I've just said to you about the economic imperatives of these markets suggests that surveillance capitalism undermines democracy from above and from below from above these data agglomerations represent a new source of social inequality is the 21st century is it not it's the digital century is it not we came into this century believing it was going to be the most democratic of any century the internet was going to democratize knowledge instead something unexpected has happened we have these vast asymmetries of knowledge to an extent that has never existed before exemplified in a growing abyss between what we know and what is known about us I call this epistemic inequality inequality of knowing inequality of what we are able to learn they have all this knowledge and of course more knowledge means more power more knowledge about us means more opportunity to control behavior always remotely never threatening us bodily not threatening murder or terror not coming to drag us off to prison if we don't agree but through the remote digital layer through the digital devices through the layer of the digital that increasingly surrounds us so now we're entering the third decade of the 21st century with these enormous asymmetries of knowledge and the power that accrues to knowledge it is on a collision course with democracy from below because in order to get the most refined predictions the predictions that approximate certainty it must interfere with human autonomy with our agency with our ability to decide what I do now and what I do next where we are increasingly subject to precognitive subliminal queues that invade our environment and are carefully engineered to bypass our awareness and therefore robbed us of the right to combat all right what happens here is a kind of wonderland we've entered a kind of wonderland where things that we thought were the case are the opposite of the case let me give you an example we thought these services were free but actually these companies think that me are free we think we're using social media but actually social media is using us right now 97% of Facebook's revenues come from its online targeted advertising markets which are a direct result of this logic that I have just described to you we think that we're searching Google we've found out that actually Google is searching us we think that these companies have privacy policies when in fact their privacy policies are really surveillance policies we think that these are innovative companies that occasionally make mistakes oops Google Google is so sorry we happen to have a microphone embedded in the nest surveillance system and we forgot to tell you we actually forgot to put it in the schematics oh we're so sorry the mistakes are the innovations finally we have been subdued by a line of reasoning that says if you have nothing to hide then you should not be concerned about any of this when the real fact is and I know that you know this because I can see you and I see complex educated thoughtful individuals sitting in this room and I know that you know that if you have nothing to hide you are nothing because everything that makes you you the source of our identity the source of our will source of our autonomy in our urge to self-determination these are resources that we grow and husband within us they are intended to be private because it is through privacy through our own experience of sanctuary that we grow these inner resources and they flourish this is the struggle from adolescence to adulthood where myself is no longer dependent on whether they like me or what they say about me but what do I think what do I believe what do I value how do I think is the right way to act this is what's within us and we hide it not because we're secretive or because we're ashamed but because only in the sanctuary of our own lives do these capabilities grow and without them a democratic society is impossible to imagine thank you for more debates talks and interviews subscribe today to the Institute of Arts and ideas at IAI TV you

Channel: The Institute of Art and Ideas

Views: 68,571

Keywords: the age of surveillance capitalism, capitalism, power of corporations, digital age, privacy in capitalism, human nature in the 21st century, shoshana zuboff, in the age of smart machines, surveillance capitalism, is capitalism the answer?, socialism vs. capitalism, what is capitalism, capitalism explained, the state of capitalism today, the power of corporations

Id: 8HzW5rzPUy8

Length: 29min 11sec (1751 seconds)

Published: Mon Nov 18 2019