Building Bots at Quartz // John Keefe, Quartz [FirstMark's Code Driven]

[Music] you can you guys hear me this is good all right okay good hi I'm John Keefe I am a bot developer and a product manager at quartz which is Kesey calm and we are a news and financial website covering the global economy and quartz has a long innovation of being incredibly innovative and so I was really fortunate to join there in just just in March my route to having developer in my title is kind of interesting I actually was a producer reporter journalist I was the news director for a long time at WNYC New York Public Radio and while I was there I saw what was happening at the New York Times Chicago Tribune LA Times where they were using data in these new ways where it wasn't just about using data to pursue a story but actually putting maps and charts and and actual databases online for the audience to look at I thought that is amazing I want to do that but in order to do that I'm gonna have to learn how to code so I started tinkering I was the news director at the time it was not in my job description to be coding so I would do it at home on my couch and one August of 2011 I was trying to do this thing you may have seen where you can put in a an address into a box and it drops a pin on a map right we're all very familiar with that well I wanted to figure out how to do that I was trying to learn JavaScript and so in the in the process of doing this I coded up this little map right here this is a map of Manhattan and these actually are hurricane evacuation zones so I had run across this shape file thought that would be useful maybe if there's ever flooding or a hurricane or something like that that the reporters in the newsroom would find this useful so wow I could use Fusion Tables and some JavaScript and I made a little box and you could drop you you could put your address in and drop your PIN in the middle of the map and oh it's cute I showed it to two people one person at work and one person at home they're like cute John cute and then strange enough that actually that was in the summer about six weeks later a hurricane started heading for New York City it was Hurricane Irene and and then so I was like hey I actually have a hurricane evacuation map on my computer maybe that would be useful so the folks that WNYC put it up on s3 so it was nice and stable and secure and it's just JavaScript so it was sitting there by itself and then the next day the mayor declared that zone a which is this orange area later maps were much more brightly colored this was not my best map ever but this orange area was evacuated so everybody went to NYC gov to go see if they were in the evacuation zone and crashed the site in fact the city site was down for a day and the only map on the entire internet that had any of this information was the one I made on my couch so this got a million pageviews in just a couple of days which is more than the site it was a hole used to get in a month and they're like hey could you keep doing that that was pretty cool so I said sure all we'll need is another hurricane of course then next year hurricane sandy came so this became my new job in the course I actually thanks to generous people in the journalism coding community generous people like yourselves who put their code online I ended up learning how to do this and now I actually have BOTS I have developer in my title which is kind of amazing and I'm so tickled by that so at quartz we're looking at BOTS and we're looking at basically four areas so when we're talking about BOTS there's a lot of definitions for that but we talk about software that you can talk with so that maybe in a chat format where you're texting or you're using a chat interface typing that maybe in speaking actually to something like Amazon Alexa or Google home and then we're also looking at the artificial intelligence and other issues behind not only these things that we can power these things with but also to help journalists do their job better and then lastly we're looking at making tools for journalists to do their jobs better some of them with AI some of them not and so these are this is what I'm going to breeze through here and tell you about how we're doing this all in the quartz bot studio so there are two people at quartz with bots in their title one of them is me one of them is Emily with Rowe she's a bot editor but they're but everybody else at quartz is sort of bought adjacent everybody has ideas for BOTS there are other people who code BOTS and so we'll talk about some of those folks do so in terms of chat then the text interface the quartz app is actually something that is very it's not your typical news app so this is actually the quartz app this is actually a little video and I'll start it they're gonna go there this is how the quartz app looks it's sort of this chat interface this is actually how we handled earnings reports so we use emoji we use this little chat interface this is not your typical news site about a year ago quartz decided this is a more interesting and sort of a breezy quick way to present the news to people and so this is one fun example each one of these earnings reports represented by emoji and some clever writing is linked to and so you can go read more about it but you could also just get it very quickly in a in a quick scan so this sort of chat interface is something that we're pretty familiar with and as a sidelight we just recently with the iOS 11 added augmented reality to this chat interface so here's our chat interface and you can click on some news objects and we actually this is the roof of quartz which is two blocks that way so we're probably looking at this bill actually here so this is Cassini represented an AR on the roof of our building we're integrating that into the app and I just put that up here because it's really cool and we just did it it's not AI or chat but it's something we're adding we're also playing a lot with Twitter and looking at how we can do journalism on Twitter you may have everybody remember this attack this we had this ransomware attack happened and the water quite wanna cry virus and they attacked computers and locked them down encrypted them unless you paid in Bitcoin to unlock your computer everybody remember this happening okay so interestingly you may have seen this so Bitcoin Bitcoin wallets they're public you can see them so you could see when people were paying the ransom right so there were three Bitcoin wallets and Keith Collins who's a computer programmer slash journalist at quartz created a Twitter bot that would watch these three wallets and tweet out how much money had been put into these wallets and convert them into dollars so here is a here's an example at this point one hundred and forty two thousand dollars had been paid in ransom so this bot was just tweeting out got a little bit of a following among people who are interested in the story people are affected by it and also people who are into cryptocurrency the one of the cool things he also did here though is he coded it so it would also tweet if somebody took the money out so who could take the money out just the criminals right just the people holding ransom so there was some speculation that it that might not actually happen that these guys didn't realize what they're getting into and that they might not ever withdraw any money except that at about 11 o'clock one night they did they took out twenty thousand dollars and they proceeded to continue to take money so this we are watching they were watching in real-time these people who were holding all these computers ransom taking their money out of these accounts so the bot actually broke the news this is the first this was the first word that this had happened we might not I don't even think we had the first story up but we had the first tweet up for sure so that was a way we're applying Twitter BOTS to do journalism and Keith has been doing that a lot at courts it's really exciting we're playing a lot also with facebook Messenger Facebook Messenger is a sort of a ubiquitous platform you probably use it in one form or another so we're looking at how news might fit into a facebook Messenger a lot of places are a lot of news organizations my echo dot won't be available apparently for my demo so we'll see what's going on as Alexa speaking to me so a lot of news organizations are putting a material in facebook Messenger we are actually more inspired to not just put our stories in facebook Messenger but find ways to use messenger and use a chat platform to do more chatty things so you may have seen pancho this is not ours this is done by beta works but this is a good example right so you can get your weather anywhere right but here's Pancho Pancho is a weather bot and if you ask for the weather Pancho gives you the weather and it's very very straightforward sure but the other thing you can do is first of all the weather comes along with some sort of attitude which happens but there's also if you just happen to type here's a weather bot you type I'm hungry and suddenly Pancho says oh you're hungry what are you gonna do about it order takeout or make a recipe okay I'll make a recipe and it says step into my kitchen okay just kidding I always wanted to say that preferably on my own TV show so you get this sort of sense it's got a little bit of an attitude a little bit of us a little bit of spunk so we're gonna grok you through this guacamole recipe and it's all about squishing it in your hands and licking the guacamole off your hands it's playful it's and it's something that you don't get from your you know normal weather app so this kind of a personality this sort of using the conversational interface is something that we we've been exploring and we are also very soon launching a edition of quartz called quartz e which is much more of a lifestyle brand and ahead of that we've launched the court C page and we have a bot attached to it so if you either go to the court C page and click message or you go to messenger calm /qz wife or court C you can get the bot and the bot right now is all things stranger things anybody had stranger things fans in here hands up stranger things okay couple of folks okay not enough y'all you have to get in so stranger things as eight of you know just drop their second season on Friday so this is that Netflix and so we decided to hook into that as an experiment right a lot of this stuff we're doing it's all experimental so here you can go into messenger calm that /qz why you can do this now and you get into the conversation and it you can recap season one or you can talk season two Emily with row our bottle etre is furiously watching all of season two right now and making new sort of experiences to go along with you and the idea is that you can come back to the bot and the bot can remind you to come back to and you can expand sort of learn along with the bot and not go past where you are in the season right so no spoilers and the idea is that it becomes another companion to the to the show and since this is all about code I'll tell you a little bit about how we're doing this we're writing this in a language called Reve script how many of you have heard of Reve script one okay yeah I hadn't either okay so riu script is a script a bot scripting language and actually you you can see this is actually the section there's running this one so up top the human me said sure to a previous question and you can see then the answer is the minus sign with the little party hat and then the intro comes in and these buttons come in and so we code it all and Emily actually writes the entire bottom n't in this way using these little pluses and minuses and routing or routing different paths through the script we chose this method because unlike the point-and-click bot constructors that you can get there are a lot of authoring tools out there this one is it's actually close to code it is code in a way and it's very writer based and so much of what we're doing is writing scripts and these experiences so it's not just a FAQ where you could sort of set it and forget it we're actually doing something new pretty much every week and emily is doing something new every day she's doing an episode a day until she gets through season two so that's happening right now so our sort of stack in a sense or lower flow is Emily's brain she's got all the idea she's watching the show isn't coming up with the ideas we write it in read script read script we put those text files on to Dexter Dexter is an authoring tool out of beta works just on the road a little bit and we've decided to use them as our BOTS altering tool mainly because of what I was saying about the writer based but we also looked at about two dozen different working tools and that's the one we decided to pick we use dialogue flow which used to be called until like a week ago API AI so if we go looking for API AI you won't find any more now dialogue flow and we use that for some of the natural language processing so like here if I say what's up it responds not much but also what is upside down and eight of you will understand that joke back to where I was and you can get back into this script so we're using natural language processing to watch for when people need help when they say silly things because humans like to say silly things too kibbutz all right so voice interfaces we have Alexa I'm actually gonna skip over Alexa how many of you have played with Alexa or know of Alexa right okay so we think that there are Alexa's pretty easy or more straightforward than some of the others to code so we're playing a lot with it we've experimented with quizzes and presenting news our flash briefing is our news offering and that comes across Alexa we I personally I have two kids at home they use Alexa if you see kids using Alexa there's no friction in how they make it work they got it and that is very clear to me that this generation will not have any problem talking to devices for the rest of their lives so we're pretty convinced that we need to figure out how news organizations can fit into these kinds of platforms a little bit about the AI that we're playing with so this is a bot that we made you know this this kind of sport of finding Trump tweets from history to be ironic about present-day Trump tweets we sort of made a bot that kind of does that irony is hard to code for so if any of you are doing that good Anya but what we did do we were able to do is use AI and processing of the tweets using a little program called tweet to Veck so it's turning tweets and directors so we can actually look for similar tweets from before he was president so that's what this does every time Trump tweets it goes out and finds the most similar tweet from before he was president it sometimes it works really really well so this is from June 11th of this year my daughter Ivanka will be on Fox and Friends tomorrow morning enjoy lined up very closely to one from 2013 my wife Melania will be on TV see today at about 5 p.m. showing really beautiful jewelry at a very low price okay so it worked it's sometimes it's hit and miss it did actually notice that the Miami Heat were in Miami without even so making no or seeing them word Miami so it's just pretty well there's more useful things that hey I can be used for ProPublica just released this last week I believe where they went through tens of thousands of press releases from members of Congress to determine using language processing and similarity detection what Congress people were interested in so here in New York we have Kirsten Gillibrand she's a New York senator and what she is most interested in that is unique to her so it's not just I'm interested in education it's she's interested in these items and other people are less interested in them so we have firefighter Grant Fire Department contained breathing what's that about anybody know why would she be interested in those things well she was a big proponent and pushed through a lot of legislation around first responders and and care for the 911 responders so that's that popped out the AI was able to detect that this can be very useful for reporters this happens to be a public thing but it's a way that AI can be useful for journalism I only have a couple minutes so I just want to say we're we use slack a lot how many people use slack okay everybody right all right same in journalism some some fields not so much and I'll just say real quick that we have developed with organization called document cloud a thing called quack bot which is a tool for journalists that allows journalists to ask the bot to do very common things it's right it also has natural language processing in it so you can say how are you today but it really it goes out and it can get data it can grab screenshots it can do a lot of the things that journalists do it's this very sort of Industry specific flack bot and we built this we use Amazon lambda to actually keep making new skills for this spot we have a really interesting thing in journalism where the open source community is really strong in journalism but the number of developers in journal in newsrooms can be pretty small and a lot of newsrooms don't have them so we're looking at ways to use this little stack using lambda functions to put tools that are already sort of out there in the wild that might be hard for a newsroom to implement because they don't have a developer put them into quack BOTS so that people can just ask ad crackpot to their slack team and then ask the bots to do it for them so that's one of the things that we're doing you can read more about this you can this project is open source so you can see how we're building it and you can read all about what we're doing at BOTS qz comm so this entire project has been funded by the a grant from the knight Foundation and so part of that means that we're sharing everything - so you can see our code see what we're doing go ahead and use our code that's awesome and then here are a bunch of links if you want to take a picture of something there you go but you can sign up for crackpot if you're in a journalistic organization otherwise just take our code there's this stranger things but you see that Bostick easy comm is our blog BOTS @qz comm is our email and that's me cool thanks [Applause] yeah for the analysis you mean like for oh so all the NLP right now we're using API AI slash dialogue flow so every time that a journalist or in a messenger app says something that isn't immediately clear we send it out to the dialogue flow I'll get used to that eventually and it comes back with an intent and then we handle it that way yeah yeah for yeah for the messenger bot yes so we looked at 25 different authoring engines different platforms and when I say we I mean my colleagues I actually didn't I wasn't part of that project they went through a lot of work we actually had a script about Beyonce that we applied to each of the different authoring tools to see which one worked best for us in the end we chose Dexter but at bat scusi comm we talked a little bit more about some of the other bought platforms that we liked but didn't fit for our purposes so you can check it out there yeah how did it counter the behavior so how do we know so what we basically what we do is anything that doesn't get caught so we have a bunch of keywords and that you might say in the course of talking about stranger things and anything that doesn't get caught by the bot script we have a catch-all and we send that whole phrase out to dialogue flow and it comes back depending on what they say we tailor that we use the intent that we get back from dialogue flow and then route the question or the answer accordingly if we get into the situation where it's not it's not matching exactly like somebody's saying something like for example there's a character and stranger things named eleven eleven comes back as like a number so we have to go in and tweak on the dialogue flow side we can retrain the model so it will recognize other things so we do that a fair amount also it's not really set up for emoji reaction right now so we've had to go in and teach it a whole bunch of emoji reactions and that this is yes this is no so that's how we're doing it so between the reef script answers and the training the model on the dialogue flow side we try to do that best that's the best we can yeah yeah so the biggest challenge right now I think is sort of exactly where we're just talking about you never really know what a human is going to do and a lot of people actually like to go in to these experiences and just mess with your mind you know and that's totally understandable I think we are looking at the complexity of handling these kinds of things like how do you make sure if somebody says I'm ready but really what they're responding to was a stranger things question and not like some other adventure question and so really detecting tracking what people have done and using that as the context for what they mean next is probably the biggest challenge we may have to put in some machine learning to help solve some of that but right now we're just doing our best guesses based on when and where they say ready yes they are fun and that is actually it's something that I've been a real proponent of actually have spent a year trying to make something new every week I got yeah so two-thirds through the air but I really see the value of experimenting and playing in a very sort of public way like publishing this out and sharing it out so we're doing that at the same time we learn everyday like with some of these experiments so what you are seeing here have sort of made it out of the house so they're better than they show some level of potential right so the stranger things bot is an experiment 100% experiment but it's a really high quality experiment and we're learning a lot from it so we can see where people get stuck we can see is this the kind of thing that anybody would would there be other uses for it so partly we are just trying to be a place that is on the cutting edge in these kinds of experiments and sharing that out so now you know quartz is it does that at the same time we are learning every day and always I'm on the product team so we are looking at how we might make products things like our other products like our app and our Alexa newscasts are supported by advertising so we do sell ads we're a commercial website so we look at that we're not selling ads in the stranger thing spot right now but what we're learning right now through being able to make these conversational interfaces could become a money-making endeavor at some point my pleasure thank you [Applause]

Channel: Code Driven NYC

Views: 83

Keywords: software, development, engineering, code, programming, startups, product, design

Id: I5FeMcIbwJA

Length: 28min 18sec (1698 seconds)

Published: Tue Nov 28 2017