Learn to Build a Chatbot with DialogFlow and Python

so uh i think it's about time to get started uh thank you everyone for joining my name is nick i'm the director of projects with ai ucf and today we have michael dupont he's a software developer for disney michael go ahead and introduce yourself great uh yes my name is michael dupont i am a senior software developer for disney i work on the emerging technologies team which i like to tell people we work with all kinds of tech buzzwords except for uh the blockchain so ai machine learning computer vision augmented reality virtual reality computer vision chat bots like we're going to talk about today great that's awesome um so if we're ready we can uh michael you can go ahead and take this stage all right uh let me go ahead and start sharing my screen because i well what i would like to do with you guys today is to actually build a chat bot from scratch uh i oh i didn't i didn't mention the intro i i am the organizer of the orlando python meetup uh given uh tons of talks there and and helped the group uh stay around and be relevant during the pandemic here oh you need to uh enable screen sharing by the way oh give me a sec and anyone who's in those groups can can tell you that uh i love doing live coding and i know i know that's a bane of a lot of people's existence sometimes especially if you say go to a conference uh but uh and that i also really love tangents so if you have any questions at any point in time ask them right then and there and we'll answer it go down a rabbit hole and we'll come back awesome and uh just so everyone's aware feel free to unmute yourself at any time if you have a question just go ahead and shout it no shame it's not it's not like lecture here this is a place where we can hang out and talk to each other okay so we're in dialog flow here and go ahead we go to create a new agent a little bit of a little bit of background on dialogflow actually it used to be called api.ai it itself was a startup and was bought by google about three years ago three four years ago uh rebranded it uh and it's the reason why they bought it is it was the easiest way to create uh google assistant skills and in my opinion still is now uh we're creating this new uh this new agent what would you guys like this agent to be about general companionship maybe like not really task oriented perhaps okay so uh just having having a friend in the pandemic yeah that would be cool relevant what should the friend be names maybe uh let's call this my companion because if if you're if you were to go out and actually make this public uh one of those things you might want to do is allow the user to select a name for the uh to for their agent to be say hey can you change your name to name and we'll that we can actually have that be one of the first things that we do here so we don't need to worry about the rest of this for this particular project oh did it cancel oh maybe i have too many just a second actually i'm already in this in this uh df demo here yes yes yes let me delete this i think i think it only allows you to have a certain number of agents uh without paying so i created i'm creating one of my uh let's say my virtual companion his name is jimbo so that's my name of choice [Music] great oh that's interesting personally i'm having trouble getting to the uh dialog flow uh page i keep hitting the um projects in google cloud okay uh so the uh this url here dialogflow.cloud.google.com as long as you're logged in to your google account uh if it sends you off to just the i think it's just dialogflow.com just you there should be a sign in button there to get started there we go i'm there come on what i'm trying to stack overflows for you yeah that's really weird i wonder if i can just repurpose one of these yeah you could just create a new intent that we could uh start with yeah maybe it's fine i think i this is probably something i haven't touched in a while uh i i do i've done some work with code for orlando so i think this was one of these uh example things that we were working on a while back anyway so we created a new agent yeah uh the first thing we need to do is you know let's pretend that these don't exist for the time being uh whenever a new agent is created we're automatically given two intents we get this default welcome intent which if we click into it's listening for the app has been opened and we'll respond to with this uh text response here how can orlando help you today and you uh you can put multiple responses in here and it will just choose one at random so i just go to up here in our window we can do open app or hi any of these oh wait a second i just do hi oh never mind uh what you are seeing down here is the other one is it doesn't actually know how to respond with high or open app because we're actually already in the app in this testing framework here this default fallback intent as the name might suggest is if your agent is given some kind of input that it doesn't know how to handle it will come down here to our default fallback intent and we actually don't have any uh responses in here yet so i can go ahead and i'm not sure how to help with that save this hey michael uh we have a question from the chat so um they're wondering what an intent is um i kind of explained it it's it's almost like a function um yeah if you want to go ahead yeah so sorry i kind of skipped over that so an intent is analogous to a function in your application so this this agent is itself an application and an intent would be a function however the agent is determining how your function gets called based on what intent the user what what intent matches closest to what the user said so we can we can get a better idea of exactly how that works if we create this new intent here we're going to call this change name and what sort of things would you as a user say to this bot to say i would like to change your name to blank or i would like to change your name you could say you know can you go by you know the name michael or can i call you michael could you go by michael can i can i call you michael so this is our training data these are these training phrases here are saying if i in here say can i call you michael and i can just do that right here can i call you michael it's recognizing it oh no it's not uh oh we haven't saved this yet that's why always important to save if we if we do this again nope i this is what i get for now starting with something clean here i'm just gonna i'm gonna clean these up yeah um i think what's happening is the program is recognizing it as a command that um isn't the one that we're trying to build right now so getting rid this is this is called a a conflict and this uh even though we i wasn't clicking into those other intents this is very likely that the number of phrases that those other intents were trained on were so vastly outnumbering can i call you this even though it was an exact match it likely had a stronger confidence that it was that other intent than it was this one it's kind of like when you try asking google to like set a timer for your pizza and before you know it's like calling up your ex-girlfriend okay so we we now are hitting the correct intent here can i call you michael we don't have a response yet uh but it did and it did identify our change name intent uh no obviously the first thing we might want to do is a text response saying sure you can call me michael however well if we save this and we so we have our intent has been saved we should now get a little notification saying the agent has trained and if we run this again sure you can call me michael now this is great if you want your bot to be named michael and only to listen for michael if we ask it for hey can you call me sarah it's still going to say yes you can call me michael so now what we want to do is we want to listen for any arbitrary name and we do this by having slots and a slot is analogous to a parameter in our function so one thing uh i there are two ways you can do this you can either set it down here explicitly or we can actually highlight the word or phrase that we want to say this is a parameter and they already have a bunch of these system entities built in that we can choose from so if i just do name we have cis.given name and then i can also highlight this other phrase down here and it already it knows which entities this intent has already been assigned to so now we have given name yes but now we want to say hey sure you can call me whatever name you just said well this value here this dollar sign given name which is our parameter name here all we need to do is use our parameter name given name save this can i call you sarah and i'm going to wait for the agent to be finished training there we go can i call you sarah sure you can call me sarah what about like a last name can it also handle that yeah so we can have uh can i call you michael dupont it's my name uh so it already recognized that michael is a give is part of our given name slot here and we can do the same thing we did before where we highlight that other one and we can do last name and now we have two here if we go ahead and yeah so that's now part of our training data so it can now listen for obviously can i call you sarah or uh smith as last name sure you can call me sarah or we can come down here and i'm actually not sure and call me given name last name i'm not i'm actually not sure yeah if it knows to choose one or the other given given the the slots that were filled there you go okay it seems it seems to be smart enough otherwise that would have come back 50 50. now let's say that we want to actually make this required what if we can anyone tell me what might happen if instead we say can i call you something different and have that be in there so can i call you something different just can i call you something different well we didn't slow uh so yeah it actually is doing something something good here where if this if the slots are not filled it's not going to use any of these text responses down here that require a slot to be filled in or to use it so we actually didn't get a response back there now either we can say we can add a response down here saying you didn't give me anything or what is likely what we actually want is if we at least make the given name a required entity we can define a prompt here saying uh sure uh what should my name be uh or specifically this is asking for first name so first name b close that so now we have a prompt in there if we save this and wait for the agent to be trained and ask the same question is now actually going to say sure what can my first name be so that and then if we actually want to use this here uh where we actually have enough in here where we can start playing around with it in the simulator a little bit so here uh we can also go in and we can define custom entities which we can do later if you guys uh if you want uh so uh we'll get we'll get to that then we'll get uh we'll also get to fulfillment in not too long but let's look at the integrations we have this is where we can take our agent that we've built in dialog flow and actually put it on different platforms obviously they really want you to use google assistant it's up top and it's big and we'll do that in just a second but look at some of the other things we can do with dialogflow telephony these are all giant uh uh call center companies uh disney actually uses avaya and uh so we can actually integrate this as the front to a call center but you can also just use this dialogflow phone gateway and they'll give you a number where you can just call your bot and i've done this for uh for hackathons for some pretty cool demos that's cool uh you got twilio here for text for for other other uses there we got text based so you can just have the standalone web demo you can create a messenger bot line telegram slack this as you can see this this was a slack bot at one point and then there's other things here uh like twilio text messaging and up until about a year ago you could even export this over to alexa i can't imagine why they removed it so we're going to do oh it sounds like there's a question oh yes a silly question but is there any way to uh export it in a different way that could be used with other things like say uh instagram isn't listed on there among the things is there a way we could export it in a way we can make it compatible later like with not listed platforms there is a way where you can get the the data out of here uh there's two main ways so if we go into our settings you can actually import and export this will export your entire project as a a series of folders with json files in them uh and then you can obviously you can use this for backup as well so restore and import as well there's also a an sdk available for dialogflow i've used the python client uh and you have complete access to your dialect flow project in a programmatic fashion uh that's your entities your intents uh basic configuration for for disney we had one project where we needed to have a series of entities that were constantly kept up to date based on uh based on what what attractions there are in the park for example and rather than saying hey a new attraction has been added let me go in and manually change things i created a script that would pull the entire attraction list down from the internal disney apis same things that are powering the mobile apps and and the website it would parse parse all that information pull down the existing synonyms from dialog flow package it into into these json files and then upload the entire thing back up to dialog flow and then i actually automated that so i never had to worry about that you know keeping my entities up to date ever again fantastic awesome nice okay so uh for testing uh we can do this in google assistant and all we need to do is this is our assistant integration tab here all we need to do is just go to test for the time being this is deploying the current state of our model to the google assistant so we have a question from the chat and he's asking how in the in the sentence can i call you something else how does it know that the word something isn't a real name oh so this is where we're actually leveraging the the built-in system entities uh we can we can actually uh take a a look at this by uh rather than going to to test things first here let's create a new intent so we we have this companion agent uh and let's say that we want well we want it to tell us a story so let's just say or uh give me some now actually what would you guys like to do can you uh first can you define what an entity is oh yeah yeah so uh let's just say that uh uh you guys have worked with things like enums before or or dictionaries that uh the the with the key values okay yeah so if we uh so if we have this companion bot here and uh let's say that we want it to tell us a story just as an example so we have uh we have an entity called story and uh so so this uh this story can have a bunch of values in it uh let's just say moby dick for one although i can't imagine why you'd want a bot to read you the entirety of moby dick uh or 1984 198 uh about 1984. one more the hobbit great so we are so we have a an entity here which is of type story that has values in it of moby dick 1984 and the hobbit if we want to use this entity we can create a new intent called uh read a story uh read me 1984. no put that in there it has already recognized that it is of type story oh and we can now do something uh well save this it will it will update our model and we can now say uh can you read me moby dick we don't have a response in here yet but it was you can see that oh wait let's change name oh so actually this is a good example here this is this is that uh the collision issue that we saw earlier so the can you part because we had because we have so many training phrases in in change name that start with can it actually has oh uh it has uh anytime it sees can now it it's going to give a higher uh possibility score to change name than it is to read a story now we can fix this by saying by adding uh can you read me a story and there's no book in here we can make this a required parameter just like we did before what story would you like or can you read me hobbit again it knows that it's a story we can save this and now if we run the same thing again instead of saying change name it should say that this is read a story which it is cool yeah i was probably picking up on the first name last name nature of moby dick for the other intent too like they probably thought it was a name it's not a story it is possible it may have thought the story was a name however uh i would be high i highly doubt that story itself is a name in the given name or last name entity that the model is trained on it probably just over fit with can with the other intent but i believe if i we can't actually go back now and see it but when it had first name given name uh in there it was probably asking hey what uh what name would you like me to call you cool okay it's really cool how you only need so few training phrases to get started like you don't need a you know to scrape reddit just to have something halfway functional you just put in 10 sentences and it it's able to generalize fairly well yeah they're i know that on on the dialogflow side they're doing a lot of uh training phrase augmentation in order to make that work and then even with all of these different uh and all they're called nlu a natural language understanding as opposed to natural language processing they are doing a lot of nlp under the hood to to under try to assess the meaning of what you're saying rather than just looking at the exact sentence structure uh but obviously the more training data you have the better it's going to be interesting so you said augmenting would that be like changing the grammar of a given sentence or adding in a different you know like at a different punctuation or articles or several different words yeah it's going to make uh it for lack of a better uh phrase uh crowd-sourced uh ways to say a sentence just based on probably all all of the different searches that get put into into google yeah yeah because they have so much data and they know how to boil it down to like it's true meaning that's why google search works so well they can really pick out like what question you're asking for even if you ask it in a bunch of different ways yeah and and just just kind of hammer hammer that home we only have three training phrases here can you read me story can you read me a story read me story but i bet we can do can you read my kids a book i bet you no okay and i i went a little too far off the training phrases there but so it hit our default fallback intent here all right it's it's usually pretty good about uh being able to pick up uh not what you said is not quite what i am trained on but i'm still fairly confident and and you can actually go in and adjust how aggressive your default fallback intent is just based on uh the default fallback intent is going to trigger if none of the intents are above a certain confidence threshold like say thirty percent or sixty percent uh and that's all that that's tunable in all of these different nle cool platforms so we're halfway through i'm i'm going to uh jump back over here to our uh our test just to just uh just to give one example of having that conversation flow or you could say the dialog flow of asking for the name because we we specified that you at least need to give it a first name as a required a required slot so can i change your name i'm not sure how to help with that oh oh did we uh oh that's right that's that's uh could can i call you something else yeah that's right so all of these different things if i was doing this in production i would go can i change your name i'm not sure how to help with that oh i'm going to put that in there as a new training framework yeah yeah and you guys when you start playing around with this you'll see how it works where um you'll be all i see like kind of what it's able to do oh there we go there we go so we we got this back it's now expecting me to say uh let's say jim okay you can call me jim elves [Laughter] okay i'm not actually sure where this came from but some oh uh can i call you something else so it it apparently it heard well maybe someone's last name is else and it didn't and something here this is actually a very uh let's say a timely example where it definitely did not interpret something as a first name yeah yeah but it picked up on l's somehow yeah so oh and here's the other cool thing um once these once your agent is in this interface here our our actions console and you can test it it is now white listed on any other google device or on your android phone or if you're on ios the google assistant app so that's great for for product testing yeah i have a home mini and i just play around with it all the time i'll be typing on my screen and then i'll just trigger it with my voice right away and we actually uh for uh in in the office we well we're not really in the office anymore but anyway uh we really love having the devices with screens on them because they do the live transcription as you're asking it so it takes out one possible did it really hear me correctly and if it didn't you can just you say okay it didn't hear me correctly let me try something different cool cool okay so we have we have a basic app we can ask it to change its name even though it's not really changing its name uh and have it read a story uh at this point i have 25 minutes left we can jump into the back end by adding a fulfillment layer this is everything that we've done so far as you've seen we haven't written any code everything is living on google servers but there are times when google is not going to be enough and you want to have some more pulling information out as opposed to just responding with whatever is in the entities to do that there are two things we need to do to start so we have we need to enable web hook which i'm actually going to clear that out and let's say let's have our our change name stay the same but in our read a story we didn't give read a story a response so there's this other tab down here called fulfillment and we can selectively enable or disable calls to our fulfillment layer on for each individual intent don't remember to save when you whenever you change anything in here but now uh we can just ignore dialog flow here for a little bit now i i sent a j or i created a gist uh with the code for uh for you guys to use and i have it just right here i'll go ahead and open this it's posted in the discord i'll i'll post it again in the chat just for anyone wants to follow along okay and i have i have some instructions in here on how to run this i'm in my conda library and all of that so this is using a library called fast api uh there's there's there's uh you don't have to use a specific framework here in fact uh the uh if you're if you've you can do this with flask or with uh starlit by itself mango doesn't matter it's it's all web i do like fast api for for new apis because they do a couple of nice things a it's async compatible in python so we actually have async def down here which is great and they also leverage another library called pydantic to uh do parameter and payload body uh validation so our code down here is not going to be called unless uh the the data that our user or in this case dialogflow is giving us is valid so it adds it that adds a level of protection so what you can see down here actually before i talk about this code let's pull dialog flow back up now that this here is enabled for our web hook and we saved it if i try to do readme 1984 it's actually going to give us some diagnostic info and we're going to have to wait for this to time out but we have this diagnostic info down here and this uh this raw api response here oh wait a second i think i'm jumping ahead of myself a little bit uh let's actually just get this running so in order to make this publicly available we're using a lovely little command line utility called ngrok all this does is make my computer appear as though it's on the public internet all i need to do to do that is i'm going to go to my desktop i have my little ngrok executable here it's free by the way uh http and we're going to mirror port 8080 so now we uh we have these forwarding urls here so any web server that i have on my computer that is listening on port 8080 is going to be available on the public internet through ngrok i can copy this as https dialogflow only allows https fulfillment and points by the way go ahead and save that you can also add authentication header values all that good stuff for uh for securing it now so if i use any graphic every day so if anyone has any questions feel free to hit me up afterwards and i'll be happy to help you with them but it's a really powerful tool that's really simple to get set up i use it for showing a friend a website i'm working on without having to like configure dns and it's just very easy to get started and it's great for this project it's another one of those great hackathon tools as well because you can have everything and the reason why we're using it here is you know if you're in production you're going to deploy this into aws or azure gcp or your enterprise cloud if you're if you're in uh if you're doing it for work but it's much harder to debug on those than it is to just see the print statements come down here okay so we're also using a a program called uvicorn is it's one of these whiskey apps spelled wsgi that rather than using fast api or if you've used flask or any of these other things they have debugging development servers this is what you would actually use in production and fast api actually prefers to use uvicorn but all we're doing here is saying uvicorn we want you to run on port 8080 which is what ngrok is listening to and the app the application is in our app uh in our app file app.pi and it is named app so that's that's all that the syntax is doing here but now uh we now we have all of this set up if we do readme 1984 we should have gotten yeah so we got uh we received a post request here which we're listening we're listening for a post at our root uh we got an unprocessable uh unprocessable entity however so we're gonna debug this a little bit and we here's where this diagnostic info should be able to come in so basically what's happening is google is sending us a whole lot of information and we haven't really processed it at all right i'm wondering uh just because this isn't this is an old agent i bet this is actually sitting on the the v1 apis that are going to be deprecated soon and i built this for the v2 however it's not it's not that big of an issue all we need to do is change how we are validating the input and it should work just fine okay uh so all if we're looking at this this is the uh this is what's being sent over and actually i can do one other quick debug thing i'm going to i think i have flask sitting in here as well if i just create a new file called app2 i'm just going to have this be a really basic flask app yeah has anyone worked with flask before anyone i think a lot of uh people interested in this project they're a lot of them are like freshmen or sophomores so they might not be super familiar but uh a lot of the libraries we're using today are pretty common so like i said if you have any questions just go ahead and ask in our discord and i'm sure someone can help you like get a flask app running or help you debug your code but flask is it's like a web server application or web server library where you can handle different requests to a web server and return whatever information that a user is looking for sorry sorry for blowing through this so quickly but i i want to make sure we have enough time to to actually make a response for you guys i think that this should be all we need just to see what's coming in so i'm going to stop this uh i'm going to actually got one more i'm going to do app.run port 8080 and then i'm just going to run this directly so python app2.pi and that should oh class just wants to know what it should be called okay so so without changing anything else if i do read me 1984 again let's see here here's the input that we are getting directly from dialogflow i just output this as a as a json file for us to for us to view so yes this is exactly my suspicion so we're looking at a version one this is not going to be the same as what you guys are using just i i i repurposed an older uh app but actually i bet there you know there should be a way if i go in here to our settings yeah so legacy v1 if i upgrade this to v2 save i bet that actually is all i need to do here so i'll run uvicorn again and we shouldn't have any issue okay i'm responding to read a story intent with one slots found 1984. that's perfect uh cool yeah so if we go back up here all this is doing is trying to validate and parse our fulfillment request here so we're going through and saying well here's the query result the things that we want in order to figure out how to respond to this we need to know our parameters so here's our story type and the value that it was looking for and also we want to know the intent now that their name property here is whatever they're internally looking at their id but we care about the display name so that's that's where we're getting if we look in here uh query result just kind of ignore the syntax for the time being but we have our request here which has here's our parameters which is the dictionary and in our in the intent we want our display name now if you're if you're doing this and you're creating your own api you have full control over here but this is just responding to the data that we are being fed from dialogflow which we don't have any control over how they uh how they create the what's being sent to us in the request the nice thing here is we now have these regular python classes that we can just pull the data from almost as though we were just referencing like a javascript style syntax so we can create this the response here which this should work for any uh albeit v2 uh dialogflow app saying i'm responding to our intent display name with however many parameters there are and then we're just uh we're joining all the parameter values so if we if we made our first name one uh require external fulfillment then we'd have two parameters in here and the values would be first likely first name last name because i believe they come in the same order as they are listed in our slots here now if we if we want to use this let's see we got we got 10 minutes left uh do you do you actually want to do uh question time after the entire talk is done that's fine um yeah if you guys don't mind this might run over a few minutes but yeah okay i can i can show just one example of actually using the display name uh to to handle different intents i think that would be very useful yeah yeah go for that that's that is really useful okay so you'll notice that we only have a single endpoint here and it's a post because we're receiving data if we if we go ahead and we set fulfillment for this intent as well uh so intent saved training has started all that good stuff can i call you [Music] steve mcqueen because that's just the disney name that came to me first i'm responding to your changed name intent with two slots found steve and mcqueen it's really cool uh so it was able to recognize that okay this is a different intent and here are the different parameters and if we want to actually respond in a contextual way to these different things we can create different handlers so let's just say this this is a we're going to handle our name and we're going to take in uh let's let's actually make this uh let's just say we're going to take in just the parameters so let's just go ahead and we're just going to print the parameters for now if i try to make these keyword arguments uh because uh actually let's let's do that right here so if i go ahead and instead of using these uh if i change these to underscores i think python will be much happier and story shouldn't need it so i can actually change this to a given name which is a type string and a last name which should also be of type string and we're going to return a string this is these are just python uh function annotations python is not strongly typed it's completely optional but in my opinion it enhances readability even if you don't want to use the optional type type uh checking libraries yeah i noticed that in some of your videos that uh you use that kind of you define all your types and i've never seen anyone cook python like that it's really interesting you're finding the type annotations are becoming far more prevalent in uh in maintained libraries modern libraries because a lot of people are using these type annotation checking tools and you'll actually see pedantic uh where it's actually checking the types against the types that we give it so if display name was none it came in with a null value it actually will say that it wasn't validated because in in python if you want to do that is actually you can give it an optional wrapper and still in python if it's not a basic type we have to import from the typing library as you see up here okay interesting um the the other nice thing with something like fast api with these typings is fast api will automatically build you your api documentation for you interactive documentation and it uses these types here to specify to actually build that documentation on your behalf wow that's super convenient that's awesome uh so one last thing oh uh we have our given name and our last name here and then we just need to add an f to the front and now this is a formatted string and then the other thing we have here is we're gonna handle our story as well and that one is just a story that is also a string and we're going to return a string there we go we so we now have these different handle functions here uh the easiest way to get this mapping done is going to be to just create a dictionary here uh to say if we are given this change name intent so change name which is our display name which we're pulling out right here uh the handler is going to be the handle name and if we get the read a story we're going to handle the story so rather we don't care about the count anymore and we honestly don't care about anything else there we can do uh we can have a check here let's just say the handler that we want to use is our handlers.get our intent and uh in in python if you uh get on a dictionary if it doesn't find the key value in there it returns none so that's a nice little check for us uh and i think i'm actually uh i'm on python 3.8 so i can actually do this oh gosh if handler or wall result oh my gosh that's the first time i've seen that used it's it's not there are a handful of times where it's very useful i in my opinion this is one of them so we can actually so we can say text equals our handler and we're going to start our uh queryresult.parameters so this is just taking the dictionary that is parameters and saying the string value is explicitly this given name so i so this is actually the an example where the parameters on the front end which is dialogflow are actually going to match they have to match what we're expecting in this function given this syntax totally optional but i think it looks it makes it look really nice yeah and then lastly we can have an else here we can say text equals uh i'm not sure how to help with that and return that okay hey we're going to rerun this because uvicorn well i didn't give it the the the command line argument to reload so this is now up to date we don't have any code errors in here and if we go back to can i call you say steve jobs okay i'll go by steve okay we changed name i'm going to just get rid of these text responses here so really what you're doing is you're offloading the functionality from dialog flow to your python application correct okay we need to get perfect so this uh i i erased the responses here so there's there's absolutely nothing coming from the front end this is being done entirely by the back end so okay i'll go by first name last name or read me the hobbit read me the hobbit let's start reading the hobbit which is exactly what we did here awesome now there's as as your application gets bigger and more complex on the back end obviously you'll want to this kind of thing would be put into a separate file maybe all of these live in separate files that that sort of thing but this this is the i'd say this is this is your basic template for how to handle all of the different intents that require going to the back end now there's nothing that we did here actually requires using the back end but uh we could ease we could very easily replace this handle story with uh let me given this story let me go check some external let me check audible uh the audible api to get a an a link to an mp3 and then tell google assistant to start playing this mp3 instead that that would be a very good use case for having a back end for this particular intent yeah you can really code in whatever you want and interface with different apis or you know other programs that you built so that you can operate them using these voice assistants yeah and the the other recommendation i will give you guys when you're when you're doing your projects let dialog flow do as much heavy lifting as it can and one example of that that i mentioned earlier is if you can use a system intent use a system or not a system in intent if you can use a system entity use a system entity rather than having to go and manage your own list of things your custom entities should be things that don't exist in the system there are tons of system intents uh so maybe it's maybe because we just don't have any added yet uh let me just create a new intent i want to call this airport because i love i love this example so let's just say where is orlando international so it should know that orlando international or we can have this be an airport so this is one of those many system intents now if i look up in the dialogflow documentation dialogflow system airport system entities reference and then i'm going to search for airport on here system airport you'll see that this system airport if it matches the name or or even any of the codes like mco would be mco or kmco would also match for orlando [Music] if if i enable this to have fulfillment on the back end i'll go ahead and save that i'm actually going to print what our query parameters are here just to show you what information we get from it it's not going to match anything so if i so when i when i test it we're going to get this i'm not sure how to help with that phrase oh and i need to also reload this this should be trained now uh and i can use a different airport name so where is jfk in new york i'm not sure how to help with that great but we should have a print down here it doesn't even it doesn't look like we do you know what i don't think you saved the intent oh you you might be right there oh yeah because i got the default file by content that is correct okay training completed good catch you're going to be hitting that blue button a lot when you're in dialogue though yep this is also why pair programming is great okay so we got the correct intent here i'm not sure how to help with that we check back here so it was able to recognize the system entity but look at all this other information we get it knows the country the city the name of the airport and then the airport codes so absolutely if this is all information that we don't need to know i cannot i can and i can repurpose this saying uh where is miami international and it also you can see the value right here uh it also knew that i can specify it by the airport code or the airport name it's these are very flexible in that way and one other thing you can do with these entities that i didn't mention earlier but will also be very useful so we have the hobbit here these are the values the uh but over here we can add synonyms so there's the hobbit uh or we can call this uh uh the the lord of the rings prequel uh and if we go ahead and save this or uh let's just we can say this uh the bleak book doesn't matter what you call uh hit the break this the book we were all forced to read in high school yeah i much prefer brave new world yeah uh so we can have uh read me the bleak book and well hey oh it didn't finish training that's why there we go yeah let's start reading 1984. that's awesome i i yeah i think that's that's all i have for you guys we're only six minutes over time and uh and give me any questions you have i'll be i'll be on as long as people have questions are there any questions um michael i have a question for you so um have you played around with the knowledge thing right there um on the sidebar have you played around with that i haven't in fact just wondering we would you like to show up a little bit i i have uh i have not played around with it uh uh myself yeah maybe i'll play around with it after this and let you know if i come up with anything does anyone have any questions about um software development or working for disney or creating these kinds of chat bots in a work environment i guess not well michael do you have any questions for us or any comments closing remarks um i'm usually so bad at this so uh dialogflow is very powerful again let it do as much work as as you can there's there's tons of other things in here i didn't even get into for example you have uh you can add follow-up intents so if you have an intent that requires a yes or no answer you can explicitly just add a follow intent that says uh hey can i would you like me to get your car if they say yes well then uh what should i do when they say yes uh and then you can you can also branch this out into uh well what if they say no or what if there is something else uh yeah you can nest all of these intense together and string together something a lot more complex yeah so even you know this nested one we're gonna add a follow-up to this one and this is how you build your your conversation tree ever anything here is going is that your high level but when you when you get into these nested ones they're only available within the scope that you specify which is very powerful the other thing that i think will be very useful if we go back to our integrations tab here there's these implicit invocations now this is you can add any of your custom intents here so you can have if we add read a story to it and we go to test it's now updating our our action or our google skill here and i didn't change the invocation name so it still needs to talk to city of orlando but if uh if i do uh talk to cigarette i know which is just what we have here as our uh open my app we're now just opening up this is hitting that welcome intent that that gets pre-populated for you but if i want to jump immediately into an action i can do ask the city of orlando which is so weird uh to read uh to read uh 1984. hopefully this should work got it getting the test version let's start reading 1984. let's start reading 1984. so you can have ask my companion to start reading my my twitter feed as an example and then that actually is a great example of having an implicit intent because you're going straight into what you want your bot to do to a specific intent that's then sending a request over to your back end to pull from the twitter api to get say the most recent five messages uh from from any given feed and maybe the particular feed or or list you want to pull from is a parameter is the slot uh and then it constructs the string sends it back to you that's a great example yeah that's a yeah these things are very highly useful and so many possibilities for what you can build with them we have a question from the chat and he's just asking are there any other examples that you would recommend we check out um any other things that you've seen that are worth looking at um for well for dialogflow specifically or for the nlu chatbot space in general uh he asked for dialog flow specifically but you can elaborate on other uh chatbot programs i i have two specific examples for dialogflow that i think if you get to the point where you're happy with what it's saying you have this screen here that you could potentially use or having things display on your phone dialog flow allows you in your intents to to have uh specific things on a platform level so you can see that uh if this is our default response here which will actually be filled by our by our web hook uh but if it's being called from google response we can add a response here to do basic things i'll actually just create a new intent here just to show this i'm just going to call it test and it's just going to listen for tests so it's highly over fit for this specific use case but i'm going to [Music] uh add an assistant custom response and this is just a basic card and i can give it this is a title so cool it's a required text this is a basic mobile card and then we can give it we can give a url uh for for an image accessibility text we can have this link out i'll have this uh link out to uh one of my websites actually you're creating like a package response with like more than just text yep if i go ahead and say if i save this oh i think it's missing oh it needs some it needs something in the default tab now if we're if we're calling this on google assistant we're never going to see the defaults or we shouldn't see the default ever i don't think and if i come back in here i'll just i will reset my interaction here let's get the test test you can see we have a basic heart here we're now leveraging the display that's awesome it said test back so the the default response there was just what it said you can have it display one thing and say another uh a great example of this is uh if you were if you wanted to if someone was asking for the the public library operating hours you may just say the library is open from 9am to 6pm but on the card on your display you may actually give the operating hours for each day of the week sunday through saturday and that's that's more information it said exactly what the user wanted but then the screen gives you the op the opportunity to provide more context around your answer however you should never your response should almost the response that gets spoken should almost never have as much information as is displayed on the screen because if you're if if it's talking too long it gets very annoying yeah yeah i've experienced that yeah it's really a user experience that you're trying to engineer into making something have a conversational element to it something that you know people want to talk to and people that are able to retrieve useful information from or interact with in a meaningful way if i get rid of this here we can look at another response this may be another another useful one you can have media content in here so this is just displaying a large image or an icon their suggestion chips are those little bubbles that appear as suggested answers or suggested actions either on the bottom of your assistant screen or on the the devices carousels are the swipeable images multiple images on the screen and then you can you can even just do custom payloads in here the google assistant functionalities to have media playback like audio or video files you can have a custom payload that sends your users directly to a youtube video on your display oh that's really cool it's really cool yeah i mean these uh chatbots are more than just chatbots they're interactive media and you can create really whatever you want out of them yep well um it looks like we're 15 minutes over michael we really enjoyed having you thanks for coming out and sharing all this very useful information uh with us um once again i'll post a link for your club orlando python and let us know if you have any uh meetings in the near future and we'll um you know we'll send some love your way as well okay uh good luck on your projects guys yeah yeah we'll keep in touch great and uh thank you everyone for coming out to this meeting i enjoyed spending time with you and learning more about these awesome chat bots alrighty uh signing off see you guys

Channel: AI at UCF

Views: 3,378

Rating: 4.768116 out of 5

Keywords: machine learning, artificial intelligence, statistics

Id: 3Ac8urt6Nps

Length: 74min 19sec (4459 seconds)

Published: Fri Oct 02 2020