Rasa Livecoding: Create a new assistant in Rasa 2.0

hi welcome to the live stream just make sure i have everything up uh today we are going to be working on creating a new assistant from uh and it looks like we have [Music] still waiting on uh youtube to catch up there we go all right uh so looks like a question already tell us how to connect roz with json api so we can embed our bots in our website or anywhere about front may not have option to fix css so um when you launch your raza server so you use raza run as the command on the command line um the sanic server you can access via um you know requests using a rest api wow uh so you can use like post and uh get requests from it hello buddy hi my dog just walked in a little 17. uh and there's a big section on um the docs about uh deployment so i would check that out hello and welcome everybody today we're going to be doing something a little bit different we're going to be starting a brand new assistant so let me start by heading over to the rock i don't know if you could tell uh my dog just ran under the lights that i have and it went um in a very concerning way but it seems to be fine so that might just be something i need to um work with in the future to a point uh uh joan from amsterdam hello new glasses they are new glasses thank you i know if you can tell but they're purple these are rosy colors yay um uh give an idea about actions so we have we're probably not going to get to actions in this video because we're starting a brand new assistant but um i'm trying to think of a good um video for i've talked about them in most of the last videos that i've done on the the dialect quiz assistant so brand new assistant what are we gonna do um i thought it would be fun to start it using the razzle playground so this is a new feature in the rosin docs razor.com playground and basically you build an assistant in your web browser and then you can download it and continue to work with it in your local ide so uh if you can see i actually have a new project it's just an empty folder called new project setup in vs code so once we have downloaded our code we'll be able to pop it right in here and run it locally in addition to having it run in this uh playground environment hi ali so let's get started oh sorry i just realized i'm not actually seeing everybody i'm seeing the top chats so what are we building um i get a lot of questions about various things um online and particularly a lot of questions about people who are interested in getting into nlp or getting into machine learning or want to go to grad school um and a lot of times like i already have that information like what i think about various things as like a blog post or like i tweeted about it somewhere so i thought it would be helpful for me if i could sit down and have a um an assistant that i could run on my website that handles just some of those faqs um and also would be like a nice you know long-term deployment and maintenance thing for for me to work on for myself because that's one of my my personal goals for this year is to spend more time working on devops uh hi alexander welcome to the stream so uh let's get started so there are five steps for us to go through the playground here is using raza 2.0 so if you haven't used rasa 2.0 yet you may not have used rules right here and i think to start with i'm going to um not change a whole bunch uh i'm going to use sort of the example here and then once we have it in a local environment i'll change it a little bit more i thought it would be helpful to go through all the parts and talk about what we might want to change um and then at some point yeah i'll do that on camera i was gonna say at some point i'm gonna sit down and like figure out what all the faqs that i want to answer are um i think it makes sense for me to do that on camera so um in our nlu file uh some things that you may notice this is now in a yaml format rather than a markdown format so yaml is another mark mark upper markdown md markdown yeah no it's definitely marked down and yaml stands for yammer ana markdown language i believe um recursive acronyms and computer scientists the thing that happens a lot um so in uh here we are providing our training data uh and do we talk to no we don't mess with the domain so um the way that raz assistant works if you're using a language for which there is a pipeline that has um a language model uses a language model if you hear like thumping and crunching it's my dog maybe down to down there all right he's just playing um so you don't need to provide that many training examples for language if you have for example pre-trained embeddings uh because by having you know hi hey hello good day and good morning and i'm going to add howdy because that's something that i say quite a bit um i can have a fairly small number of examples and still get pretty good results because you're leveraging these pre-trained uh features basically that you can you can use in your assistant uh amir says markdown yeah thank you i don't know i just had a moment of empty brain uh faisal says what are you gonna teach us today we're starting a new assistant uh i'm going through all the parts uh so we have a couple of intents here so we have this one which is greet so um intents are things that the user wants to do so these are things that the user would say to us they'd say hi hey hello example also probably hello with uh an e instead of an a as well so i'm just adding some examples here to make to help our our assistant handle a wider variety of inputs uh and then we have a subscribe intent so it looks like here people are um interested in subscribing to a newsletter so i want to get the newsletter can you send me the newsletter can you sign me up for the newsletter uh i'd maybe add something like can i get on your mailing list or something like that and then so again these are things that the user wants to do inform is our third intent here and this is when the user is telling us something that is informational some information that they want us to remember and the intent names can be anything that you like um inform tends to be a pretty common one for people to use for hey here's some piece of information and if there's different pieces of information that the user might tell you generally they get looped lumped together under the inform example um in form intense right so if we were asking for um let's say email and also you know what's your first name um those would probably both go under the inform intent and you would detect where the names are and detect where the emails are um so here are examples of people just um you know saying random email addresses um and this example.com is a domain that is used for examples and you can actually just go to it um and it's just sort of like example.com it's just sort of kept as a public good for people to use as examples so it is a real website but it uh you know you're not gonna be messing up with with anyone actually getting emails there all right so this is the training data for the things that the users are going to want to say and uh responses so these are the things that the assistant would say and you could generate these using natural language generation it's possible in raza in general i would recommend writing your responses by hand as sort of a good baseline um in particular i would recommend not using neural language generation and then solve serving those raw responses to users just because they're very unpredictable and um in the majority of applications not not being predictable or like actually correct is uh you know a problem so some responses are uh assistant can greet so uh can say hello how can i help you or hi so you see we have some some multiple uh responses here and i believe by default if you have multiple responses and no rules um which responses used will be picked randomly uh and then utter ask email so asking for the email was your email address utter subscribe um text i've subscribed and then the email that was extracted to the newsletter and then text you've been subscribed the newsletter will be sent to email so we've got two utterances for utter subscribe and two utterances for our degree and one utterance for utterasque email and these are things that will be sent from the assistant to the user stories uh so there's only one story here and the intent is that there's like one um step through of this you know example um so the story here is called greet and subscribe uh the user says hi the assistant says utter greet uh the user says hey i want to subscribe uh the um i was wrong about not talking about actions today by the way and then we're going to start a form that's going to collect information and we're going to continue to loop through that form until all of the slots of the form is asking for have been um build so slots are little pieces of information that you store over the course of a conversation um that for example the the address or if you were saving first names to you know um you know add hi first name to your newsletter that would be a slot um if you had like a newsletter and a mailing list and like a weekly newsletter and people were choosing which newsletter they want to describe to that could be a slot so any piece of information you're going to store over the course of a conversation and use in some way um would be stored in a slot and what a form does is you say hey here's the slots i want to fill here's the the things i want you to say to fill those slots go through and get all these pieces of information in order so a form is basically like a little mini story that you don't have to write and maintain forms used to be a um a separate thing that you needed to run an action server for and now they're part of raza open source so you no longer need to do that so let's look at the form um so what the form here is doing is it is just getting the email from the user um so we could have multiple forms here this one is called newsletter form uh and it's trying to get information to fill this slot email and it's going to get this information from text so just from the text that the user inputs as is without trying to for example identify an entity and then use that to fill a slot an entity is you know something that has a name that you would know about ahead of time so um let's say you could subscribe multiple companies uh through the same portal and someone's like oh i wanna you know subscribe to the company y newsletter i wanna subscribe to the company x newsletter so company y and companyx could be entities that you could identify in the text and then um use that to change the way that your assistant is uh you know handling the interaction uh and then finally rules so uh we have a couple of things here so this one is to activate the subscribe storm subscribe forum so uh rules are more rigid than stories so stories are a suggestion for how you want the conversation to go if the user is saying exactly what you expect them to say and you identify them as saying exactly what you expect them to say exactly the way it's portrayed in the story so if we have exactly you know this pattern of utterances then it'll just step through the story but if the user starts to say other things then um the the core policy which decides what to do next will pick what it thinks is most likely to be the best answer rather than just continuing to go through and do the actions that we say for the assistant to do so um stories are you know training data that train machine learning policy that you would set yourself um that's a little bit more flexible rules are going to be very rigid so um when x happens we always do y in general i would say use rules as little as possible but when you need them they're there and one place to use rules is to serve forms so here when the user is going to say hey i want to subscribe to your newsletter we're going to start the newsletter form so this one uh and then we're going to continue uh providing the newsletter form until it's been filled so here we have one slot that we're interested in and we're going to keep asking until that slot is filled uh and if we had multiple slots we'd make sure they were all filled and then uh to submit the forms you might have noticed hey we're in a we're in an infinite loop here rachel it's never going to end so in that uh um infinite loop once we finished it we're just going to uh stop doing it so the active loop instead of being the newsletter form is just gonna be nothing uh and then we are going to subscribe so those are all of the things that we need to specify for our assistant to work if we were not using a form if we were not using rules you wouldn't need to do those like the very very baseline is you need your nlu data so what intents are there what are some ways that people would express this intent you need your responses what is your assistant going to say and you need your stories what do you imagine conversations might look like all right so let's train our assistant so this is training the assistant using the data that we've provided above and the steps that we've provided above and we'll go through and give it a shot check with the chat uh amir says inform is a common intent that raza understands with no custom intent recognization of a threshold and also no fallback there is that right i have that experience using maza i don't know let me check i don't believe in form is treated separately but i could absolutely be wrong about that to the rosidox ctrl k to search inform [Music] intent pop this open while the the training is happening um [Music] information information information uh for the cuisine and number slots no intent is specified so you can add examples to the generic and form intent you need to annotate the cuisine example today blah blah blah only a few examples are shown for each intent let's see [Music] yeah i don't think it's necessarily treated any separate but i believe when you initialize a rasa assistant by default you get greet affirm deny inform and bought query like are you a bot so it is one of the ones that i believe is created by default when you initialize in a bop but i don't think that it's treated separately um by by the core in the nlu if i'm wrong about that somebody let me know in the chat all right our assistant is trained uh dr engineer says what do you say about neural engine rules while using raza neural engines rule i'm not a hundred percent sure what you mean by that um so our i'm guessing what you're saying is how do you know what to say next um and are you using deep learning and the answer is yes um so the policy that we we recommend for figuring out what to say next is called ted and we have some youtube videos that vincent did that goes into super deep detail all of the implementation you know ins and outs um and that is a transformer policy for uh figuring out what to say next so it looks back over the course of the conversation and you know uses the training information that you've provided and as your your assistant is um being used with test users and you add more stories and you you know annotate more data and retrain it will use that as well um so sort of the uh our our recommended pipeline with the ted policy is using deep learning transformers are a um family of a deep neural network um it's sort of like uh cnn's rnns and transformers so they're as distinct from from each other as that um and then the rules are something that we so again knew in rasa 2.0 we had a lot of requests from them from users um and it's also just generally useful to say hey when this happens the assistant will always do this and we know it for sure and not like why it's sarcastic in most the time but you know maybe sometimes depending on other situations it might not so um yeah i hope that i hope that answers your question it's a little sometimes just like the chat it's a little bit hard to follow what people are asking just because of the format all right uh all right let's talk to our assistant so i'll start by saying howdy which is an example that we provided and it looks like that was uh correctly identified hello can i help you uh can i get on your mailing list so this should be um should activate the subscribe form i think is what it was called what's your email address fake at exist example.com you've been subscribed newsletter will be sent to fake example.com and i don't think it'll be able to do anything if we say thanks yeah so um here i've said thanks so this is an intent that is not been shown to assist in before and by its response we can tell that probably it thinks i was saying hello so the most likely um intent was identified for this turn was greet and the assistant's like okay i've been greeted i should agree back so definitely some some room to uh to grow there and we can now download the project so uh if you don't train your assistant first you can't download it and that's something that uh i discovered so i'm just gonna pop this out and start that download [Music] waiting project.zip fantastic i think i might be able to just pop it in here will that work hey fantastic okay can i unzip it here i'm gonna mess around in my file system so i'm just gonna pop up nope yep all right we're good uh so i have extracted all and here is my project and here's all the things that's in our project um and you'll notice there are more things than the things that we um uh actually trained so let's go through and look at all these files and talk about what they are so the config um i mentioned the policies that you could set that you uh your your assistant figures out what to do next and uh that's policies or what these called dialogue policies it's shortening for um and that sort of determines the course the conversation so the memoization is hey if we've seen this exact theme in stories just keep going like that's what we want people to do just follow that path the rule policy is hey if one of our rules is triggered that we've provided then do do the thing the rule says to do so we're using rules to serve our form here so we have to have the rule policy and then the ted policy which is that transformer driven policy that i mentioned are driven it's a neural network the family of neural network is transformer i have um a couple of youtube videos on transformers and various applications um that's the whole thing i have to say there uh and then you can also set some hyper parameters so uh max history that is sort of like the attention span of the assistant um if you make this bigger it's gonna take longer to train you might add like a little bit of additional latency but you also might get better results if you have a conversation that continues for a while um ebooks is just sort of like how many rounds of training basically and then we also have uh the nlu pipeline so the the policies and the dialogue policies um used to be called part of rasa core is what do i say next uh the nlu pipeline is hey here is this human language i want it in a format that the computer can um hand and handle and understand so um some things here first of all it's in english um we are i believe by default where we assuming are we using spacey here i don't think we are um but this will determine some some things in the internals if you have a language that we don't have any like sorry i just realized the audio was horrible for the last little bit sorry it should be fixed now um i'm here to say is max issue i don't understand is it a hyper hyper parameter in the core pipeline it is a hyperparameter for the specific policy so this is the hyperparameter specifically for ted and if i had other i don't think there's any hyper parameters for the memoization rule policy but if i had them they would be tuned directly underneath them so for each policy the tuning happens directly underneath that policy similarly for these you know different parts of the you know you pipeline uh this number of epochs applies to the diet classifier this number of epoxy applies to the response selector and so on um is the keras policy still possible i believe the keras policy is deprecated but i would double check in the docs because i i will admit i don't remember all of the specific pipeline changes between 1.10 and 2.0 okay and the pipeline takes all of the um you know human input and turns it into a computer readable format so it's doing a lot of things um this one is breaking the text apart into words in this case because it's english um but basically every time there's white spice it's like oh those are separate entities i should treat them differently um the red red dex featurizer uses regular expressions so you could tune this quite a bit yourself if you had um say specific character clusters you were looking for or something like that uh and then various featurizers so all of these are uh creating are taking that text input and extracting um numbers from it basically doing some transformation to cause it to be numbers instead of words if that makes sense so i have a video that came out on monday on what featurizers are more broadly um the diet classifier so this is um a classifier that uh does both intense so what is the user trying to say and entities um so what you know things are the user referring to that we know about that we might want to you know use in the future for different purposes um and then a little bit of tuning entity synonym mapper hey these two things mean the same please treat them the same basically uh and then the response selector to respond select listen it's really here um credentials we're not serving this anywhere um if we were um we would you know follow the instructions in here and then in the um deployment section of the docs to talk about what to do um by default uh rasa servers are launched with the rest channel so somebody asked about about json so when you start a lawsuit a rasa server it has you know the ability to take rest requests um so that's the easiest way uh but if you want to use a specific channel like facebook or slack or um sakuraio which is sort of like an alternative to rest or don't know what mattermost is but if you wanted to use it this is how you do it i think it's a bot service um bot serving service as a as this as this area domain so these are all the things that our assistant knows about um so this is going to say hey after 60 minutes reset the session they ain't coming back we don't need to remember nothing um do we want to carry over the slots to the new session so um if somebody you know reloads the page do we want to remember what they say um intense so these are all the things that our assistant knows that users wants to do it knows they want to say hi that they want to give it information and they want to subscribe to the newsletter and that's it um and uh i think actually we should add a bot challenge so let's do that and i'm going to file a bug for that because this should have a bot challenge by default is it the 28th today right playground bot challenge oh this pencil i think this will also be a good example of how to extend the assistant um slots so we have exactly one slot here and it is saving the piece of information email within the text of the conversation um and uh if we did not carry over the slots then we would need to um enter the email in every time if that makes sense so it makes sense to carry slots over particularly if you have like you know a customer service assistant and you know you're going to be passing it over to human and back to the assistant and you want to make sure that you you continue to have the information that the user has provided over time if you want a specific slot to change what happens so let's say um in the example where we have like the company that you want to subscribe to the newsletter of um that would influence the conversation because you'd launch a different form and like store the email somewhere separately for the company x newsletter and the company y newsletter if that makes sense uh and then here is what the bot says so this we um this we edited ourselves if you remember in the playground um actions so the assistant can say hi it can let you know that you're subscribed um and the other ask email you're like hey why isn't that defined under the actions um so email is the name of a slot and we have a form and the form when it's asking for a slot will ask using this utter underscore ask underscore name of the slot template so we don't need to hey talk about that especially because it's being handled within the form uh and in the form we have only one form here it's called newsletter form uh it wants the email slot it's the only slot that we have and it's gonna get the type from text and then the version of our system is 2.0 endpoints we did no we didn't do endpoints sorry we did uh credentials um so if you wanted to um you know a pass information back and forth if you wanted to save your stored models somewhere uh if you wanted to uh you know if you were having your custom actions uh run somewhere else and you wanted to point your assistant to where your custom actions were being run um you could do that um end point of the action so this is where you would really um you know handle you know passing information back and forth and be like hey assistant i need you to look here for like the history of the conversation hey assistant i need you to look here if you want to execute an action um so right now we only have a form that's being handled within uh rasa open source if you want to run code like any code that you need to um yeah any code that you need to um just use your assistant and have it do what you specifically need it to do um that would go in the actions.pi file if if you're using python it can be any language that you want but we have an sdk for python so that's easiest um and this is where you'd put the code that um you would you know you created an action and then whenever you needed that action to be triggered you would add that to your stories um you'd add it to your list of actions in your domain file and then when that you know particular action was identified as the most likely next thing to do that code would be executed um by making a request to the action server which if you're running not locally um you would provide that request your the url to send that request to in the endpoints folder wow um and points this one right here uh yeah that's that uh and then forms so here's a list of all our forms we only need one and all the slots that it needs it only needs one uh and then responsive so these are all of the responses that the user can give and i believe you can have this information it's fine to have it in two places but you can also have it only in the domain you don't necessarily need another forms and responses file if you don't want them uh data this is where our nlu data that we slightly uh edited so these are the examples of what we think our users are going to say rules this is the rule for executing the form action and then stories here's how we think uh these interactions are gonna go so let's uh add a little example so we have another intent that i called bot challenge if i remember correctly and this is just a way to make sure that whoever's using our assistant if they want to know if they're talking to a bot they can get correct information about it so bot challenge uh and i'm gonna add some examples and my examples will be are you a bot am i talking to a human is this a chatbot um so these are just some things that somebody might say if they wanted to uh i need a pipe here i think i need a pipe here there we go um so i just added the upright pipe character there because i was like the syntax highlighting is different i think i did something wrong and it's is fixed it's fine um so these are examples of things that somebody uh might say if they want to know that they were talking to a bot uh and then i'm going to go into the domain and i'm going to uh create a new utterance and i'm going to call it and i'm just going to have it say i am a bot powered by rosa uh and then i'm going to add that to my actions as well utter i am a spot so a new thing that our assistant can say uh and then i'm going to uh in my stories i'm going to add a new story let me make sure i got that right that i don't think i think it's on space there we go story are you a bot uh and it's only going to have two steps we're gonna get intent dot challenge so this is the name of the thing that i think the user might say and then we'll have the action um yeah excellent all right so let's save that now uh we've made a change to our assistant it's not going to change the conversation yet to do that we are going to need to uh um retrain question uh have you ever worked for game ai no i've never worked in gaming um emma who's the community manager who read the um uh behavior with us last two weeks um i believe it was worked in gaming maybe you still have as well but no i've never worked in never working game um so i'm gonna take a quick second second and just turn to the chat how do we determine the value of max history um so it's a hyper parameter you would tune it you would try different hyper parameters if you thought that um you know more history would be helpful for your assistant um and then see what happens using you know your evaluation criteria uh amir says um i built a custom connector while integrating with uh you know messaging channel what are the new channels added in raza 2.0 i don't know if they're off in my head that sounds like a question for the docs i i just go to the docs and check for connectors uh alexander says what policy and pipelines would you recommend for language with no pre-trained models i recommend training custom embeddings um and then there's a walkthrough on that on um i don't know there's a walkthrough there's definitely more discussion of it in the docs is the word that i was looking for uh how to evaluate chatbot systems that is a great question um yeah so in terms of like from an engineering standpoint how do you run tests the way that you would have a test is you have an end-to-end conversation you know how you want it to go and you check that every time you update and change your system this end-to-end conversation or conversations do do what you expect them to do in a broader sense how you know if your system is good really the most reliable way is to have human testers who you know provide feedback on the system so what we recommend at raza is once you have a prototype you share it using raza x which is just a nice interface that you can use to share your assistance and then have test users use your assistance and once they have the data that you know their conversations you go through and manually annotate it oh this conversation broke down you know that intent was identified incorrectly fix the data annotations maybe provide additional examples of conversations going well retrain your assistant you know serve it to users fix the errors retrain serve it to users and have this sort of cycle of almost human in the loop um development where you're continuing to use and improve on the um how the assistant is actually doing in-situ in a broader sense how do you know when it's good like what metrics can you use for evaluating chatbot success um just very practically what do you want your chat bot to be able to do and when people encounter it do they do that thing like do they successfully subscribe to the newsletter do they get their questions answered um do they you know buy a hoodie or whatever it is that your chatbot's trying to help them do um yeah so i would say human evaluation and are people doing the thing that you're trying to help them do with a chatbot it's a big question uh amir says every time i change the domain i have to retrain is there any way to tackle this with no retraining no unfortunately because the um every time you train a model it's based on the specific things that you set up to train it right so the specific and all you the specific uh you know intents that you have the specific things your assistant can say um once you train that you need to retrain your assistant just because otherwise uh it's going to assume that the things are as they were before you change that file if that makes sense uh paul says after we collect the data using forms is a good practice to run a method to perform an api call or should we just use the form to call an action that performs the api call uh run method i depends on what you're using the data for i would say if you're using the form data to you know make a query i'd probably do that as an action but it depends on the api uh prerequisites for learning raza um so we actually have a raza for beginners course that starts you like from from zero i would say if you have a particularly complex assistant um knowing some python would be helpful uh particularly if you're gonna want to customize a lot of stuff um if you're deploying your assistant i mean that's you know uh i am myself learning a lot about devops so being good at googling questions or like using a search engine to find answers to your questions and figuring out how to ask technical questions i would say is actually probably a a better prerequisite than python i was like how to execute a story only when a slot is set or when the value is null uh use a rule for that um and if the it's a whole series of things you can have a rule with multiple parts okay so um let's let's train our assistant and see if this bot challenge uh works good uh you can't see because it's behind me but in the the bottom corner there where it says fresh rasa install conda um that is a conda environment that i have that has rasa 1.10 installed on it and if you're working on windows we have an installation video that walks you through installing raza um and it asks you to do anaconda environment so this is my conda environment and it's the it's the wrong one so i'm gonna move myself back over um i need to make sure that i'm running this in an environment with raza 2.0 and let me try to remember the cond command uh i think it's conda list enves does that sound right i want to see a list of all my environments because i don't remember what i've called it nope that's not it hmm okay so this is the packages in the environment okay so this is listing the packages that i'm using to create my con environments which is not what i'm asking um i think okay let's see if i can do this so i'm going to deactivate my current um my current conda environment and i think it's fresh raza install underscore two i think is what i called it oh okay yeah okay that seemed to work so uh let's first of all um i am currently one directory above uh where my project is so if you see here i've listed all the files and i can see a folder called project in a file called project zip so if i try to do raza train here i am going to get an error probably because i don't have the files that raza is going to look for in my current directory yep so see it's looking for the config.yml that doesn't exist so it does exist in the project folder and i'm just going to do raza train and train a new model and see how it go so you can see that we actually have a model in here this is the one that we trained online in the playground all right failed to evaluate validate data.nlu please make sure the file's correct in all mandatory parameters are specified here are the errors found um intent and example was not defined let me see pretty sure i messed up i t e and t and t okay let's try this um so i did example instead of examples and i think that's why we got the error yeah there we go okay now it's training so it's training the nlu model now which is what's going to allow it to identify the um the information that the users provide and then it'll train the the core model which is the you know domain policy the domain policy the dialogue policy that says what your assistant should do next 2dm fast says does anyone know how to program in python okay some people um yeah i would say the um i mean like how much python you know really depends on how complex you need to know depends on how complex your system is going to be um if you're going to be doing a lot of you know really involved stuff then knowing where python would be helpful i mean or you don't have to use python you could use any programming language you like but you're probably gonna have to do your own sdk oh people are helping me with the the conda ends list sorry i didn't see that until it was too late uh which one is better for normalization spacey tokenizer or whitespace tokenizer can we use them both no you need to use one tokenizer and the tokenizer that you use should depend on what the rest of your pipeline depends on so for the config uh so for uh the diet classifier i believe these are all required so the diet classifier expects to see these featurizers in this order in order to work so the specific parts of your pipeline before your nlu you know models the ones that you're actually training are going to depend on the model that you train and which ones you should use with which model will be uh talked about in the docs good question uh uh friso says for prerequisites nothing if you don't want custom actions just for raza you can learn from videos without any prerequisites well thank you um hopefully uh 2dm fast says i need someone with python programming experience um i guess post a job ad for python dev uh thank you all for the uh help with my conda that i did not see yet uh his own says example using numbers please um i remember this this is the thing where like one for like the the text example for like press one for x that apparently is very popular on whatsapp it's in my it's in my big list of tutorial ideas um i don't want to promise a timeline but i do know that people would like it so that's where that's where i am with that um oh and ahmed says examples instead of example thank you if only i'd read the chat faster uh dr partha is there any embedding used in raza uh yes you can rather not by default but um yes pre-trained embeddings for word embeddings are a big part of the different um many of the different you know nlu and core components um and if you want like a lot more details the algorithm whiteboard series that vincent does is really really good and goes into a lot of depths about you know and the internals and what they look like let's talk to our assistant now let's see if it works so uh shell so this is going to launch a very like simple example right in my uh right my browser there we go and you can see it's starting a server so this is a sanic server that's serving our assistant uh i'm gonna start by saying hi uh the assistant replies hi uh i'm gonna say are you a bot so i'm i this should be identified as a bot challenge and the assistant should say i am about powered by raza oh did not all right [Music] let's pop back up to our stories spot challenge i'm a bot yep that's right uh let's check our nlu yeah so this is an example that is explicitly provided in our nlu um oh i see what happened i see what happened sorry um so what's that let me check my stories no that's not the case i was thinking that maybe when we say a the assistant immediately asks for our email address to subscribe but it doesn't so i should have had the intent subscribe here so let's restart and do command c let's restart our raza server and i'm going to do debug so that we get a lot more information about what's going on and why this is happening right say hi uh and from here i mean i realize there's a lot of information that we get back uh where is hi where's the last thing that i said there we go so here's the last thing that i said and here's all the things that the assistant did uh before the next utterance was made so doo doo doo doo uh first of all we're starting a new conversation um we're you know creating a lock which means that you like during the course of conversations things aren't going to change underneath you basically um creating a new tracker for the id so the tracker store is what stores the history of the conversation starting a new session for conversation id uh and then we're starting the session so by me saying hi i've kicked off the creation of the conversation um remember that we're storing our slots across sessions so it's storing all of that information uh and right now we don't have any information set in our slot so uh by default uh slots are initialized as none there's no trading model for response selector the component is either not trained or doesn't receive enough training for data yeah um i don't think we need this because we don't really have uh we have multiple responses but i believe at this point they're just going to be selected randomly uh adding following selector to key messenger blah blah blah blah blah blah trackers for events okay um so here's the history of the conversation so far so the user said hi that's me um and uh before that the uh assistant was doing this action listen so at any given time the assistant is doing an action and just sitting there waiting for user input is action listing it's a built-in there's no applicable rule so our rule policy was checked um and then what happens next is going to be predicted using the memoization policy so if we've seen this before keep going and then the predicted next action is utter greet so we've seen in our story that user says hi okay i should say hi i'm very confident that that's the case confidence goes from zero to one um and the assistant says hello how can i help you and then next we're going to wait for user input and now it's waiting for us so i'm going to say are you a bot here and see what this intent is identified as okay my guess is that the intent here is not going to be uh are you a bot okay response selector okay so the intent is yeah so the intent is being identified incorrectly it is being identified as subscribe and not are you a bot so i'm going to add um i think more examples for bot challenge to see if i can get a little bit of a better uh can you tell me the newsletter kind of get on your mailing list okay are you a bot is this a real person you're a bot aren't you all right so now that we have some additional examples um let's restart retrain and then i think we'll launch a raza x instance um see how that goes i may need to update raza x which is why i haven't used it at first raza train so i've added i've just added some more training examples we shouldn't need to change our core policy because we haven't changed the order that we say things in we should only need to change our nlu policies and i'll u models because i've added more additional examples uh a link for learning mazda from scratch i'll get the um those are for beginners i'm gonna pop that in the in the chat once i find it um one sec i'm just having a little bit of a hard time pulling this up [Music] i know i tweeted about it though so i'm gonna see if i can there we go all right um so i'll post about it in the chat and that's probably the best way uh to get started with frazzle because it walks you through the process and should be a little bit more helpful uh we'll have to install raza reinstall raza nope you can update instead of reinstalling if you would like how to handle multiple intents at a time in raza you can do it it's a little bit hacky um there's a section on multiple intents on the dock so i would i would check that out uh oh yeah faces like it deluxe always great advice um uh krishna i'm trying to build a chatbot which can answer frequent error solutions how should i go about it there's a lot of different ways so i don't have like a good example off the top of my head of what will work uh quick um although the uh hairs on suggests the faq bots as an example and that could definitely work as a structure for it uh gender says if you try directed out high i would believe that it would work as there is a story followed by the high intent yeah the thing about stories is that they are they're guesses right um so the problem i think is not with the um the core because the intent was identified as subscribed so let's try it again and see if we can get the intent to be identified correctly because that's when i went back and i looked through all of the the things that the assistant did um if the intent had been subscribed which is what it was identified was as that would have been the right path to this story that makes sense all right let's try again okay still happening so let's see if this is the uh user agree intent subscribe yep so the problem we have here is that our intent is being uh identified as subscribe rather than the assistant is picking the wrong thing to do once it gets the bot challenge so intent form 10 challenge did i in fact spell this correctly when i put it in my domain always a potential challenge and i believe you can still have underscores in raza 2.0 um [Music] interesting um yeah i think that's a good question so let's see what happens if we do it on its own i think it's a not a bad idea of a gender uh raj says how to handle a for loop efficiently in a context like adding an item for a cart i would do that as a form okay are you a bot will we get the right uh hmm okay interesting uh let's try something else is this a real person that should restart the conversation right did i put the slash backwards sure did all right hmm okay so uh the intent is consistently being uh identified as subscribe all right story yeah steps yeah that's right that's right okay i don't know why i'm looking at the stories i know that the problem isn't the stories maybe if i move this higher is it possible that i've messed up the yaml for sure it's possible 100 it is possible but i'm not getting any i'm not getting any um syntax errors in the ammo let's try retraining and see if that does it i don't think it should uh i'm just there we go uh just seeing if we can get better results oh great i forgot that we were not in the right uh uh directory and also every time we train a model fyi it's saved as a new model so we do the the versioning for you automatically okay so bot challenge is definitely being identified as an intent and included correctly so it's not that it's not being uh detected problem solving uh i think story's not correctly built running your training is not done proper that's possible um i don't have any tests written i don't think i can just test it without the tests all right let's see i don't know okay yeah i don't care about that okay yeah i don't have any tests written so it just doesn't work ah let's re 1v so like a little verbose a little information but not like the whole thing because debug gave me like a lot of info [Music] on me try a slot is deprecated and we'll be removing raw's open source please change that type and configure okay i'm just not gonna mess with that because that's the email slot i don't care about that um are you a bot is this a chapa also apparently 1v is not the flag for uh oh that just might be the server flag that might not be the the right flag for um the shell hmm uh amir says can you check rosa shell nlu for sure okay step one type command correctly [Music] office says if there's a form in the story and there is another action before the form while filling a form somehow try to predict the previous action and put that action in the form even though we didn't call it um so you can have a form be interrupted and then go back to filling it out we did that in the uh the assistant so let's see is this a bot interesting interesting so what i see here so here are all of the uh intents so it's being identified as intent subscribe with confidence 75 ish um i have a video on confidence if you want to know more uh on in the nlnu for developer series uh but if we look here at the intent ranking so this is the ranking of the intents by how likely they are or how confident the model is it's not likelihood sorry um number one to subscribe number two is in form number three is great uh conspicuously missing here is bot challenge what did i mess up okay so the reason that we're not getting uh we're not getting the correct answer to the bot challenge is because the intent is just not being considered did i miss it somewhere in the domain extremely likely all right let's see uh i don't think we need to say that it's unfuturized i think i can just get rid of that actually um i'm probably wrong about that intense great inform subscribe bot challenge maybe if i take the underscore out maybe the underscore is messing things up try that uh and then let's update it here and then update it did i not save [Music] did i not save my file my domain file before i retrained because that could very well be the problem all right uh control c blah blah blah brother train one more time um [Music] you sure that you loaded the proper model it should by default load the most recent model uh but i think deleting all the models is a good idea because we could just be using them all that we trained uh peace out uh it could just be using them all that we trained in the playground okay so i definitely do need to have most recent call run by run code i mean pi module in main blah blah blah in train uh additional arguments line 55 train dot pi comma divide run run until complete train async dot pi imported up high get domain itself gets stories imported up high get stories exclusion percentage so i'm just running through these and um all these like raws it up high imported up pi comma dot pi stuff um those are internals so i didn't change those those aren't the way that they uh they should be so in theory uh okay so it is something in the domain um it was from yaml path domain line 144 i don't think we have that many lines no butcher don't ah don't mean not reserved nice oh okay yep this is just because i took the unfeaturized out remember when i just said that it wouldn't be a problem uh let's just look up slot types real quick raza slot types excuse me that's the legacy one box raza so what an expression [Music] browser trained nlu control k nope okay uh slots types raised if the slot type wasn't valid helpful what are the valid slot types initial value value conversation that will be featurized intense influence predictions of the dialogue more features with zero blah blah blah blah blah and slots by its last name any spots object ah right objects slot objects all right concepts of slot the type text oh i think it is a text all right let's do that and then try retraining and see what it happens i'm pretty sure that's why we got those errors that we just got all right so it's training okay the version key is missing in the training data file will be delivered 2.0 2.0 yep it is [Music] okay and that's from the from the previous one looks like it's training pretty good all right we have a new model we know that that's the model that we're going to be using because it's the only one that we have and let's try raza shell nlu and see if we actually have all the intents that we have provided data for uh sahib says how can we get different utterances for different questions i placed in the same intent they're actually kind of the same but sometimes their answers are different that's why i'm asking i mean if the let me check something real quick i remember people were talking about it i don't remember if it was actually added uh yeah okay so one thing that might work for that is a hierarchical intent so uh so we do let's see if i can find it um so you have like uh beans sorry um so you have like a hierarchical hierarchical structures final hierarchical texture there we go okay so if you have um a hierarchical intent structure um you can uh do that using rasa open source components that might be a good way to do that if they really like our separate versions of the same thing if that makes sense all right let's see if this works is this a box yay okay fantastic we did fix it so i guess it was i don't know maybe we should have deleted all the models to begin with and then just retrained a new model mysterious sorry ctrl c is to to close the chat not control k control k is to uh search all right um yeah so let's really quickly uh run through a conversation uh where we use this new intent and new response and then we'll wrap it up for the day because it's been a little over an hour at this point uh i'm here says you can make different other responses in domain.yaml under the template selection and also utter declare under actions so let's start by saying hi to our assistant uh is this a bot and we should get the correct answer here great and now we should be able to say can i sub to the newsletter uh fake example.com okay um so this particular story um saying hi then asking if it's about then starting the form was not shown in our stories but it can handle um this sort of like new set of interactions in this order um that's the end of the sentence uh because we can um handle different orders because of the way we do our dialogue policy wow okay i'm gonna go make coffee excuse me put my glasses on thanks so much for joining me today guys i hope this was helpful um walking through the process of starting a new assistant adding some stuff changing some stuff uh and i'm gonna spend some time thinking about what sort of things that i want in my my faq assistant that i'm gonna be working on and yeah so tomorrow we're going to be starting a new paper um i think it's just going to be me this time and it's about um attention and saliency so a lot of sort of like explanatory models of what is this um you know using um are using attention as in the transformer attention is all you need sort of measure but there's another measure of what is important called saliency that has more of a i'm familiar with it from the linguistics literature and it has a lot of meaning so basically it means um the things that are a human is considering important when making choices of doing things what information is most influential i guess is a good definition i don't know it's very overloaded so i'm looking forward to the paper we'll start that tomorrow and then on friday we'll be back with more live coding so i'll see y'all tomorrow for paper reading friday for live coding and until then um if you have any questions head to the browser forums and i will see you

Channel: Rasa

Views: 18,072

Rating: 4.8476191 out of 5

Keywords:

Id: xK1I9z52IB4

Length: 75min 38sec (4538 seconds)

Published: Wed Oct 28 2020