Virtual Agent Academy: Keywords to NLU

hi everyone welcome to virtual agent academy good morning good evening wherever you are i wanted to introduce you to our steam colleague david from the nlu product team who will be presenting today's topic um very exciting on migrating to nlu so david i'll turn it over to you thanks michelle uh good morning good afternoon and good evening everyone um very nice to be here again and uh looking forward to telling you something about the virtual agent academy and then in specific to moving from keywords to nlu and um this is a like an entry level to mid mid-level academy session so don't expect to go too in-depth here but we will walk through the basics as we have a lot of customers out there that are getting started on a virtual agent and nyu and would like to know how to proceed so let me walk you through the goals for today so just quickly brief briefly go through getting started in some good practices then getting to understand what the migration to nu actually is then i'll actually show you through an exercise to go from keywords to nlu and then talk a little bit about how you could measure success all right um getting started and recommended practices um so make sure that you are familiar and look around the virtual agent and nlu community uh there's a wealth of information there you can ask questions you can give answers there's a quick start index for all of the the articles that are around there and not only articles by the way it's also references to now learning or the docs to quickly get you started as you find your way in the virtual agent and niu domain and topic there's a great number of articles out there also with very advanced stuff that you really would like to take care of so make sure that you have a check in that index article the index article is is continuously updated and i know that there are several articles out there that aren't being referenced yet but we'll uh update that in the in the coming week or weeks all right um first and foremost think big but start small as you do your va and nlu implementation you really have to take care and and and approach things step by step and use an iterative approach so um the first step would be to make sure that you have an owner and you have governance in your organization around who gets to basically uh create what and maintain what that is a critical starting point then obviously you need to define uh what your goals are and also the scope and timelines obviously and don't start out too big as we will get to right make sure you agree on your timeline and um first and foremost if you really get into va and you want to make sure you have an impression uh the biggest difference between a search and a virtual agent is the fact that you can actually greet people and make that personal so start the conversation with a personal and intelligent greeting don't forget to gather feedback as you go along right it's critical every step in the way whether you're a weekend or a year in the feedback is critical into tuning and making sure that the goals that you set out of are still having a chance to be met and you can actually set new goals along the way start off with five topics with each iteration and then move forward forward and forward this way this gives you a chance to basically understand how the five topics land inside your organization and also are able to basically grasp the feedback that's coming in because it's only supposed to be around these five topics or unless the feedback is on a topic that you totally missed out on those five topics should be your main volume drivers as well right that's that's also important we'll get to this later a little bit make sure that you define some metrics so that you understand if you're meeting those goals all right as you go along you scale up you use nlu and this is what we're going to talk about today to be able to basically add more and more topics to the virtual agent and then have usage analytics in place to make sure that you understand usage and how they compare to your goals all right let's talk about natural language understanding so you may have seen this many times already so i won't go in depth but natural language understanding basically a set of utterances that have an intent and sometimes an entity that your end users will use in order to have a conversation with your virtual agent and in which they want to get work done faster and work means they want to get something done or they're having a problem that they want to get fixed etc so they have an utterance they say something and then they have an intent with that and they just want to make sure that that what they utter is resolved quickly and swiftly and we will use both intents and the entities in order to make the flow more let's say human-like as you can see here but also faster because you can skip steps along the way um i think another important aspect to touch upon is that with the addition from with from nru compared to just keywords you can actually introduce your organizations and industry lingo into the conversation and have that work seamlessly driving both the flow and the improvements to the flow and the expectations from your for your end users so how does this work so you may need to make sure that you activate the appropriate and applicable nau plugins there's uh you can follow the docs uh there's various very good articles on the community that will help you with this section you create a new nlu model actually create a new application in this in the studio and then with that you create a new niu model then you create new intents or better you just import existing intents for example for itsm hr or csm out of the box you then map those utterances to an intent right you don't need to do that if you get the out of the box because they already will have a set of utterances but you may have specific utterances the way you speak about things in your organization that you would like to add to that intent you then make sure you train and test the model so be sure to have a set of test utterances right what's a good set of test utterances well you should for every intent right if you have let's say 20 10 to 25 utterances in which you train it make sure you also have five to ten test addresses enable nlu for va that's the easy part because that's basically just flipping a switch as you will see in the exercise then you map the intent so the nlu intents to the virtual agent topics you then test the topic discovery and tune the nlu mark and under topic discovery i'd like to add some specific information as you can see here this is a set of intents or topics that get a certain nou score that you can see under the confidence column and uh the highest obviously is the the the topic with the highest confidence and in this case this example there is actually six topics that are above the annual u threshold of 0.7 that will all get pushed to uh va and then va the the virtual agent determines based on the delta which in this case is 0.00 which is smaller than the delta so it means that the the delta the difference between the printer issues the first and the second intent printer issues and check it check it ticket status is smaller than uh the the default meaning that all of the topics above the threshold will be presented to the end user in the menu that is critical to understand so it's not just the first or the second intent in this case because the other four are outside of the delta no because the second one is closer than or is close enough to the first intent all of the topics above the threshold will be shown to the end user as long as they have a matching topic that's also a important note to make there's a big article on it by nabil on the community so read up all about it i will make sure that the pdf becomes available after the session all right so what's the difference between keywords and nyu it's when you have your printer issues you basically need to start typing a sentence and along the way um the virtual agent will check whether or not there's a keyword match and will present in a menu the topics that might match the sentence that you're typing in this case you can see that currently printer issues has been matching and then it will allow you to make a selection based on a menu or however you've designed the conversation with nlu you could actually skip a number of those steps because it picks up on the intent that you that you already typed and it immediately goes down into the solution or into the conversation of the intent to the point where it either runs into let's say a question or an answer and in this case it will immediately point to yes i know what you want you need a printer installed so this is what you need to do so let's move to the exercise and i will switch to my screen here so basically i have um i will make it a little bit bigger i have installed uh all of the plugins around virtual agents and nlu have also been installed uh because i just needed the conversations to be placed i have a number of uh topics in here that are openit ticket email issues password research as you can see vpn related but foremost what is important is let's say the setup topic so the way that you have like a fallback scenario or a greeting and that's that setup topic uh greetings is an interesting one that i briefly wanted to show you right so this is the default one which basically comes uh out of the box if you don't do anything it's just simple two messages i'm your virtual agent let me know it's not personalized uh anything like that right so if you want to change that you just go into general settings and then you have your setup toppings and you can look at the greetings topic and then you would want to change it but right now we don't have anything to change it to so the way to go about that is just search for the right greet so in this case i'd like a dynamic reading topic but because they're all um basically static you can't really change anything because they are part of the in this case the the itsm solution you need to duplicate them so i'm already installed i have my application open va academy and i'm now going to go and say greeting dynamic saving and then publishing it all right as you can see it's part of the category setup topics and that's why when i go to general settings i actually can go greeting dynamic save it and then if i go back to my agent and reload it actually knows me and it already sees that there's uh there's some more information that he liked to show me so this is a more personalized meeting and you can go all out here improving that so that's a a quick start and a quick improvement to just making sure that it becomes more dynamic all right the second piece is to make sure that we move to um to nlu but before that i just wanted to quickly show what happens if i say hey um the printer installed right that's where i get my menu right so this is still without nou in just keywords and i select okay what do i have i can't find the printer and then i get my selection so let's go back now we are satisfied with what we have but we want to enable nou so we go to the niu settings also under general settings of the virtual agent we click enable nlu and you have to select your provider in this case servicenow we only have english in our organization at the moment so let's leave it at english we could add german and french for example in the paris release but right now we just stick to english and then before we can start mapping and changing the existing topics to nlu power topics because as you can see right now if i go in here we still we we've moved away from keywords and now we see a [Music] nlu model here but we can't select anything because we also don't have it and i have the wrong scope which is not a problem i'll just switch but first i want to make sure i switch to my studio so here's my studio and i'm going to create a new nlu application or part of my application is going to be nou based the full model includes both virtual agent for example and nlu so if you're creating in a specific domain and in this case the greeting dynamic i think i actually should have created it in global not in not in the in the scope of my application because it's itsm specific so um i'm just going to call it uh rnau model so i don't have to limit it to itsm i could add all my company's intents here and then instead of creating new intents i'm just going to import intents from itsm and i know that i have printer issues i want an open it tickets i have some challenges with my email issues those are the top five lift and password reset so these are the five topics that we're going to start off with and importing that will take a little while give it a little moment as we go along might be a good moment if there are any burning questions that i need to answer marcel i could pick one up okay it looks like we have one open question and it was more around um what assets are available maybe to partners through the partner portal on maybe scoping guides or information about past presentations so david is that something we post to the community after the presentation yes i can i can talk to it on the community after the presentation [Music] there is currently nothing partner specific that we have out there that's not available to all but i'll reply to it in more detail on the community after the session okay great and the other question that typically customers will ask is um let's say you want to include i.t intents as well as hr intents you know maybe some finance intense is best practice to keep the model separate or is it to keep everything under one model so that you can train and test it against the other intents within that model yes it it depends uh depends on your organization and the way you've set up the organization how you're organized sometimes organizations are organized in such a way that they are different departments and they don't want the intents to be part of the same model because they want different people working on the on the model and owning the model so that's sort of the governance side of the house and then also it becomes a let's say a manageability challenge if you have one single model with let's say uh 50 60 intents that that also becomes kind of unmanageable from that perspective if especially if there are multiple domains involved so um it depends is the uh to be to the honest question the honest answer i can give at this moment right it depends on um on what you really want if you start off right then start off with one model and um you can always let's say branch out from one model into multiple models as you grow or as you basically scale out to other domains i think actually that's more of a good guidance more on that to come though in in subsequent uh sessions it's not i think this is a good question that we will answer in a follow-up where we talk a little bit more let's say on the 201 for um managing your uh your model and and conversations so it's a good suggestion all right so i've been publishing it and let me just test i need a printer installed all right so i think nyu is working at least on the application side of the house let me just go back into the machine and go back into my virtual agent designer i go back to global because of most of the topics i actually created in global so let me just go to printer issues and printer issues as you can see here this is the paris release so if you're on new york or orlando you won't see this tabbed interface it will look slightly different but the properties page is something that you all know about and this one is important because you can actually pick your niu model here as you can see because we've published the the model we can now select it in the select box and then in this area this is where you select what intent should be associated to this topic right if you don't pick the right intent here you're going to get awkward results so be mindful so when i save this and i publish this i now should be able to get a different experience in my environments so let me see if i can rewrite need a printer installed all right okay so hey we've improved a little bit right we didn't need to pick a topic it found the right topic um but right now it's still asking okay what's the issue that you have and so hey i need the printer installed so i would still choose this one and the reason for that is that we did not use the entity matching and we've not configured that and that is a different academy session that was actually the last academy session that i did so i'll make a note that i point you to that as we go along at the end uh you can find the whole session around how to set up entities and basically how to improve this so that you could actually with the can't find the printer on my machine or the need a printer installed utterance you get a direct match to the need to install a printer and your end users basically get served in in one flow instead of having to answer two three or four questions so um you have to repeat these steps for each and every topic as you go along so in this case right we have password reset we check the properties we select the model and we select the intent publish and as soon as i publish it we've actually moved away from keywords to nlu open it ticket same thing there you select your model nlu model select attend intent and hit publish now i do want to point you to something if i quickly go back to my printer issues all right um if you click on nlu intent because you're dying to see what that does it actually brings you to the intent as you saw it uh up in the no mod it's basically the studio version that now gets brought here so you can also do some testing here and train the model from within the va interface the languages is more important right because here you can actually see what the entity mapping does [Music] so you need to pick what utterance it matches the user issue right so it matches the printer issue right uh there we go and then you select what english word ties to what entity if you have any in this case right did not ditch this step work you we don't have any um entity for it so you don't select it you save and publish and then you've actually matched your made your entity mapping successful there we go so as you did this now you're thinking okay so what what did we improve on compared to where we started off how do i measure success and uh that is uh challenging um but there is an answer to uh to to helping you out there as well right so for example when you have your topic discovery so why didn't this utterance bring up the right topic right what should i do to make sure that we get a match um keywords seem to be having better accuracy why isn't nlu working and can i add the exact utterance to the model right those are questions that you may have or that your end users may have or that your admins may have um we have a dashboard that's available um that is contains metrics on the on the topic but at the moment we're revising the dashboard and we're rethinking the let's say the metrics then the indicators that are on that dashboard so um as we come up with an improved version in the coming months make sure that you take these tips home uh till then and basically is to make sure that you have your utterances um in such a shape that you don't have too many utterances right so even the out of the box are around 40 to 50 and sometimes that's that's even too much you could start off with with 10 20. around 20 is is like a a good a good size and it's more important that you have the different phrases from let's say um the different ways you say things inside your organization to have those than to have a a high number of similar phrases inside your utterances um still remove those if they're ambiguous right um it's also important that your intents are specific but do not have a high overlap with other intents so if you say hey i have a request intent don't also have a request new password or request access to this application intent because both are will have various a lot of similarities when it comes to requests um so that's important um and make sure that your your utterances should be actionable right they should have an intent um you don't need to add things like i want or i need you you can add sometimes they they help improve the let's say the accuracy but it depends on the on the intent all right having said that let's move on to a quick q a because we're already at the end of the session i see okay we've had a lot of q a activity um these have all been answered in the q a um but just to recap so we had a question around um using the fallback topic to connect with a live agent um and that's really if if nlu isn't going to correctly predict the intent or it's something you don't currently support fallback is a great option to then give the user the option to search or connect with a live agent um so i would say there make sure you use the the context variable live agent application in the fallback to give users the option to connect and then another question is around associating new topics with maybe out of the box intense um and just getting into how those are mapped so david did you want to talk a little bit more about how those are used and you know it's it's a one-to-one mapping between topic and intent um yes there is there's just a one-on-one topic right you don't want to have a an intent that can you can't actually have an intent that maps to two different topics um at least not ones that are active and published um so you just it just doesn't make sense also from let's say a conversation perspective right you do want to go into a conversation where when the person asks for something like i want to i need a new laptop or i need a new monitor or i have a i want to get my password reset and you do want to make sure that it only matches with one of the topics in your environment and the relationship to out of the box or custom yeah i think it's custom intents uh you need because there's um in your organization you may have a high influx of certain requests or incidents that are of particular nature that we don't cover with our out of the box topics and intents so then you have to create your own the ones that we do create we see a lot in the domains that we serve whether that's csm or hr or itsm all right i think that wraps up our question oh wait i spoke too soon we have one more um a question around can you comment on the entity type of open-ended entity recommendations or tips on use of that particular entity that's new for paris um well and we we do wanna i can't comment on it um but i think that's um indeed for a future session a very good one to pick up on i think that's the let's say the entities 201 uh last session of the academy we did the 101 and this one is a very good one to address in the in the 201 especially since there's a lot more to entities than i talked about in the in the last academy so if you don't mind uh waiting up a little longer on that one it's in the in the pen so to speak okay and a couple other questions around yeah i i i can see them uh let me let me just answer them as we go along um yes so um let's let's take uh make sure that we address also the languages discussions right so how do we talk about those languages and how do we add them and how do they translate to itsm out of the box or hr out of the box i think that's also a good one to to to cover in in the next two or one session in a way to add favorites to multiple virtual agents um i don't know exactly what you mean with that one so i can't answer that specifically if you could elaborate a little bit that would be awesome and then the last one let's say vishal by enabling nlu do we need to map every topic to intent an llu model like for the greetings topic no you don't need to do that you don't need to map it to the nlu model david i can answer this for the specific one so you don't have to but that means um this topic won't be discovered so you can publish it but without having an intent paired with it that means it cannot be discovered within utterance so for example anything else that that setup topic should not be discovered by an nlu this is why it's not really mapped now the greeting um if you think of this use case where your end user might say hi how are you you want to answer him with the greeting again in this case it makes sense to map it but some some topics like anything else or fallback yeah you don't want it to be discovered via an utterance i hope this answers and no it does not discover automatically that's what what vr michelle is saying um so um on both sides right when you have let's say intent discovery um so that's a new app that's in the in the app store it basically tells you what topics and intents you should be looking for in order to add as next to your uh model um to cover it doesn't build the intents or the topics and the topic flows the conversations for you right now so it's just a means of discovering um let's say that the direction you should be creating entities and our intents and topics all right um i think that's oh there's that's it yep yeah i think david there was one more on virtual agent on multiple portals that we asked was to keep favorites on both without adding manually um so i'm going to take a stab at interpreting that one so what some customers may want is is you know virtual agents on separate portals they may want certain topics displaying from the show me everything menu but not necessarily displaying everything but maybe relevant topics to that portal and then through intent discovery everything becomes available so there's a couple ways to do that you can you know as part of the greeting topic use the portal context variable to kind of drive you know greetings and what topics display per portal um or you could simply you know say i want these topics to show on this portal and maybe remove these other ones so there's a couple ways to do that um and there's some information out in the community to to kind of guide there but that is something that comes up quite frequently when there's there's a separate portal situation all right i think that's it marcel all right well thank you so much everyone and uh we will see you again in two weeks so stay tuned

Channel: ServiceNow - Now Community

Views: 1,280

Rating: 5 out of 5

Keywords: ServiceNow, ServiceNow Community, virtual agent, chatbot, VA, NLU, configuration, config, natural language, demo, webinar, getting started, basics, migration

Id: Y9Ov4qIeKa8

Length: 36min 50sec (2210 seconds)

Published: Thu Oct 08 2020