Creating bots with Bot Framework Composer

okay so hello on the presentation about both framework composers so my name is gosha boschenska i'm originally from poland but i live in the nottingham or new nottingham i live in hawk norway where ada lovelace is resting place so i'm also ai mvp of the 365 developer and i really interesting with new technology and anything new shining is coming so today i will talk to you about what our bots what is bot framework and also what we can do with both framework composers such as how we can start how we can build and how we can publish it so starting with that what are bots the bots are application that can automate our task more easily they have a conventional interface can go wherever your audience is going to so it can be on on web on mobile or any where they are so it's not on one side only and supporting board technology is maturing because we're going to a lot of technology and a lot of data around us and we need answer to have now and both is something what can help us to get this at this moment and if we think about both there are a lot of chatbots that we are already using every day so that such as siri on our iphone if we have one of them or google now on android there is as well the alexa one so she can speak at your home or any room that you have cortana or your workplace if you're using windows machine or messenger on facebook is as a really popular one to answer that basic question about your company but if we think about bots it's something new probably we are wrong so it's starting in 1950 when that turing taste team come and it was checking if machine can be intelligent based on text only conversation then in 1966 the first board created by mit ai laboratory came eliza it was first human simulation conversation and she was kind of therapy for users in 1972 came parry he was a person simulated a paranoid schizophrenia so it was a little different than eliza in 1988 come jabberwocky and it was designed to simulate natural human chat in an interesting and turning and humorous manner and in 1996 1997 it was released online so when internet was came on in 1992 doctors esbastio it was first ai speech synthetic program created for ms-dos and is as well was a psychologist that interact with digital voice and 1995 came alice so it's artificial linguistic internal computer entity so it was natural language processing bot and she could apply a lot of patterns to human input but she not passed the turing test and next come in 2006 ibm watson a lot of machine learning things and as well bots in 2010 came siri two years later google now in 2015 alexa the same year cortana and 2016 bought for messengers in 2016 i also came thai i don't know that you remember but this was a bot created by microsoft it was she and she should be a teenager and learn from people how to be a teenager but unfortunately it came with really nasty nazi personality so after a few hours microsoft needed to get down the spot from twitter so this all what we're talking about it's called conventional ai and there are agent that can do at conversional with the humans with us and the example of this one is both framework from where we can start working on so when we're thinking about both we need to think as a normal application so we need to plan what the bots will doing we're going to build this test that it's going okay with us if not go back to build and negate test if we are happy we're going to publish it add connectors to connect with other things like teams slack or other channels that user will use and x evaluate like all programs need to be evaluate and back to build and do this things again so for example pizza order bot pizza revolt first when user comes to great user and then ask user what they wish to order they can select the pizzas what we offer prompt their pizza size and what kind of pizza they want over crafts then if there are any source type user would like to prompt for toppings on and ask of added extra one if they would like to so creating until user will say no yeah so this is repeat one and next ask for cheese because maybe someone doesn't like this on the pizza how covered the speeds are full or half if they're any side orders also we can create a repeat loop until there will be no more side orders ask for uh what kind of delivery they would like to have ask for payment confirm order and we can finish conversation so this is the easy way of how we can plan the boat what we need to think about when we're going to order a pizza so we've designed user experience we also need to think what we would like them to see so for example it should be only the text one or the user would like to speak to the bot or have some rich user controls that they can click on them or add extra one kind of adaptive cards like audio animation carousel and any other what we would like to have and one thing what is good to think about if we have really big business is to create the itunes hub to hand off to humans because if you want really something from the board and it's not getting your answer correctly you would like to speak to human to real person that will answer all of this question for you so we have eigenhob and uh two ways the argent hub will check that there is any human ready for answer if not then going to bot or we can create a message router that you first talking to both because maybe you will get more answers from this and then go to agent hub and find out there is any human that you can talk away and both framework when we're going to start we're going to start with input so user going to tap speak or click anything using some channels or if you would like to use some devices as well and this all is based on azure bot services and this is enough to create a board and from this we can start how it will work the extra one what we can go is add cognitive services such as speech q and a language understanding vision we can also brand a personality of the board and extra bits we can add such as any data sources or add skills such as mail tasks connect to an api office graph or azure directory and when we're going to chat with both it's always going with http to check so when user join conversation is going uh with http post if it's okay and a collect connection is established it's going to body join and saying some greetings information such as hi and our user will start first and saying hi and then both replayed you said hi and still will that connection will be established until we finish conversation with bot and both framework is an open source project so if you are a developer you can work on any language you like so node.js java javascript python c-sharp everything is on github with sdk that you can use in cli tools there's emulator web chat controls a lot of great documentation and as well bodybuilder samples that you can go and see how the bot working but we came today to not talking about coding but about both framework composer so the difference is that it's designed and built on conversational experience so you don't need to know the coding is visual outer in canvas and it can integrate with all as understanding tools such as lois or qna maker it's using language generation to create a response for port and again it's open source and you can find all code on the github and as well as cross platform so it doesn't matter what kind of os you like you still can use spot framework composer so one on bit is language generation so with language generation we can go and create what response will be so like grading prefix it will be high and or hello and both will just randomly select one of this and then we can just create some if else of the time of the day what is coming and then connect this buff together in the response with the replay also the good one is language understanding if we more would like to go with user experience so we can do three things we have utterance so it's something that user might say such as hi hello or good morning and we have entities as well for things like fun light or location like paris roma or other things that we know that this will be a location that we're looking for and the intents it's represent what we're going on so if someone will say for example good morning we will know that it means grating and on this intent will create the response from bot and this all is on natural narrower language understanding concept and i really like this gif because it's showing how machine learning can start working and finding all information only based on on tags so it's really great creating and training from on microsoft side with language understanding to find this a little bit the extra one what we also can add is q a maker so it's easy question and answer so any question user will ask we can add them and they can find in our board and as well we can connect q a maker and luis together and with a query we can create a dispatch and ask where it should go so if we know that it's a q a maker or it's more like lewis and more specific question or something what it can do for example switch off the light and bot framework composer is based on adaptive dialogue so we have things like recognizer we have action what bot can do and we can we have triggers that when we someone will type anything will run this dialogue and actions of the bot there is also a memory of turn dialog and user that we can use and remember to have this flow of conversation between us and both and as well we can add a lot of other services example is cognitive services but any api that we're using it's really easy to add and work on them there is also personality chat so this is in preview and you can use this in your boat so there are things like how your boat will go with your answers but if you create a q a maker you have this personality chat automatically that you can add so it's really good and friendly response that you can add so how to start with composer i mentioned everything is on github so this only what we need to install is.net core sdk version 3.1 bot framework emulator to see how it works and board framework composer we can as well build this composer locally so we can download the code and from code we can run it then we'll be open the website and uh extra one what we need to install is node.js and yarn to install bit and start this composer but if you would like to work together with your team you can as well host this on your site uh that's what you need to do it's an extra bit so probably you need to have azure subscription you need to have vm with install all necessary information and with nginx you can run this and host composer on your site and work together with your team so this is how composer looks so we have some menu we have the triggers that we run we have navigation and other property that we could add so let's go and see how it works so this is the both framework composer and this what we have when we see we have a lot of examples that we can come and start with so let's see for example echobot so i would go with bots so we can change a name we can add description and this easily is going uh to run for us so next we can go and start and see how the bots working so in this board we have a no intent so anything you will write you it will go back and say this exactly what you type and as well we have some grating so when we join it will come with response welcome user the response we have in both responses so we have welcome user and it's in common so it's saying welcome to the echo board sample so let's run this one i have a version uh not that update every night so the extra one probably what you will not see is open web chat so it's extra and the new one and here when you as well can go and it will be answered for you but i don't want to restart so as well you can open this in the both framework emulator so you're just going to click test in emulator and it's going to open for you and with this one we have more information that we can see so we have a debugging screen so it's going to say what kind of message you send what is for sending from you the bot state and this all pause and waiting if we're going to restart we as well can go with freestyle conversation with new user id or we can go with the same user as well so let me close this one so this is the basic board when we can go and start working on so what we have here we can add for example new dialog and we can say for example help so if we don't know what both will doing we can start with always help is the best one and we can go with send response so we can just type i'm here to help you and to run this we need a trigger so we're going back here to add a trigger and this bit is with luis so we if you would like to use luis we need to add all this information but for now we don't want to connect to any of them so the easy way is to change recognizer type to regular expression okay and now when we add trigger we just asking what is the name so we can add again help and this is what we will type how it will work so now when i restart oh sorry one more we need to start a dialogue so we're going to start a new dialog and here we just select what dialogue we're going to start so it's this one what we just created so let's restart one again so we can see that it's going to starting now it's running we can again test in emulator it will open for me and now when i type help it's going back with this dialog what i created so this is how easy you can create all dialogues and run with triggers so let's stop and there are also a lot of examples that we can go with them so we can create a to do with lewis or dialogue questions so or just simple to do with simple to do it will not connect to luis but will create everything for you so we have a dialogue like add to the two uh clear to do's delayed and show so it's really good to go through this all examples and see how them they works and we can see how all of this response will going back with every of them and again we just easy can go and start board and it will go into creating everything for us and we can test how it works so always the first one starting is a little longer because it's going to build and do everything so hello actors that's in emulator always when you click this one it's open new one but you don't need to open and close you can just click restart a new conversation here so again we can start with help if we don't know what bot doing so we can do add a to-do and test and it's add a new to-do list so we can check and come back help okay so it's like to be fun so with all this response we can see what they're doing so when we go to common and help function we have here all this information we can also go and see this in the code how it works so like here we have if the user have one to do is going to show this one order is different so when we have this board already we can go and publish so with publish we need to select new profile if we don't have one we can create so we're going to add a new add the name so it's our name it's nothing specific and this one we can host this is on web app azure function or power virtual agents so it's really up to you where you put this bot so i will choose azure function and then is going to create resources for me so only one thing here what people don't like is that you need to use your work or school account you cannot use your personal one for when you have for testing but this is the idea how it's going so how we just use my work one i'm going to authenticate okay it send me message so now i can create a new azure resource group so i can select where it should go i can add a new name of my resource i can add the name of research group that need to be unique so let's add some name and i can choose as well the region so i will choose uk south for example but if i would choose uk south i need to uh have different louis one region so lewis is only on west u.s west europe and australia is so on this case until it will not come to uk one i don't want to send between regions so i will back to west europe so makes our quick review and the base what we need is we'll create for me it's application registration because it's new application my azure function when it will be run and microsoft bought channel registration and then extra bits what i can add if i will work with uh bots it's azure cosmos db application inside to see how the both working and adding an extra data to analyze blob storage if i would like to send all information and host them somewhere or have attachment files that should be sent language understanding so there are two language understanding authoring is when we would like to only put information about what user will say and what intent will have or azure predicted account when we would like to give ai to predict what user will say and then analyze what they say and how they interact with what they typing and extra one as well we can add q a maker to add our q a we don't need to do this but we can add oscillator and then we're going to click done so now it's provisioning this all azure resources so let me check on my azure now okay so provision success i have here a bot test and it's create for now the function but the bot is not published yet so i need to back here to publish click here and publish selected bot i can as well write a comment and click ok and now it's going to publish and giving me all information what is doing so now build my bot next will be zip then and when it's zipped it will be pushed to azure function so it's little take a while and i can see then next here that slowly will show uh all information so the main thing is both channel registration what we'll use and we configure our board later so here we have channels that the bot will start working okay it's still adding so the first one we have a web chat that is automatically added but extra one we can add a direct line if you would like to put them on our site so we can click and done and it's added for us we can add them as well on themes and configure them and publish and have all information how to publish to a tenant catalog so we can go save already we can start using them and test it on on this channel so it's really really easy uh there are as well more channels such as alex facebook but then you need to little bit know how to create application on the amazon or facebook site to create all of this information but all of them you have here so you have only one bot that you create input framework composer and then you can put on other channels there are more channels as well that you can add here but you need to add a little more coding uh to board also this what we can go is add configuration and authentication if we have uh we want authenticate bot so we can add some the name of the connection that will put the bot composer and we can set select the provider so how you can see there is a lot of them that we can use as azure active directory dynamics facebook office 365 spotify or any other that user can come and authenticate with their user and login that they have for the services and i'm going to show you one of this bot that i created so it's office 365 merc robot so how it works uh when user will try to get a profile info or meetings information other stuff is going to start with authentication so we're going to get a user token and here is my connection name so only this connection name i need because all information about how to get i put in the config so in convinc i have my app id i have my password and only just what you need to authenticate when you add all these bits to the bot registration and then assign the user token and i'm logged in so like if i would like it next to show profile i need to call uh first to get uh authentication and next create a simple uh simple api call so in this one i'm calling the graph so it's going to get information my graph url authorization with user token that i i will just get from it and how it go with value and what is going back and then based on the respond if it's okay so it's 200 i have all these properties such as display name mail given name job title mobile phone and other stuff and then show it on the profile as well here what i use on greetings i have user one so let's go to both response and this what i'm using i'm using that uh adaptive card so adaptive card is simple adjacent and it's easy to go and create so when we go to adaptive cart okay let's try on this one adaptive card so get started we have here a card designer so you don't need to log in you don't need to do anything you just have this portal one and you can just come and type what you're working on i using version 1.2 in this case and you have here all information how you can put the visual things for you what kind of image what kind of information it will be go so to do that you need to just create a json and after that with user you need to add activity called attachments and parts to json this what you just created so let's see how the bot is working test in emulator okay and now it's going back with my hello message so i'm not type anything it's just i joined the conversation and it's going back so i can click help and it's again going back with another adaptive card so i would like to see my profile so it's typing uh exactly this what entity i put and i need to login and it's going to login me with my credentials so it's now singing me it's remember so i will copy this bit put my number and then i have all my information that it's going back and this is the simple how it works and how you can add an api that really easily work for you and to do that as well because it's using uh adaptive it's using a sorry it's using active directory so you need to go to active directory to get this all information go to your app registration and here you're going to create a board so when we go to authentication we can see information how the user can authenticate and here are a few bits so for this board i create that it can create with any azure id doesn't matter from which tenant and as well can create with personal microsoft accounts for example skype xbox but as well you can create that it will go only on your azure id and no one else will come or only with work or school azure id and next you need to add api permission the max api permission what you can add is 30 so you need to think uh what you're adding if not then you need to create new azure new application for that so you need to add so to authenticate the base one is open id and if you would like to get a profile user is profile but extra bits as well you can add just clicking add permission and as you can see there's a lot of permission you can add using active directory only for this i use all the microsoft graph delegate permission and here what you can add and what user can see so this is resources that you can use so it's a boat framework composer on microsoft docs there are some step-by-step how you can create a weather boat a boat emulator when you can download them q and a maker when you can go and lose micro software as well and you can go out step by step and go with this and adaptive card what i show you today so thank you very much for listening my presentation and you can find me on my website twitter and instagram if you would like to thank you can we somehow create hybrid bots where the middle part this is the interesting thing where the middle part is taken care of by an actual human and then go back to the bot for for instance payment or completing the the query session yes the answer is yes but i'm not sure how you can create this with both framework composer with normal bot framework you can do it easily but with this one i'm not sure but the main answer is yes you can yes perfect right so um when you're using the bot framework i'm presuming it's cognitive services that are powering the bot framework under the covers is it is and and if that is correct is there a good way to limit the cost of cognitive services in azure so bot framework not using cognitive services until you want to use this so it's not using this what main thing it's using is um how i create that my board so bought that so this is the simplest one without anything so i have only up service plan because i need to have my application i create a function so it's on the function one and both channel registration so only this three is for basic bot if you don't want to use luis q a or any other cognitive services cool i think we have a follow-up question on that would it be okay that i move it mark to yeah go for it up here because the question is is the actual bot in the function or is it in the channel registration or is the bot channel only the link to the consumers of the bots so the real boat is in azure function because their their code is and azure and both channel registration is to consume the bot and add more channels and distribute them for other channels and care about authentication and other stuff but the code is in the function or web app or power virtual assistant i think that it's called because you can choose where it should be but the real code is it need to be because it's a code and then we come immediately to the next question what kind of code do we get how about version control and can you connect the bot framework composer to some kind of repository like you do with data factory for instance so let me one moment okay so let's see this echo board so this is the code what was creating and i'm going to open this individual studio code so we have our dialogues knowledge help and this is in q a for example uh we can have in elegy so language understanding and he here's kind of saudi code what we're using so this how it's going to pack all of this is as a zip and pushing and it's later understanding so it's not simple like code code because it's based on a lot of json files and led understanding and other recognizers things but it's still accessible code on my computer that i can check into my repository of choice yes i think yes so the one thing on here is when you're going to publish you can create your uh thing so you can create your published profiles and next you can pull from selected profile as well if there is any changes but it's still i think it's still to cooperate it's still they working on this that's why they created this bit that you can create a site on your virtual machine when you can cooperate with all of this but if i will pack this one as a zip and send to you you can still use the bot perfect thanks so next question it's slightly off topic and it may be a little unfair to ask you because it's not what you've come here to talk about but what is the best way to deploy bot code um created through the bot framework and i'm specifically thinking of pushing the code through devops so i i'm wondering whether arm would be you know sufficient to push all your bot code through devops to azure so because there are two ways it's bot framework and bought framework composer yeah i'm not sure about both framework composer because i'm not working with devops on this bit but with normal both framework it's like normal application and you can go exactly the same as normal app that you working on right okay okay cool okay the longer we develop our bots the more intelligent they get do we have a limit on how smart they can be when does the code or options take too long to run so it's pretty much unbearable to load the bot or use the box so there is something called skills what you create so if you know the area of microservices it's something similar so you create a smaller board that will doing their skill things and you can separate them and then you can use together but you know it's only our imagination how body will look like but we can separate this bit to not go here and see too many dialogues and overwhelming ourselves cool so next question since bots can work with and connect with other box it almost feels like things can get quite complex very quickly if this assertion is true are there any ways of working to keep things simple first design your board and think what exactly you would like to achieve from this one and then create it so i was sad from this bit so because it's true you can over complicate it and you can really find yourself like there is too many things and i couldn't find how you can work on it and i i agree you can do this really really easily that's why that first bit for both in my opinion is to think and design what it should do and if it's more complicated create a skills and then add to your bot so it will be less complicated when you come and see even if it will be still huge it will be smaller great thank you related to that skills does it make sense to create for instance a log in bot and then create a chain of bots in order to create these complex flows probably the meaning is does it make sense to create one bot to handle one part and then when you need to expand you create one master bot that actually asks the slave bot the sub tasks so we have a bunch of tiny bots that know their subject their skill or do we want to have everything in one big large box no no no you can create a skill so uh on microsoft site if you came there is some samples of skills so for example calendar one uh what you can do like creative meetings getting your events schedule and meeting with someone else so this kind of skill is only for calendar one and this also how you can create like kind of models what the bot should do and then get them together in your boat so we could actually create a model that has the skills about cars so whenever my main model detects this is a question about cars it goes to the car skill bot with the car yes oh cool i think that looks kind of useful don't you mark yeah that looks really good to me i'm really looking forward to trying it out it is it's one of those many things on our list i think mikhail that we need to try out okay if you will have any question you can come and i will try to help you you well you don't want to say that to us goes yeah because we'll be speaking to you tomorrow yeah i like speaking about god i think that was the end of the questions for for today again thanks a lot garcia this was really interesting and well really thanks this was a wonderful interesting interesting and i really have to try this out yes thank you for having me and i hope you will do some what and you will show me you definitely will

Channel: Hybrid Virtual Group

Views: 213

Rating: 5 out of 5

Keywords: Bot Framework, Azure, Bot Framework Composer, Cognitive Services, QnA, Bots, Bot, Artificial Intelligence, Microsoft, QnAMaker, Luis, Power Virtual Agents, Functions, LU, Language Understanding, NLU, Natural, LG, Language Generation

Id: vnw57c0hHzo

Length: 47min 5sec (2825 seconds)

Published: Mon Aug 09 2021