Copilot Studio Dynamic Chaining and Generative Actions in Power Virtual Agents

hi there folks in today's demonstration I'm going to show you how we can build a power virtual agent using Dynamic chaining and generative actions what does that mean well it's new functionality it's in preview instead of building out topics with trigger words you're going to use gen AI in order to identify the appropriate action whether that's a standard or a premium or a custom action or a power automate flow to call to deliver that functionality to your end user so using this new functionality the power agent will dynamically choose an action and use generative AI to populate the properties of that action so I'm going to build you a demo with three different actions a weather API a jokes API and also a PowerPoint slide flow and before you know it you'll have a fully functioning power virtual agent that's able to deliver up some pretty nifty functionality using this new feature so if that's something that interests you please make sure you like And subscribe and without further Ado let's let's jump into the demo so I'm going to kick things off from our recently rebranded co-pilot Studio I'm going to create a new co-pilot we'll give that a name and whilst you can see here we can add websites for conversational boosting with generative answers today's demonstration is all about Dynamic chaining with generative actions so I'm going to create this and then we'll have a look at the new functionality so with the bot Now set up the first thing to know is if I jump across onto the generative AI tab we can see we have a slightly redesigned screen it includes boost in conversational coverage for generative answers which is not what the demo is about but it also includes this Dynamic chaining with generative actions which is the exciting bit the preview bit that will enable us to bring in flows and actions using generative technology so I'm going go ahead and and enable that and I'll hit save so those changes saved if I jump across onto my plugins preview and for those that were Eagle eyed you might have noticed a new icon at the top here that which is the new tracing mode we have a screen here that now shows shows us all of our topics and all of our plugins and new trigger types so previously we had triggers that were based on the topic trigger words but now you'll note that these topics are based on Dynamic chaining and what that means is the description of our topic if I go into the goodbye topic right now you can see the description here says this tool can handle queries like byy by for now byy now goodbye no thank you and goodbye what we previously knew as trigger words are being replaced by now natural language so our co-pilot can recognize the topics to identify and use in this more Dynamic conversation string so if I jump back into this new plugin screen and we have a look at the plugins tab here you'll see at the moment I have no plugins but today what I'm going to do is I'm going to add in a plug-in for the weather I'm going to build a custom connector which will have a thirdparty API for jokes and I'll also bring in a flow one that I recently built that would generate a PowerPoint SL slide deck from a description so I'll show you how we can bring all three of these into our chat bot today so we can have a generative actions conversation using Dynamic chaining so at the moment if I go into the test on the left hand side and type into the bot tell me a joke for instance of course it's going to say I can't answer that in some description oh you're welcome very interesting but if I try and ask it for the weather in abine for instance again is going to have absolutely no clue and therefore it's saying I'm sorry I'm not sure how to help you with that can you try rephrasing cuz of course it doesn't have any logic and traditionally we would have created a topic about the weather we would have called a flow we would have got the answer back and then we would given an a standard response but for this I'm going to go into add an action and I'm going to add in the weather action so what we see here are all the actions that we have across the Power Platform the ones that you see in power autom in power apps and as I scroll down we can see at the top we have get forecast for today so I can click on that and add this action and whilst there are a series of default values that are accepted we can go to next here we can have a quick look at some of these action details we've got the plug-in name the important bit in order for power virtual agents to recognize the topic is this model description so getting the forecast for the current day in the specified location but as we scroll down again there are inputs and outputs so inputs refer to the input Properties or parameters of that action so of course to get the weather you need to know the location you also need to know the units and if I go into edit we can see at the moment both the location and the units are set dynamically using AI but what I'm going to do for the units is to change that to set a value it's asking me do I want to confirm and that value I'm going to choose as Imperial so I'll go ahead and save that and then we can have a look briefly at outputs again if there's anything you want to store and use further on in your conversation you can go into edit here and you can manually add the outputs available from this particular action we also have the ability to respond to the user after running the action and of course I would like to know what the weather is so at this point in time we have a couple of options or three options AI can generate it AI can generate it as an Adaptive card which is currently grayed out but will soon be available you can create your own message or you can create an Adaptive card so I'm going to leave it on the dynamically generate a message and this will really help demonstrate the power of this new functionality so I'll go back to the previous page I'll hit finish and we'll bring this action into our new chatbot so previously we have asked what the weather is if again I ask uh what is the weather but I won't ask in a particular town or city the nature of dynamic chaining is that it recognizes the fact that I want to access this Microsoft uh weather action and and uh it's going to search for any of the properties that are missing now remember I pointed out at the top there's a new icon it's the new tracing mode if I click on that we can see that there's missing information for the action hence the question here from our bot and if I put in Aberdine we will hopefully get a weather forecast for Aberdine so there we go Heavy Rain which is correct and we have a high and low temperature now if I was to ask what's the weather in Dundee for instance I could say how about dunde so I've not actually asked about the weather but I've followed up on this conversation that I've currently got going with a bot and you can see already that it's identify were in the context of weather and therefore search for dunde in that particular action returning to me a summary now if I was to ask for the top temperature in Edinburgh again this is a weather related question let's see if it'll identify specifically the top temperature in Edinburgh today is 46° f so rather than given me a summary about the weather it's recognized the fact that we're talking about the weather here it's called the endpoint that endpoint will return a host of information in the standard output that you'd see in power automate for instance and because I've asked specifically about that top temperature I've got a very specific response about that top temperature in edin Edinburgh today so that quickly demonstrates the nature of dynamic chaining and generative answers we've had a conversation about the weather we've simply attached an action we've fixed one of those input values and then the location is completely Dynamic based on the questions that I'm asking the bot so let's now see if we can bring a power automate flow into our chatbot so if we jump back across onto our plugins I'll go into add here and we'll go and add in a plugin and at the top right you can see I can create a new flow now the good thing about this if I create the new flow just like you get the experience in power apps and other places we get the popup with the new design designer for power automate and we have the trigger to run a flow from co-pilot and we also have the respond to co-pilot so if you think about the way that this works we have the trigger you can pass in inputs from your conversation into your flow then you can do lots of stuff in the middle here by adding actions and then you can respond back to your co-pilot to let it know that that particular flow has completed now there are timeouts on this I think it's 120 seconds 2 minutes before your flow will time out so make sure that whatever you do between the trigger and the response it's going to happen promptly anything you think is going to run beyond that you can run after this particular response and I'm going to show that in the flow that I've built based on the video that I did on generating PowerPoint slide decks so with that flow now opened the flow called create me a PowerPoint slide pptx you can see I have that runoff flow from co-pilot if I go into parameters I have a request for both an email and the requirement ments the reason for the email is because I want to notify that user proactively that their slide deck has been created and then I have the requirements which is basically the brief description of the slide deck now these two parameters and descriptions here are very important because it helps the AI model recognize the data that it's looking to extract and pass into this action dynamically so if you've ever used the descriptions here before and being quite flippant about it and not bothered well for generative AI make sure you give it some that's meaningful and help that that language model identify your properties so we have our input properties for our runoff flow we have a response in this case is just a basic response back to say it's okay and then we have the contents of that demonstration I've done in my previous video which I'll have in the description But it includes keys for various apis and it uses the open AI service to generate some slides populate that slide content merge the slides create a file and also hit hit up a text to speech service so we can get a recorded audio file now at the very end of it you'll see I have an action here called post a message into chat or Channel and the reason I've got that in here is because it allows me to post a proactive message back into that chat bot so you'll see that I've configured it here to post as the power virtual agent it's currently a preview value for the post as previously you've probably used the flow bot or a user so we're posting as a power virtual agent we're posting in a chat with a bot and we're posting to the bot that I've called the Gen actions POC now you'll note that I'm pulling through the email address that's the reason why in the trigger I have the email because I need to know which user I'm notifying in the chatbot that their slides have been created and you'll see that the slides have been created and are available here and to play the transcript click here now there is some HTML sitting behind this I actually built it in the classic designer so at the moment it's not editable but you can see if I hover my mouse over there is is a dynamic file path there so for both the two links we have a link to the output of the body path which is for the create file of the PowerPoint file and the create file for the audio file so HTML unfortunately at this point in time needs to be updated in the old classic designer using this switch but that's all I want to cover today about this particular flow I want to jump back into my power virtual agent and show you how we can now use the power of these generator actions and dynamic chaining to create a conversation and Trigger the creation of our PowerPoint slide deck so I've created my new power automate it's called create me a PowerPoint slide deck which we can see here if I couldn't find it of course I could search or I could filter and if I click on that particular flow I can bring it in and just like I did with the weather you can see again I have a description so it's a service to generate slide decks or PowerPoint presentations which will enable the AI to identify the particular use for this flow and then as we move down we've got inputs the email and the requirements and again outputs if I wanted to save anything from the response but like before I want to go into the inputs and where the email has been set up to retrieve a dynamic value I'm actually want to use a system variable if I go to set as value we can confirm here click on selecting a variable into system and because I've enabled the security and the teams Channel it means I can use the property for user. email so I'm actually able to now pull across the email address of the user in teams that has sent that request for the power automate which therefore means that they will get the notification when their set of slides have been generated and the same with apply with any instance where you want to use that proactive messaging within power automate so I can hit save and we can go ahead with finish and this is now bringing that into our power virtual agent we can now see the flow added in there to create me a PowerPoint slide I could of course make the request here from our test designer but what I've actually done is I brought it into teams and we can see it's already attached here I've been playing about with it earlier to make sure it all works of course as I always do and if I ask the question um what is the weather for instance we can just test that same functionality and hopefully it'll ask me the city that I'm interested in so what's the location I'm going to go with Aberdine again so there we go the weather forecast is still heavy rain and a high temperature of 47 fenhe but what about can you create me a PowerPoint deck and of course I've got an input parameter so I've not provided the description so generative AI has said what is the brief description of the slide deck so because I'm hot off of watching the pantomim in Aberdine let's go and ask for a slide deck about the history of the HMT theater in Aberdine and pantos on a five page slide deck we'll go ahead and hit enter and we can see that our slides are being produced as we speak so if we go back into Power automate and we go back onto the details pane we should be able to see that our flow has been triggered um it's currently running there we go 16 seconds ago so if I pop that open all that's happening now is everything that's within this scope is running which is all the generative I stuff the stuff that I've demonstrated in my previous video but ultimately it's grabbing some slide content based on that prompt that I provided which has come in Via our trigger so you can see the email address you can see the trigger about the history of h&t theater in Aberdine and pantos on a five-page slide deck that's going through this process it will generate some images using generative AI merge the presentation slides create a couple of files and then if I go back to our power virtual agent when it's complete I should get that Pro active message popping up on my chat in teams because I've added this virtual agent into teams to let me know that my slide deck has been created and remember this is a personal experience this bot is available to potentially everyone in the organization but because it's me that's made that request it's me that's going to get that notification because I've grabbed that email address from that user variable as a result of using the bot in teams so we can see now that the slides have been generated and that they're available I click here I can also play the transcript if we jump back onto the flow we can see the the time it took for all these stages so it did actually take 7 minutes to produce which if I had put all of these actions between the trigger and the response the overall flow would have timed out the bot would have timed out and this whole setup wouldn't have worked so by responding immediately back to our co-pilot to let it know that that process has been kicked off we're ultimately using this post a message back to a chat or a channel proactive messaging it's called called with power virtual agents which is a preview feature it lets me Target that bot that particular recipient in which case is me and a personalized message which includes the links to those two files if I go and click on the link of course it will pop open our PowerPoint presentation which is now opened up for us on screen so this isn't the actual hmd theater of course it's all AI generated images but some interesting uh points there throughout this slide deck that's being generated using that that process and of course if I click on this link here we will get the audio file today we begin a journey into the rich history of HMT theater a remarkable venue in abedine so that wasn't my voice that was text to speech based on the generative AI that was produced for that very simple prompt that I Pro provided to our power virtual agent in teams and at request about the history of HMT theater in Aberdine and pantos on a five-page slide deck so so hopefully you can see how we've managed to bring in that functionality into a power virtual agent using a single power automate flow the input parameters one of which is fixed the email address based on the user. email and then we have the dynamic value which is obviously the description or purpose of the PowerPoint deck that we're producing so finally I would like to try and bring in this joke API which is quite simple it's an API that produces a random joke if I refresh on screen here we'll see that we're getting lots of different jokes off the back of it so I'm going to build a custom connector using this API and then we'll bring that action from that custom connector into our bot and then it'll be able to start producing jokes and make us laugh so the first thing to do is for me to grab this URL so highlight and copy contrl + C I need to make sure I'm in that same development so it's a US environment here and I'm going to create a new connector from blank and we can call it the joke connector I'll continue with that and the first thing that you might notice is we have a new screen stage four for AI plugin so this is important a bit like the description of the input parameters and the purpose of the flow making sure that you provide data in here so that the model can recognize its purpose so just like you normally do with a custom connector you can specify the icon color and give it a description I'm going to call this the joke connector and then I need to put in the host now got the full URL here I need to trim off the https colon SL slash so I only need the beginning of the URL so I can strip back this jokes random the base URL can be jokes and the end point I'm wanting to hit for the random joke is random so I need to grab just that word random hopefully all of that will become make more sense in a minute so the host is this URL the base is for jokes security there's no authentication so whilst in a normal Enterprise scenario you might set up oo or use an API key I'm going with a public joke API here so no need for me to set that up I can go into the definition and this is where I'm going to create a new action we're going to give it the name random joke the description is very important so tell us a random joke and then their op operation ID is going to be random joke so of course I could add multiple actions in here based on what the API can do it can tell specific jokes based on categories it can produce a random joke which is what I'm wanting to bring into my chat bot today and if you had an API that equally had loads of different actions you could bring them in so the next bits we need to uh import from a sample so I need to get based on that specific URL so here you would paste in the full URL in this case this one here and then the header is based on what we're passing in which is nothing so I can go ahead and import and we will see hopefully everything going in okay we can go into the default response import from a sample we've already got a sample here overall on this tab so I can contrl C on that Tab and jump across into here and paste it in so this is what the response is going to look like four different properties in this object we can import it and that's all we need to do on this particular screen over on the AI plugin we need to give it a description again so tell us a joke and the preper so the description here is to provide a random joke from a public API so this is all in preview at the moment but it's going to help with our model identify the purpose of our connector the purpose of the actions that we have on that connector as we bring them in now I can just click on create connector and then finally I can go ahead and test so with the connector now created I can click on the test tab here I need to create a connection so that I can get access to this as me as a user the connector has been built by me it's been shared to me therefore I can create a connection to it and then I can test the operation and we can see we've got a response back with our joke body so everything is set up working for our random joke action in our new custom connector called the joke connector so if I jump over onto our bot again and if I ask it to tell me a joke will probably confuse The Living Daylights of it because of course there isn't an action there that it can use for this purpose and in fact it's asked if we wanted to end the conversation and if I look at the tracing mode it's gone down that goodbye topic based on some of these phrases and the description but instead if we go into plugins and we go and add and we go and choose our joke action that we've created so I'll search for joke you can see that there's my random joke action for the joke connector and it will bring through the details hopefully if I go to next the model is based on tell us a random joke which is obviously the the string that I set or the description I set when configuring the connector I'm not going to change in the inputs um as there are none and I'm not going to save the output anywhere so I'm just going to go ahead with a finish and by doing this it should then give our bot the ability to tell some random jokes so if I refresh my bot and I'll ask it to uh tell me a joke and even though I've got the spelling wrong hopefully it will still recognize my intent why can't bicycles stand on their own they are too tired hilarious how about tell me another again too fasten the keys but hopefully it's recognizing my intent good why or what do elves post on social media elfies if we look at that tracing mode you can see it's calling out the random joke action if I was now to switch conversation and ask about um how warm is it in Edinburgh it should recognize that I'm asking specifically about the weather forecast and as you can see in the dynamic chaining log we have the fact it's pulled in that forecast action for Edinburgh and brought back specifically the top temperature if I was then to ask to uh create me a slide deck we can see that it will now hopefully choose the action related to creating a slide deck we have some missing information which is the requirements of that parameter of our slide deck and of course if I provided the details of the slide deck my flow would get launched and it would create me a slide deck and pop back a proactive message into my team's chat to let me know when those slides have been generated and when that transcript is available to play a

Channel: DamoBird365

Views: 1,631

Keywords: power platform, power virtual agents, power platform copilot, copilot studio, generative actions, dynamic chaining, chat bot, ai, GPT, generative ai, microsoft copilot ai, generative ai use cases, Generative actions, Dynamic chaining, Pva

Id: FJkNdD5pcdU

Length: 24min 9sec (1449 seconds)

Published: Sun Dec 10 2023