Mastering Generative Answers in Copilot Studio

generate advances in co-pilot Studio allows you to connect to your website SharePoint upload documents custom data sources all of these things and have an instant AI based chat co-pilot and the way that it's written in the documentation it's easy and awesome but what I've found in the comments and I've been playing around with this is that sometimes the experience is a bit so I'm going to take you through how to get the most out of this so that you can get back to that much more a happy place with it because you really can so first up why generate ADV anwers why would you even use this when you're creating a co- pilot or a chatbot traditionally we used to have to build out topics here's a particular topic if people put in these particular key phrases or trigger phrases then we want to go down this conversational path and we absolutely can still do that but when you do that chances are you're really only aware of maybe the top 10 20 30% of questions that your users might be asking and then you've got a long T tail so our walrus here is very smug on the top knowing all of the things that the users are going to want to ask but there's depths below that he doesn't know about and this is where generative answers can come in because you can point it to all of these other different data sources and have the AI find the answers rather than having to author topics the things that are not those core things that the wol Russ wants to be in control of so how this works is at the moment it is working with the GPT 3.5 turbo large language model this is an open AI foundational model so the starting point is that you're just using a generic model it's not specifically trained on your organization's data and it is hosted on an internal Azure openai service so that means that this stuff is included in the license that you pay for co-pilot Studio which is us about $200 per per month per tenant as a starting point so the things that I'm showing you here are all included in that license if you start to get into plugging into your own Azure AI or custom sources that's a whole other different thing but we're not going to go there just yet so what we've got here is all of this is contained the language model is in there you're not choosing it it's hosted included in that service and you're not having to set any of that up so that's the really easy starting point for you we're using a concept here called rag retrieval augmented generation what that means is that you're not training a whole language model on your data rather you are augmenting it supplementing it with that business data so by pointing it to your public website or your SharePoint site or by uploading documents you are grounding I do love this expression grounding a co-pilot I'm amused by that every single time but you are grounding the experience in that data so this is a way of making a custom co-pilot without actually having to do custom work you don't have to go away and you know train a model yourself which honestly is very very resource heavy and intensive so you don't want to do that unless you've got a lot of expertise and a lot of budget for the whole thing this is a really good way to get that in the context of your organization's information very quickly and very cheaply it will actually pick up the context and we'll have a look at this a little bit later cuz some of this is a little bit limited but it knows as the user is chatting it knows the last 10 turns in the conversation for context and if you've got multiple sources so if you're pointing to both a website and a SharePoint repository and a couple of uploaded documents it's actually going to pull the top three answers from each of them feed all of that stuff back into the model for it to bring that output back into the system so the way this works is that inside co-pilot Studio there is something called a conversation boosting topic when you toggle on that says you want to use generative an this conversation boosting topic is created and there are two ways that you can use this inside your co-pilot the first way is to use it as what's called a fallback Topic in other words if the chatbot the co-pilot doesn't understand what the user is asking for just use the generative answers traditionally in a copilot or chatbot the fallback would be call speak to an agent or something like that so what this is doing is allowing you to use that AI so if we think back to our wol Rus before where we've got him on top knowing you know the top 20% of the answers that we want this can be your Iceberg underneath this can be everything else or you might not even need the wall Rus at all you can actually just build a chatbot using this with no predefined topics and just say my chatbot my co-pilot is whatever is on the website and the user just go ahead and ask it or you can mix it up and say I'm going to specify these particular topics and then this is my fallback you can also within this use the generative answers inside a particular topic so if you have one of those core topics that you are actually saying here is the user's intent we've got an example here of something working at looking at University course information if this is something to do with the international relations course we want generative answers pointed to say this specific document or this specific part of the website so you can actually use it both in the scope of a single topic or as a fullback if it doesn't find a topic or just as the whole co-pilot as you like now there are some limitations with this and if these limitations are too limiting for you then there are other more advanced tools which will cost you more and take you more to set up of plugging into Azure open AI connecting to custom data sources but within the scope of what we're doing here you can actually do quite a lot so you are able to work with public websites your starting point needs to be really a top level domain and we'll have a look at the implications of that you can't set it up to point to a website that's multiple levels down however if your top level domain is in that good hierarchy it will get there so we'll take a look of that and then you have a maximum of four sites that you can point to documents you can upload documents I think this is really quite limited we'll take a look at why in a moment but if you've got documents up to 3 Meg that's probably the biggest limitation of this you can upload those what this is doing is using dataverse search to index the documents to help you find context in there and then it will return the results but we'll see it can't actually provide a link to the document itself much better if you want to work with documents is to have them on SharePoint or one drive because you actually get a much better experience you've got those same limitations there of 3 megabytes per file you can only list four SharePoint sites but the bigger limitation here is that this needs to be internal so this won't work if you're creating a co-pilot for external use but if you're doing doing and I think a lot of this honestly is internal use cases you need your employees to have access to your knowledge that's stored on SharePoint of some kind then you're going to get a much better result using SharePoint than you are with the document uploads so I'm going to take you through each of these in turn and sort of show you what the experience is and how you can get the most out of it so the first example here that we're going to look at is the public data on a website so I have started here with a website that is um local where I am in Australia this is called The Better Health Channel and I pointed to a website here that has a a lot of information about various health concerns that people might have so this is pointed just straight out of the box here all I've done is gone in and say here is my co-pilot point it to that website and off we go with answers I didn't create any topics didn't do anything and then it comes back with those things and you'll see we've actually got a link in here and that is a link directly to the website and that's the website content where it's coming from now I mentioned earlier that the URL needs to be not more than two levels deep so when I've created this bot I've just put in better health. vi. goo. Au that's it but it's found the results one 2 three levels down and I've seen examples where it's gone further than that so it is actually able to kind of go further down in there and retrieve those results what you can do though this is more than just being able to sort of search and retrieve cuz the typical thing you go in you ask it a question but this is generative AI you can actually ask it to generate that content in a different way so you can switch from here into then saying please explain it to a child and now we've got a different answer again noting all the way this is always giving you those references back in there so that you can see what's going on now there's another option within here called content moderation where you can set the content moderation to high medium or low the example I used before I had it on high high content moderation is set to give you much more accurate answers certainly for anything like this where you're looking for University course information or health information most businesses scenarios I think you're going to be better served by that low content moderation is going to give you more creative less accurate answers so depending on your scenario you can play around with that here I actually tested this out on this information set it to low made absolutely zero difference so than if you're working with less well structured sources of information this might make a difference if you're pointing to social media or uh conversation chat Forum type things you might get something different here and you might want to play around with that but for these much more sort of structured article based type facts I found it doesn't really make much of a difference now one of the biggest problems I find that people have when they first come in to start using this is this error here I'm sorry I can't help with that let's take a look at an example here hello I'm a university bot how many years is The Bachelor of international relations I'm sorry I'm not sure how to help with that can you try rephrasing so there are a few reasons why this can happen the first one if you've only just created your Bot I know we get all over excited we go in and create the bot put in the website doesn't work straight away just just give it a few minutes go have a beverage of choice as I often say come back and then you will find this working so first thing just give it a few minutes if you're still still getting a zero answer then these are the other reasons why that could be happening the first thing is and this is the example I'm about to show you that there are no results the data source just doesn't contain the answer then remember so this is not just connected to the whole of the web this is a foundational model connected to your data it's not just going rogue and finding everything it can be filtered so remember this is based on Azure open AI if you're asking at something rude I'm not going to swear because this is YouTube and I'm not going to get myself in trouble with this video but if you're asking inappropriate questions of your co-pilot you could get that answer as well and then it can also be filtered by that content moderation so where we looked at that one previously for high medium low then if you've got it set to high it's very alert to not giving you random madeup answers which is a limitation of uh generative AI some of the more recent models are much better with that so that's another reason for it as well generally speaking unless you know you're doing something on the edge of inappropriate content it's going to be either that you're just too quick with setting up your Bot and you haven't given it a few minutes to process itself or that there are no results and that's the example of what's going on here so this is a local University uh in Australia here where I have just put in the top level domain and I've asked it about a certain course but when I actually go into the University website so you'll see the domain the website I've put in here anu.edu.au when I find the information about the course look at this programs and courses. anu.edu.au it's actually a different top level domain that's why it's not finding it so depending on how your website is structured this is one of these things that you're going to need to be aware of you can add up to four so you'll see the difference here I've now added that one in and bingo now I've got my answers so if your website is structured in a way that you've got a lot of those top level domains and nothing over it this probably isn't going to work for you but if you can add up to before or point it to those things so just be aware of that if you're getting zero results that that might be one of the things that's going on so the next thing we can do here is upload documents this one is kind of cool also quite limited so the idea here is that I'm just going to take a documents I'm working now with a series of policy documents and this is a very typical HR type scenario I've got the leave policy the grievance policy the Performance Management the all of those kinds of things sitting in here so what I've come in here here and said how much parental leave can I take and this is coming back with that answer but what I've done here is just I haven't pointed this to anything I've just uploaded this document into my co-pilot and it's able to give me the answer the thing here is giving you the citations so you can see that we've got those in there and it just says citation one and if I click on citation one I get this experience which isn't awesome if you're a user this is this is actually the right information it's a word document that I've uploaded but you've lost all your formatting you've lost everything in there so this is quite good if you just want to get the facts and get the answers it's actually really good if you want to be able to just very quickly upload a document but then if you want to maintain your co-pilot you're going to have to upload updates to that document and change it to be honest I'm I'm struggling to see good use cases for this I think the best use case of this is that it's simple and easy and doesn't need any authentication as we have a look at the SharePoint example you are sort of working with a use case where you need authentication but the experience is much better from the user point of view so if this is a use case you've got a an idea for let me know in the comments here I think this is one of these ones where it is just sort of quick and easy it's fun to play with I'm less inclined to feel like I would be using this one in real life so let's take a look at SharePoint and one drive if you're getting value out of this video please give me a a like on the video it certainly helps it reach other people and subscribe to the channel I'm doing heaps more on Co pilot studio so SharePoint or one drive this is a much better experience and what I've done here for comparison is exactly the same thing I've uploaded exactly the same documents but this time I've put them in and I've pointed them to a SharePoint site document library in SharePoint rather than just uploading the documents into the co-pilot so this is giving me the same answer how much parental leave can I take and you'll see that we've got all of that sitting in there and then it's actually listing the leave policy document but the difference here is that that's a link to the document now again this is the document on SharePoint this is my test so I'm not showing any authentication here but if you actually publish and use this spot the user would have to do that authentication to prove who they are and then they're only getting access to what they have access access to in SharePoint leaving that aside The Experience here is that I can click on that leave policy and now that's going to take me directly into that document that is such a better user experience than what we've got with the previous one where we just had a chunk of text on the screen so if your use case works for that internal validation then I think or authentication I think this is a certainly a far superior experience than just uploading the documents now there is another experience in here if you are using the co-pilot for Microsoft 365 that's the paid license where you're paying $30 per user per month and you've got copilot working across team themes and Outlook and word and PowerPoint all of those kinds of things then you will likely have going on in the background something called Microsoft 365 semantic search that is something that is actually working to give you a better indexing experience and a better search experience for all of those co-pilot experiences this is a screenshot I've taken from a video I'll put the link in the description here from The Fabulous guys at Microsoft mechanics explaining what this is and how it all works so if if you've got that going for you you will find you're probably getting even better results from your SharePoint search in here so if you've got the combination of the Microsoft 365 sorry the co-pilot for Microsoft 365 license and co-pilot Studio you're going to get the best of both worlds as I said screenshot from another video here all of the things I'm showing here I don't have the full co-pilot for Microsoft 365 license enabled I've just got the co-pilot Studio I'm actually working in a trial to do this but if you're just buying that outright right even without the full co-pilot for Microsoft 365 you could do it but there is some kind of benefit in the The Connection in there now one of the absolutely key ways that you can have a better experience with generative answers is to use this piece here called custom instructions if you've done anything else much with chat GPT or AI you might have come across this concept before this is all about saying that you are essentially giving a a meta prompt to your co-pilot to say you are acting as a HR assistant and you are friendly and lovely and you know always happy to answer people's questions giving your AI a Persona and a role to play and those kind of meta instructions on how you want it to respond really does influence the response and really will give you a much much better experience of doing this so let's take a look at a couple of examples here using the same documents I was using before those answers were coming back earlier with just the facts all good but you might want a bit more than that so I've used something here that is actually one of the samples that Microsoft show on their website of how to do this as a starting point but I've added a little bit more to it act as an assistant in the human resources department and you are knowledgeable about all of the policies in the department you want to help people find the right answers and then we're giving it some specific parameters around saying always GRE always greet users warmly and use smiley emoji this does get a bit much pretty quick but just to show you something you can do positive language use emojis exclamation marks so look at the difference that that makes when I go through and do that hello smiley face I'd be happy to help you so we're now getting the same answer but it's coming through with a much much different tone and also still giving us those references in there so then what happens is we can say let's say I'm the manager this time generate a single paragraph script which I can use for conversation about a performance issue with my staff member now this is where you want to think about that sort of custom instructions because I've still got it there going oh you're friendly and smiley and using emojis and this is not so great right now I'm a manager and I mean okay of course I'd be happy to help you with that um here's a script you can use so I asked it for a single paragraph and it's going on and on and on and you know like again the little kind of um celebration Emoji in the middle of something about a performance review not so good so what we're going to do now is change this and have a look at the difference in this custom instruction so I'm going to come in here now I'm saying act as a senior HR advisor to the management team you see the difference in tone here I've taken the references to the Emojis out to see what happens now you can only have one of these custom instructions per co-pilot so if you've got something that's acting both for managers and for staff you just want to find that moderate in between but you might might also want different co-pilots for different types of uses and to sort of prompt it in this different way behind the scenes let's look at the difference here this is much more appropriate right when addressing an issue with a staff member it's speaking to me in that kind of much more uh managerial advisory type of tone in here it's still going on and on so the other thing that we can do with the custom instructions is to ask for a limit to how long it should be answers should be no more than 500 characters long please be brief so let's try this again certainly a whole lot better I did actually do a character count and it's more like 700 so it's not quite on the money but you can use that custom prompt to actually really the custom instruction sorry to really narrow it down to a certain length if you want the bot to be more brief have a play around with that this is one of the most important ways and one of the that you're going to get better results out of generative answers the other thing that's really important about generative answers is prompting now this is tricky because this is on the user side and getting users to understand how to prompt correctly is tricky if you've got customers kind of out of your control but you might want to provide some suggestions or guidance alongside your co-pilot if it's internal certainly sharing tips with each other about how this works is going to lead to a much better experience so here's an example here back to the earlier one where I'm coming in and I'm an employee this time and I haven't asked it a question remember the reasons why we get the I'm sorry I can't help you with that I've got a grievance with my manager I feel like I'm being discriminated against my manager ignores all my suggestions and talks over me in meetings he does not do this with anyone else what can I do if you actually have a look at the policy about discrimination or the policy about grievances it doesn't specifically mention being ignored or being talked over or those kinds of things so it's simply just not finding the answer in there so if you've got a use case where you want people to be able to do this kind of thing you're going to need even more knowledge in there for it to refer to or to train people that this is how you're going to do your prompts so again if I've put in something here I need to complain about my managers showing discrimination against me now it can do that so again with this smiley face but I'm really not liking the smiley faces I know people do love putting emojis in things though so it's worth showing you that so now we've actually got it's working on the eeo policy which is the Equal Employment Opportunity policy and it's finding those things and talking about grievances so I didn't even need to use those specific words but I needed to use a prompt that allowed it to find that whereas the first one uh just just didn't do that so then the next thing you can do with the prompt even though earlier when I said you know create a single paragraph it didn't go very well for me the prompt even without having it in the custom instructions can actually give you something here where you can say generate three to five bullet points so you can ask it for a much more specific output and you'll see here that it has actually come back with that generate three bullet points to prepare me for a meeting with my manager to discuss it and here we go 1 2 3 and we've got those things in there which is actually a much better outcome than the the great big long thing and again like we did earlier with the please summarize it for a child please summarize the grievance policy for me in a way a 12-year-old could understand and there we go so just always remembering that this is not just search and finding facts that this is actually able to do something much more sophisticated than that in rewording things give me bullet points on how I should approach this explain it to me in simple language and you're going to get that outcome now it can handle multi-turn conversations this one a little bit limited so far I'm hoping we'll see some um Improvement in this is certainly better than the last time I tried it but let's say we've got uh an example here back to the one we were looking at before where I've asked it as the manager to say please re please write out all of those things for me to help me with this conversation and then I want to make it shorter so I can come in here and say please write it as a script for an in-person meeting I'm sorry I'm not sure to how to help with that so again it's not quite getting it it is supposed to understand please write it as I have seen examples where it can do that follow on but asking it to please rewrite in that way doesn't really do it however what are the progress Milestones I need to document so this is one where we're talking about a performance Improvement plan and as I go through in the performance Improvement plan then it's got the thread of that conversation already from the earlier conversation and I can just ask it a follow-up question and it's very successful at doing that so if you want to get the most out of this then these are the core things here to remember to get to the happy place understand where this rag this retrieval augmented generation concept fits what are those data sources that you're connecting to to what's the use case for it is it the fallback for the whole bot or is it particular topics where you want it to be make sure you get those custom instructions right play around with that you'll get a lot more out of it and if you are in that situation where you can help your users learn prompting that is going to be a happy place too if you need something that is less limited than all this check out my video here on how to use Azure open AI with your data and more content coming on generative actions custom instructions let me know in the comments here what else you would like to learn plenty plenty more to come on co- pilot Studio thanks for watching

Channel: Lisa Crosbie

Views: 3,894

Keywords:

Id: W53TE5EtBQg

Length: 25min 32sec (1532 seconds)

Published: Tue Mar 19 2024