How to use Azure OpenAI on your Data with Copilot Studio

Microsoft co-pilot Studio has a feature called generative answers which allows you to upload documents put in public websites and get instant AI generated answers over those data sources but there are circumstances where you want more than what you can get out of the box with this low code experience and co-pilot studio is a low code experience but it is designed to be able to reach through into some of the more sophisticated tools for when you do need to extend so if for instance you want to work with documents in an external repository and you want sophisticated search or you want to connect to a database somewhere else and again you want that really sophisticated search rather than just a single point of retrieval then one of the things you can do is to connect as your open AI on your data and bring it in and I'm going to take you through step by step how to do that and what you get out of it this is a great slide that Microsoft have provided as part of the co-pilot Studio implementation guidance I'll pop the link in the description for that full pack this is taking you through the way that it works with the low code tools where you can connect to public data SharePoint uploaded documents and custom data and this is the piece that I'm now going to go through so you'll see here that when you bring that as your open AI on your data you've got a lot of other things with those more sophisticated AI type of search tools things like vector search and semantic search which can go much deeper and give you much more accurate results so this is a use case for those really large complex type of data sets and what I'm going to go through in the demo here first is connecting up a database I'm actually going to use a sample database so if you want to follow along with this you can do exactly the same thing then what I'm going to do is bring in Azure AI search to create an index over that data and talk about some of those indexing options from there we're also going to have a resource for Azure open AI where we can bring the latest and greatest GPT models across working with our data here and then finally we're going to bring all of that back into the co-pilot Studio experience so this is this nice blending of starting with a low Cod tool going deeper into other things and bringing it back in there so that you can have that handoff and not have to start again so I'm here in my Azure portal I've got some resources that I've already got spun up here I've got my AI search aure AI search Resort and I have also got a Azure open AI resource in there as well let's first take a look at the search resource and what we want to do here is to bring our data into it and index it using that more sophisticated search functionality so we'll come up here and say import data and you'll see that you've got a lot of different data sources here that you can choose from depending on where your data is including some preview things there with SharePoint online which I will try in another video for now now I'm just going to choose the sample data set and you'll see that we've got a real estate sample set and a hotel sample set I'm just going to work with the hotel sample set here you've got options to bring other sophisticated search and AI Tools in here as well so we've got enrichments if you want to be using things like extracting people's names an entity extraction feature or location names we've got detecting languages translations key phrases so all of these things you can use in com combination this is all much more sophisticated than just simply pointing at a SharePoint site and being done I'm going to skip all of those for now what happens is I get my database showing up here with all of the things that are going on so you'll see that I've got a database here with multiple tables in it it's got the hotels in there we will scroll down a bit and you'll see that we've got other tables so we've got a table with the hotels has related rooms and all of those things are in there as well we can just have a look at the full list there and then each one we have got a check box in here of whether it is going to be indexed whether it's going to be part of the search so you can go through and set all that up to be the way that you want once you're satisfied with that we're creating an indexer now because I'm working on a sample data set here I can't have it automatically updating but with your real data set you can set up that schedule for being able to have the data updated into the index all right I think we are good to go let's click submit on there we get a notification here saying that the index was successfully created always love a success message and now when I come back in we can have a look at the indexes on the side here and see that that's come through and that's successful there's 50 rows in that table and we are ready to go now if you want to work with more sophisticated searching tools you've got things in here around working with semantic search again trying to give you an overview of getting started with this we'll go deeper into those things if you're interested let me know I can make some more content on this this is all very new for all of us learning all of this stuff all right now we're switching across into the Azure open AI Studio no more dark mode I'm afraid so what we've got here is the ability to deploy whatever language model we want to work with so the first thing I'm going to do is create a new deployment and then we're going to use that deployment of an AI model over the top of that indexed data that we just created and bring of those things back together so I've got no deployments in here at the moment we're just going to click here to create a new deployment I will select the model now what you have access to here is going to depend on what geographic region you're in I'm building this in an Australia East geography I've got access to the latest and greatest here I'm going straight for the GPT 432 in there there are different costs as well so you do need to weigh up the cost of these models against your use case if you've got really complex high value use cases some of those models are going to be worth paying for others you will get just as good a result with some of the other model so it is worth understanding that as well we give the deployment a name and click create and then again a success message excellent all right from here what we're going to do is test it out so I'm going to go over into my chat playground here and see what happens so this is giving me an experience of test ing things I haven't yet connected up the data this is just my GPT 4 Turbo model so we'll show you what it can do and then I'll show you the difference when it's connected to the data and this is a core concept here of being able to augment your language model you don't necessarily have to do custom models you can augment that with your data and have a much more close experience to what you want to do so let's come in here and pop in aquarium I'm going to New York for 3 days lucky me my budget is $300 a night I need something that sleeps three people I would like to be near the major tourist attractions and will click Send now this is just using the GPT 4 language model this is just predictive it's actually just going through and putting things in you'll see you can spot straight away I'm recording this February 2024 thankfully please make sure to check the hotel's Co protocols not top of mind anymore so this is not currently scaring the internet and finding things this is just a predictive language model giving the answers but we want to do better than this we want to connect it up to the data source so let's do that and see what difference it makes so over here add your data and now we're going to bring in that indexed data that we did in the first part here so I'm going to click here and say let's add a data source I can choose from any of these things I've already built all of that in the first part in that Azure AI search we can come in here and choose the search service so that was the one that I built it on and then I can can choose my Azure AI search index and that's the index that we created earlier you do need to acknowledge that this is going to start costing you money please make sure you check your costs when you do this and have all of the right alerts set up from here we're going to map the data into what we did in the index into here so it knows what it's working with so where is our content it's actually in the description and category Fields the file name is the hotel ID the Ty is the hotel name and I actually don't have a URL in this data set so we'll click next and then we've got the different search types I've only got basic keyword search set up here but if you've gone into those more sophisticated tools of using the semantic search then you'd be able to select that from here so we will click next we get a nice confirmation message save and close all right let's see what difference this makes so we saw what happened before let's clear the chat start again with a whole new convers ation we'll pop the same question in here and see what we get that's different so straight away very different answer we're going to work on the friendliness of this in a moment and there is one reference in here so this is how we can tell now that it's actually connecting to that data source and not just making stuff up it's actually connecting so this is a really good way of mitigating against that sort of hallucination but also having now that gp4 model augmented with our data but I didn't have to train a custom model I'm just adding it I'm giving it extra things what we can do here is if I click through on that it's actually giving me the reference so all of these tools that give you these citations can also just really help the user validate that they're getting the right information want to give this some personality though and this idea of adding a system prompt is another core concept of how you can get the most out of your AI experience so we'll click over here onto the prompt and you'll see the system system message here you are an AI assistant that helps people find information I mean sure let's do better than that you are a travel agent who is expert in finding hotels and tourist attractions that match what your clients are looking for your tone is friendly helpful and chatty now you can actually go quite deep here and the more information you're putting in this concept of asking the AI to act as a certain type of persona can completely change the experience you get and is another really good way of getting better results than just typing something in we will apply those changes update the system message in there and now let's give it another go and see what happens so we'll pop that in there look at that look how different this is now so what we've got oh New York the city that never sleeps so now we're getting that kind of Chatty enthusiastic Persona that we gave it we are still getting the reference we have got some references in here to going seeing things on Broadway which honestly is one of my favorite things to do just before before we go any further let's have a look at some of these configuration options so you can see that we've got the GPT 4 model in here if I go into the parameters we've got different things so again with the low code experience you don't have control over this as we go into these Azure AI tools I've got control over things like for instance temperature which sitting at zero means it's very accurate it's very deterministic we know what we're going to get out of it whereas if I slide that up to the top which is one it gets very cre ative so depending on your use case here 7 is usually the default for most of these things but given that it's connected to data it's defaulted right down to kind of a be less creative that also means that you will get the same answer every time whereas if you slide that scale up you're going to get a more creative and a different answer each time let's ask it one more thing because it's suggested I'll be right in the heart of Broadway I am interested in that what Broadway shows should I see on Valentine's Day I'm sorry but I can't provide real time information so what we've got here a really sophisticated language model connected to our data but it's not connected to the internet yet so it can't get that current data we're going to mix that up and bring that in in The Next Step so what I'm going to do now is take this model so I've got a combination here of the large language model the data indexed through the Azure AI search with any of those other sophisticated things that I wanted to do and I'm just going to push that back into my lcode app experience so we're going to go here deploy to a new Microsoft co-pilot Studio co-pilot you depending on when you're doing this you might still see some old names power virtual agents is what that used to be called don't worry about that it might have changed by the time you watch this now this comes up in a new co-pilot experience and it's asking you to give a co-pilot a name you will see here welcome to Microsoft co-pilot Studio name your new bot to connect your Azure open AI resource that's what you want to see at the time of recording this is only in US environments but hopefully coming to other geographies near you soon co-pilot name we're going to come in here and put in the name of the co-pilot it's only working in English again at the time of recording and I click create we give it a second the bot will deploy in the background and once that's done we get those lovely success messages that we like so much so what we're going to do now is see what this is done let's have a look at the plumbing so I'm going to come over here have a look at my topics and plugins so these are all of the topics you can actually use copilot Studio to create specific topics as we go through and say you know if the user asks these types of things take them down this conversational path we've also got in here this system topic which is this conversational boosting topic where this has gone this is sitting here as as unknown intent so at the moment my co-pilot is just a this is all it's doing I haven't created any other topics but I could have something around travel I might have something else that I want people to ask so depending on your use case you could actually have a conversational thread and then you bring this in as part of that or this could just be effectively a fullback or or the entirety of the co-pilot is just to connect to that you've got choices in there so what we have got here is that we are using generative answers on this data source here so if I open this you'll see that we've got the Azure open AI let's have a look at the properties for that and these are the properties that came through from where we set it up you can actually change this in here now if you want to because we've got that model available to us and all of those other values that we had in there around the index and what it was mapped to including our personality in here you are a travel agent who is friendly and chatty that's come across as well all of that is editable in here one last thing I'm going to go back into my data sources I want it to be connected up to be able to see what's going on we've got a public website in here so I'm going to put in the the broadway.com website and see what it does because that is now going to point to that this works best on websites that actually have some information if your website is largely just sort of marketing text and it's multiple layers it will only go sort of two layers down in the URL if you've got something that's largely graphics and navigation you won't get a good result out of this but if you've got websites that actually have a lot of substance fairly close to the top then this is a really good feature to use now you might have noticed up until now that I have actually been working off pre-recorded screenshots which I don't normally do but all of those Azure resources there's costs there's time to deploy I just wanted to make sure that all of that was uh under control I now in my live co-pilot and I'm giving it the same prompt here I'm traveling to New York on my own I enjoy art and theater where should I stay and what should I see while I'm there and this is going to draw on the data source as well as the website that I've put in there if you like any of this please give this video a thumbs up it helps it reach other people and I really do want to do a lot more content on co-pilot studio all right that sounds like an exciting trip we've got that nice personality coming through here as for things to see uh we've got citations so I can click on those things and see the information about the hotels Let's test out what happens with the website here so we're going to come in here and say what should I see on Broadway with a six-year-old and again it's it's able to determine whether it's going to that data source or whether it's going to the website you can have multiple websites in there and it's going to find that information so here we go we have got Broadway offers a magical world of Music we've got Aladdin Back to the Future that's actually a really good one I did was lucky enough to get to see that the Lion King and if I click on that it's actually taking me through to that best Broadway shows for kids which is where it's getting that information from let me know what else you would like to learn about co-pilot Studio thank you very much for watching

Channel: Lisa Crosbie

Views: 37,834

Keywords: azure openai, azure ai studio, azure openai service, azure openai demo, azure openai chatbot, azure openai tutorial, azure ai, azure openai on your data, microsoft copilot studio, copilot studio, copilot studio demo, lisa crosbie, learn with lisa, generative ai, azure openai use your own data

Id: FmRmdCpnq8M

Length: 16min 58sec (1018 seconds)

Published: Mon Feb 19 2024