Build plugins for Copilot for Microsoft 365 with your own data

hey everyone my name is rabie Williams and I'm A Cloud developer Advocate um and today I'm going to show you how you can build plugins for Microsoft co-pilot so like Rob mentioned plugins allow you to expand skills for co-pilot and there are several ways you can create plugins for our platform as a developer you could use uh teams message extension Power Platform connectors or open AI plugins to build on your existing technical Investments in Microsoft 365 platform Integrations or you could create new ones that align with emerging industry standards but today our focus in our Pro developer story starts with teams message extensions now for those of you who don't know about message extension which I highly doubt I'm just going to keep it in simple terms a message extension is a capability in teams that allows you to enhance how you send a message now we have lots of them already in our store and for sure I I'm pretty sure many of you have already built one but let's see an example all right so we created this message extension northwin inventory uh to help with the inventory management of this fous company called northwin Traders um I'm in chat with AA and Bob my colleagues at Northwind and I'm searching for chai and this is how the message extension looks like I have this adaptive card now I've shared with my colleagues I can go ahead and ask them to restock products Etc so this is what it does you don't have to open up an app uh you can directly in a conversation bring uh business data and then you could make uh updates to that data what makes up a plugin for co-pilot now there are these three building blocks as you can see here first one is the app backend which is your logic basically so it could be a teams uh bot based message extension or it could be an API based message extension which we will cover later on in the session and the second one is the Manifest file which in my opinion is probably the most important file for co-pilot because this file tells co-pilot what your app is capable of doing and the third one is adaptive card which basically forms the response from co-pilot which has all your business data dynamically populated and you can take action in them right now this demo that you saw earlier the northwin inventory um it works with a single query parameter meaning that it can only do one query like search based on a product name because you saw that I was searching for chai that's the product name and then with that I can get the results back from my API so um if you have that as a plugin which this message extension I forgot to tell you that this is already a plug-in that you can use in co-pilot but now if you're going to use that you could start off with prompts like fine chai in Northwind or send me details of tofu again you're just emphasizing on the product name but this is a developer show so we want to show you something more exciting where you can take this message extension and make it next level plugin so let me just show you that so this is my message extension app um as you can see here I'm in the Manifest file the second uh building block the Manifest version is 1.12 which is which needs to be upgraded to 1.13 and above for it to work as a plug-in so we're looking at the old uh mage extension still when you come down below you can see that there is a section called compos extension now this is the message extension definition in manifest file so you've got this bot command you've got bot ID and then you've got the command called Product search so you basically go ahead and do a search on product based on parameter product name so that's what happened before so the chai is the product name and with that it invokes this bot command and brings all the list of uh products so this is now how it works so you also have a backend bot code which will take this and process like I mentioned now let me show you the enhanced version of the plug-in so I'm in manifest version 1.16 and scrolling down to the manifest definition oh I'll just stop by and say that all the descriptions will be pretty elaborate because this this is where co-pilot gets all the information from manifest file saying okay this app is supposed to do this so whenever the user types in an a prompt it knows okay that is the intent so this has to be elaborate like most of our as developers we just write you know if it's northin inventory we just say North an inventory app description and this is what we do most of the time but you can't get away with that now because you need to put a description for co-pilot to know what your app is supposed to do and then we come down to the message extension section SE you've got now inventory search instead of product search because hey co-pilot is able to process more than one parameter so in teams and Outlook your message extension can process only one query parameter but co-pilot can process multiple so now we have product name category name status Supply City stock query and you'll see that we''re even giving hints to copilot as of what possible values we would get so if your prompt has something like get me stock for um uh chai if it's about 50k units or something it would know how to process that information so you can give all that in the description um so this is how it works now and then in your Bot code obviously you have to go ahead and make sure that you're passing all these parameters into your API calls so this is what we've done with uh the enhancement of this whole app now enough with the talk let let me go ahead and show you some code sorry my demo so I'm using co-pilot to perform two tasks in one go first I want to generate a list of all the beverages that are currently low in stock second I want to know whether these products meet North V's minimum order quantity policy to accomplish this co-pilot analyzes the policy to determine what it means to be compliant I'm not told uh co-pilot what it means so it finds that a product is compliant if it's reorder level is greater than 25 units co-pilot then retrieves the low stock Beverages and checks whether they meet this rule it generates a report for me and I really love how it puts all these emojis um and then it's actually going ahead and showing which is compliant and which products are not compliant it also gives me a summary of the policy and a link to the document the policy document so that I can go ahead and verify it myself now I was a bit fast there because I wanted to take a closer look at this prom prompt so the first line co-pilot is able to understand that this is okay this is calling a plugin so it's able to do that and then it will translate that natural language into query parameters and then invoke the bot of that plugin now let me show you how that is in my app that is running locally I'm now opening up visual studio code which is where the teams toolkit is currently running my application here we can see that I've actually logged this um query parameter object so that you can actually take a look at what happens when it's translating and um you can see here that the category name is mapped to Beverages and inventory status is low stock based on that one prompt and then it will go ahead and call my bot for it now let's revisit the second part of the prompt which involves compliance so copilot has successfully located the relevant documents summarize the document for me it and um you can see here it added the reference to the document and you can go ahead and click that and verify and it's even got a little blurb for me to read through the document so in a way what it does is it takes your business data from the plugin and then it searches uh files emails Etc and then you know comes it all comes together and gives you the right information so that you could make Better Business decisions and get work done faster right let's move on to another one because this is fun um so I have another uh work to do I have to find all the low stock products coming from Sydney and right now copilot is trying to interpret this prompt doing its translations converting it into query parameters calling the uh bot and everything happening um at the same time it will find out okay this is that one product that matches the criter criteria it will give me the Adaptive card right here in the conversation um and I'm about to take an action to restock but then I had this thought okay what is the payment term with this supplier obviously I have co-pilot I have powers so I go ahead and ask you know what is the payment terms for uh this Supply good day made and um it goes ahead and searches all my files and my emails and everything and it finds that there is something there there is a agreement with this uh particular supplier so it goes ahead and brings all that information and even highlights the part that I should be interested in that is the payment term and as you can see it also gives me um a little blur of what it is and then the link to the PDF file let's go ahead and click on that link and open up the PDF file this person who's doing all this job is really busy so you know they can't really read a document but hey here we are and um right now you can see the payment terms over there and you know it's it's all good um and then let's go ahead and restock this product so I can go back to the Adaptive card here take an action uh put the quantity in and then click on restock and as you can see here the Adaptive card will refresh and on the spot and it will update the status from low stock to on order so everything is done in the same conversation you don't have to open different apps different documents everything is done now thank you by the way this Northwind inventory app or plug-in um is available open source for you to go ahead and try out if there are developers here now to wrap it up let's think about what value plugins bring you can connect co-pilot to your third party application you can retrieve realtime data like the product information stock information something you need upto-date information about and then you could also help your users take action then and there in the conversation with co-pilot and obviously the probably the most important is if you already have a message extension you could reuse those uh technical Investments and create a plugin without doing anything from scratch of course you can do it from scratch too

Channel: Microsoft 365 Developer

Views: 11,413

Keywords: Copilot, Microsoft Copilot, Copilot for Microsoft 365, Copilot demo, Dev, Microsoft developer, Plugins, microsoft plugins

Id: HpgKKf6Ruek

Length: 11min 18sec (678 seconds)

Published: Tue Dec 19 2023