(bright sound) - A lot of the information you work with today might sit outside
of Microsoft 365 locations. So how can you safely
bring that information into your Microsoft
365 Copilot experiences to ground generative AI results? There are two primary options. First, you can use
Microsoft Graph connectors to integrate the data sources you want with Semantic Index for Copilot. This option pre-indexes defined
read-only information used for retrieval with your prompts to drive the most relevant
AI generated responses. The second option uses
managed and trusted plugins which can be a few different types, including OpenAI plugins, Microsoft Teams message extensions, and Power Platform connectors. And with plugins instead of
pre-indexing information, they optimize for real-time
API based data retrieval and they can optionally
even write back data. Both options follow Microsoft's comprehensive security
compliance and privacy approach while respecting the
boundaries that you define within your organization's data services. And both types can even work
together when responding to a single prompt. In fact, let me show you an
example using a connector and a plugin simultaneously. Let's say I'm a help desk technician in Microsoft Teams and I want to respond to all users facing an issue with DNS. I'll start by submitting a prompt to help me find who's impacted and to write an email for me that I can send
them with instructions that reference our internal
knowledge base article. Copilot has taken my prompt and is generating a
response, in this case, by pulling in information
from both a plugin configured for Jira which we're
using for issue tracking, as well as a Graph connector
for an intranet site which we're using as a knowledge
base for our help desk. And it responds with a
matching DNS related ticket along with the steps for flushing DNS and even an email it authored
that I can use to respond to our user with the matching open ticket. You'll also see at the bottom of the response that both
the plugin's Jira ticket and the connector's intranet
KB article are referenced. Additionally, if I scroll up and click on the Jira
ticket details, you'll see that my plugin also
presents an adaptive card to show more information and it even allows me to take action. I'll hit "Edit issue" and
right from within Teams, I can change the record
in Jira, for example, if I sent the email to the user already
and resolved the issue, I can update the ticket status
from "To Do," to "Done." And by hitting "Save,"
it'll record that change to our Jira system. So we just saw how you can
extend Microsoft 365 Copilot to external data sources coming into the Microsoft Graph using connectors and two-way integration using plugins. Next, let me show you what was going on behind the scenes
and how each one works. The first option where we
referenced our intranet article, leveraged Semantic Index for Copilot which works with Microsoft 365 Copilot by default over internal
Microsoft 365 data. When you submit a prompt, Copilot brings in the Microsoft Graph with a Semantic Index to
correlate activity signals, retrieving indexed information
uniquely relevant to you. It presents the combination of your prompt with what was retrieved to the large language model to generate an informed response. Importantly, Semantic
Index works on two levels. By default, it indexes information within SharePoint inside
your organization's tenant. And second, each user also
has their own separate index for email and their
individual important documents like files shared with them,
where they've commented or were at mentioned, and more. This two-pronged approach
is what increases relevancy, allowing Microsoft 365 Copilot to orchestrate AI generated
results that are unique to you as a user, again, based on your approved
access to information. And a great way to demonstrate
Semantic Index for real is by showing two sets of
search results in SharePoint, one without leveraging the
Semantic Index and one with it. Here we're searching on
Project Melbourne Shiproom. Notice we have a typo. On the left, without the Semantic Index, you can see results are retrieved based on keywords and ranking as
well as last modified date for relevance and a
file summary for context but it's not the specific
information we're looking for. Conversely, on the right
with the Semantic Index, instead of a list of links, notice the informative
answer right at the top. You can see that the index has
mapped semantic signals such as people relationships and recent activities
surrounding the content the user has access to, to help determine the intent of your search and formulate an answer. So you can see how Semantic Index improves the search experience
across Microsoft 365 and that includes Microsoft 365 Copilot. So how do you take advantage of Semantic Index for your external data, like we saw in our example? First, if you already use Graph
connectors in Microsoft 365, they will work without any changes. If not, before you
connect to external data, from the Microsoft 365 Admin Center, you do need to enable the service or start a trial and from
there you can add a connection. There are options for enterprise websites and a few other common data
sources listed here as well. For example, if I open ServiceNow, you'll see that there are three options for ticketing, knowledge base,
and catalog, but in my case, I'll set up an enterprise website and this will show me a few simple options to define the site, who
can search the data source, and whether connector
results are shown in search. Then for even more options, I can go into advanced settings for
more granular configuration with options to define meta
tags, specific exclusions, semantic property labels, and
customizations to the schema. And once you have a
connection up and running, you'll be able to see integrated results using SharePoint search
with the connected site. In this case, I'll just
perform a simple keyword search for troubleshooting instructions
and results are found in SharePoint locations as well
as from the connected site. And here is the DNS article
that was used as part of the retrieval process to respond to my prompt from before. Beyond Microsoft Graph
connectors with Semantic Index, let's look at our second option for data retrieval with
Copilot, using plugins. Again, starting with how they work. So with a plugin configured,
when we submitted the prompt, Microsoft 365 Copilot
leveraged that plugin to retrieve information
directly from its source, again, in real time without
pre-indexing the data. It was also able to perform
additional actions on behalf of the user and with respect
to their permissions, like changing the status of the ticket. This time, I'll demonstrate
the plugin by itself without connection to our knowledge base. In Microsoft Teams,
I'll write a new prompt to look up VPN related
tickets in our Jira system and submit it. Then Copilot responds
using that information and finds two tickets in the
system related to VPN issues. And as I mentioned, plugins
work without pre-indexing and there are three types. These can be OpenAI plugins, Microsoft Teams message extensions, and Power Platform connectors. Now I'll explain how plugins work with a short tour of their code. All three of these types
of plugins are similar with similar packaging and components. This is an example of a
Teams app manifest file that uses NuGet Package
Manager to find app packages and their details. You can see that it refers to a YAML file. And in that file, is
where the API is defined for connecting to the
external data or service as well as any actions it can
take on behalf of the user. Also note the natural
language descriptions of the API get operation
and its responses. These are used by the LLM to assess what the plugin can do to help respond to a prompt. And because these are portable
components and can be used for OpenAI, Teams, and Power
Platform, as a developer, you'll be able to maintain
a single shared code base. Each will leverage the same
underlying open API spec. Then to make plugins available in your Microsoft 365 environment, you'll do this from the
Microsoft 365 Admin Center in the Integrated Apps page. Here you can choose from existing plugins from the store, just like you
would get other apps today. You can also upload your
own plugin packages directly in the Teams Admin Center. Either way you add the
plugin, you're in control of who can access the plugin
and its underlying data. So those are the two main
options available to you for safely bringing in data
external to Microsoft 365 into your Microsoft
365 Copilot experiences to ground generative AI results. To learn more, check out
aka.ms/CopilotExtensibility and keep following Microsoft Mechanics for the latest in tech
updates, thanks for watching. (bright sound)
