[MUSIC] SCOTT GUTHRIE:
Well, good morning, everyone. It is great to be
back here at Build, and we're looking forward to
spending a week with you. As Kevin and Satya have
talked about today, AI is going to
profoundly change how we work and how every
organization operates. Every existing app is going
to be reinvented with AI, and we're going to see
new apps be built with AI that weren't possible before. Kevin just walked us through the Copilot stack and explained the different
layers involved in building solutions that take deep advantage of
large AI models. I'm going to build on his talk and walk you through how
we're going to make it easy to develop these AI
solutions using Azure, GitHub, and Visual Studio. We're going to show a
lot of code today in my talk and introduce
you along the way to some amazing new capabilities that we're announcing and
releasing here at Build. Let's start by talking about
Visual Studio and GitHub. The Visual Studio
family of products, which includes
both Visual Studio as well as Visual Studio Code, are now the most widely
used development tools in the entire world across
all languages and platforms. GitHub is used by over 100 million
developers and is literally the home
of open source. GitHub also now provides
developers and enterprises with an integrated end to end developer platform that can
be used to collaborate, automate, and secure
DevOps solutions. With GitHub, you can build
amazing AI solutions, including all the Copilot capabilities that
Kevin just talked about and use it to
transform your businesses. Now one of the most exciting new capabilities that
we've launched with GitHub is a new service
we call GitHub Copilot. GitHub Copilot was the first
solution that we built using the new transformational
large AI models developed by OpenAI, and Copilot provides an AI pair programmer
that works with all popular programming
languages and dramatically accelerates
your productivity. This spring we
announced a new update for GitHub Copilot that
we call Copilot X. Copilot X further
enhances Copilot using the latest GPT-4 models and provides immersive Copilot
chat functionality, Copilot for pull requests in automatic unit test
generation scenarios. It's truly amazing and what I'd like to do is invite
Thomas Dohmke, the CEO of GitHub on stage
to show it off in action. Please welcome, Thomas. THOMAS DOHMKE:
Good to see you. SCOTT GUTHRIE:
You too. THOMAS DOHMKE:
Hello, Build. It's so great to be here in person. I'm Thomas and I'm a developer. What we have witnessed
this past year with GitHub Copilot has been a step change to develop
our productivity. Software has eaten the
world now it's AI's turn. But the autocomplete
Power Platform program has always been just
the starting point. We swing the doors wide open to a new age when natural language powers
the coding experience. That begins with
GitHub Copilot Chat, a key feature for
GitHub Copilot X. Let's see some real code. For this demo scenario here, I've inherited some Python
code from a coworker. It's pretty purely documented that we've all
been there, right? With GitHub Copilot, I can simply write code by
accepting code suggestion. Here's a little unit test. I can just press the Tab
key a couple of times and I've written a
full unit tests in seven lines of code with no learning the
code and whatnot. But what I can do with Copilot is to provide
additional context. I can't tell Copilot
what I want. Well, until now, because now we have Copilot
chat sits right here. In Visual Studio Code, it suggests task based on
natural language and it can interact with it down here to both learn and build
while I'm coding. We have another file here. It's called Unknown PY, and it has these three
cryptic regular expressions. I have no idea what
those are doing. Let's see if Copilot
can explain them to us. Copilot is rendering
my response. You can see at the
first expression is validating email addresses. The second, one is
validating phone numbers, and the third one is
validating strong passwords. Makes sense, I guess. Let's highlight this and make this code a
bit more readable. Copilot analyzed my code. You can see here, now my regular expressions
get nice names, I get nice comments. It looks much
better, but I think we can do even a
little bit better to make the next person that takes this code
understand it even more. Let's say we want
to separate out the validation functions
and add more comments. Here's even better code. My regular expression still
have the good names and I get three new methods enough to validate email,
phone number, and password. I think that's much better. I can just pull up here
and take this code over. Now I have a much
better documented file that I can pass on to
the next co-worker. I got a third file and this
seems to be passing expenses. [APPLAUSE] I got a third file and this
seems to be passing expenses. Let's see if this
actually works. Oh, I got an error, it can't unpack the values. Let's see if Copilot can
fix the bugs in my code. That's a couple of issues. Seems like the split
method is splitting the values as a comma separator. But in my comment here, I had said I want
the space separator. Then it also says the
date time module is are imported probably because I
only highlighted the methods. If you look at here,
the fix isn't split. My fixed here and split is here so I can just go of course here, I'm still the pilot and
fix this one method, and now my code runs. Cool. While we're here, I can also ask
Copilot to generate some unit tests for
me and I can do this because slash commands. Instead of typing all
this stuff on stage, I can just type tests. Now Copilot analyzes all
the code paths in my code. It looks at the comment, and you can see it
here generating multiple unit tests
for my class. In fact, there's one
for invalid dates, one for invalid values, command lines, single
lines, empty string. It does all that
boilerplate work for me that I don't want to do. Now I can take this, click this little three dots here and insert
this into new file. If a new unit test file
and it can of course, now use Copilot and Copilot Chat to build
this out even more. Well, thank you Copilot for
doing all the work for me. [APPLAUSE] Now I know some of
you may be thinking, when can I get my
hands on all this? We want to bring this
whole experience, all of Copilot X as broadly as we can and
as soon as possible. But right now we have
quite the waitlist. While don't worry, I have an Oprah moment for you. You get a free Copilot Chat. Everybody here right at Build in the audience gets early
access to Copilot Chat today. [APPLAUSE] Use Copilot and Copilot Chat both in Visual Studio and Visual
Studio Code and thank you so much for being here today.
Back to you, Scott. SCOTT GUTHRIE:
Thanks Thomas. We're really excited to see what everyone here is able to
do with GitHub Copilot. As you just saw, Copilot, dramatically accelerates
developer productivity. In fact, 46 percent
of all lines of code written by developers
that are using GitHub Copilot are now
fully AI generated, 75 percent of developers using Copilot are feeling they can now focus on more
satisfying work. No one really loves debugging regular expressions and you can now enjoy your jobs more, 96 percent of
developers report being able to complete repetitive
tasks a lot faster. We now have over a
million developers using GitHub Copilot around the
world today and again, we're excited to have
everyone here in the live audience be able
to take advantage of the latest GitHub Copilot Chat capability starting this week. Now, we've seen with
Visual Studio and we've seen GitHub Copilot and
Visual Studio in action. Let's now look at how we can use these incredibly
powerful tools to start building AI plugins and as you heard from
Satya and Kevin, we're embracing an open
plugin standard that provides plug-in interoperability
across ChatGPT and all of the Microsoft
Copilot offerings. With GitHub and Visual Studio, we have an incredibly
streamlined developer experience for building these AI plugins. Let's watch a demo that
shows all this off. SPEAKER 1:
Plugins enable developers to integrate custom applications and data into the chat
experience for users. Let's say we have
an e-retail store that sells outdoor equipment. When a user asks a question
about outdoor products within ChatGPT we might want to provide them with a list of related
products that are available. Now let me show you how
easy it is to create a plug-in using Visual
Studio Code and GitHub. I could fork and clone the
repo on my local machine, but instead, I'm going to
use GitHub Code spaces. Codespaces are
GitHub's one-click developer environment
in the Cloud. They're fully configured
to your repo and they're free for up to 60 hours a month, which makes it a great fit for
open source contributions, exploring new stacks or working
across multiple devices. Now let's crack it open
and see what's inside. The code for this sample plugin actually looks pretty simple. It uses FastAPI, a Python web framework to expose some rest API endpoints
that are called by ChatGPT. For this demo, our plugin
is just going to return product information when users ask about them in the chat. I'll start by just loading some fake products from
the products.JSON file. I'm pretty new to
Python and I just love how Copilot helps me
as a pair programmer. I'm able to stay in the zone and just focus on the big picture. Once we're done with our API, all we need to do to
transform this project into a plugin is add a
little bit of metadata. Well-known ai-plugin.json
has a description of what the plugin does and
includes things like a logo that will be
shown in the response. The same file can be used to
create plugins for ChatGPT, Microsoft 365 Copilot
and Bing Chat. Now let's add a
break-point to our API. Once the plugin gets called, I can observe the flow from
the request and the response. One of the great benefits
of Codespaces is that I don't have to do anything special to install
the dependencies. I can run and debug the project and
everything just works. When our project starts, we can see that Codespaces has opened the ports for
our FastAPI server. This is really helpful for
debugging web apps and APIs. By default, the endpoints for our codespace are only
accessible to me. They're secure by default. But I can also
configure them to be accessible by my team
or my organization. Or I can choose to
make them public. For the purposes of this demo, I'm going to make
our FastAPI port public so that
ChatGPT can call it. Once I have the URL copied, I'm going to create
a new chat in ChatGPT and then install the
plugin via the plugins door. This is really simple. All we specify is the URL to our codespace and ChatGPT
will take care of the rest. Now that our plugin
is installed, we can start experimenting
with the code inside the chat. To check it all
out, all I need to do is start asking
questions that might match the natural
language instructions we gave in metadata we
exposed for our plugin. In this case, it was
climbing products. Because our plugin is
running inside a codespace, we can even interactively
debug it as well. As you can see, our
breakpoint is being hit inside VS Code as soon as
our plugin is invoked. We can step through the
code, inspect variables, and really determine if there's any issues with the
way that our code is behaving based on what's
being sent by ChatGPT. The experience for building
a ChatGPT plugin inside VS Code and GitHub Codespaces
is incredibly productive. It's a lot like
building a web app. The experience we showed here was for building
a ChatGPT plugin. But the experience for
building a plugin from Microsoft 365 Copilot and Bing
Chat is exactly the same. Now let me show you
the same plugin we just built running
inside Bing Chat. Microsoft 365 Copilot also
supports the same plugin, and you'll hear more
about that tomorrow. [APPLAUSE] SCOTT GUTHRIE:
Plugins are incredibly powerful way to connect your apps
and data to AI and you just saw how we can use
GitHub Codespaces to really deliver an amazing development
environment and Copilot, to help you code and
actually build it. As Satya announced this morning, we now have one plugin model, extensibility across, again,
OpenAI's ChatGPT service, as well as all of our
Microsoft Copilot experiences. Tomorrow morning you'll
hear from Rajesh and Panos who will share
more about Teams and Windows and how you can use the exact same
extensibility model to build great experiences
within them as well. You can use this
extensibility model to enable plugins within copilot experiences you build within your own apps as well. A little bit later
on in this keynote, I'm going to talk
about how you build your own Copilot experiences using our Azure set of services. Now, in that last demo, we use GitHub Codespaces, to both code the plugin, but also to do a simple
test of it within ChatGPT. Because Codespaces was
running in the Cloud, it was really easy for
ChatGPT to connect to it and you have a nice interactive debugging experience
as part of it. This works really
well for getting started and for simple
development scenarios. But once you go into
production and as you're AI plugin gets a
little bit more sophisticated, you're going to want
to be able to have a backend that's
going to be able to scale it out and there, you're going to need
an elastic Cloud. Azure provides a great set
of Cloud-native services, including powerful
Kubernetes, as well as container-based
solutions that you can leverage to do this. With our GitHub actions
capability inside GitHub, you can easily set up now a secure Cloud-based
CICD solution to automate and
secure the building, the testing, and
the deployment of your plugins to these
Azure services. We have lots of organizations
out there taking advantage of this
capability today, I think a great example
given we're talking about AI scenarios though,
is ChatGPT itself. ChatGPT is a great
example of using this GitHub and
Azure combination. OpenAI uses GitHub for their development and they
use Kubernetes on Azure, as well as our Azure
Cosmos DB database service to run and scale the actual
ChatGPT service as well. Obviously ChatGPT has captured the world's imagination
this past six months, it's the fastest growing
consumer product in the history of the web and want
to do now is show another demo on how you
can also deploy AI apps using that same GitHub and
Azure combination as well. SPEAKER 1:
Now that we're ready to publish the plugin, we need to run it
somewhere that's secure and scalable
for our customers. At the end of the day
an open AI plugin is just a web service. We're going to use
Azure Container apps to host the production version of our plugin and GitHub
Actions to test and deploy it. Now, I could go through the setup steps
in the GitHub UI, but today I want to show
you how to configure GitHub Actions in Azure
all within the terminal. We'll use the new
Azure Developer CLI or AZD for the steps. AZD is a command line tool that accelerates the process
of building apps on Azure by providing developer-friendly
commands that help automate each stage
of the developer workflow. I'm going to start by making
sure I'm logged into Azure. Next I'm going to initialize the repo with an AZD template. Templates include everything I need to provision
the resources for our plugin and deploy it to Azure Container apps
using GitHub actions. The component that was added to the repo is the GitHub Action. Now, I always have a
hard time reading YAML, so let's use Copilot to
help us make sense of it. It looks like our
action is going to provision Azure Container Apps, and all the other
services we need for our production scenarios
and I also see that it's referencing some external
variables so I won't accidentally be checking in
any secrets into my repo. Now I'm just going to push
my changes to the repo. Let's take a look
at what happens when one of these actions runs. I'm going to open
the actions log for our repo to see
our action running. I can see it's provisioning all the resources we need
and then I can check out the results to get the new URL of our Azure Container Apps
where our plugin will run. Now that we have our plugin running on Azure Container Apps, we're able to scale to
millions of users with the confidence that
our plugin will run securely and with
high availability. Our continuous
delivery pipeline and GitHub Actions enables the
team to collaborate and commit changes while ensuring
that the updates are tested and deployed in a
repeatable, secure manner. SCOTT GUTHRIE:
We walked through how you can clone a sample AI plug-in repo, add code to it, set
up a CICD pipeline, and deploy it into Azure. As you've seen, everything
is pretty easy to do, and what's even better is you
can start doing it today. The exact same repo
that we've done in all these demos is
now public on GitHub, and all the
capabilities that we've shown in terms of
Visual Studio and GitHub and Azure are now available for all
of you to start building great AI
plugins with it. The beauty is it'll work
with not just ChatGPT, but with every Microsoft
Copilot experience, as well as the Copilot
experiences you build yourself. We spent some time looking at
how to develop AI plugins. Let's now switch gears and deep dive into what
we call AI orchestration. Now, as Kevin explained earlier, AI orchestration
involves grounding, prompt design and engineering, evaluation, and AI safety. These are the core
fundamentals for how you create great Copilot
experiences. This is exactly how Microsoft created all of our
Copilots as well. If you think about the
pace of AI innovation that you've seen from
Microsoft over the last five months, it's been
pretty intense. We've introduced GitHub Copilot, Dynamics 365 Copilot, PowerPlatform Copilot,
Microsoft 365 Copilot, Security Copilot,
Nuanced Bing, and more. That's a lot of Copilots. People often ask me,
"What's in the water? How are you producing so
many Copilots so fast?" The reason we've been able
to move at this fast pace is because we built all of these Copilots on
top of one platform, which is Azure AI. This is the exact same platform
that all of you can also leverage as you build your own Copilot in AI
experiences as well. Azure AI includes several
categories of AI capabilities, including our advanced
Azure AI infrastructure. Azure OpenAI Service is our newest capability and became generally available in
January of this year. The Azure OpenAI Service enables developers and organizations to access and use the most advanced
AI models in the world, including both
ChatGPT and GPT-4. All of these Microsoft Copilot
experiences that you're seeing here at Build are built using these Azure
opening AI models. Again, all of the Microsoft
Copilot offerings are built using the
exact same Azure OpenAI Service and
APIs that you can all use to build and scale your
own AI solutions as well. Now, what makes
the Azure OpenAI Service so powerful is that it enables you to ground and fine tune the AI models
with your data. We're making it easy to do this with all of our
Azure Data services, and this was to get much more intelligent and tailored outputs from the AI models. Obviously, as you do it, one of the questions we
get asked a lot is, what happens with my data? The bottom line is, it's really important as you're grounding AI with your data that you
trust your Cloud provider. Obviously, Microsoft and
Azure AI runs on trust. We're commited to doing all
the right security work, the privacy and compliance work across everything
that we do. We've been doing this with
our Cloud for over a decade. Our approach to AI
is no different. When you use the Azure
OpenAI Service, your instance is isolated
from every other customer. Your data's not used
to train or enrich the foundation AI model
that's used by others, so you don't have to worry
about anyone other than your organization
benefiting from AI that's trained on your data. Your data and the AI models are protected at every step by the most comprehensive
enterprise compliance and security controls
in the industry. That's why more than 4,500
customers are already trusting and using the
Azure OpenAI service today. It's the fastest growing
service in Azure's history, and I want to show a
quick video of a couple of customers that
have already built some amazing solutions with
it. Let's watch a video. [MUSIC] SPEAKER 2:
With Azure OpenAI Service, developers can unlock
innovative new ways to delight customers and offer completely new experiences
with generative AI. As a leading agreement
technology provider, DocuSign set out to
help customers and signers better understand
their agreements. With a single-click, DocuSign extracts
key datapoints, creates an AI
generated summary of a contract and even
allows users to ask questions and get answers
quickly so people can review and sign agreements
faster with more confidence. Thread helps IT
technicians improved customer communication
by eliminating tickets in favor
of conversations that began in the applications
they use every day, like Microsoft Teams with
Azure OpenAI Service. Within two weeks, Thread
developed a tool that automates time entries and streamlines resourcing and service
request assignments, saving technicians
over an hour per day. Typeface unlocks the creativity
of their customers to tell unique stories faster and
easier than ever before. Using Microsoft Azure and
Azure OpenAI service, Typeface provides customers like Vasanthi Cosmetics with a
seamless experience to craft captivating content across
marketing channels while staying true to their
distinct brand voice, and visual identity. After learning about the
Santi's brand using images, sample messaging, and
branding guidelines, Typeface uses Azure
OpenAI Service and other services to provide stunning images and
compelling copy in seconds that is
right on-brand. Azure OpenAI Service makes
the impossible possible for small businesses and startups as well as large and
established brands. [APPLAUSE] SCOTT GUTHRIE:
You just saw some great examples of companies that are already using the Azure Open AI Service within
their products. As you heard in some cases, it was done in weeks. Let's now dive deep into
how you can as well. This week at Build, we're introducing our
new Azure AI Studio. The Azure AI Studio makes it incredibly easy to ground
your AI models with your own data and to build retrieval augmented generation
or RAG-based solutions. This enables you to build
your own Copilot experiences that are specific to your
apps and organizations. You're going to see, we've
made it really easy to do so. Please welcome Seth Juarez onstage to actually show
us how to do it live. SETH JUAREZ:
How's it going, my friends? You're probably
sitting here thinking, "This has got to be
really hard for me to do, I want to build my own Copilot." It turns out that creating your own private
company Copilot that understands your data with Azure OpenAI Service has
never been simpler. In this scenario, we're the Contoso
outdoor company, we're trying to make
Contoso happening again. Notice that when I ask
it a question that's specific about a return policy, because here let me type
it, I'm a fast typer, but notice I'm asking about camping and I'm wondering
if I'm going to get dirty, and now that I purchased
the Trail Master X4 tent, I want to know, can
I send it back? Notice that the response
that it gives is effective and it has the
right documentation. How do you do such a thing? You might ask. Well, it
all starts with Azure. You go to the Azure portal and create a new Azure
OpenAI Resource. This is just like any
other Azure resource. What you do when
you get this is you get this Azure AI Studio, which is a new experience
which makes it incredibly easy to ground your
AI models with your data. I can use any of the
available models. Notice here I have
GPT-4 and GPT-35-turbo, amongst others
that are on there. Now, this experience allows
you also to test it out. I'm using the vanilla
ChatGPT here, and notice that I asked, when I ask it about camping, it has a pretty good answer and you can see these
responses in there. The question then becomes
what happens when I ask a specific question about the return policy for
the Trail Master X4 tent? Well, notice it doesn't know. How do I fix this? How do I put data in there? Well, you've heard
a lot from a Kevin about something called
the RAG pattern, which is where you
retrieves facts, inject them into the prompt
or augment the prompt, and then what you do is you
generate a new response. Now, the thing
that I want to say about this studio as
you're looking at it is this is your very
own Azure AI Studio. It only benefits you. No one else can see your data. This is just for you. This RAG pattern that
Kevin talked about is such a common pattern that we've simplified it to
just a few clicks. Let me go ahead and
click "Add Data", and then go to add data source. Look, I got excited and
started doing it backstage. Notice that when you
add your data source, you can select from a number
of different data sources, all the modalities
available to you on Azure. Also with cognitive search
as you heard about before, there is a new vector embedding available to
you right in there. Also, the cool thing
about cognitive searches, it can crack open
your PDF documents and your Word files
and be in there. Let's keep going. I acknowledge that I'm
connecting to an Azure service. Now, let's map some
of the data together. There's the content, there's the filename, there's the title. This is the best
part of programming. Hit "Next". Here's
a cool feature. This is a cool
feature that we also added into this
experience where you can limit responses to the
data content that you retrieve or you can let the service also augment some of that
information as well. I'm going to leave it off.
Let's hit "Save Data" and now I'm going to ask it some of the same questions from before. I'm really concerned
about cleanliness and camping apparently.
Who wrote this? Notice that I'm getting a better answer that's
grounded in data, and also what I ask it
about a specific policy, return policy for the
Trail Master X4 tent, I want you to notice
something about the answer, which
is really cool. Notice not only do
you get an answer, but you get the citations
from your actual data. What happened is when
the query went in, it retrieved your data, injected into the prompt, and provided the answers. As you can see,
it's pretty simple. Once you've got this working
in the playground and interaction flow is set up, everything is exposed as an API. You can use it in any
application and service it natively in whatever
existing application you have like this one. Taking full advantage of the power of Azure
AI. It's pretty cool. [APPLAUSE] SCOTT GUTHRIE:
Thanks Seth. Seth just showed you how incredibly easy it is to ground the Azure OpenAI service
using your own data. Keep it private and specific, just your organization so
only you benefit from it, and then this
enables you to build great Copilot experiences that are specific to your apps, specific to your organizations, and the great thing is
what Seth just demoed is now in preview
for you all to use. [APPLAUSE] SCOTT GUTHRIE: Data grounding is
a critical part of how we build applications. As Kevin covered in his talk, AI orchestration also
includes prompt engineering, and prompt engineering involves constructing the
prompt and the meta prompt that you provide the AI model to
produce a stronger, more specific response
for the user, and this week at Build, we're introducing our new prompt flow support in Azure AI. Prompt flow provides end-to-end
AI development tooling that supports prompt
construction, orchestration,
testing, evaluation, and deployment, and it makes it incredibly easy to
leverage open source tools and frameworks like
Semantic Kernel and LangChain to build your
AI solution as well. Let's go ahead and take
a look at prompt flow in action and Seth back to you. SETH JUAREZ:
Let's do it. Before remember we were looking at
just the documents. This time we want to look
at a little bit more. So this is the same e-commerce
application from before, which could be any SaaS
company or product. Notice I'm going to
ask it about jackets, but this time I'm going
to be logged in as Jane, who has a very specific
shopping cart. Remember before it
was just a bunch of documents that were used
to augment the prompt. But can we do more? Can we add more information
into the prompt? It looks like we certainly can. Notice that we're getting a
good response to our data, as well as it's telling us how well it
pairs with the stuff in our actual cart and they are actually in there
which is pretty cool. To do this, we are using a brand new Azure AI
feature called Prompt Flow. It's pretty cool. This is really cool we should
clap a little bit. It's so cool. This
is really cool stuff. This is a powerful
suite of tools and services designed for prompt engineering regardless
of your experience in machine learning or AI, it's the first end to end prompt engineering
tool of its kind. It includes tooling for prompt
construction, evaluation, deployment, testing,
and even monitoring. With Azure AI Prompt Flow, we can fetch data
from multiple sources and fetch it and put it
directly into the prompt. Whether it's structured,
unstructured, it's pretty cool. I'm going to show
you three things. There's a ton of things in here, but I'm just going
to show you three. First the graph, the
prompt and then the model. This here is the
prompt flow graph, and what's cool about
this is you can get structured information
coming into here like a JSON file, and you can flow information
down into the prompt. Data flows down into the prompt, like Flow Prompt,
but we called it Prompt Flow. That's
the first thing. Let me show you the
retrieved cart node, which is pretty cool. I'm going to just
show you one node. Notice that this node
takes the cart ID and calls this minimal API, and this minimal API
is written in .net, so it looks like I closed it. Why would I ever do
that? I'm so sorry. It's a minimal API that you can call that has C-Sharp
in the backend. Now, the reason why
I wanted to show you this really quickly is to show you how extensible this is. If I can call an API, I can do anything as I'm
doing things in the prompt. In fact, that API that I'm calling that
you see right here is exactly the same API that manages the
cart on the website. It's exactly the same thing. I'm not even kidding.
It's the same thing. I could show you the backend, but I forgot to open
Visual Studio Code. The thing about this extensibility that's
actually really cool is that because of that we can extend to things
like LangChain, Semantic Kernel, or other
tools that are data based. Forgive the pun.
That can pull data into your prompt or
really anything else. That's the first thing
I wanted to show you. The second thing I
wanted to show you was the actual prompt. This is the prompt
that we're building. Notice that this prompt
is a bunch of text that gets data from the
previous nodes. As I zoom in, notice
that it's enumerating the items of the cart and
putting it into the prompt. That's how it knows when it generates the actual response. This then gets put
into the model, that then is, the
answer is retrieved. This model can be any of the OpenAI models that
you saw before, including any OSS models
that you deploy in Azure ML. Let's go to the actual model call this is where
we call the model. We can run it directly in here. We can also bulk
test and deploy, and if you want to
see the responses, you can see the
responses right here, which is the answer
we saw before, as well as the actual trace of where and how long things take. Finally, you can deploy
this and you can use it in your app as
a single endpoint. In summary, prompt flow
empowers developers to create richer language experiences that integrate data from
multiple sources, providing personalized and contextually
relevant information to your users with its
focus on simplicity, flexibility, and
user engagement, prompt flow is the first prompt engineering
tool of its kind. First end to end prompt
engineering tool of its kind. I'm excited to see
what you build with it. Back to you Scott. SCOTT GUTHRIE:
Thanks Seth. Prompt Flow is an end to
end system for building, testing and deploying modern
AI-driven applications. As you just saw, Prompt
Flow works not just with our Azure OpenAI Service, but also with thousands of open-source AI models as well, and this week at Build, we're also releasing a new
Azure AI model catalog that makes it
incredibly easy for you to also use and consume every open source model out there across your
Azure AI solutions. Now Prompt Flow and Azure AI can also help you test and evaluate your AI applications to help ensure that your prompts
are running safely, and in this new era of AI considering safety is
not an optional feature. It's really a requirement, and it can't be an afterthought
you really need to design with safety from
the very beginning, and every AI system
that we build at Microsoft is designed to uphold our AI principles,
and this week, we're announcing our
new Azure AI Content Safety service to
make it easier for you to also test and evaluate your AI deployments
for safety as well. Now the Azure AI
Content Safety service provides the same technologies
that we use internally to build our own Microsoft
Copilot experiences and so you can also benefit from all the learnings
that we've had in terms of making sure
that our products are secure and safe. To share more, let me
introduce Sarah Bird, who leads our
responsible AI effort at Microsoft to talk about how we're doing this.
Please welcome Sarah. SARAH BIRD:
The recent breakthroughs in foundation models have also transformed what's
possible in responsible AI, enabling us to develop new
state of the art tools and technologies that previously we only could have dreamed of. We've built on top
of these AI models, deep understanding of text, image and multimodal content, and have developed a
new Azure AI service to automatically detect
undesirable content, making it easier to keep
online communities, applications, and
the AI systems safe. We know from our
own experiences, building GitHub
Copilot, Bing Chat, and Microsoft 365 Copilot, that safety is a key part
of building any copilot. We use a defense in depth or
layered approach to safety. First, we've worked
with OpenAI to build safety directly
into the model so it can recognize problematic queries and
respond appropriately. Now, let's talk about
the next two layers; the layers that you control. The safety system
and the Meta-Prompt. I'll start by showing
you the safety system; Azure AI Content Safety. Let's take a look. SPEAKER 3:
Azure AI Content Safety is an API based product that can be deployed as part of
any application to monitor for harmful
content in real time. I'm in the studio so that we can explore how it works
interactively. For the Contoso bot, most people are
going to be asking appropriate queries about
products they'd like to buy. For example, here's a user query looking for an ax to cut
a path in the forest. If we ask Content
Safety to score this, we can see that it
recognizes it as safe. However, one of the challenges
with language is that small changes can completely
change the meaning. But the power of new
foundation models and Azure AI Content Safety is that it can understand
these differences. If I change even
a single word in the sentence and
rerun the query, you can see that Azure AI
Content Safety understands that this query is not safe
and it would reject it. Here, I'm using the default
settings, but if I want, I can adjust the severity
levels to meet the needs of my application and deploy this through the
API in production. Contoso uses Azure OpenAI, so Azure AI Content Safety
is already built-in. Returning to the Contoso bot, I can test these
same queries again. Here, I'm going to start with a recommendation
for cutting a path. As you can see,
the Contoso bot is prepared to recommend products
the customer can buy. However, if I
pretend to be a user with the same harmful
query as before, I can see the safety
systems in action, and my bot knows to reject it, protecting Contoso
and its users. SARAH BIRD:
Pretty cool, right? If you're using an
open source LLM, you can call Azure
AI Content Safety directly using the API. However, if you're
using Azure OpenAI, you're starting with
the first two layers of safety from the beginning. Regardless of what
model you're using, the next area that you want to focus on is the Meta-Prompt; system level prompt
that you feed into the model to
control its output. Here we have an example
of the safety portion of the Meta-Prompt for the Contoso application that Seth showed. As you can see, it's simply
instructing the model in natural language to control
for grounding the response, for controlling the
tone, for safety, and to prevent attempts to break the model's
safety systems. But what I really
want to show you is how you can engineer and experiment with these kind of Meta-Prompts using
Azure AI prompt flow. Let's take a look. SPEAKER 3:
I'm going back to the Contoso retail flow that Seth showed earlier to experiment with the Meta-Prompt and add the safety section
I just showed you. Rather than just push
it to production, I also want to test it first. Built into Azure AI is a system for testing
prompt variance. As you can see, I'm
testing two variants here. Variant 0 contains the
new safety section. Now I'm going to use Prompt Flow to see how these two compare. Prompt Flow provides
built-in metrics, which makes it easy
for me to evaluate. Let's look at the results. First, I'm going to look at
the outputs of the system to understand how these two
prompts stack up qualitatively. I can see immediately that there are differences in
how they behave. If I go to Example 3, which is the example of a user trying to
jailbreak the system, I can see that Variant
0 outperforms Variant 1 because Variant 0 refuses to give information
about its prompt, while Variant 1 is
successfully jailbroken. Now that I've looked
at the examples, I want to understand
how these two compare overall by looking at metrics. I can see that Variant 0
is outperforming Variant 1 across the board with higher
scores in groundedness, relevance, and coherence, which means it's the
clear winner for me. Now I can confidently
deploy to production and bring the power of AI to
Contoso shopping experiences. SARAH BIRD:
Here, you've seen the combined power of Azure AI Content
Safety integrated in Azure OpenAI and
Meta-Prompt evaluation in Prompt Flow to build
safety into Copilot. This is just a
glimpse of some of the new responsible AI capabilities we
have in Azure AI. We're making it easier to build real-world mission
critical, safe, and secure AI solutions that you can trust and use
in your business. Scott, back to you. SCOTT GUTHRIE:
Thanks so much Sarah. In this new era of AI, as I said earlier, safety
is not an optional feature, it is a requirement. It can't be an afterthought. You need to design
with safety in mind from the very beginning. With services like Azure
AI Content Safety service, we're providing you with
powerful capabilities that are built-in to Azure AI, automatically integrated in
Azure AI already for you, that'll help you be safe and secure as you build
your AI solutions. Now that we've
covered the Azure AI and the Copilot stack, let's now dive deep onto our
Azure AI infrastructure. Now, we've invested heavily
over many years now to make Azure the place to do
cutting edge AI innovation. Azure is very much now the world's AI supercomputer and it's the
infrastructure that's powering not just the
ChatGPT experience and all of the AI products and services coming
from Microsoft, but it's also the
AI supercomputer that was used to train the large foundation models
that OpenAI produced. To do this, we've built a purpose-built AI
infrastructure to deliver reliability and
performance at scale. We've built the largest AI model training
system in the world, one with tens of thousands of interconnected GPUs
and fast networking. The picture here is one of our new NVIDIA Hopper
based GPU systems. Azure was the first Cloud
provider in the world to deploy and offer Hopper based
GPU systems to customers. The picture here you can
see a lot of cables. These are InfiniBand cables and they're really important for our GPU clusters and it's one of the ways that we really differentiate
ourselves from others. Neither AWS nor Google
support InfiniBand today. InfiniBand gives Azure the
highest network bandwidth, and lowest network latency of any Cloud hyperscaler
in the world today. That picture shows you
what the servers are. Let's switch gears now
and take a look at what one of these data center
facilities looks like. The picture on the top right, you can see one of our new data centers that we're building to add to our Azure Cloud
region in Dublin, and those little things
are tractors and cars. It gives you a
sense of the scale. That building is
just one data center that we're commissioning and
adding into that region. To give you a sense
of the scale that Azure is now operating at, we'll now bring live more than 120 data centers
this calendar year alone. That's one new data
center every three days. Most of these data
centers, like this one, are ones that we're
designing and building ourselves and multiple data centers that make up
what we call an Azure region that you
can deploy across. Now, our Azure region in Dublin is made up of many
data centers like this one. In total size, it's
already over a mile and a half long and half a mile wide, and areas two
million square feet of data center floor space that's already live that you can deploy your
workloads in today. To put two million square
feet in perspective, that's the equivalent of 35
football stadiums in size. All just in one
region inside Dublin. Dublin is less than
three percent of our total data center footprint
around the world today. We now have more than
60 Azure regions live. Dublin is just one of those
dots there on the map. It's more regions and
in more countries than any other Cloud provider. We're bringing all of the
scale and all this power to you so that you can build
amazing solutions on top of it. But we also know that with all that power comes responsibility. We started our
sustainability journey at Microsoft more
than a decade ago and we're committed
to leveraging 100 percent renewable
energy by 2025. Our data centers,
including all of the new AI infrastructure
that we're deploying, are very much part
of that commitment. As an example, our data
centers in Sweden use free air cooling to
reduce carbon emissions. While in Arizona we're
using solar power. In Finland, where we're using heat energy produced
by the data centers to warm homes in the winter
in Helsinki to help further reduce the
carbon emissions from the local
community as well. Microsoft, including all of our Microsoft Cloud
data centers, will be carbon negative by 2030, and by 2050 will have removed all historical carbon emitted by Microsoft since
our founding in 1975. [APPLAUSE] Now the Azure AI infrastructure is being used for many different
AI models and scenarios. We're talking at
this conference and today a lot about
the great work we're doing with OpenAI
and LLMs on Azure, but we're also using this
exact same infrastructure for many other AI models and
experiences as well. Another unique AI partnership
we have is with NVIDIA. NVIDIA is bringing its
AI software to Azure. By integrating NVIDIA's AI
enterprise with Azure AI, organizations are
going to all have a secure enterprise-ready
platform to quickly build and deploy custom
machine learning models. NVIDIA's Omniverse
Cloud provides a full stack
environment to design, develop, deploy, and manage industrial
metaverse applications. This gives enterprises access to the full stack suite of Omniverse software
applications and NVIDIA OVX infrastructure and combine it with the scale
and security of Azure. Azure is the only Cloud provider offering Omniverse Cloud today. With Omniverse Cloud on Azure, organizations like BMW are now able to build
truly virtual factors. The BMW team can aggregate data into massive
high-performance models, connect their domain-specific
software tools and enable multi-user live collaboration
across factory locations. They can run real-time
digital twin simulations, virtually optimizing
layouts, robotics, and logistic systems even
years before a factory opens, and it's all possible from any location and on any device. All the AI innovation
that we've seen today is built on top of a
foundation of data. Data is really the
fuel that powers AI. AI is only as good as your data, and so it's never been
more important to have a great data analytics and data management
foundation in place. This week at Build, we're excited to announce
Microsoft Fabric, a unified platform for data analytics really
designed for the era of AI. With Microsoft Fabric, we're
bringing together Power BI, Data Factory, and the next
generation of Synapse. Fabric unifies all these
analytics tools at every layer into one seamless product with one experience and one
common architecture. You literally have
unification at every layer, unified data, a unified product experience for all your data professionals to collaborate in, a unified governance
and security, providing a single source
of truth for everyone in the organization and a
single way to secure it, and a unified business
model to help ensure that all your resources are used in the most cost-effective way. This complete platform really delivers data analytics as a
software as a service model, just like Microsoft 365, making it much easier
to stand up and manage, with
frictionless onboarding, automatic optimization
and integration, built-in security
governance and compliance. It's lake-centric, and has
an open data architecture, and it has deep integration
with Microsoft 365. What's great is it's designed
in particular for the era of AI with built-in
Copilot support. This Copilot support
enables you to use AI to understand and reason over data in extraordinary new ways. I will show a video of some of these new
capabilities in action. SPEAKER 4:
Let's take a look at how Copilot in Power BI is empowering everyone to quickly
find and share insights. For example, say
I'm an HR analyst, who is constantly
handling requests for my management to do analysis
and provide answers. It can take hours and even days to respond with new reports. Now with Copilot and Power BI, I can simply describe
the report I want to build and get
insights in seconds. Maybe we need to break
down somewhere employ data to understand demographics
and hiring trends. I just describe what I need, and Copilot will
automatically analyze my data and create a new
report based on my needs. Immediately, I have a
beautiful Power BI report that has been
automatically created and is fully interactive. I can start slicing and dicing
my data to explore deeper. Copilot added the charts
and slicer as I asked for, but also conveniently
gives me options to adjust the output to
get just what I need. For example, let's
switch to ask for metrics and trends in a
slightly different layout. Automatically, Copilot
updates the report. Let's go ahead and
keep this report page. It goes much further. I can also ask questions such as why is our
attrition rate going up? Copilot responds by adding a new page to my report
and tapping into Power BI's built-in advanced
analysis capabilities for finding key influencers
for variables in my data. In addition to giving me
a summary of an insight about different employee types having different
attrition rates, I can easily explore Power
BI's key influencer visual to see what's driving the biggest impact in our attrition rate. I can even automatically see the most significant segments in my data just by clicking. Now let's go back
to the first page and finish up our new report. I'm going to ask
Copilot to make it look like our existing
exact dashboard, and instantly it's applied
the same formatting and style and adjusted
the layout to match. Best of all, I can still
interact directly with the Power BI report that
Copilot has created. For example, if I
wanted to manually change this bar chart
to a TreeMap on my own, it's easy to do with
just a few clicks. Let's add employee
functional area, and now I can filter
by this new field. Finally, to make the report even easier for my
team to understand, I'm going to ask Copilot to add a rich text description of my data right inside the report. This narrative summary
is fully dynamic, and not only does Copilot highlight interesting
insights for my data, it will update the summary
every time the data is refreshed or people
filter the report. Just like that, in seconds, I've created a report
that would have taken hours or days to do manually. Copilot built on top of
Azure OpenAI is truly revolutionary for how we empower everyone to find
and share insights. [APPLAUSE] SCOTT GUTHRIE:
What you saw there is how we're able to organize data in
Microsoft Fabric, and in turn enable this amazing natural
language experience in Power BI on top of it. Microsoft Fabric is
designed for the era of AI, and its serverless
data management engine is tuned for advanced
AI scenarios like that. With Microsoft Fabric, we have a deep
commitment as well to open formats and APIs. Fabric comes with a SaaS
multi-Cloud data lake, what we call OneLake
that's built-in and automatically available
to every Microsoft tenant. OneLake is built on open formats including
Delta and Parquet, meaning it's compatible
with solutions like Databricks and other
open-source tools. Since we know that many
customers want to build AI solutions using data that might already have been
stored in other Clouds, we've also built Microsoft
Fabric to work across Clouds, including AWS and in
the future Google. I'm incredibly pleased to
announce that Microsoft Fabric is now available in public
preview for everyone to use. There's some great sessions
on Microsoft Fabric, and you're going to be
able to learn a lot more about it later this
week at Build. We have an exciting
future ahead of us. Azure in the Microsoft
Cloud is the place for every developer and
every organization on the planet to be able
to innovate using AI. We're really looking
forward to building some great AI solutions
with you together, and I hope you have
a great rest of the Build and a great rest of the day. Thank you very much. [APPLAUSE, MUSIC]
