(gentle upbeat music) - Up next, we take an inside
look at Azure's OpenAI Service that lets you leverage large
scale generative AI models based in Azure that have
a deep understanding of language and code as
you build new applications. Now we'll unpack the core
concepts for interacting with models, using prompts, and demonstrate how you can
use Azure's OpenAI Studio to experiment with and test your models before bringing them into your code to deliver differentiated app experiences, all with Azure's enterprise-grade security for your app's foundation. And joining me today for an in-depth look at
Azure's OpenAI Service is Pablo Castro, distinguished
engineer on Azure's AI team. Welcome back to Mechanics. - It's great to be back,
and in the studio this time. - Thanks so much for joining
us back on Mechanics. So OpenAI, it's really
started to gain momentum over the past couple of years, you know, provides this really
large, extremely rich set of base AI models that
can accelerate the process of constructing natural
language powered experiences, that can, for example, greatly
improve how you interact with your apps and extract
knowledge from data. Now, this is an exciting
area that Microsoft has invested a lot in. So what are we solving for then with Azure's OpenAI service? - There's definitely a ton
of potential and we see lots of applications powered
by Azure OpenAI today. From a developer's perspective the thing that excites me the most is how it helps build new experiences with several cutting
edge models like GPT-3, a large language model that
generates content based on natural language input, Codex, which can translate natural
language instructions directly into code, and
DALL-E 2, a new model that generates realistic images and art from natural
language descriptions. And when we combine OpenAI with Azure into the Azure OpenAI Service, you get the core building blocks you need for production-grade applications, from how we host OpenAI
at scale for you in Azure to enterprise-level security. We're also giving you a great environment to try things out and iterate. The Azure OpenAI Studio
lets you experiment and test your ideas with OpenAI before bringing them into your code. And when you're done experimenting and you know what you want,
you can call the service from your code just
like any other REST API. And security is built in with capabilities spanning
strong authentication, role-based access control and the ability to configure virtual
networks, private endpoints as you would for any Azure service. We've also incorporated tools
for supporting responsible AI with content filtering in the service. For example, if your input
includes inappropriate content, the Azure OpenAI Service would catch it. And this works for any
generated output too. You can use this to detect and mitigate harmful use following
responsible AI principles which you can find more
about at aka.ms/AIprinciples. - So can you give a few examples of how we're using OpenAI
in apps and services today? - Sure, OpenAI actually
powers a number of experiences at Microsoft today. For example, co-pilot in the
Power Platform enables users to author Power Fx commands or
even reach automation flows. Another great example is the Designer app we recently announced, which
uses the DALL-E 2 model to interpret your text descriptions to generate images and even artwork. Let me show you. I'm here
in the Designer preview. You can find this at
designer.microsoft.com. I'm going to add a design
headline, say DALL-E 2 and here I can generate an
image from natural language. I'll type grass field with trees and you'll see Designer gives
me some nice recommendations to augment it. But I'll keep things
simple and just add cows. Now if I want to change the look, I can add as a cartoon and that's starting to
look closer to what I want but I'll change the cartoon
to a blocky video game and that's exactly what I wanted. Now I just need to choose a design. I'll pick this one and you'll
see our progression of images. And just to see a little bit more detail, I'll select this second one, so I can zoom into the
middle of the screen and now you can see
more of a closeup view. These were all generated on demand. Designer interpreted what I typed and it created new images
each time I iterated. So these apps aren't
just putting an app frame around the API. OpenAI models like this can help you build more dynamic, interactive and
differentiated experiences. - And that's really an important point because we're not training models in the traditional AI
machine learning sense, we're just interacting
with them using text. So can you explain a bit
more behind that concept? - Yes, these types of models
use text for interaction. They have been trained on
huge amounts of written text and can use their accumulated knowledge to perform a number of tasks directly. We call the text input
to a model a prompt. The simplest form of this
interaction pattern is to give a model a string
of text or a prompt and ask it to complete it. It could be a partial statement,
a question or anything you want the model to add to. In cases where the
model has enough context to generate good outputs,
that might be all you need. This is sometimes called zero shot because the model responds with no additional input or training. That said, sometimes you do want to guide the model a bit more. You can do this by giving it examples of how you want it to behave, such as a question-answers
pair, and finishing the prompt with the one you wanted to complete. This still requires no actual updates of the weight in the model. The examples are learned on the spot. So this is often referred
to as few-shot learning. - So how easy is it
then to integrate OpenAI as part of everyday apps? - Oh, it's super straightforward. OpenAI can be used to
create many new experiences that were impossible before. My favorite example is
how you can transform the entire surface area of an app into something you can converse
with using natural language. Let's use Minecraft,
the game as an example. And if you've played Minecraft,
you might be familiar with the text commands you
can make during gameplay to manipulate the world around you. The challenge is that
there is a lot of commands. I'll run /help to see all
the possible commands. Here you can see there are
dozens of different commands that you would need to memorize and this is in addition to the more than a thousand
different materials and lots of different entities. One command is slash
/summon, which is used to spawn a new entity into
the world from cows to ship and then /field to build
structures using blocks and so on. The nice thing about Minecraft
is that it's extensible, so you can write custom mods. What I'm going to show you is how we created a mod called
conjure, which acts as a plugin so that when you write a
natural language message, it'll pass that command to
the OpenAI Service in Azure using the Codex model.
This interprets the intent and finally passes back
the generated command that Minecraft can natively understand. So let's try this out
for real in Minecraft. I'll ask it to make a
cow and there's our cow. It converted my request to create one. That was pretty easy though because it just converted
my command to summon cow. Now let's be less precise and type, "Make an animal that goes moo." And OpenAI knows about the world and it was able to translate my intent using the sound a cow makes to tell Minecraft to summon a cow. And beyond the summon command, it can also interpret other requests. I'll ask it to make a
transparent wall and there it is. Even though transparent
isn't part of a block name, Codex knows that glass is transparent and it connected the dots for us. It executed the fill
command and as you can see, it even came up with relative coordinates. So what we just saw is how
we can use OpenAI not only to let you interact using natural language with an application, but
also how the models knowledge of the world in general can
really enrich the experience. - This is really a great
example of the possibilities for interacting with apps
using natural language. Now aside from building the mod itself, was there anything else you had to do? - Remember that we discussed before the importance of prompts? So for this I needed a very simple prompt. This asks Codex to
translate natural language into Minecraft commands. At the top, I made a statement
in English about my intent and then I gave it a few examples such as building walls
using relative coordinates and summoning animals. This is like a few-shot scenario. You'll see that even though
I defined that "Make a fish," in my case translates to
/summon tropical fish. It could take that tiny piece
of information and generalize to all similar cases, even
if the details vary a lot. So these prompts apply to
all the animals in Minecraft, even the sounds they make or other indirect ways you
might use to refer to them. This really shows the
power of few-shot learning with the OpenAI Service. This whole thing was just a
few hours of work with my kids. - And it really shows
how just a little bit of guidance actually gets you the outcomes that you want using your prompt. - Exactly and these models
are already in use out there. For example, Codex powers the
GitHub co-pilot capability that helps developers write code faster. Let me show you. Here's a portion of the
code behind my Minecraft mod in Visual Studio Code and I have the GitHub
co-pilot extension enabled. The mod is written in
Java and this is the part that takes the text that the player wrote and sends it to OpenAI. So here I have the HTTP request that calls the OpenAI Service in Azure. What's missing is actually
sending the request and then processing the response. But I can use co-pilot to write that code. Here Co-pilot is predicting
this entire line of code. I just need to hit tab
and I can keep going and just by hitting the
tab key a few times, co-pilot wrote all the code we needed, from login to parsing the JSON response and producing a result. - And I got to say that's super impressive just using the tab key in that case. So can you show us how you
might put all this to work with an app that you
might build from scratch? - If you're building a custom app, you just need a few lines of code to call the OpenAI Service. Let's take the example of
a green energy company. We need to keep up to date on the latest trends and insights, which requires us to
build an understanding across lots of unstructured
data sitting in documents from various sources. I'll explain how we've
architected our app first and then I'll show you how
it works and what we did. In this example, the user
can ask the app a question using natural language. The app then calls Azure Cognitive Search to discover text-based documents
related to that question. This ranks the documents and
returns the top candidates but to answer some types of questions, we'll need data across multiple
of these top documents. So it passes the top candidates
to the Azure OpenAI Service, which reads and understands
these top documents, leveraging the GPT-3 model
to generate an answer, which is then passed
back to our application using information spread
across this relevant documents. So let me show you this in action. I'll start in the browser and you're looking at the simple app, but the real beauty is in
the power of information it can bring to you. I'll do a quick search for renewable sources
expected and actual for 2022. And it finds a few relevant documents each containing parts of the answer. And at the top you'll
see the generated answer that grabs elements
from multiple documents, like 22% from the first doc and first half of 2022
from the second doc. Notice there is a third unrelated doc that was not included in the summary because GPT-3 realized it was not related to the question. Now I'll show you how
easy it is to call OpenAI and generate the cross-document
answer which you saw. I'm in Visual Studio and
here are a few lines of code that do all of the work. First, I take the user query and send it to Azure Cognitive Search. Then I take the results from search, which are multiple documents
and I construct a prompt from them using a template. Finally, I send this prompt to OpenAI to generate the summary answer. So with very little code
we could leverage OpenAI. And this same process could be used to very quickly solve other problems like, entity extraction or summarization that otherwise would've
required you to pick, train and deploy custom machine learning models. Here you more or less just
build a prompt and you're done. - Right, I can really see
how useful this would be across any number of
research-based activities. That said, though, in this case, you actually knew what to do. So if someone's watching and
needed to start from scratch, where would you even begin? - The Azure OpenAI Studio
is a great place to start. It lets you experiment with prompts before you build them into your apps. To use the OpenAI Studio, first you'll need an Azure subscription. You'll need to apply for access to Azure OpenAI using this form at aka.ms/oai/access. After you create an OpenAI resource, it'll link you directly
to the OpenAI Studio. In the studio, you'll start
by creating a deployment. You can think of a deployment
as your own service endpoint for OpenAI that your application calls. In my case I'll use an
existing deployment, so we'll skip that. You can see I already
have two deployments, one using davinci2, for
natural language generation and another one using
Codex for code generation. Going back to the home screen, you'll see the studio offers
a few examples to get started. Summarize Text, Classify
Text, Natural Language to SQL and generating new product names. In the studio, we also
have the playground. So let me head over there. First I'll choose one of
the deployments I created. I'll choose davinci2. Then I'll select an example
from the dropdown menu and I'll start with this example for generating a new product name. You can see the example
uses a few-shot prompt, with a product description and seed words to help it understand
and generate results. In this example, we're
looking to name shoes that can fit any foot size, with seed words like
adaptable, fit and omni-fit. When I generate the results, it comes up with omni-fit shoes, perfect-fit
shoes and all-fit shoes. And so you can see how this can be used to help people with
their creative process. - How would you then
go from these examples that you just showed to
maybe the OpenAI search app that you showed before? - Well, the playground helps you test and iterate on your prompts before you incorporate them
into your custom application. Let's use the summarized text
example as a starting point. This example prompt is
about neutron stars. By looking at the structure of the prompt, I can see the pattern they
follow and learn from it. We're using the Davinci model, which is the largest of the GPT-3 models and optimized for text understanding. Now you can see this long
paragraph of text as input. The example adds Tl;dr at the
end to suggest to the model that this should be completed
with a short summary. Let's run it. And here, you see the generated
output about a neutron star. So I'm giving OpenAI instructions for what to do in plain English. Coming back to the cross-document
summarization example, let's guide the model to
read a set of statements so it can answer a question based on those statements,
just like a human would. If I go to Visual Studio and grab my prompt and paste it here, you'll see our prompt,
including the placeholders, our code replaced that runtime in the app. This is real content, so it's pretty long and hard to follow. So instead of running that one, let me just paste a shorter version that has the same structure, so we can more easily see what's going on. I'll ask it to complete the task, which is to answer the question about the percentage of homes with solar. And you'll see a detailed answer with a few regions called out. The answer was accurate but I'd like to see a shorter
version of the answer. And this is where I can
provide an example answer to further refine the results for the style that I'm looking for. So first, I'll undo the last
run to clear the answer. Now I'll paste in the
same statement from before but you'll see that this
is a different question with a very concise answer. From there, I can generate a new result for the first question and
you'll see that it learned from our example and returned
a very concise answer. So the playground in
the Azure OpenAI Studio helps you experiment and iterate. You can test your prompts
for accuracy, refine them, and then bring them into
your custom applications. And if that isn't precise enough, one thing that I didn't have
time to demonstrate today is Azure OpenAI's support for fine-tuning where you can specialize
one of the base models with additional data that you provide. So you have the full spectrum, from zero-shot to creating
your own fine-tuned models. - And all of what you've
shown today to be clear, is completely grounded in
security and responsible AI. So now that you've got hands
on with Azure's OpenAI Service, can you explain for everyone watching how they might get started and learn more? - The best way to get
started is by trying it out. We've made it easy to
sign up for the service at at aka.ms/oai/access. And from there you can start
using Azure's OpenAI Studio. - Thanks so much, Pablo,
for joining us today and showing us some highlights
of what Azure OpenAI can do. Of course, keep checking
back to "Microsoft Mechanics" for all the latest updates. Subscribe to our channel
if you haven't already. And as always, thank you for watching. (gentle music)
