Introducing your copilot for the web: AI-powered Bing and Microsoft Edge

[Music] hi everyone welcome uh it's great to see all of you in Seattle in person uh we have an unbelievable show I see Scott Guthrie's even wore his red shirt so we welcome you to the Azure kubernetes container service 2023 Lodge no don't worry we'll have some fun we will have some fun you know Scott's not coming up to show you code on screen anytime soon but look it's an exciting time uh in Tech uh you know the broad Contours of this next platform uh are just getting clearer and clearer each day uh the advances what's possible uh that's what obviously excites us uh in our industry but we're also grounded in what's happening in the broader world there's no question uh there are enormous challenges out there in fact it's reminds me and I've spoken about this before of the very founding of Microsoft in 1975. in fact the when the popular Electronics cover came out with the Altair which of course our Founders picked up and ran with it and created essentially what's the software industry as we know of it today that same week new suite had a cover with President Carter trying to fight off uh the three-headed monster of uh inflation recession and an energy crisis um and today we'd have something similar you would have ai on one cover and then we'll have those three challenges Plus for good measure we can add a few more and so we as Microsoft we as the tech industry have to sort of really ground ourselves in how do we relate one to the other in other words can we use technology to overcome the challenges that people and organizations and countries face that's really the pursuit here and in that context I would say uh I just want to share a couple of anecdotes which give me great hope quite honestly it gives me personally a lot of satisfaction around working at Microsoft working in this industry to push the state of the art of Technology the first one you know obviously when Sam and his team late last year launched chat GPT that's the only thing anybody your friends and family wanted to talk about throughout the holidays it was just crazy uh house you know it was like the Mosaic moment the closest we've come it's what been 30 years now uh since when Mosaic launched which I distinctly remember um and it was very exciting time so I I was on all I went on a holiday first and then I was in the first week in India of January I was in India uh and on my on Jan 1st I look at my news feed and I see this tweet uh that Andre H karpathy put out uh who is you know ex-open Ai and Tesla and uh and it's now an independent AI developer and uh he had this thing where the product that really he was most excited about the previous year was GitHub co-pilot and he was saying how 80 percent of his code was being generated by this I would say first at scale product built on nlm technology and you know this doesn't mean he's 80 somehow not doing his work in fact it's he's getting so much more leverage uh and in fact recently we crossed a hundred million developers on GitHub and think about this right there are 100 million developers on GitHub if we can improve their productivity just like how andrage was able to sort of you know observe and then let's say in the next decade we double that number maybe double it again right that we get close to a half a billion developers what Economic Opportunity it would create because there is not a meeting that I go to today with any CEO CX or any organization who's not looking for more software developers more digital skills that's the currency in every part of every every sector of the economy in every country of the world that's the opportunity we have to be able to take this technology and make a difference but then the next day I went to Mumbai and then I saw this demo this was just for me the most profound thing that I've seen in a long long time the demo was actually built by the ministry of electronics in India because they're building out a digital public good their idea is look India has got you know multiple official languages and they wanted to democratize essentially access to language translation so they're building it out as a digital public good in fact Microsoft and Azure and Microsoft research are all involved in that project and so this is basically speech to text text to speech across all of the languages in India and um so they showed me this demo where a farmer uh speaking in Hindi Express is a pretty complex thought about how he had heard about some some government program and wants to apply for a subsidy uh that he thinks he's eligible for and so it's a pretty complex prompt query the technology does a good job it goes to the bot recognizes the speech comes back and says you know what you should go to this portal fill out these forms and you'll get your subsidy so he says look I'm not going to go into any portal uh I'm not going to fill out any forms can you help me and he does it and then I was told that a developer said okay you know what let us daisy chain a model that was trained on all of the documents of the government of India using GPT and with this speech recognition software so basically two models coming together to really help a rural farmer in India trying to get access to a government program look I grew up in India I dreamed every day that someday the Industrial Revolution will get evenly distributed across the world and here I was seeing something so profound something that was developed by the folks at openai and the west coast of the United States a few months earlier used by a developer locally to have an impact on a rural farmer that to me is what gives me meaning and I think gives us all meaning in our industry so it's just fantastic to see that and now of course we've got to scale it scale it and scale it with a real understanding that we can't break things right so it's about being also clear-eyed about the unintended consequences of any new technology in fact that's why way back in 2016 is when we came out with the AI principles yeah both we as Microsoft and our sort of Partners at open AI deeply care about this in fact the entire Genesis of openai is from that Foundation and so we built these principles but we've just not just put those principles as a document but we've been practicing it because that's the only way technology gets better in this particular case when you talk about AI it's about alignment with human preferences and societal norms and you're not going to do that in a lab you have to do that out there in the world and so it starts by the way with design decision once makes when you think about AI you can have the human in the loop you can have the human on the loop or you can have the human out of the loop those are decisions we as product makers build into the products and so we whenever we've come up with some new things in new models we in fact put a premium on human agency when you think about these generative AI models remember one thing they are prompted to do things the prompting comes from a human being we get to in fact help them prompt we get them to take the draft that gets generated they should review the draft they get to review the draft they get to approve the draft using judgment we want to give them as many perspectives as many ways to regenerate so we want to take even just the design of AI products as a first-class construct and build that into our products but that's not sufficient we realize that you have to right at the pre-training stage when it comes to the data the pre-training itself the model itself has to be safe the safety system around the model uh the application context all will matter and so we absolutely and you'll see that today we take all of that as first class things which we want to reduce not just to principles but to engineering practice such that we can build AI That's more aligned with human values more aligned with what our preferences are both individually and as a society and so what is it that we should do and what should we build I think that this technology is going to reshape pretty much every software category we know I mean we've seen that right when we've saw if you think about the web we've had what three at least very distinct platform shifts that have shaped the web the web was born on the PC and the server and then it evolved with mobile and cloud and now the question is how is AI going to reshape the web um and each time in each one of those faces some real foundational technology layers sometimes I describe them as these organizational layers of Technology were born right the browser was the first one without the browser the web would have not been as popular as it is today same thing with search organize the web and then when it came to The Mobile Generation and mobile and cloud in fact these super apps especially outside of the United States uh around messaging became the way people consume the web uh we also had uh app stores so the question is what are the constructs what are the organizing layers going forward you'll see some of that today we think there are two things that are emerging one is this conversational intelligent agents I think they're going to be things that we are going to have everywhere we go all computer interaction is going to be mediated with an agent helping you in fact we're going to have this notion of a co-pilot that's going to be there across every application canvas inside of an operating system shell in a browser and so we want to show you some of this innovation starting with how it's going to reshape the largest software category on planet Earth which I've been working on for a long time which we're very very excited about search and it's a new day uh in search it's a new paradigm for search rapid Innovation is going to come in fact a race starts today in terms of what you can expect and we're going to move we're going to move fast and for us every day we want to bring out new things and most importantly we want to have a lot of fun innovating again in search because it's high time with that let me turn it over to Yusuf thank you thanks Satya uh it's great to be here with all of you today we've been working on something we think is pretty special and we're just eager to share it with you in my time at the company I've been lucky to witness a few important moments and I think this might be another one of those We Believe asakya said that we can improve the way billions of people can benefit from the internet created by really an amazing team across Microsoft from the core web engineering folks in mikhail's group to the brilliant Folks at Microsoft research to the tireless people that work on the Azure AI supercomputer we think of it humbly as the next generation of search and browsing infused with AI and assembled as an integrated experience we are going to reimagine the search engine the web browser and new chat experiences into something we think of as your co-pilot for the web now co-pilot is a critical word because we believe in the empowering nature of AI in which you the individual are in charge now what it is is not even as important as what it represents and that is for us an aspiration to unlock the joy of discovery The Wonder of creation and that feeling of empowerment from being able to harness the world's knowledge at the center of this new co-pilot experience is an all-new Bing search engine and Edge web browser and it's going to do four things for you first it's a better search it's to search you know and love but it's better because it's AI powered second not only does it give you the search results but it will actually answer your questions third we're going to make it incredibly easy to use we're going to let you chat we're going to let you just talk to it naturally and last when you need that spark of creativity Bing can generate content for you automatically to help you get started all right so let's talk a little bit about the opportunity in Search and why we believe we're at the start of the Next Generation oh no no sorry about that we're on to the next generation in all seriousness with over 10 billion queries a day we all know the search engine is an incredible tool we do and yet as the web is grown we've run into challenges people are overwhelmed increasingly with too many links when they're trying to find simple answers in fact 40 percent of the time people click on search links they click back immediately that's a sign they're not finding what they want and most notably we have to adapt to search versus the other way around turns out searchworks better if you give it fewer keywords you might be surprised to know that 75 of all searches are three keywords or less but come on we shouldn't be surprised search has remained fundamentally the same since the last major inflection when we moved from a directory approach to search to an algorithmic approach to search the user experience and the underlying approach are essentially the same as 20 years ago few keywords and you get millions of links but as the world around us has changed the way people use search has changed we all know this people are trying to use search to do more than it was designed to do to illustrate that let's just look at a breakdown of search queries by type to understand this point search queries today you can bucket them into serve three types of categories the first are what we call navigational this is people searching for a website so when you need to renew your driver's license you need to go to the DMV when you want to tell your friend about that great pizza place you send in the web link that's what search was designed to do and it does it super well the second third of categories are we what we call informational these are things like what's the weather forecast what was last night's sports scores what's today's stock price search works great for that but for anything more complex like hey I've got to pick up this love seat at the store is it going to fit in the back of my car search is going to fall short on that and then the final third we'll Buck it into what we call everything else these are typically more deep research in nature this is things like trip planning or shopping complex shopping an example query here that people are putting in is hey can you recommend a five-day itinerary to Mexico City and for these search Falls far short of the desired goal what this means if you just kind of do the simple math it means that roughly half of all searches aren't delivering the job that people want if you if you go on that 10 million queries it means that every second 50 000 people searches potentially go unanswered this is why we this is why we believe it's time for a new approach in search and for the billions of queries that are going unanswered we've seen new attempts to try and address the problem as you all know there are vertical search attempts Amazon has done a better job for shopping YouTube is great for video Reddit is a great place to come get advice and the benefits of search are well known it's fast it's timely and there's a great business model and then more recently they've been another Vector more disruptive ideas like leveraging AI to answer questions directly and to generate content these are amazing as well they show what's possible but what if you could get the two to come together Not only would you get two things in one but we think you could actually solve problems of age and we think you could get to something that is really one plus one equals three and we have done that with the new Bing I want to share with you four technical breakthroughs the team has achieved to make this come to life first through our fantastic partnership with Sam and the brilliant team at open AI I'm excited to announce that Bing is running on a new Next Generation large language model one that is much more powerful than chat GPT and one that is customized specifically for search it's unlikely that you've had a chance to play with and we can't wait for you to try it second we've developed a proprietary way of working with openai that allows us to best leverage the power we call this collection of capabilities and techniques the Prometheus model the core idea here is that both that training and at run time we engage with the openai model more intelligently through our knowledge of the web via the Bing index and some special query techniques we're going to dig into this a little bit more later but the benefits are the following first we can improve the relevancy of Answers by feeding in and better tuning queries given our understanding of the web search index next we can annotate the answers with specific web links and citations we can get you more up-to-date information because search crawls the web every day and we can improve understanding of geolocation finally we can increase the safety of the answers as well by catching queries at initiation and then checking that again a delivery of an answer next we've been making steady improvements on the Bing algorithms for years and we test these with independent judges and it shows that RX search experience is on par or better than any search experience when you take away the brands but a few weeks ago something special happened we applied the AI model to our course search ranking engine and we saw the largest jump in relevance in two decades and we believe we can continue to drive breakthroughs as we improve the models and then finally we are reimagining how you interact with all of these capabilities across search browser and chat by pulling them into a unified experience I want you to think about search coming together with answers search coming together with chat and search coming together with the browser as we all know and as the folks at openai taught us the user experience is as important as the underlying technical platform all right enough talk you guys ready to see it in action let's show you how we're going to enable the copilot for the web now before I do I want to call out two things for clarity first because there's so much I'm going to show you and so I you don't have to watch me type every search I recorded these searches live just yesterday second in case there are Skeptics in the room you're going to get a chance to play with it directly put your hands on it and type the same queries as well as your own favorites right after this presentation all right you're all familiar with search so I'm not going to show you a search I want to show you what you can't do today with today's search I'm going to focus on answers chat and the ability to create but first let me introduce you to the new Bing home page you're going to notice some subtle but important changes first we have an expanded search box capable of accepting up to a thousand characters because now Bing works with natural language and you saw a little hint to chat which I'm going to get back to in a second now I want to set up the first search scenario my daughter and I we both love art she's studying art at school and I'd like to stay connected with her on our mutual passion last semester she was learning about Mexican painters I'd like to get a quick summary of the most influential Mexican painters and their Works to learn a bit more about the topic if I type the full query of what I'd like to know in today's search here's what I'd get and so I'll just type in compare the most influential Mexican artists and their top paintings and you'll get what you expect right some links it's fine but we can do better let's try this now in the new Bing what you'll see as we pull up is first you see the web results here on the left but then on the right you start to see how we start to compile the answer and what you get here now is we have the ability to highlight these web links we can annotate the results and that's because we're able to go in and apply our index onto the answers there in other words the answers and the search on one page has saved me a huge amount of time this gives you a little bit of a sense of what you can do now you've seen some of this before you might say hey I've seen some of this let's show you how we can do some additional things I'm going to show you another query so where we use the timeliness of search so let's go ahead and ask about events in Scottsdale during the Super Bowl and what you'll see is we get back an answer here where we have events and we're able to do that because Bing crosshweb notice how we can find not only that the Super Bowl is played in Glendale on the 12th but then the events like cardi B Super Bowl party that's on the 10th also shows up so we were able to pull these things together so you get start to get a sense of how we can build on what's today with the Bing index now I'm going to show you a few more of these types of answers quickly so you can get a sense of the power and The Time Savings from Bing when I'm running an errand like the example I gave to you earlier I can ask me to determine if that new love seat from Ikea is going to fit in the back of my Honda Odyssey and what you'll see is being can actually find the dimensions of the love seat the Interior Space of the car and then make an estimation as to whether it will fit in this case I'm choosing an example of where Bing does not know the answer and we know and we know that we can't be definitive about it and the reason I'm doing that is because we know we won't be able to answer every question every time but Bing can still provide some helpful information as you can see on this answer we also know we'll make our share of mistakes and as we begin to roll this out so we've added a quick feedback button at the top of every search so that you can give us feedback and we can learn another example when I'm shopping I can ask Bing to search find and compare the top three selling pet vacuums listing the pros and cons and just take a look look how great this answer is it has all three of the products I'm looking for super helpful pros and cons and think if you had to compile that how much time that would take you to do and as you can see at the top of the page we still have the advertising in this example because we know when people are shopping Those ads are helpful and finally on this one if I'm cooking and I realize I've forgotten a key ingredient for in this case for example eggs for my cake recipe Bing can not only find the exact the egg substitutes it can get me the exact amount for each ingredient and and take a look at this I love this you can actually see for example if you go with vinegar and baking soda the cake is lighter and more fluffy these are just little helpful tips that every day help make your life a little better these are just some examples and you can start to get a sense of how with answers we go far beyond what you can do with search day we can actually help you get what you want to get done now let me tell you about how Bing goes further to help you with particularly complex questions for which there's not a precise answer I wanted to introduce to you the new chat experience in bing I think of this as search with your own personal helper to help you refine your query until you get exactly what you're looking for this comes in handy for activities like trip planning and shopping research let's start with shopping so I'm going to look for a 65 inch TV again you see our ads at the top the result the links on the left and the answers here on the right and you can pick whichever you'd like we give you a good set of answers but now I want to refine this query so I can do that by going to chat now I can either swipe up with my fingers or look up here at the top of the screen we have now a new chat scope and with that with one click you are now into chat look how beautiful that is search to chat just so seamless and now we take away all of the content that was in your place and you focus you on your query the search box you can't see now that can take up to 2 000 characters so you can really you can just talk to it you can just ask for it so in this case let's say I'm going to ask for a gaming optimized TV all I have to say is which of these are best for gaming and we remember all of the context we know that we're talking about flat screens we know we're talking about 65-inch TVs and look how Bing starts to come back it does all the queries on my behalf and comes back with a great answer and I just want to highlight a few things for you since we know you're asking about gaming TVs we pull out oh this one has a game Optimizer this has game mode and so we make that really helpful I'm on a budget I'll ask it to adjust it for which one of these is the cheapest again thing knows the context and it just goes in and re-fires the queries so easy you just talk to it and you can refine your shopping experience and again we find the prices here I didn't know you could get a flat screen for under 500 but that's a good deal there on Bing if you're looking for a TV all right so we think that's going to make shopping and shopping easier let's talk about travel and before I jump in I want to kind of just remind you have you remind yourself when you're going to plan a trip to a foreign country think about all the things you go through travel times what sites do I want to see regulations to observe budget our research shows were on Bing people spend on average weeks to even months to plan a trip and to use our organizational tools I'm going to show you how we make that so much easier with the new being I'm traveling to Mexico for my cousin's wedding and with the new Bing I now don't have to start with something that's dumbed down like Mexico City travel tips I can ask for what I want first let me just compare that against what you get in today's search engines so I'll type in this long query of what I really want and you know you get what you expect links to go try and find find the answer for yourself right but we can do much better let's try it in the new Bing so I'll type the I put in the long query which is essentially create an itinerary for a five-day trip to Mexico City for me and my family and just like that Bing goes to work and just take a look at how it starts to compile it starts with day one and we we put in there look arrive in Mexico City check into your hotel go check out maybe the Playa de artes you know have some lunch then there's day two and you see isn't this just so much better as a starting point and look if you want to learn more if you're like hey I don't love these five days no problem down there we have links where you can go and learn more and uh and we put in some nice touches in there as well and again now let's say business travel changes oh I only have a three day I don't have to go back out there and figure I just say hey make this a three-day trip and Bing reflows that recommendation into a three-day trip and now let's now let's just have some fun let's say okay yeah I'm still trip planning um we like to shop where can I shop and you get some shopping recommendations like to go out at night all right make the most of the trip where's the nightlife and you get a list of nightlife you see this is just so much better than today's shirt search to start uh for your travel planning let me show you a final example of uh chat and how I think the new language models are going to maybe help bring the world a little bit closer together understanding different cultures is often done through the Arts like music and literature and I've been fascinated by Japanese Traditions satis shared with me one of his favorite searches from a while back on poetry so with that I'm going to use that as inspiration so without a clear idea of what I want necessarily I'm just going to type in a simple prompt top Japanese poets and Bing starts to respond with a nice list and you see this it does a great job of mixing the Japanese language and the English language and it knows since I'm currently in English to do that right away I learned about this poet matsuo bashu turns out he is uh one of the greatest Haiku masters of the world I love how it not only lists his name but we go ahead and we give you one of his famous haikus in Japanese and we Auto translate it in English great and I could say I'll learn a little bit more about matsuo so I can tell me a little bit more we give you another jumping off point that's great and then I can say hey tell me about another haiku now what I want you to reflect on is look how easy this is to discover something new normally I might not have done this I might not have gone to learn about something new in the world through search because it's just you know it's cumbersome to click on links and have to deal with foreign languages but this is what we mean by unlocking the joy of discovery all right finally when what you are searching for doesn't exist and you need that spark of creativity Bing can generate the content to help you get started we just finished planning that trip to Mexico and now what I would like to do is I'd like to share that information with my family they're all over the world but I can simply asking hey write an email sharing this itinerary with that I've researched and and put it in the thing for me to be able to send to my family so notice here how the email starts it's a great email personalized touches has the trip highlights it'll close you with a nice heartfelt message it's just a great way just saves a bunch of time on your everyday work now for my family uh English isn't necessarily always the first language so I can ask Bing to just translate that in Spanish and with a simple request just basically say hey translate it to Spanish big knows to take that entire email and itinerary and convert that into Spanish in fact Bing can translate automatically in over a hundred languages my sister I'm going to admit to you she's a better Spanish writer than I am but I might just impress her with this one here and even if I don't at least I save myself a lot of time from having to type the mail our focus with Bing is to help generate content and inspiration that helps you with your daily life and I'm going to give you a couple more examples uh let me show you one here I want to create a weekly meal plan from my family a four that has vegetarian options and caters to those that don't like nuts since I'm not the healthiest of eater I'm going to admit to you here on occasion this has given me a great uh list Day by Day breakfast lunch and dinner with all of the ingredients so much easier to help you start like to create a meal plan that says hey let me get to a healthier meal plan uh now this there's some things in there I learned when I did this query which is chia seeds which I don't normally eat that sparked another idea to show you let's say I want to get this grocery list by grocery section so all I can do is say give me that grocery list by grocery section and Bing now takes that exact menu for the week and puts all of the ingredients I need by section by grocery section so when I go shopping that can be super efficient right so you see all of the ingredients there it's a great it's a great help and then finally if you're looking for family activities and you're struggling for ideas Bing can help you with fun things like spontaneous trivia games my family and I were into music so I asked Bing to create a 90s music trivia game and just look how great some of these questions are and the answer so just creates that game for you and uh um we were having a little debate backstage about one of the questions that comes up here which is uh who wrote the hip song uh Jump Around is it crisscross or House of Pain and I call it out because it shows you how clever it is because for those of you who know your 90s hip-hop Kriss Kross wrote a song called jump but House of Pain was the one that wrote jump around that's how clever Bing is so Bing is going to help you make great with your look great with your family and have a bunch of fun so let's summarize where we are so far you've seen better search you've seen complete answers you've seen an incredible new chat experience and the ability to spark your creativity all with Bing your AI powered copot for the web but but but but thank you but what if that's still too much work what if you could have that co-pilot right alongside you so is there at the ready at any time you want it aware of the context in which you're in and what if you could get that Copilot on the 1.4 billion Windows PCS on the most used application on the PC the browser well you don't have to wonder team has created it I want to introduce you to your AI powered co-pilot in Edge we've just updated edge with a new look and feel and new AI capabilities as you can see here it's sleeker it's lighter and you're going to notice now that we've integrated Bing in a really cool new way let me show you here I am on the Gap website I'm browser on in my new edge browser and I want to read a gaps quarterly report so I can navigate down to their earnings click on Q3 and up comes the 15 page uh Gap PDF it's pretty long I won't have time to read all that what I'd love is a summary of the key points I want to show you how now with the power of Bing's AI capabilities within Edge we can help with one click I can open up the sidebar and now as you can see at the top of the window we have two features we have chat and compose let me show how chat works I can use chat in Edge to Simply ask it to give me the key takeaways of the page I'm on so I'll just say key takeaways from the page and Bing and AI can now read that PDF and look how great it comes up with the summary of the key points here their earnings the fact that it's going to reaffirm full year guidance very very cool a massive Time Savings but now I want to compare this with say Lululemon who also has their third quarter earnings Bing can now call out to the web pull information from outside of this page bring it into Edge compare it with the information that's on this page all with an edge and I asked it to do it in a table and look how amazing this is just like that in one table I can get an answer this question think about how much time that would have taken otherwise let me show you we can take it one step further a top use case we've learned from our friends at open AI is that developers are really being more productive with chat GPT so here we're on a stack Overflow website discussion board to learn a little bit about programming and in this case I'm researching tips on how to parse the Json file as we read through we find this great little code snippet and I'm like oh it's fantastic except it's in Python and we need it in rest all we need to do is highlight that text have it automatically you know copied over into the edge sidebar and now being inside of Ed says what do you want to do we'll say hey rewrite this code in Rust and with that simple command Bing can go and take that code and rewrite that automatically in the new programming language this is amazing GitHub copilot has been a huge boost in developer productivity imagine what the co-pilot can do for people everywhere on any page one final thing to show you not only can you better consume information but you can better create after our big announcement I'm going to want to write a LinkedIn post let's say so I'll just click create on the post and up comes with a creation dialog in LinkedIn but now I can open up the Bing sidebar and you can see here I now will go to compose and I'll just give it a prompt I'll say hey introducing the new AI powered Bing an edge let's make that enthusiastic and generate a draft and just like I need help with enthusiasm and just like that you get a little draft and I can edit it and then with one click it copies right over into my post dialog I can add some hashtags to get it some some juice and just like that I've created a post so all of these amazing new capabilities and what we think is a revolutionary new experience world-class search the ability to actually get answers to your questions Made Easy with integrated chat and the ability to generate content when you need it to spark your imagination brought to you not only when you're searching but everywhere on the web courtesy of the new edge browser with your co-pilot for the web we aspire to unlock that Joy of discovery that Wonder of creation and that feeling of empowerment of being able to harness the world's knowledge thank you on that on the presentation [Applause] in a moment two of our engine engineering leaders are going to come up and unpack a little bit about the technical details of how we built the new Bing and our approach to responsible AI but before I do I'd like to invite Sam Altman to come up and share his thoughts on this moment and our joint work welcome Sam thank you Yousef uh it's great to be here the the new Bing experience really looks fantastic and the depth of it doesn't come through until you get to use it so I hope you all will enjoy open Ai and Microsoft have been working together for more than three years now we're so grateful to have a partner that shares our vision our values of building an advanced AI That's safe and we'll have a very positive impact on society thanks to our partnership and azure's AI infrastructure we've been able to make pretty significant strides in our research which has led to the creation of systems like Chachi PT Dolly and codex but we want to make the benefits of AI available to as many people as possible and that's why we've worked with Microsoft to get this AI technology into the hands of millions of people through Azure open AI service GitHub co-pilot and starting today Bing this new Bing experience is powered by one of our next Generation models that Microsoft does customized specifically for search it takes key learnings from chat GPT GPT 3.5 and the new model is faster more accurate and more capable we all search for things many times a day and Microsoft has created and shipped a much better more useful and more enjoyable search experience I feel like we've been waiting for this for 20 years so I'm very happy it's here I believe that using AI to transform critical tasks like these is going to greatly improve our productivity and day-to-day quality of life I think this is the beginning of a very new era going forward we are excited to continue to collaborate with Microsoft very far into the future the two companies share a deep sense of responsibility in ensuring that AI gets deployed safely it's very important to us we're eager to continue learning from Real World use so the great better and better AI systems you've got to do that in the real world not in the lab and our teams will continue working together to set standards for the use of these systems across the entire industry thank you very much and now I'll hand it over to Dina [Applause] Yousef hello I'm Dina Saunders I'm the product leader of the team who brought together the magic behind Bing combined with open ai's most powerful model to date Yusuf introduced you to Prometheus I'm here to tell you about the technical Innovation behind the experience there are five aspects contributing to the newbing the first is a substantial under the hood change to every layer of Bing's technological stack the second is a proprietary system called Bing chat orchestration the third is state-of-the-art prompt generation the fourth is around inference and a brand new interactive user experience our last development is the infrastructure to scale for the opportunity ahead today we'll touch on three of these let me start with Bing chat orchestration this is what we use to gather information needed to answer your question the chat orchestrator ingests your long semantic query and fans out multiple searches in fact you might remember as Yusuf was sharing earlier you could actually see in the user experience the searches that Bing plus Prometheus is issuing on your behalf in addition to searching we also inject additional sources of information to help inform the model let me pause here and say that is incredibly important we're actually pulling in fresh data news answers contextual signals such as your location and the context of the conversation to feed into the model to help ground the information that the model is then using to reason over with this fresh information some of the experiences that Yusuf showed you earlier such as a sports example or inferring on recent events those would simply not be possible there's more though this orchestrator in parallel actually parses the documents that it finds and identifies important and relevant pieces of information this orchestrator is by nature curious and if it comes across something particularly interesting or insightful it actually starts that same cycle again this virtuous Loop results in a package of information that informs and fuels the responses you are seeing this magic combination of reasoning capabilities with fresh contextual data Sparks the Innovative and groundbreaking combination that powers the newbing next I'll talk about inference and the new interactive user experience the new interactive experience seamlessly Blends search and chat we feed a prompt I.E a proprietary set of instructions into the model which synthesizes the information reasons over all of the context that we've gathered to form an answer to your question this overall process is called Model completion think of it like when someone knows you well enough to complete your sentence we run model inferences to generate tokens I.E Words which we stream to in the user experience real time as you saw in yusuf's examples these words combine with sentences to form their response to your question and when appropriate Prometheus enhances the text answer with citations to sources and Rich structured answers from Bing and use this presentation you saw that these inferences display in a brand new Innovative dual interactive user experience that blends new conversational elements a classic search experience and a new interactive immersive chat mode the last aspect of our differentiation and Innovation is around the infrastructure we have to scale our differentiation comes really from taking this Innovation that we just shared with you today and being able to ship this at scale to millions of users worldwide earlier I talked about how we touched every aspect of the tech stack to deliver this all of those steps that we just talked about those are done in the order of milliseconds served at the speed of search latency when Yusuf was showing you the search page you could see near instant search results checks to post on the conversational insights all streaming in this is possible because we are built on Azure the world supercomputer the best and most trusted Cloud platform available this was a huge effort incredibly hard behind the scenes teams were working on powering and building out machines in data centers worldwide we were carefully orchestrating and configuring a complex set of distributed resources we built new platform pieces designed to help load balance optimize performance and scale like never before there is no other company that can do this like Microsoft and Bing as we roll out the product globally we're starting out small focused on learning and evolving with the help of your feedback we understand that with this major technological shift there will be new challenges along the way we're committed to learning with you on behalf of the team we're delighted to share this moment with you and with that let me introduce you to Sarah who's one of Microsoft's renowned responsible AI experts [Applause] I'm Sarah Byrd I lead our responsible AI engineering team for new foundational AI Technologies like the Prometheus model I was one of the first people to touch the new open AI model as part of an advanced red team that we pulled together jointly with openai to understand the technology my first reaction was just wow it's the most exciting and Powerful technology I have ever touched but with the technology this powerful I also know that we have an even greater responsibility to ensure that it is developed deployed and used properly which means there's a lot that we have to do to make it ready for users fortunately at Microsoft we are not starting from scratch we have been preparing for this moment for many years since 2017 we have been investing in a cross-company program to ensure that our AI systems are responsible by Design which has pushed us to innovate across our entire portfolio of products to solve responsible AI challenges through that we have created a foundation of responsible AI product implementation patterns that we can build on top of we're also not new to working with generative AI Technologies we've been using early versions and office and Bing for spell checking and grammar rewriting for years obviously lately it's only getting more exciting in the past year we launched the GitHub copilot GA based on the Codex code generation model the Bing Creator powered by the Dali image generation model and our own Florence multimodal model that's a lot of generative AI and with each one we have learned more about the risk the generative AI Technologies can bring including well-known ones like the perpetuation of stereotypes and bias as well as novel risk such as jailbreaks and we have developed mitigations to address them in developing the new Bing we are building on our years of operating large-scale consumer and Enterprise services and our deep partnership with openai however for this product we went further than we've ever gone before we have marshalled the full strength of our responsible AI ecosystem scientists researchers ethicists engineers and legal and policy experts to work together as a single team to develop approaches to measurement and risk mitigation strategies we've added responsible AI to every layer from the core AI model to the user experience first starting with the base technology we are partnering with openai to improve the Model Behavior through fine-tuning second we've built a state-of-the-art safety system Bing has been maturing its defensive Technologies over many years of operating a web scale search engine these include AI systems that understand user queries and classify documents to ensure safe and quality results for users we have combined these with newer AI technologies that we've created in Azure to moderate generative models We are continuing continuously retraining all of these safety models by mining Anonymous Bing query logs and using the new openai model to generate thousands of example conversations and we can update our defenses in minutes to respond to gaps we find or changes in the world the result is a safety system that enables us to act and understand data at every level of the application so we can defend against intentional misuse and mistakes from the AI finally at the application layer we are iterating on instructions as part of the meta prompt to guide the model to produce great responses and we have designed the user experience to ensure users understand and are in control so how do we know all of this works measuring responsible AI harms is a challenging new area of research so this is where we really needed to innovate we had a key idea that we could actually use the new openai model as a responsible AI tool to help us test for potential risk and we developed a new testing system based on this idea let's look at attack planning as an example of how this works early red teaming showed that the model can generate much more sophisticated instructions than earlier versions of the technology to help someone plan an attack for example on a school obviously we don't want to Aid illegal activities in the new Bing However the fact that the model understands these activities means we can use it to identify and defend against them first we took advantage of the model's ability to conduct realistic conversations to develop a conversation simulator the model pretends to be an adversarial user to conduct thousands of different potentially harmful conversations with Bing to see how it reacts as a result we're able to continuously test our system on a wide range of conversations before any real user ever touches it once we have the conversations the next step is to analyze them to see where Bing is doing the right thing versus where we have defects conversations are difficult for most AI to classify because they're multi-turned and often more varied but with the new model we were able to push the boundary of what is possible we took guidelines that are typically used by expert linguists to label data and modify them so the model could understand them as labeling instructions we iterated it with it and the human experts until there was significant agreement in their labels we then use it to classify conversations automatically so we could understand the gaps in our system and experiment with options to improve them this system enables us to create a tight Loop of testing analyzing and improving which has led to significant new Innovations and improvements in our responsible AI mitigations from our initial implementation to where we are today the same system enables us to test many different responsible AI risk for example how accurate and fresh the information is of course there's still more to do here today and we do see places where the model is making mistakes so we wanted him to empower users to understand the sources of any information and detect errors themselves which is why we have provided references in the interface we've also added feedback features so that users can point out issues they find so that we can get better over time maturing a new technology takes time and collaboration which is why we're very excited for this next phase where we can share the technology with the real world we look forward to seeing users unlock its full potential and getting feedback from stakeholders across Society to help develop this into an essential tool for the future our goal is to continuously Advance the state of the art Youssef back to you Sarah all right to close we have one final piece of news to share I'm pleased to announce that the new Bing is live today for desktop on limited preview what this means is that we're going to make Bing available today for everyone to try on a limited number of queries and then to sign up on the waitlist to get access to the full experience we're also starting with a select group of folks who will have access to the full experience right away all of you here in the room you're all going to be on that list so you're all going to get to try with try it right away and in addition then we plan to expand to millions of people in the coming weeks we're also going to be launching our mobile version today is an important part of our journey but it's just the beginning as Satya said we intend to innovate quickly get your feedback and continue to bring new Innovations and capabilities to everyone welcome to the new being an edge thanks very much thanks for being here [Music] [Music]

Channel: Bing

Views: 849,092

Keywords: Bing, Edge, Microsoft, AI, copilot, chat, search, answers, search engine, AI copilot

Id: rOeRWRJ16yY

Length: 55min 9sec (3309 seconds)

Published: Tue Feb 07 2023