ChatGPT-4o Comes to Azure AI Studio

if we're optimizing power and efficiency across every layer of the stack from the data center to the network our latest latest data center designs a purpose built for these AI workloads so that we can effectively and responsibly use every megawatt of power to drive down the cost of AI and the power draw uh and we are incorporating Advanced Data Center cooling techniques to fit the thermal profile off the workloads and match it to the environment and the location where it operates um at the Silicon layer we are dynamically able to map workloads to the best accelerated AI Hardware so that we have the best performance uh and our custom IO hardware and server designs allow us to provide dramatically faster networking remote storage and local storage threr you know this endtoend approach is really helping us get to the unprecedented scale in fact last November we announced the most powerful AI super computer in the cloud for training using just actually a very small fraction of our Cloud infrastructure and over the past 6 months we've added 30 times that supercomputing power to Azure yeah it's crazy to see the scale and of course we're not just scaling training uh fleets we're scaling our infrance Fleet around the world quadrupling the number of countries where Azure AI services are available today and it's great to see that at the heart at the heart of our AI infrastructure are the world's most advanced AI accelerators right we offer the most complete selection of AI accelerators including from Nvidia and AMD as well as our own Azure Maya all dynamically optimized for the workloads uh that means whether you're using Microsoft co-pilot or building your own own co-pilot apps uh we ensure that you get the best accelerator performance at the best cost uh for example you know you see this in what has happened with gp4 right it's 12x cheaper and 6X faster since it launched uh and that's you know the type of progress you can you know continue to see how you know to continue to see the progress as we evolve the system architecture it all starts though with this very deep deep partnership with Nvidia which the entirety of the co-pilot stack across both all of their Hardware Innovation as well as the system software Innovation together we offer Azure confidential Computing on gpus to design to be really help you protect sensitive data around the AI models end to endend uh we're bringing in fact the latest h20s to Azure later this year uh and we will be among the first Cloud providers to offer nvidia's Blackwell gpus b100s as well as gb200 configurations and we're continuing to work with them to train and optimize both large language models like GPT 40 as well as small language models like the 53 family now beyond the hardware we are bringing nvidia's key Enterprise platform offerings uh to our cloud like the Omniverse cloud and dgx Cloud to Azure with deep integration with even the broader Microsoft cloud for example uh Nvidia recently announced that their djx cloud integrates natively with Microsoft fabric that means you can train those models using dgx cloud with the full access to fabric data and Omniverse apis will be available first on Azure for developers to build their industrial AI Solutions we're also working with Nvidia Nim industry specific developer services and making them fantastic on Azure so lot of exciting work uh with Nvidia now now coming to AMD I am really excited to share that we are the first Cloud to deliver General availability of VMS based on AMD Ami 300X AI accelerator it's a big milestone for both AMD and Microsoft we've been working at it for a while and it's great to see that today as we speak it offers the best price performance on GPT 4 inference and we'll continue to move forward uh with Azure Maya in fact our first clusters are live and soon if you're using co-pilot or one of the Azure open AI Services some of your prompts will be served using Maya Hardware now Beyond AI our endtoend systems optimization also makes Cloud native apps and the development of cloud native apps better right 6 months ago is when we announced our first general purpose Arm based compute processor Microsoft Cobalt and today I am really excited to announce the public preview of cobalt based VMS you know Cobalt is being used for video processing and permissions Management in Microsoft 365 helping power billions of conversations on services like Microsoft teams already and we're delivering that same arms based performance and efficiencies to many customers in fact including elastic mango semen Snowflake and teradata in our most recent Benchmark data and Tas our Cobalt 100 VMS delivered up to 40% better performance than any other generally available arm-based VM so we are very very excited about Cobalt getting into the market now let's move up the stack to the foundation models look with Azure AI we offer the broadest selection of Frontier and open source model models including llms and slms so you can choose the model that makes the most sense for your unique needs and your application needs in fact more than 50,000 organizations use Azure AI today yeah it's great momentum and it all starts though with our most strategic and most important partnership with open AI just last week open AI announced GPT 40 their latest multimodal model which was trained on Azure it's an absolute breakthrough it has text audio image and video as input and output it can respond and just have a a humanlike conversation that's fast and fluid it can even be interrupted mid sentence uh gb40 is also the top performing model on benchmarks across a variety of modalities and it's always going to get you to the best answer uh it has state-of-the-art performance at understanding the information you actually provide in your prompt cuz that's really what it matter what matters uh what open a i demoed last week I mean that was just pretty magical stuff and yesterday we showed how co-pilot will leverage GPT 40 in the video I'm just going to play notice that you can share your screen or session as a prompt in co-pilot and have co-pilot help you in whatever it is that you're doing let's roll the video hey co-pilot how's it going hey I'm doing great and it looks like you're about to dive into some Minecraft ready to build explore and maybe dodge a few moms yeah I'm trying to figure out how to make a sword my son plays this game all the time and I have no idea what I'm doing no worries I'm here to help you impress your son I see you've got a crafting table in front of you which is a great start to craft a sword you'll need some materials can you open your inventory just press e on your keyboard yeah do I have what I need let's see you've got some sticks perfect for the swords handle but it looks like you're missing the material for the blade you can use wood Stone iron gold or Diamond let's start by gathering some wood or stone shall we head out and find some resources yeah that sounds great oh my God what's that oh no it's a zombie run you need to get away from it fast either Sprint away or quickly build the Tower of blocks straight up to get out of its reach if you can find a shelter or dig into the side of a hill do it now just make sure it can't reach you all right I think I might be safe now cool that was a close one great job finding shelter it's really cool to see that you know the magic here is that co-pilot is your personal assistant that's helping you with any task helping you stay quite frankly in the flow in the moment whether it's a game or a document or a you know line of code and as openai innovates our promise is that we will bring all that Innovation to Azure 2o in fact the same day that openai announced GPD 40 we made the model available for testing on Azure open AI service and today it I'm excited to say that it's generally available on Azure AI you know what this means though is that now we can have these groundbreaking apps that all of you can build using this capability one of the coolest things is that now any app any website can essentially be turned into a full multimodal full duplex conversational canvas let's take a look all right let me ask the agent here hey Seth how's it going what can I help you with today I am fantastic I'm sorry I'm in such a hurry I'm going to go on an overnight camping hike where we're camping all this it's going to be dark cold should I should I use these shoes it's the mountains it's probably going to be cold hey no worries let me take a look ah those are the trar hiking sandals while they're fantastic for summer hikes and keeping your feet cool they might not be the best choice for Co oh my good okay I'm sorry to interrupt you pick the best thing that I should use and can you just add it to my car just one product no worries let's go with the Tre ready Hawking boots they're durable provide great ankle support and most importantly they'll keep your feet warm in the cold mountains I'll add them to your cart now ready for check out I see that amazing while Seth was you know Seth was in a hurry but there you go GPD 40 kept up with him it's incredible I really want to thank the open AI team uh for their partnership and and really the responsible approach to Innovation helping our industry move forward Sam will be here in fact joining Kevin in a little bit to talk a lot more about what's coming right because that's the exciting stuff how do you all sample what comes next uh we also I brought we bringing lots and lots of other models as well from coher and data bricks and Desi meta Mistral snowflake all to Azure AI we want to support the broadest set of models from every country every language I'm excited to announce in fact we're bringing models from coher g42 ntity data Nixa as well as many more as models as Services because that's the way you can easily get to managed AI models uh and we all love open source too in fact two years ago uh at Bill we were the first to partner with hugging face making it simple for you to access the leading open source Library uh with state-of-the-art language models via Azure Ai and today I'm really excited to announce that we're expanding our partnership bringing more models from hugging face with text generation inference with text embedding inference directly into Azure I Studio and and we're not stopping there we are adding not just large language models but we're also leading the small language Revolution small language model Revolution you know our five3 family of slms are the most capable and most cost effective they outperform models of the same size with the next size up even across a variety of language reasoning coding as well as math benchmarks uh if you think about it by performance parameter count ratio it's truly Best in Class uh and today we are adding new models to the 53 family uh to add even more flexibility across that quality cost curve uh we're introducing 53 vision of 4.2 billion parameter multimodal model with language and vision capabilities it can be used to reason over real world images or generate insights and answer questions about images as you can see right here yeah and we're also making a 7 billion parameter 53 small and a 14 billion parameter 53 medium models available uh with Fi you can build apps that span the web your Android iOS windows and the edge uh they can take advantage of local hardware when available and fall back on the cloud when not simplifying really all of what vs developers have to do to support multiple platforms using one AI model now it's just awesome to see how many developers are already using 53 to you know do incredible things from am Solutions the Thai company uh that I mentioned uh earlier the ITC which has been uh built a co-pilot for Indian Farmers to ask questions about their crops epic uh in healthcare which is now using fi to summarize complex patient histories more quickly and efficiently uh and another very very cool use cases in education uh today I'm very thrilled to announce a new partnership with KH academy uh we'll be working together to use 53 to make math tutoring more accessible and I'm also excited to share that they'll be making K Amigo their AI assistant free to all us teachers let's roll the [Applause] [Music] video you forgot already felt like I was in a place in my teaching career where I felt like I was kind of losing my Sparkle and I would just feel really defeated when I looked out on the classroom and I would see students that just didn't look engaged teachers have an incredibly hard job and what we think we can do is leverage technology to take some of the stuff off of their plate to really actually humanize the classroom by some miracle we became a kigo pilot school with new advances in generative AI we launched kigo the point is to be that personalized tutor for every student student and to be a teaching assistant for every teacher I started to build these more robust lessons and I started to see my students engage we're working with Microsoft on these fi models that are specifically tuned for math tutoring if we can make a small language model like fi work really well in that use case then we would like to kind of shift the traffic to fi in those particular scenarios using a small language model the cost is a lot lower we're really excited that kigo and especially in the partnership with Microsoft being able to give these teacher tools for free to us teachers is going to make a dramatic impact in US education I think we're going to make them the innovators the questioners isn't that really just why you wake up every morning right because that's our for our future our next generation and to me that's [Music] everything you know I'm super excited to see the impact uh this all will have and what Khan Academy will do and Sal's going to in fact join Kevin uh soon to share more and I'm really thankful to te for teachers like Melissa and everything that they do thank you very much you know of course it's about more than just models it's about the tools you need to build these experiences uh with Azure AI Studio we provide an end to-end tooling solution to develop and Safeguard the copilot apps you build uh we also provide tooling and guidance to evaluate your AI models and applications for performance and quality which is one of the most important tasks as you can imagine with all these models uh and I'm excited to announce that Azure AI studio now is generally available it's an endtoend development environment to build train and fine-tune AI models and do so responsibly it includes rebuilt in support what is perhaps the most important feature which is in this age of AI which is AI safety Azure AI Studio includes the state-of-the-art safety tooling you know to everything from detecting hallucinations in model outputs risk and safety monitoring uh it helps understand which inputs and outputs are triggering uh content filters uh prompt Shields by the way to detect and block these prompt injection attacks um and so today we are adding new capabilities including custom categories so that you can create these unique filters for Proms and completions with rapid deployment options which I think is super important as you deploy these models into the real world uh if when an emerging threat is you know appears Beyond Azure AI Studio we recognize that there are Advanced applications where you need much more customization of these models for very specific use cases and today I'm really excited to announce that Azure AI custom models will come giving you the ability to train a custom model that's unique to your domain to your data that's perhaps proprietary uh that same Builders and data scientists who have been working with open AI brought all the five advances to you will work you know with all of you to be able to build out these custom models the output will be domain specific it'll be multitask and multimodal uh Best in Class as defined by benchmarks including perhaps even specific language proficiency that may be required now let's just go up the stack uh to data ultimately in order to train fine-tune ground your models you need your data to be in its best shape and to do so we are building out the full dat data estate right from operational stores to analytics in Asia we've also added AI capabilities to all of our operational stores whether it's Cosmos DB or SQL or postra SQL uh the core though is this of the intelligent data platform is Microsoft fabric uh we now have over 11,000 customers including leaders in every industry who are using fabric it's fantastic to see the progress with fabric you get everything you need in a single integrated SAS platform it's deeply integrated at most fundamental level with computer and storage being unified your experience is Unified governance is Unified and more importantly the business model is Unified and what's also great about fabric is that it works with data anywhere right not just on Azure but it can be on AWS or on gcp or even on your on premise data Center uh and today we are taking the next step we're introducing realtime intelligence in fabric customers today have more and more of this real-time data coming from your iot systems your Telemetry systems in fact Cloud applications themselves are generating lots of data but with fabric anyone can unlock actionable insights across all of your data estate let's take a look look introducing realtime intelligence in Microsoft fabric an end to-end solution empowering you to get instant actionable insights on streaming data at its heart lies a central place to discover manage and consume event data across your entire organization with a rich governed experience get started quickly by bringing in data from Microsoft sources and across clouds with a variety of out-of-the-box connectors route the relevant data to the right destination in fabri using a simple drag and drop experience explore insights on pedabytes of streaming data with just a few clicks elevate your analysis by harnessing the intelligence of co-pilot in Microsoft fabric using simple natural language make efficient business decisions in the moment with real time actionable insights and respond to changing Landscapes proactively allow users to monitor the data they care about detect changing patterns and set alerts or actions that drive business value all your data all your teams all in one place this is Microsoft [Music] Fabric and we're making it even easier to design build and interoperate with fabric with your own applications right right and in fact we're building out a new app platform with fabric workload development kit so that people like ezri for example ex Le have you know who have integrated their spatial analytics with fabric so that customers can generate insights from their own location data using EZ's Rich tools and libraries right on fabric right this is just exciting to see it's the first time you know where the analytics stack is really a first class app platform as well and Beyond fabric we're integrating the power of AI across the entirety of the data stack there's no question that rag is core to any AI powered application especially in the Enterprise today and Azure AI search makes it possible to run rag at any scale delivering very highly accurate responses using the state-of-the-art retrieval systems in fact chat GPT supports for you know where gpts their assistance API are all powered by Azure AI search today uh and with buil-in one Lake integration Azure AI search will automatically index your structured data too and it's also integrated into Azure AI Studio to support bringing your own embedding model for example and so it's pretty incredible to see aure search grow over the last year into that very core developer service now let's go up to developer tools nearly 50 years after our founding as a developer tools company here we are once again redefining software development right gith up co-pilot was the first I would say hit product of this generative AI age uh and it's the most widely adopted AI developer tools 1.8 million Subs across 50,000 organizations are using it and get up Co with get up copilot we are empowering every developer on the planet to be able to access programming languages and programming knowledge in their own native language uh think about that any person can start programming whether it's in Hindi or Brazilian Portuguese and Jo they bring back the joy of coding to their native language uh and with co-pilot workspace staying in your floor has never been easier uh we are an order of magnitude closer to a world where any person can go from idea to code in an instant uh you start with an issue it creates a spec based on its deep understanding of your code base it then creates a plan which you can execute to generate the code across the full repo that is multiple files at every point in this process from the issue to spec to plan to code you are in control you can edit it and that's really what is fundamentally a new way of building software and we're looking forward to making it much more broadly available in the coming months and today we're taking one more big leap forward you know we are Bridging the broader developer tools and services ecosystem with co-pilot for the first time we're really thrilled to be announcing get up co-pilot extensions now you can customize get up co-pilot with capabilities from third party Services whether it's Docker Sentry and many many more and of course we have a new extension for Azure to get up co-pilot for Azure uh you can instantly deploy to Azure to get information about your Azure resources uh just using natural language uh and what co-pilot did for coding we're now doing for infra and Ops uh to show you all this in action here is Nia from our GitHub team Nia take it away thanks seia GitHub co-pilot gives you suggestions in your favorite editor like here where I'm writing unit tests co-pilot is great at meeting you where you're at regardless of the language you're most comfortable with so let's ask for something simple like how to write a prime number test in Java but let's Converse in Spanish using my voice in Java look at that gracias co-pilot co-pilot is great at turning natural language into code and back again but what about beyond the code with the new GitHub co-pilot extensions you can now bring the context from your connected systems to you so now I can ask Azure where my app is deployed I could ask what my available Azure resources are or I could diagnose issues with my environment and this isn't just for Azure as Saia announced any developer can now create extensions for GitHub co-pilot and that includes any tool in your stack including your in-house tools keeping you in the flow across your entire day actually 75% of a developer's day is spent outside of coding Gathering requirements writing specifications and creating plans let's show how GitHub co-pilot can help with that live on stage for the first time so typically my day starts by looking at GitHub issues looks like we want to support a rich text input for our product description let's open workspace and get some help with that co-pilot interprets the intent of the issue to see what's required and it then looks across the entire code B base and it proposes what changes should be made this specification is fully editable and the whole process is iterative but actually this looks pretty good co-pilot can now help us build a plan on how to implement this change all right that's a great start but we must not forget about our documentation so let's edit the plan and have co-pilot update our readme and then we can even get C- palat's help in starting to implement the code for us now this was just a simple example but in a large Enterprise codebase there are tens of thousands of files and dozens of stakeholders involved and that means meetings so many meetings where work space helps you focus on what you need to change and by the way as a developer I'm always in control I can see exactly what changes co-al is proposing and I can even get a live preview all right let's test out the input all right this looks great so I can go back and I can edit my code in BS code or I can submit these changes as a pull request to share with my team GitHub co-pilot co-pilot extensions and pallet workspace help you stay focused on solving problems and keeping you in the flow

Channel: Microsoft DevRadio

Views: 1,154

Keywords:

Id: Sr4E8s-hpp4

Length: 29min 34sec (1774 seconds)

Published: Thu May 23 2024