Ask the Experts: Azure Arc and Azure SQL Managed Instance | Data Exposed Live

[Music] hi i'm anna hoffman and welcome to this episode of data exposed live we are super excited to have you joining us today from wherever you're streaming from whether it's learn tv twitter or twitch or youtube or somewhere else we're not aware of oh or if you're streaming in from the microsoft reactor we thank you for joining us today um today is a special episode because we are actually doing an ask the expert session so today i have on two very special guests i think you all might be familiar with them um but we're going to be talking about azure arc azure sql management and our goal is really to help disambiguate or demystify all the different options and other questions you might have really related to anything we treat this kind of like an ama so you can ask whatever questions um you like uh now as we get started the first thing i want to do is bring up our wonderful speakers so uh first i'm going to bring up uh buckwoody buck thanks so much for joining us hello anna how are you i am good how are you every day is just amazing good i'm glad to hear that um can you like tell our viewers for those folks that might not uh know you can you tell our viewers a little bit about what you do and kind of what your focus is sure um i'm buck woody and i'm a data scientist here at microsoft i work on the uh sql the azure data team we call it nowadays because it's much broader than just sql server on premises although it still includes all that and i work on some of the adoption efforts developing courses doing sessions like these teaching i also run the community the azure data community and a quick plug for that aka dot ms forward slash data community all one word if you go there you can join in some of the uh close to 100 user groups around the world and countries around the world 90 plus thousand people around the world in this definitely come and learn those i also run the mvp program for our group and have some great folks working in that and then of course from time to time i'm super privileged and honored to be on data exposed yeah definitely uh we have a series we do together called something old something new so if folks watching haven't seen that before uh definitely something uh you know without any bias at all i would recommend people trying i highly recommend it dot ms forward slash s-o-s-n something old something new awesome all right cool thanks so much buck um so we also have with us someone you might also know and that is uh bob ward bob thanks so much for joining us hey anna thanks for having me here today really excited to be on data exposed again uh super excited to be on a show where hashtag ask bw means ask bob ward it's a very great way to type the show oh buck or buckwoody or bw wait anna did not tell me that you were woody and then you know bob ward whoops oh well so yeah hey thanks jenna for telling me buck was going to be here too i guess we both have to talk together today great oh it's like the parent trap you have to trick people it's a parent trap exactly um excited to be here today yeah we're excited to have you and bob just for viewers who aren't familiar with some of the stuff that you've worked on in the past can you can you share with us sure um this is kind of crazy to say this anna but this is my 28th year at the company all working on something called sql server so somebody did ask me recently somebody doesn't is not a technology person and said what do you do at microsoft and i said well if it's a microsoft product or cloud service and has the word sql on it i pretty much work on that whether it's internally working with our team to talk about uh customer feedback or new releases of what we're doing in this space or uh being with people like yourselves uh in front of the camera right or an event uh talking about sql so anything sql edge to cloud i kind of work on that kind of thing awesome great and uh thanks so much again both of you i think we're gonna have a really great show um i do want to put that out there as just a reminder to folks tuning in that this is an ask the expert so we're expecting lots of questions we're hoping for questions um from you all so please uh keep your questions coming in questions or comments even sports related comments we're taking today um so it looks like anders peterson says yes bob i am wearing my packer shirt because he knows i guess a big packers fan or something yeah that's unfortunate for anders to do that i'm sorry that he feels the need to wear a jersey of a lower team a team lower than the cowboys well you know to me i mean i i don't really know a lot about sports ball but bob's been educating me and if i understand it correctly um well your picture's right there behind you you're you're a big fan of the buffalo team buffalo what are they called yeah that would once again be an incorrect comment by yourself uh that is a buffalo because i am actually broadcasting great from the great state of colorado today so buffalo being the theme of course in colorado but no i am not a fan of the buffalo bills oh okay dallas cowboys cowboys all right so we will take sports question but you know you'll see me kind of try to reign us back into azure sql and azure art topics and kind of to get us started like just to get people's questions kind of brain's kind of melting so they have some questions uh bob i thought it would be great like if you could start by telling us like what actually is azure sql and then maybe drilling in on a managed instance yeah thanks santa uh let me know if you can see this okay right here i've got a a slide to kind of go over some of the aspects of managed distances can you say that okay yep perfect you know i think the thing the right thing to do to make sure we understand what managed instance is is to show we've been doing this for a while and i've also been doing this for a while is to make sure you understand how azure sql managed instance fits them into the portfolio of azure sql itself azure sql is not a specific service it's a suite of services whether it be running a sql server itself in a virtual machine we call that also infrastructure as a service or running a complete sql instance in the cloud we call that azure sql managed instance or just having a database in the cloud which is called azure sql database if you look in the middle here azure sql managed instance is really quite frankly the combination of running a full sql server instance with managed capabilities i want you to think in terms of what it looks like from a managed management studio perspective when you connect management studio into a managed instance it looks just like sql server but there's going to be a couple of things that you don't see for example in object explorer like you don't see always on availability groups and that's because we build one for you automatically should you pick the right service to your options so you don't even need to manage that aspect so think of the fully managed service from microsoft behind the scenes platform as a service as a sql instance and in fact if you look at their best for modernizing existing apps we kind of feel like you can migrate your sql server today into a virtual machine with sql server that's kind of the quickest way to get into the cloud you can also migrate into managed instance but we call it modernization because you're taking advantage of new things in the cloud they're going to make things easier for you to run your application now i kind of put this together anna a little bit because people have asked me okay that sounds interesting but from a feature perspective or from a user experience perspective is it sql server so think about this it's a pre-installed sql with an abstraction away from the operating system and all the infrastructure so again we just pre-install sql for you it is the same engine as sql server say sql 19 but it's ahead of the game a little bit ahead of sql 19 because it's version list imagine a world where you don't patch anything anymore we take care of all that for you and the t-sql language that language you know and love is almost 100 compatible and the minor differences that exist i've got a little shortcut for you to go take a look at that so i think of it this way a fully managed service is sql plus the cloud to help automate some things now there are other features that you might find interesting that our instance level features again this is a complete instance for you whether it be agent resource governor link servers machine learning services all of those things that you know that are outside the scope of a database but part of your instance you get access to that with azure sql managed instance and you may have seen things like bi services ssrs as and is that you use you get those with your sql server licenses those are not part of managed instance but they integrate well with those services whether you run them on premises or in the cloud so that's kind of a way of looking at like the whole feature set of sql server banana i kind of picked on one area that i think is very interesting of how we automate things or make them manage and that is availability so a built-in complete availability system whether you pick a general purpose option or business critical option think of it like a failover cluster instance for general purpose or a complete availability group system for business critical and we back it up with money we back it up with a service level agreement to make sure you have this four nines capability to keep you available all the time and then we have this amazing redundancy story system databases by the way which a lot of people don't see that today with availability groups that are included in our replicas you can set up auto failover groups across regions think of that like a distributed availability group set up for you and then this is really great your application complete connection abstraction built in you don't install a listener or any of those kind of type concepts then from a disaster recovery perspective how about a complete built-in database backup system for you automated completely and even keeping your backups up to 10 years completely redundant whether it be geo-redundant zone redundant or local redundant but you do have some backup control because it is a managed instance so you can make your own copy only backups and you can restore backups from any sql server or other managed instances into this environment and then you can recover easily a very simple point in time or source system or even recover deleted databases that got accidentally dropped or doing restores from geo regions like if you had backup setup for geo redundancy you had a problem in a given region you could actually go restore things from those regions so so anna a complete sql server but managed for you with instance level capabilities all running in the cloud this one slide alone tells you what the value proposition alone could be for an azure sql managed instance yeah definitely i mean i think there's a lot there and i think a lot is comforting uh especially for our sql server people when they see that is it sql server and they see a lot of the capabilities that they're familiar with uh coming to be a thing and manage incense wow i'm gonna pause because like we have so many questions coming in right now so let's let's start going through some questions i brought buck back up in case uh buck you have some answers to these questions so all right let's just get right into it the first question comes from karthik on youtube and karthik says hi all can we accept expect sql manage instance business critical tier specifically with more storage yeah karthik thanks for joining us today thanks for asking that question you know we just announced 16 terabyte support for general purpose managed instances and we fully intend to increase storage capacities also in business critical not ready to announce a specific time that we're doing that but i promise you that is one of our intentions to increase that storage capacity for a business critical service tier in the future awesome thanks uh thanks for that bob and thanks for your question karthik uh this next question comes from meatball seros uh they say what happens when the four nines are violated so i think this was about availability you said at least 99 99.99 of availability right thank you meatball for asking the question what a great what a great handle yeah honestly it's called a service level agreement because it's a contract when you sign up for subscription and then you deploy managed instance you now are part of our service level agreement and if you violate if we violate what we've told you as those four nine capabilities you get credits back to your azure subscription from a billing perspective you can imagine that we have a very high incentive to not make sure we violate those slas but yeah it's a complete system setup so if you think that's something that's happened for you you can contact microsoft support and apply for service credits awesome thanks and thanks meatball uh actually meatball clarified that his name is george so okay thanks george uh all right let's move on to more questions so this question came in from twitter from gary i think this would be a good one for you uh buck let me just grab the question i filled it it is here it is how can you get java applications to connect to a managed instance using an azure active directory account okay good stuff uh java connected to managed instance so something you want to think about first of all is that you need to be using the microsoft jdbc driver for sql server that's the number one thing uh because you can't just connect to it with just about anything because you're gonna lose some of the active directory integration that you have there so that's number one make sure you have the latest one by the way uh version i think it's 7.2 has the latest and you're going to have like um various authentication mechanisms or connection mechanisms that you can signal within the driver connection string uh things like um active directory msi and that'll let you do uh azure sql databases synapse analytics uh from an azure resource with identity support enabled which is really nice uh you can also do the msi client id within that um you can do it for the older version with version six uh you can do the um active directory integrated and you just put authentication equals active directory integrated and then you can connect to azure sql database or synapse analytics using integrated authentication and to use that though you need to make sure you federate your on-premises uh on-premises active directory adfs active directory federation services with active directory in the cloud and that way you can get that back um once that's set up you can connect that way now something else you need to think about when you're using that library so that's the connection mechanism i think we may even have a short link that we can put up for you that has the the pointer to exactly what to do and even the download location uh for the driver but something else you need to think about is when you program against it there are some considerations between java and a database um java has different data types than sql server well so this r or python or anything else so we do have a document as well another another short link maybe that has up there what kind of data type mapping you want to be very careful there especially for dates you don't want things to flip around or do something unusual the other thing that you want to figure out is how you manage the result sets whether you're going to persist them down or how you're going to handle transactions um also uh table value parameters become interesting in java so we've got some information on that and then uh always recommended and i don't wanna i wanna make sure i always cover this uh we have always encrypted um support for java uh you definitely want to read up on enforcing that security should not be an afterthought it should be something you do in specifics i think we've put those links up there for you that hopefully that will that will help out um java and any database always interesting to connect to awesome yeah thanks and i think it's cool too that we do have these programming guides depending on what language you're using like we're not just saying like you have to develop with net or you know you can't use sql and what's also nice is all these drivers are kind of the same whether you're using azure sql database address sql manage instance uh sql server or azure arc which we'll get to uh soon um i think this kind of relates though to another question we got in from the forums um and that question was uh you know i'm using manage instance attempting to create a linked server against synapse on demand but not getting a connection bob is this something you think you can help with yeah thanks you know it is calling out the fact that you can do linked servers with managed instances because it's an instance level uh capability and we do support the ability to actually connect that to a synapse system so there's a lot of reasons probably why somebody can't get the connection to make that work so i know that uh one thing that i found very handy one of our colleagues yovan puppets has a very nice blog where he details out exactly the steps you need to do if you have a managed instance and you'd like to make that connection to a synapse system to azure setups analytics and the reason why that's important is that you don't have to move data you'd like to be able to query data in synapse and see that maybe even together with a sql server managed instance so there's two ways to do it one is through external tables actually uh through managed instances and the other is a linked server javon has some very nice detail steps of everything you should do i recommend you first go to that blog and make sure you've followed all those steps as they described and see if that doesn't resolve your issue awesome great uh thanks bob and hope that helps okay moving along some of our questions uh this next question comes in from uh nss dharma any options for monitoring a path instance freely hmm monitoring like a like a managed instance freely so first of all if you have tools uh you can even consider the fact that inside managed instance there are capabilities in the sql engine like dynamic management views query store things of that nature that just comes with your managed instance so you yourself could use your own tools to go query that information and look at it so for so from a monitoring perspective there are things that just come with the engine and manage this and supports those that you can choose yourself yeah one thing to consider though that it's not necessarily free is a really nice technology called sql insights and i think you've had elaine maybe on your show here to talk about that think of that i think of let this this way as a perf mod in the cloud a perfmon in the cloud that allows me to monitor azure databases as your virtual machines with sql server and manage systems and even see this information together so if you're looking for maybe a lower cost solution maybe consider sql insights that's in preview today but if you really said look i just don't wanna i haven't managed instance i need to do something that doesn't cost me any money remember all the built-in things that just come with it that you both have access to and management studio all the reports work as normal you've got your you've still got your activity monitor so all of the all of the surface areas you had for monitoring before none of that changes um and if you want a sort of an enterprise free monitoring solution a lot of companies these days are going to grafana and kibana in fact we use some of that ourselves uh for certain things but as bob pointed out uh trust me i used to write these systems all the time i've written so many uh multiple management and you know checking logs and all those things from disparate systems you're not going to get close to what you can do in insights and sometimes it's worth the cost of admission just to get that done and this is not an expensive product insights is not like some really heavyweight expensive products so if you want free you have everything as bob pointed out that you've always had you're also able to connect up anything that can read a dmv that you're using today but if you want an enterprise space thing definitely take a look at insights it's really going to be hard to beat grafana cabana is a roll your own you'll have to build it uh but it is free awesome thanks both yeah there was a follow-up question on sql insights uh karthik asked does sql insights have any alerting mechanisms uh specifically for sql agent job failures on manage instance yeah sql insights is more about monitoring like performance and telemetry type data to be honest with you uh there there could be there are some specific metrics across everything that comes with the managed instance so some of it could be like availability or some of that nature but i'm not aware of a specific counter that insights has to look at agent job failures i have to double check in there yeah so you have to use a different mechanism that you normally would use today with sql server to see did my agent job fail that could be mail for example i was getting ready to say yeah that was built right in and in your job step you have exception reporting and you also have success reporting i do both uh whenever i do a job i do not just say if you fail let me know that's a terrible idea uh because what if that system isn't working it'll never let you know when you think everything's okay while it's a pain to get an email every morning that the server's not out of space and everything ran and my dbcc checks ran all those sorts of things it's a pain to get those i know when i don't get those i need to if you treat it like bob you know you've got kids if your kid gets really quiet that's bad right you go find if you're if you're i had a little t-shirt on my daughter and it said if this t-shirt gets quiet come find me and so that's the way i do it with database alerting so i use the ones that are built right in for agent failures although you can still monitor that with other tools i tend to rely on something really close to the metal i know we've talked to the team that runs insights we've added chatted in the future there you know could we add in more about uh failure uh detection or errors and things of that nature not just performance telemetry data so that's that's something that the team is considering but today you know as buck just talked about like database mail the things that just come with agent alone those are the best mechanisms to monitor uh failures yeah anders anders says uh server with thousands of jobs failure alerts over one email seven report all jobs uh yeah that that can be that can be hard but there's ways to do that there are ways anders to to do that i think you're saying that you do have ways to do that i don't know if you guys have even looked at windows desktop automation uh free tool for the windows operating system and i use that to go through and just do regex type stuff against logs and things works out works out really well nice awesome great i love all the questions so keep the questions uh coming in um there's another question that i saw from ak and they say for a new development i believe for the constant flexibility of azure sql database is the best am i correct and mi is good for migration purposes so i think this is more like a what to use when between db and mi yeah that's a great question okay thanks for asking that because you know even though we internally think we've tried to get that message out it's it's great to keep reinforcing it so don't get me wrong it's always possible to do new development for a managed instance there's just no question but it is fair to say that we at microsoft feel like we're targeting more new cloud-borne apps to use azure sql database because you're correct in one aspect there is more flexibility for things in azure database for example serverless you have the ability to have a serverless database we can even pause computer if you're actually not using it or hyperscale where you can do really huge read scale and get massive sizes because you're just focusing on the database itself so so for new cloud-borne app development there's no question we tell customers we think one of the best options is azure sql database but don't get it you know let's be let's be clear about managed instances it is a complete sql server instance but guess what you know why we built managed instance the name of the project was called cloud lifter because at that time a few years back our customers told us hey i have agent jobs or replication or things that are instance level capabilities or even even some t-sql things that azure database doesn't have so i want to migrate my sql server to the cloud but i can't do that to azure database because of the limitations hence born the idea of cloud lifter but keep in mind one thing as we said earlier you're correct it's a great way to do migration we even support things like online restore migrations almost zero down time to do that but it is a modernization play as well so it's not just you moving a sql server there it's actually getting better faster more efficient and more cost effective when you go that route awesome thanks and hopefully uh that helps uh aka uh george i see you have a lot of questions i'm not sure all of them are gonna be great for this uh outlet but let me let me just show them to bob and buck and then you guys can kind of decide uh how you want to move forward with them so one of the questions george asks is about how to do tdd in complex environments uh another question did you want to take that one yeah i think that one um test driven development is difficult in mixed environments and hybrid um i wonder if it would be useful if i gave you an article that talks all about it um there you go anna i don't know if you could paste that it's huge yep we'll share that in a twitch so so the the the bigger the uber here uh we have a we have a series of resources for you called the cloud adoption framework uh this is a you could just go out and bring that and find the the headwaters to it but we go over every programming paradigm every data distribution paradigm every anything you can find in an itil catalog anything like that and we cover it and how you do it in the cloud so there's uh there's a whole article on this and you are going to cover the test-driven development cycle um the big deal here is the definition of the word done that's always true in a tdd environment but in the cloud you have to define done early but there are some examples in there of the code where you can do test driven development in a complex environment like the cloud awesome i don't know whether it makes sense to also discuss with tdd but you know i think devops you know azure is the place to do devops right so to me i kind of relate devops a little bit with this question some as well and we have some really great content anna you brought people on the show before that talks about how azure sql database can be a great target for things like devops environments it's funny you mentioned that because i think that's related to the other questions that george had which is how do i convince my superiors that xp or ado uh is overkill for a database that is on the order of thousands of items um i wish we could help with um fixing fixing our customers environments i go into some and you know you have to make sure you don't shake your head while they're talking um it is hard all you can do is present the logic uh and talk the terms that they understand so i've often they have a hammer and it's like we were told to do this so we're going to use this to do everything whether it's a tool or procedure so what i do with those kind of things i try to explain the rationale of why one-offs are going to happen uh that you're going to have some things that are not like others and and you're going to have to do this in cost and benefit analysis that's what they understand it costs this much but you get that much uh companies usually deal in deltas they don't mind spending a million dollars if they make two million dollars but if you just come in and say we need to spend a million dollars they're not talking to you you need to talk about we're gonna make a million dollars today with this cool solution right so uh whenever you do that just do a cost benefit analysis form if you're not familiar with how to do them just go out to your favorite search engine name bing and just type in perform a cost benefit analysis for a technical solution and you'll come back with some great simple examples awesome um thanks so much uh buck uh just for those of you that are just joining us i thought we just had a big spike in people join so welcome uh thanks for joining us by the way we are doing an ask the experts right now so from wherever you're streaming in go ahead and put your questions in and we are answering them as they come in i want to take one more question and then i want to make kind of shift gears towards talking towards azure arc uh but this last question i want to make sure we clarify because it's important uh karthik says i heard that sql mi general purpose is not for production workloads and the support sla is different from sql mi business critical can you please clarify yeah karthik i'm saying i'm glad you asked that because we want to make sure we make it crystal clear about why we'd use these different options for managed instances versus the other so the the comment about not for production workloads is definitely not true i want you to think of managed instance general purpose as a failover cluster instance just built for you right using like share storage so it has more capacity because of that for example we in preview today for 16 terabytes of storage for that option business critical is a scenario where we're building replicas non-shared storage right local storage so the failover time is going to be faster in those environments it turns out the slas are the same for both these right now so business critical offers replicas it's going to be a little bit faster failover because of the fact that we're always doing replicas plus you get one free read replica so if you need an environment with a little faster performance you're gonna have less space though you have faster performance for say a latency sensitive applications but you also need like a read replica that just comes with business critical built in plus a feature like in-memory otp only comes with business critical think of it like from an audition point of view but we have plenty of customers that are using managed systems today with general purpose in production and it serves their needs just fine awesome thanks and thanks karthik for that question um all right so we're gonna transition a little bit uh johnny skew is ready for it uh we are going to talk about azure arc because we've been hearing a lot about it and sometimes we've noticed there's some confusion so uh bob i would love if you could tell us you know like what is azure arc and you know there's a few different options so what how do we think about like an azure art sql server versus an azure arc managed instance yeah thank you anna so you know um we just made an announcement at the hybrid digital event here a few weeks ago about azure arc and it comes on a lot of different flavors so uh i don't have a specific site about azure arc for sql server but think of it this way think about an existing sql server you begin to attach to azure and take advantage of things like azure defender in your environment but anna what has happened is managed instance is a great technology running in azure public clouds and government clouds but some customers told us i love the concept of the managed part of the instance but i need to run it in my environment i need that feel of azure and a managed service but it cannot be run in the azure public cloud our government cloud i need more control of that infrastructure thus was born the idea of azure arc data services we announced the general availability of azure arc enabled managed instance just recently today it's technically in preview because the bits itself the actual launch of the bits available to you will be at the end of this month so it's pretty quick here coming right we also announced a preview of azure arc enabled postgres in this environment so think about these concepts always currents one of the biggest ones remember we talked about a managed instance being versionless well how about a version list in your environment how about the ability to deploy a managed instance very easily how about like deploy from the portal a managed instance just like you do in azure but that deployment will be pushed down into your environment but that same feel of deploying quickly and picking scale would be there as well and then because of the fact that we're connected to azure you get a unified management experience to see all your different sql assets because this is now part of kind of the azure family again though running in your environment and we want to make sure that you understand that the same security as tried and true for you in sql for managed citizens will exist in your environment as well in this azure arc scenario and then here's the kicker cloud billing how about for the first time ever paying for a sequel based on cloud billing running in your environment you're connected to azure so we can then track your usage and we can bill you for your azure subscription again for a managed instance running in your environment now i'm going to defer talking about the any hardware any kubernetes for a second because i'm going to let buck talk more about that but we're going to use the power of kubernetes to make this vision possible i kind of found that this slide might be interesting for you anna it kind of helps decide it's a little bit of what to use when remember you see here the managed instance the azure sql database even elastic pools the azure infrastructure you want a full instance just a database and notice it says here in azure microsoft managing that behind the scenes but what if you wanted it to not be an azure so ford is a good example of a company that's done that they came to us and said we love the managements and story we need to be able to run though the managed instance in our environment we said you know what if you can bring up a kubernetes distribution that you like could be many different flavors we think we can make that actually happen so it's any infrastructure now that you manage we can again i give you that feel of managed insulin with those capabilities i talked about you get the full sql server engine but you get automated hadr for example running on kubernetes powered by kubernetes running in your environment so i'd like to think this way one of our my colleagues denocker told me he said we're trying to bring you managed instance but not an azure but give you that feel of azure it's like mirroring what sql managed instance looks like today and that is the promise what azure arc we're trying to bring to customers today god show wow this is pretty cool and i think you know like a lot of people are going to be interested in this especially if you can't run uh in in azure like you mentioned um we did get a few questions but i was just wondering like is it possible to do a double click on this architecture because i'm still like trying to figure out like what that means as far as like it's azure sql manage instance but it's also a full sql server instance but it's not in azure but it's from azure right um why don't i take a stab at that um i'll tell you what uh i've got a little architecture diagram here that we can show if you like and it's it's not um it's not comprehensive and it's not you know got every little piece because it would take all day but let's talk about this so i i see a couple questions are like wait a minute your environment bob just whizzed by oh you put it in your environment what what is what is your environment well i'll tell you what it's in kubernetes and uh kubernetes as we've been pointed out by george who speaks a little greek i think from time to time uh means governor or pilot uh which is he's absolutely correct and in fact the people that named it uh pilot uh then don't like saying the whole word kubernetes so they call it k-8-s or k-8s because there's eight letters after it so they named it something that they immediately shortened well anyway uh kubernetes is an interesting environment such that you can declare in a manifest just a text file hey i'd like some groupings of things running and make sure they're always running and run them in a little environment and i'm just going to tell you what to do when you do it very similar in sql we say select star from table and we don't tell it where on the hard drive or how much memory or any of that it just goes and figures all that out we declare what we want in sql it's a declarative programming language and it goes and does the work um and this is the same you're gonna say i'd like an environment go do that and it will stand up storage and it will stand up things called pods which contain containers one or more containers and the containers can do just about anything but it'll keep them all together and you're thinking wow this sounds interesting um and does this scale or whatever well a little company i think was um google you may have heard of them i've heard of them from time to time they're running their enterprise on on on kubernetes right so you deploy kubernetes you do that wherever you want if it's standard kubernetes or even open shift we'll run it yes we will run azure sql in amazon or google or on premises or wherever it runs it's completely up to you and this is mind blowing so let me take a look at how this how this works so what we do first is you deploy kubernetes somewhere you figure that out there's there's an api that kubernetes uses that allows you to talk to it and uh the standard tool that's used there is something called kubecuddle kubectl.exe or if you're in linux just coop ctl and this is a command line interface that lets you talk to kubernetes well we also have azure data studio or even if you're deploying inside azure kubernetes service which is kubernetes running in azure you can use the azure cli to talk to that to that same api so once those tools are stood up and you've just got regular old everyday kubernetes running somewhere the first thing we'll do is we'll do an a z command that talks through the cube cuddle cd cli the interface the kubernetes api it will run out to the microsoft container registry pull down some containers set up some pods and create a deployment and the first one of these things we have is something called the controller this is a service that's going to know about everything else it sort of does all the work to talk to everything else now what i'm showing you right now is something called direct connected mode and and i'll explain the other version in a bit but after it gets itself up and running and it knows where it is and it's running on kubernetes yes that could be in uh it could be an amazon if you want uh or anywhere else then it starts bringing down other containers and pods and services things like monitoring and logs hadr scaling backups things like that brings all those down and this is awesome and so this works great and everything's running and you're saying okay great i can do this in kubernetes today where's the azure part we also bring down something into another area called azure arc integration now this is the magic sauce here in the direct connected mode what will happen is this i've sort of built this out and i'll explain this complexity in just a moment here uh coming down to the azure arc integration are any deployments you want or any actions you'd like to take also that that amazing advanced data security if you haven't played in this i'm a security guy from way back have my cissp and all that all that comes down you get all your gdpr reporting all of your enforcement everything you need comes down to that arc integrated area and what goes back up to azure is your inventory of all the resources any billing that you have going back and forth the logs and metrics from the system are going up there into insights so you can look at those things and so on and then any backup retention work that you're working with you can now work with that with the portal with the azure command line interface uh with azure data studio or with azcli commands and see all of this running let me see what have i forgotten oh oh that's right bob's favorite thing databases so now what you can do once you have this all running wherever if you're a bank and you have to be on premises but you want those reports from security and you want a bill in one place and you want the logging viewable anywhere and so on you say i want that thing that bob's been talking about this managed instance and i want it with the aha just give me that and you just put that right in the file down it comes it deploys uh from the controller and the azure arc integration now everybody's talking and as bob pointed out in preview today what we have is also postgres hyperscale this is the distributed version of postgresql if you're using that and allows you full integration that way and carries along through this system here's the amazing thing bob mentioned devops recently and all of these things can be put because it's a manifest because it's a file you can snapshot your environment and version it so you could version your environment as you go forward with devops uh allowing everything to maintain now keep in mind you get this evergreen sql mi so this whole thing of oh we waited ten years to upgrade and now we have a three-year migration project i was talking some of the day they just finished a seven year migration project i don't even want to think about that i can't do anything that feels good for seven years that that's just stunning to me that just goes away that problem just kind of goes away so with this uh again this sort of a larger architecture there's also obviously some fine-grained pieces to this but this is just mind-blowing i mean the way we can do this now so you're getting azure wherever you need it all the azure goodness wherever you need it and the mi goodness that bob's been talking about and if you need it post for sql as well awesome wow that is pretty cool i i mean i i i think there's a lot of questions that are coming in i just wanted to ask one more question before we get to some of those questions like this is a lot like is it hard to get started is it easy to get started how can people start playing with this yeah great question anna and and uh in fact uh what do they say a good question is the question i know the answer to and a great question is on the next slide uh so we have something called the azure arc jump start and we have it in github so it's freely available we give you a zero to hero uh ability to start with nothing and go up to the deployment types and we walk you right through this uh there's also some uh various scenarios kind of a supermarket we call it uh so that you can pull these things down say oh i want to look at one that looks like this and so you can pull those down that's just there at https azurearcjumpstart.io if you go to that you'll get a beautiful interface walks you step by step through deploying this so you can definitely get started there all the documentation all kinds of things and look for more training and other documentation coming as we get closer to sort of the release date that we talked about awesome uh thanks so much buck that was really helpful we do have some questions that come in but i also wanted to comment that we had some comments about you uh all pretty funny so i'm gonna share them [Laughter] i can see buck having some music related sequel database on azure arc i think there are instruments in the back we're just waiting for the you know theme song for azure arc yes i meant to talk to you about that i want to change up your theme song i was working with garageband today and i have some great ideas it involves uh like a lot of dubstep i'm seeing you as a step type yep that's that's definitely me and then another person i actually laughed at this one they said oh that guy who worked for contoso well yeah i left microsoft for a while and went to contoso to work in their medical division it didn't work out um it was i think it was a special hire you know just to meet some kind of thing they had and and they sent me back yeah so for those of you who uh aren't getting the contoso reference but came on data exposed a few weeks ago to talk about azure arc so that's another good plug so if you want to learn more about azure arc you should subscribe to our youtube channel and you can uh you can actually go the link yeah there it is thank you um so if you go to that page you can actually go watch this episode of data exposed live where buck is an employee at contoso a fictional company and he walks through all these use cases uh for azure arc with someone called dj data uh so it's a great episode i would definitely check it out if you want to learn uh more about azure arc um but that being said let's take a look at some of the other uh questions that have come in a one there is on ledger i can talk to that a little bit if you'd like for me to talk about um yeah the the the ledger feature that we're announcing for azure sql db allows you to have a off blockchain experience inside a database so what do we mean by that um so you have ethereum and the other blockchains that you can use and these are incredibly useful for banking but they're also useful for a myriad of things especially quality controlling manufacturing medical manufacturing all these kinds of things can use a blockchain but you may not want to be part of a of a public one because it's just out there and it can be slow and brittle and so on uh so we've created something called sql ledger and we now have inside the database the ability for you to make your own private blockchain experience uh for any of your data that you're doing there what this does is gives you ultimate all actions audited immutable all actions audited so no one can change anything without someone tracking that it's a really great feature and we do see people that are going to start using this uh for fiat currencies even to make their own transactions uh available awesome yeah and we we're seeing a lot of uh excitement around ledger and we have some dxposed episodes and some big those live episodes as far as our security series uh you're gonna want to check those out this next question comes in from oren who says how can users connect azure manage sql so i'm not sure if this is azure arc or azure sql manage instance so maybe we can address both but while having external ip addresses changing as they are working from home yeah i'll address that anna so there's a couple ways to connect into a managed instance one of those that we recommend is through an azure virtual network so then you would take your environment and try to make sure that your connectivity whatever that external ip address looks like is just part of a virtual network that ties into azure itself that's the more most secure method there is a public endpoint though there's a public endpoint that you could use to connect to a managed instance with a specific port there's something called a network security group where you can set up a set of rules to decide what ip addresses or range of ip addresses could be allowed to get in that public endpoint but again we kind of recommend using the virtual network approach which is the most secure way so there are ways to do that no matter what kind of ip address you're using to ensure that you're part of a virtual network that appears or connects into the virtual network of azure managed instance it was one of the big reasons another uh thing that customers wanted from us because at the time azure sql database did not have private link so this is a more secure way to do managed instance connectivity and so that's our recommendation and we have great discussions and documentation on this but hey i think you've done a data exposed live episode with a row hit on a network connectivity into things like azure database and managed instance that somebody could watch and understand how that works yeah and it looks like uh oren has clarified his question and says i was talking about managed sql not managed instance so i'm thinking oren can you correct us like are you talking about azure sql database specifically because you also mentioned it has no v-net native connectivity i'm just not really sure what you mean by managed sql that goes i really do recommend go back and edit you know you can go in your youtube series so rohick nyhak one of our colleagues he's kind of our connectivity guru and our team yeah yeah i don't know how many episodes you did with uh rohit and uh but he goes through exhaustive diagrams and so forth about how to connect anywhere from where you are into azure sql database or managed instances so i think that's what we're found there yeah i think that's the episode that he's looking for if he goes and checks it out so definitely oren go check out the stuff that uh that anna did with rohit um we you and i did of course azure sql for beginners and azure sql fundamentals so azure sql for beginners has like 60 videos and we have specific videos where you have visualizations about all the connectivity options to go into database so that could be a help as well yeah i think uh oren just to wrap that like the biggest thing you want to make sure is that whatever you're trying to connect from has access to either the virtual network where the private endpoint is deployed if you're using private endpoint or you have some sort of specific access setup either using virtual network rules or firewall rules so uh but bob and both mentioned uh we have done deep dives in a few places so just go to our youtube channels and find those um and i i think that will be helpful for you um awesome okay so the next question uh comes from aka i says this may be a silly question can quite a few things which we achieve with azure arc also be achieved if we open express route or vpn cytosite connectivity between my data center and azure yeah i'll i'll take a stab at this and then bob you can jump into um absolutely as far as like if you think of end result um i would like to query a database from on-prem and i want to do that without having to go through a v-net and so on absolutely you open up express route you're connected to your v-nets inside of azure so that's great uh the thing is there's a lot of environments where you're not allowed to do that so while you can get managed instance you absolutely could do that up in azure you could get sql db synapse azure ml storage all kinds of things if you just connect an express route man you're in good shape that way to get the same result but we're talking about environments where you can't do that or where you want to be able to take your environment and check it in ci cd with a kubernetes flavor or you want to run it over in another cloud or you want to run it on premises or all of the above you want to connect everything that way and you want to do it declaratively so i think you may want to separate the mechanism for what you do versus the goals of what you do so it's less about oh i can do this here and this here you probably do that in a lot of ways it just depends on what your requirements are and what your constraints are we always deal with requirements and constraints what it must do and what it's not allowed to do someone may say oh it must have that great azure security but the constraint is can't run in azure uh in that case this is exactly for that this is what ford that bob was talking about told us uh we love all that we want to do those things but nope it's got to be here make it work here and so we did yeah let me just add in that you know to be clear about the connectivity between arc and azure the whole idea is to arc together something you have to run your on-premises but still get the flavor of azure whether it's being able to deploy things in azure through a portal experience or cli and have it show up in your environment or to send data up for billing metrics logs and so forth when i think of express router and other options you talked about i think about connecting my applications into things running in azure so if you're running things in arc in your environment you probably already have a way within your own private network to connect those apps to those things awesome thanks uh thanks both um this next question i lost it oh i got it this next question is a little bit off topic but we're gonna take it for some light comedic relief a washington sports ball fan here this is andy leonard uh bob will the cowboys become cow men yeah andy great question today really on topic um i don't know anybody would be a sports ball fan of any washington team so i don't even know why you would pick that so it's a great question except the cowboys are already men you know we're already very very well established and you know i'm gonna go on record here today that we will beat washington this year that's not i think i think they're gonna win the world cup i really do oh my goodness i mean they could right i mean they could go all the way right but what's the phrase you say i hope all team people i just hope everybody has fun yeah i hope they let everybody play and everybody has fun let everybody play that's your yeah me play and have fun right exactly yeah well thank you uh andy for that nice distraction um and you know it's always interesting to talk about sports ball on a data show yeah that's what we call a sports ball yeah um all that being said um let me see we did get another question that came in earlier and this one i think is interesting it says what's the management differences for azure arc verse azure sql interesting one so um i'll take a step back chime in right so remember what we said in the beginning an azure sql managed instance with that key managed is that we are going to provide management capabilities like auto ha auto dr uh version list those are like three big things that stand out to me deployment uh those kind of type things that's about azure sql maintenance instance we want to bring those capabilities into your environment with arc that's why we call it azure arc managed instance right we want to give that same feel the differences are you own the infrastructure now you own the kubernetes uh pieces that are required to do that so in order in other words for us to do auto dr to do auto backups you have to provide all the storage to make that happen whereas of course in the cloud we're doing all of that behind the scenes so from a management perspective uh sql server itself all those capabilities you know and love all that just exists there in both those environments it really comes down to the automation for management services we're providing that are the same but but the biggest difference is you run the environment so the infrastructure pieces all the automation to do that are your responsibility now hey bob can i let this be a great question yeah okay anna can you show them my great question look at that uh it's almost like it was teased up or something it was almost like it was teed up um it wasn't by the way i just so you all know this actually was hidden we were going to show this and decided not to uh but i real quick like a bunny just unhid it um so if you look here you'll see sort of the management and i think what bob was pointing out is very important it's the it's the level of management you care about as far as tools management studio azure portals and things like that command line things third party tools those are all fine they're going to connect in at different levels but for instance in sqldb you're not going to go worry about running out of drive space on the base operating system we'll we'll handle that for you but in in other things you might right if you're on prem you might you might actually care about whether you've got enough room on the d drive to store your database so you can kind of see the different levels of uh management comparisons that we have here that'll tell you where you take care of it and where we take care of it uh quick question on this i see that there are some like very light blue check boxes some dark blue check boxes some half and half what what does that mean go for it bob yeah so look at uh for example sql managed instance azure arc uh elastic scalability so it's true that we're gonna be able to have scale because we're running kubernetes but it's not gonna be the same scale as in the cloud because we run the entire infrastructure so the half ones are more about you own some of that we're going to try to give you those same capabilities in azure but you own a piece of it look at automatic hadr i was just talking about this we can make the dr happen at a sql perspective but outside sql if you don't set up all the storage right then the dr may be meaningless whereas remember we talked about geo redundancy like in azure cloud we automatically take our backups and spread this across azure data centers in various regions you would have to do that yourself right that's successful because you and the infrastructure if you look at azure sql management for those big light blue ones data sovereignty that's a good one because it could be that the azure public cloud doesn't meet your sovereignty needs actually that's one reason why people do that whereas in azure arc you control where this this resides so you have complete control of the sovereignty issue this is why there's a difference between the dark blue and arc and then the lighter blue because in many azure public clouds or government clouds it may need meet your sovereignty needs but there could be some regions where we don't support you and therefore you need to go run this since another great reason why to run arc yep absolutely and it uh to put a top on that um it just depends on what you want to care about what do you want to control what you want to care about and if you you're like look i'd really like hdr handled then then you pick the options that'll do that for you awesome well i think that was a great question to kind of end on i think we answered a ton of questions this was super useful but bob thanks so much uh for joining us today uh i think we'll probably have to do this again if you're willing um and in the askbw ask bob ward show the ask the askbw woody show well would you tell me ahead of time that he's actually going to be on your next level now you know yeah okay don't tell him ahead of time you won't show up yeah um well to our viewers we want to thank you for joining us bringing all your great questions uh engaging with us it's useful for us to see what people are asking just as much as hopefully it's useful for you to see what we're saying as answers it's been a great show we want to thank you so much for joining us on data exposed live uh we got some uh thank you's coming in so thank you all uh for joining and to our viewers be sure to subscribe to our youtube channel we stream every wednesday at this time we also release new episodes on thursday so with that being said we hope to see you next time on data exposed [Music] you

Channel: Azure SQL

Views: 333

Keywords:

Id: rOBfbci64AU

Length: 60min 22sec (3622 seconds)

Published: Wed Jul 21 2021